From 89f9e4fc2109d7163461f0ea0f6ed771809dd9e1 Mon Sep 17 00:00:00 2001 From: Dudu-0223 Date: Mon, 13 Jul 2026 16:17:36 +0800 Subject: [PATCH 01/73] feat(fs-search): hide grep glob without rg --- docs/config-catalog.md | 2 +- ...6-07-09-bash-backed-grep-glob-discovery.md | 20 ++--- docs/tool-catalog.md | 4 +- examples/coding-agent/cordis.yml | 7 +- .../core/tools/tests/gen-tool-catalog.spec.ts | 13 ++++ packages/fs/README.md | 4 +- packages/fs/tool-fs-search/README.md | 12 +-- packages/fs/tool-fs-search/src/index.ts | 52 +++++++++++-- .../fs/tool-fs-search/tests/tools.spec.ts | 78 ++++++++++++++++--- scripts/gen-tool-catalog.ts | 72 +++++++++++++++-- 10 files changed, 218 insertions(+), 46 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 53db69125c..5ee653103e 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -871,7 +871,7 @@ export interface Config { } ``` -Source: [`packages/fs/tool-fs-search/src/index.ts:59`](../packages/fs/tool-fs-search/src/index.ts) +Source: [`packages/fs/tool-fs-search/src/index.ts:62`](../packages/fs/tool-fs-search/src/index.ts) ## `@deepseek-ai/dsh-tool-skill` diff --git a/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md b/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md index bc8a452c91..4644f3e07a 100644 --- a/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md +++ b/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md @@ -10,13 +10,13 @@ Search output also has two distinct budgets. The tool needs enough raw `rg` outp ## Decision -`glob` and `grep` are model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, backed by the bash seam, not by new `ctx.fs` provider methods. The package registers model-facing filesystem discovery tools, but execution uses `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)` with fixed `rg` command templates assembled by the tool. The tool layer owns schemas, argument validation, shell quoting, result parsing, result formatting, retention, formatted-result spill handoff, and timeout declaration. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution across local, sandboxed, or remote bash implementations. +`glob` and `grep` are conditional model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, backed by the bash seam, not by new `ctx.fs` provider methods. At plugin load, the package checks `command -v rg >/dev/null 2>&1` through `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)`; if the command exits nonzero, the package logs a warning and registers neither tools nor prompt sections. A probe that cannot start, times out, aborts, is killed, or produces no exit code fails plugin load loudly because that is a broken bash executor rather than an absent optional binary. When registered, execution uses the same `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)` flow with fixed `rg` command templates assembled by the tool. The tool layer owns schemas, argument validation, shell quoting, result parsing, result formatting, retention, formatted-result spill handoff, and timeout declaration. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution across local, sandboxed, or remote bash implementations. The tools do not use `ctx.bash.start()` and do not create model-visible background tasks. They run as ordinary foreground tools from the agent loop's perspective: the tool call returns only after the `rg` command exits, times out, is aborted, or fails. `defineTool({ timeoutMs })` declares the cooperative tool-call budget, `@deepseek-ai/dsh-timeout-policy` enforces it through `exec.signal`, and the tool forwards that signal into the bash request before `resolve()` / `run()`. The bash backend's own timeout remains a second safety cap; whichever aborts first wins. The tools align `path` with Claude Code's search tools while binding resolution to the bash workdir, not to `ctx.fs`. The tool derives the bash request workdir from `exec.agent?.session.header.cwd`, mirroring `dsh-tool-bash` and `dsh-tool-fs`; when no session cwd exists, it omits `request.workdir` so the bash implementation applies its configured cwd or process cwd through `resolve()`. For `grep`, `path` is an optional ripgrep target and may be a file or directory; omitted means the resolved bash workdir. For `glob`, `path` is an optional directory search root; omitted means the resolved bash workdir. Relative `path` values resolve against that workdir. Returned paths are displayed relative to the resolved bash workdir when possible and are intended to be follow-up-readable only in co-located deployments where the bash workdir and filesystem `read` root are the same workspace. v1 documents that deployment requirement but does not perform runtime cross-service validation. Remote or virtual filesystem search is deferred until there is a shared workspace/root contract or a provider-specific search backend. -The package does not inject `fs`. It injects `tools`, `systemPrompt`, and `bash`; it deliberately reads `spillStore` with `ctx.get('spillStore')` instead of static inject because formatted-result spill is optional. Existing `@deepseek-ai/dsh-tool-fs` deployments that only want `read` / `write` / `edit` do not need to load bash. +The package does not inject `fs`. It injects `tools`, `systemPrompt`, and `bash`; it deliberately reads `spillStore` with `ctx.get('spillStore')` instead of static inject because formatted-result spill is optional. Existing `@deepseek-ai/dsh-tool-fs` deployments that only want `read` / `write` / `edit` do not need to load bash. Deployments that load search need `rg` available in the bash executor environment for the tools to enter the model-visible schema. ### Package shape @@ -79,9 +79,9 @@ The `path` field follows the same split as Claude Code: `grep.path` is a file-or Raw `rg` stdout is an internal transport detail. The tool requests `stdoutMaxBytes: rawOutputMaxBytes` through `ctx.bash.resolve()` and parses `stdout.text` only when the executor returns untruncated stdout within that cap. If stdout is larger than `rawOutputMaxBytes`, or the executor still returns `stdout.truncated`, the tool fails with a clear search error telling the model to narrow `pattern`, `path`, or `include`. The tool never exposes raw `rg` output or bash raw spill paths to the model. -Only stdout is a parse source. Stderr is diagnostic text for invalid patterns, missing `rg`, and search failures; if bash truncates stderr, the tool uses the retained stderr tail with a truncation note and does not read `stderr.spillPath`. +Only stdout is a parse source. Stderr is diagnostic text for invalid patterns, runtime `rg` disappearance after registration, and search failures; if bash truncates stderr, the tool uses the retained stderr tail with a truncation note and does not read `stderr.spillPath`. -If `ctx.bash.run()` reports `aborted` because the tool timeout or caller cancellation fired, the tool returns a structured failure rather than pretending there were no matches. If bash reports its own timeout first, the tool likewise fails with a clear timeout message. Nonzero ripgrep exit semantics are tool-owned: exit 0 is success with matches, exit 1 is success with no matches, invalid pattern / missing `rg` / inaccessible search workdir are failures. +If `ctx.bash.run()` reports `aborted` because the tool timeout or caller cancellation fired, the tool returns a structured failure rather than pretending there were no matches. If bash reports its own timeout first, the tool likewise fails with a clear timeout message. Nonzero ripgrep exit semantics are tool-owned: exit 0 is success with matches, exit 1 is success with no matches, invalid pattern / runtime `rg` disappearance / inaccessible search workdir are failures. Search failures use a package-owned `HarnessError` subclass with `SEARCH_*` codes, not `FsErrorCode`, because these tools are not `ctx.fs` provider operations. The v1 vocabulary is `SEARCH_INVALID_PATTERN`, `SEARCH_FAILED`, `SEARCH_RAW_OUTPUT_OVERFLOW`, and `SEARCH_ABORTED`. Model argument validation failures such as missing required fields, blank strings, or unsupported negated/list `include` values remain ordinary tool argument errors. @@ -116,7 +116,7 @@ Line 12: ... (Full grep result stored at: /.../session-abc123/9f8e7d-grep-results.txt. Use read with offset/limit, or grep this path to search within it.) ``` -If the complete logical result fits under the inline cap, no formatted spill artifact is created. If the complete logical result is too large but formatted spill is unavailable, the footer says that the result was capped and the complete result could not be saved. The `truncated` / omitted count is a budget fact, not an incomplete-search fact; timeout, invalid regex, missing `rg`, inaccessible workdirs, raw-output overflow, binary skips, and parse failures stay in tool-domain error or incomplete fields. +If the complete logical result fits under the inline cap, no formatted spill artifact is created. If the complete logical result is too large but formatted spill is unavailable, the footer says that the result was capped and the complete result could not be saved. The `truncated` / omitted count is a budget fact, not an incomplete-search fact; timeout, invalid regex, runtime `rg` disappearance, inaccessible workdirs, raw-output overflow, binary skips, and parse failures stay in tool-domain error or incomplete fields. ## Alternatives considered @@ -138,22 +138,24 @@ If the complete logical result fits under the inline cap, no formatted spill art **Expand the bash seam with a raw-output reader first.** Rejected: a portable `readRawOutput(ref, maxBytes)` API would add reference lifetime, permission, and backend storage semantics. A per-run `stdoutMaxBytes` request is the narrower seam: search either receives complete stdout within `rawOutputMaxBytes` or fails clearly. +**Always register and report missing `rg` only at execution time.** Rejected: a model-visible tool schema is a promise that the deployment can attempt that capability. If the bash executor cannot find ripgrep at load, the safer surface is no `glob` / `grep` tools or prompt guidance. Execution-time missing-`rg` classification remains as a defensive fallback for environments that change after registration. + ## Testing -- Tests prove an aborted `exec.signal` reaches the bash backend (same-reference spec assertion plus the `SEARCH_ABORTED` result), and cover command construction/quoting (malicious patterns, paths with spaces, leading-dash values, quotes, newlines, glob metacharacters — unit assertions plus a real `bash -c` round-trip for every hostile value), `grep.path` as file and directory targets, `glob.path` as a directory search root, invalid pattern handling, no matches, malformed `rg --json` output, matched-line preview truncation, raw-output overflow, timeout/abort, formatted spill success/failure, the package-owned `SEARCH_*` error codes, and the no-background-task invariant. +- Tests cover registration-time `rg` probing (probe success registers both tools and prompt sections, nonzero probe skips both tools and prompt sections with a warning, infrastructure probe failures reject plugin load), prove an aborted `exec.signal` reaches the bash backend (same-reference spec assertion plus the `SEARCH_ABORTED` result), and cover command construction/quoting (malicious patterns, paths with spaces, leading-dash values, quotes, newlines, glob metacharacters — unit assertions plus a real `bash -c` round-trip for every hostile value), `grep.path` as file and directory targets, `glob.path` as a directory search root, invalid pattern handling, no matches, malformed `rg --json` output, matched-line preview truncation, raw-output overflow, timeout/abort, formatted spill success/failure, the package-owned `SEARCH_*` error codes, and the no-background-task invariant. - The first-party tool-owned spill precedent is covered directly: spill backend present, spill backend absent, `saveText()` failure, and missing spill owner. - The package has real Loader-path coverage for the namespace plugin export shape (`name`, `inject`, `Config`, and `apply`, with no default export). -- A real-executor integration suite (`dsh-bash-local` + a real `rg`) verifies the world: hostile patterns stay inert, per-session cwd resolution, VCS-metadata exclusion, modification-time ordering, and real ripgrep stderr classification. It self-skips where `rg` is not on PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor suite alone carries the per-file 100% coverage gate. +- A real-executor integration suite (`dsh-bash-local` + a real `rg`) verifies the world: hostile patterns stay inert, per-session cwd resolution, VCS-metadata exclusion, modification-time ordering, and real ripgrep stderr classification. It self-skips where `rg` is not on the test process PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor suite carries registration and execution coverage for missing `rg`, plus the per-file 100% coverage gate. - Snapshot gap note for the transcript-visible spill notice: this landed with the gap note, not a snapshot. The snapshot tier replays the acp-agent tree, and adding the search plugin there changes the assembled system prompt — every golden would need re-recording with a real key, which the implementing environment did not hold. The spill notice's exact transcript text is pinned by unit tests (`formatGlobOutput`/`formatGrepOutput` and the through-the-registry spill tests); wiring the plugin into the acp-agent tree plus a `test:snapshot:record` pass is the follow-up for the next key-holding session. ## Consequences -- `glob` and `grep` are model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, not `ctx.fs` provider methods and not part of the existing `@deepseek-ai/dsh-tool-fs` root plugin. The package injects `tools`, `systemPrompt`, and `bash`; it does not inject `fs`, and `ctx.spillStore` stays optional via `ctx.get('spillStore')`. +- `glob` and `grep` are conditional model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, not `ctx.fs` provider methods and not part of the existing `@deepseek-ai/dsh-tool-fs` root plugin. They register only when the bash executor can find `rg`; the package injects `tools`, `systemPrompt`, and `bash`, does not inject `fs`, and keeps `ctx.spillStore` optional via `ctx.get('spillStore')`. - The schemas are exactly `glob(pattern, path?)` and `grep(pattern, path?, include?)`; search caps and timeout are defaulted, validated Config fields (`globMaxResults`, `grepMaxMatches`, `grepMaxLineBytes`, `rawOutputMaxBytes`, `timeoutMs`). - The tools execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)`, forward `exec.signal`, never call `ctx.bash.start()`, and never expose a bash task id. The bash request workdir comes from `exec.agent?.session.header.cwd` when available; the resolved `spec.workdir` drives execution and relative-path display. - The tools request `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam, parse only untruncated stdout within that cap, and treat over-cap or still-truncated raw output as a clear search failure; raw `rg` output is never exposed to the model. - Oversized complete formatted results are saved through `ctx.spillStore.saveText()` when available while inline results stay bounded; spill failure, a missing backend, or a missing owner preserves the inline result and reports the unsaved remainder — never an `isError`. -- The package README, the generated config catalog, and exported JSDoc document the Config fields and `SEARCH_*` codes; the coding-agent example ships the tools (the acp-agent tree waits on the snapshot re-record above); the fs group README records the co-located bash/filesystem deployment requirement. +- The package README, the generated config catalog, and exported JSDoc document the Config fields and `SEARCH_*` codes; the coding-agent example ships the conditional tool plugin (the acp-agent tree waits on the snapshot re-record above); the fs group README records the `rg` availability and co-located bash/filesystem deployment requirements. ## Risks diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index 382eb04c4c..5b6e281b1a 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -20,7 +20,7 @@ This table connects model-visible tool names to the plugin package and service s | `@deepseek-ai/dsh-tool-bash` | `bash`, `bash_kill`, `bash_output` | `ctx.tools`, `ctx.bash` | `tool/call`, `tool/result`, `context/message via agent.inject() for background completion notices` | - | The bash/bash_output/bash_kill tools are model-facing consumers of the bash executor seam. | | `@deepseek-ai/dsh-tool-cordis` | `cordis_inspect`, `cordis_mount`, `cordis_unmount` | `ctx.tools` | `tool/call`, `tool/result`, `live plugin-tree mutations (mount/unmount)` | - | Ships in examples/cordis-agent only (a deliberate opt-in — mounted code gets the real ctx, see docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md). Plugins the model mounts may register ADDITIONAL model-visible tools at runtime; the request-header ToolsDelta logs those tool-set changes. | | `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after successful file operations`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The tool schemas above are identical with or without the policy plugin. | -| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | +| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | | `@deepseek-ai/dsh-tool-skill` | `skill` | `ctx.tools`, `ctx.skills` | `tool/call`, `tool/result` | - | - | | `@deepseek-ai/dsh-tool-subagent` | `subagent` | `ctx.tools`, `ctx.subagents` | `tool/call`, `tool/result`, `child session events through the chosen provider` | `subagent`, `subagent_fork` | The registered tool name is the load-time `toolName` config (default `subagent`); the schema above is that default. The shipped example agents load this package once per subagent backend, so the model additionally sees `subagent_fork` (bound to the fork backend) with an identical schema — see `examples/coding-agent/cordis.yml` and `examples/acp-agent/cordis.yml`. | | `@deepseek-ai/dsh-tool-todo` | `todo_write` | `ctx.tools`, `owning Agent session` | `tool/call`, `todo/write`, `tool/result` | - | todo_write is session-owned state; UIs render the latest todo/write event as a checklist or ACP plan. | @@ -428,7 +428,7 @@ Search file contents with a ripgrep regular expression. Returns matching lines w Source: [`packages/fs/tool-fs-search/src/index.ts`](../packages/fs/tool-fs-search/src/index.ts) -glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. +glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. ## `@deepseek-ai/dsh-tool-skill` diff --git a/examples/coding-agent/cordis.yml b/examples/coding-agent/cordis.yml index b23d785439..698ae97403 100644 --- a/examples/coding-agent/cordis.yml +++ b/examples/coding-agent/cordis.yml @@ -133,9 +133,10 @@ - id: tool-fs name: '@deepseek-ai/dsh-tool-fs' -# Bash-backed discovery tools (glob/grep): fixed ripgrep commands through the -# local bash executor above — not ctx.fs. Capped results save the complete -# formatted list through the spill backend below (ctx.spillStore, optional). +# Bash-backed discovery tools (glob/grep): if the local bash executor above +# can find rg, register fixed ripgrep commands — not ctx.fs. Capped results +# save the complete formatted list through the spill backend below +# (ctx.spillStore, optional). - id: tool-fs-search name: '@deepseek-ai/dsh-tool-fs-search' diff --git a/packages/core/tools/tests/gen-tool-catalog.spec.ts b/packages/core/tools/tests/gen-tool-catalog.spec.ts index 43490b99dc..b77cbdfd8d 100644 --- a/packages/core/tools/tests/gen-tool-catalog.spec.ts +++ b/packages/core/tools/tests/gen-tool-catalog.spec.ts @@ -61,6 +61,19 @@ describe('gen-tool-catalog collectToolCatalog', () => { expect(bash?.source).toBe('packages/bash/tool-bash/src/index.ts') }) + it('harvests search tools without depending on the generator process PATH', async () => { + const oldPath = process.env.PATH + try { + process.env.PATH = '' + const catalog = await collectToolCatalog() + const search = catalog.find(entry => entry.pkg === '@deepseek-ai/dsh-tool-fs-search') + expect(search?.schemas.map(s => s.name).sort()).toEqual(['glob', 'grep']) + } finally { + if (oldPath === undefined) delete process.env.PATH + else process.env.PATH = oldPath + } + }) + it('records the shipped `subagent_fork` alias in a note (config-driven tool name)', async () => { // `tool-subagent`'s registered name is the load-time `toolName` config, so // the shipped agents surface this one package as both `subagent` and diff --git a/packages/fs/README.md b/packages/fs/README.md index 039cb39ae9..ba57eede25 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -8,9 +8,9 @@ The filesystem stack: a provider seam (text IO + atomic mutation with an optiona | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | | `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) | -| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools, backed by fixed ripgrep commands through the bash seam (`ctx.bash`), NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | +| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). ## No timeouts on file IO diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index 2f9df200de..81e0042e92 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -1,20 +1,20 @@ # @deepseek-ai/dsh-tool-fs-search -The **model-facing filesystem discovery tools** — `glob`, `grep` — backed by the **bash executor seam**, not by `ctx.fs` provider methods. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a bounded, workdir-relative result. The package injects `tools`, `systemPrompt`, and `bash` — deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. +The **model-facing filesystem discovery tools** — `glob`, `grep` — backed by the **bash executor seam**, not by `ctx.fs` provider methods. At load, the package probes `command -v rg` through `ctx.bash`; if the executor cannot find ripgrep on its `PATH`, it logs a warning and registers no tools or prompt sections. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a bounded, workdir-relative result. The package injects `tools`, `systemPrompt`, and `bash` — deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. ```ts ignore-check -// Default deployment: a bash executor, then the discovery tools. +// Default deployment: a bash executor whose PATH includes rg, then the discovery tools. await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local -await ctx.plugin(ToolFsSearch) // this package — registers glob/grep +await ctx.plugin(ToolFsSearch) // this package — conditionally registers glob/grep // Optional: a spill backend makes capped results fully recoverable. await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local ``` Why bash-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution (local, sandboxed, remote); this package owns schemas, argument validation, shell quoting, parsing, retention, formatted-result spill, and timeout declaration. The tools never call `ctx.bash.start()` and never expose a bash task id — the call returns only after `rg` exits, times out, is aborted, or fails. -## Deployment requirement: co-located bash + filesystem +## Deployment requirement: rg + co-located bash/filesystem -Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. +The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin load; otherwise `glob` and `grep` are absent from the model-visible tool schema. Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. ## Config @@ -43,4 +43,4 @@ Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMax ## Errors -Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (missing `rg`, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. +Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. diff --git a/packages/fs/tool-fs-search/src/index.ts b/packages/fs/tool-fs-search/src/index.ts index 8c33d5770a..5930890b7a 100644 --- a/packages/fs/tool-fs-search/src/index.ts +++ b/packages/fs/tool-fs-search/src/index.ts @@ -1,6 +1,7 @@ /** * The model-facing filesystem discovery tool suite (`glob`, `grep`) over the - * bash executor seam (`ctx.bash`). This single plugin registers both tools. + * bash executor seam (`ctx.bash`). This single plugin registers both tools + * only when the mounted bash executor can find `rg` on its `PATH`. * * ## Bash-backed, not a `ctx.fs` provider method * @@ -12,9 +13,11 @@ * parsing, retention, formatted-result spill, and timeout declaration; the * bash executor owns request defaulting/capping, subprocess execution, * process-group termination, environment scrubbing, raw output capture, and - * backend substitution. The package injects `tools`, `systemPrompt`, and - * `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read opportunistically - * with `ctx.get()` because formatted-result spill is optional. + * backend substitution. At load, the package probes `command -v rg` through the + * same bash seam; if ripgrep is absent, `glob` / `grep` and their prompt + * sections are not registered. The package injects `tools`, `systemPrompt`, + * and `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read + * opportunistically with `ctx.get()` because formatted-result spill is optional. * * Returned paths are displayed relative to the resolved bash workdir and are * follow-up-readable only in co-located deployments where the bash workdir and @@ -80,6 +83,9 @@ export const Config: z = z.object({ /** The shape after schemastery applied the defaults. */ type ResolvedConfig = Required +/** POSIX-shell builtin probe for the ripgrep binary in the bash executor environment. */ +const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + /** Every search cap counts items/bytes/milliseconds — a positive integer, or retention and timeout arithmetic misbehaves silently. */ function assertPositiveInteger(name: string, value: number): void { if (!Number.isInteger(value) || value < 1) { @@ -87,8 +93,38 @@ function assertPositiveInteger(name: string, value: number): void { } } -/** Register the `glob`/`grep` filesystem discovery tool suite. */ -export function apply(ctx: Context, config: Config): void { +/** + * Check whether the mounted bash executor can find `rg`. + * + * Nonzero exit means "not available" and disables this optional tool suite. + * Infrastructure failures stay loud: a deployment with a broken bash executor + * should not silently lose tools in a way that looks like a deliberate skip. + * + * @param ctx - plugin context whose `bash` service is the executor the tools will use. + * @returns true when `command -v rg` exits 0, false when it exits nonzero. + */ +async function ripgrepAvailable(ctx: Context): Promise { + const spec = ctx.bash.resolve({ command: RG_PROBE_COMMAND }) + let result + try { + result = await ctx.bash.run(spec) + } catch (error: unknown) { + throw new Error(`tool-fs-search: ripgrep availability probe could not start: ${String(error)}`, { cause: error }) + } + if (result.aborted || result.timedOut || result.signal !== null || result.exitCode === null) { + throw new Error('tool-fs-search: ripgrep availability probe did not complete') + } + return result.exitCode === 0 +} + +/** + * Register the `glob`/`grep` filesystem discovery tool suite when `rg` exists. + * + * @param ctx - plugin context; registrations are effects scoped to this plugin. + * @param config - resolved plugin configuration from schemastery. + * @returns when ripgrep is unavailable, resolves without registering any tools. + */ +export async function apply(ctx: Context, config: Config): Promise { // schemastery (Config) has already filled every defaulted field. const resolved = config as ResolvedConfig assertPositiveInteger('globMaxResults', resolved.globMaxResults) @@ -96,6 +132,10 @@ export function apply(ctx: Context, config: Config): void { assertPositiveInteger('grepMaxLineBytes', resolved.grepMaxLineBytes) assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes) assertPositiveInteger('timeoutMs', resolved.timeoutMs) + if (!await ripgrepAvailable(ctx)) { + ctx.logger.warn('tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered') + return + } applyGlobTool(ctx, { maxResults: resolved.globMaxResults, rawOutputMaxBytes: resolved.rawOutputMaxBytes, diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index 9131940de5..648eb19585 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -2,12 +2,12 @@ * Consumer-surface tests for the search tools over a FAKE bash executor and a * FAKE spill backend, exercised through `ctx.tools.execute()` so nothing * bypasses the tool registry. The fake executor makes every seam outcome - * scriptable — truncated stdout with/without a raw spill path, abort/timeout, - * signal kills, ripgrep exit codes — so these tests verify schemas, argument - * validation, shell-safe command construction, workdir derivation, signal - * forwarding, `SEARCH_*` error classification, retention, formatted-result - * spill handoff, and the no-background-task invariant. Real-`rg` behavior is - * pinned separately in integration.spec.ts. + * scriptable — registration-time `rg` probing, truncated stdout with/without a + * raw spill path, abort/timeout, signal kills, ripgrep exit codes — so these + * tests verify schemas, argument validation, shell-safe command construction, + * workdir derivation, signal forwarding, `SEARCH_*` error classification, + * retention, formatted-result spill handoff, and the no-background-task + * invariant. Real-`rg` behavior is pinned separately in integration.spec.ts. */ import { describe, expect, it } from 'vitest' @@ -31,6 +31,8 @@ import { toWorkdirRelative, } from '@deepseek-ai/dsh-tool-fs-search' +const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + /** A successful run result over the given stdout; overrides script the failure shapes. */ function runResult(stdout: string, overrides?: Partial): BashRunResult { return { @@ -52,13 +54,18 @@ function runResult(stdout: string, overrides?: Partial): BashRunR * create a background task. */ class FakeBash extends BashExecutor { + probeRequests: BashExecRequest[] = [] + probeSpecs: BashExecSpec[] = [] requests: BashExecRequest[] = [] specs: BashExecSpec[] = [] startCalls = 0 + probeResult: BashRunResult = runResult('') + probeError?: Error handler: (spec: BashExecSpec) => BashRunResult = () => runResult('') override resolve(request: BashExecRequest): BashExecSpec { - this.requests.push(request) + if (request.command === RG_PROBE_COMMAND) this.probeRequests.push(request) + else this.requests.push(request) return { command: request.command, workdir: request.workdir ?? '/work', @@ -69,9 +76,14 @@ class FakeBash extends BashExecutor { sandboxMode: request.sandboxMode, } } - override run(spec: BashExecSpec): Promise { + override async run(spec: BashExecSpec): Promise { + if (spec.command === RG_PROBE_COMMAND) { + this.probeSpecs.push(spec) + if (this.probeError) throw this.probeError + return this.probeResult + } this.specs.push(spec) - return Promise.resolve(this.handler(spec)) + return this.handler(spec) } override start(): BashTask { this.startCalls++ @@ -113,18 +125,36 @@ class FakeSpill extends SpillStore { interface SetupOptions { config?: ToolFsSearch.Config spill?: boolean + probeError?: Error + probeResult?: BashRunResult } async function setup(options: SetupOptions = {}) { const ctx = new Context() + const warnings: string[] = [] + ctx.logger.warn = ((message: unknown) => { warnings.push(String(message)) }) as typeof ctx.logger.warn await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(FakeBash) + const bash = ctx.bash as FakeBash + if (options.probeResult) bash.probeResult = options.probeResult + if (options.probeError) bash.probeError = options.probeError if (options.spill === true) await ctx.plugin(FakeSpill) const fiber = await ctx.plugin(ToolFsSearch, options.config) - const bash = ctx.bash as FakeBash const spill = options.spill === true ? ctx.get('spillStore') as FakeSpill : undefined - return { ctx, bash, spill, fiber } + return { ctx, bash, spill, fiber, warnings } +} + +/** Assert plugin setup rejects without letting Vitest pretty-print a live Context on failure. */ +async function expectSetupRejects(options: SetupOptions, message: RegExp): Promise { + let thrown: string | undefined + try { + const loaded = await setup(options) + await loaded.fiber.dispose() + } catch (error: unknown) { + thrown = error instanceof Error ? error.message : String(error) + } + expect(thrown).toMatch(message) } /** A stand-in agent whose session header carries the given cwd (and a stable id). */ @@ -152,13 +182,37 @@ function matchLine(path: string, lineNumber: number, lineText: string): string { describe('registration', () => { it('registers glob and grep with their prompt sections', async () => { - const { ctx } = await setup() + const { ctx, bash } = await setup() + expect(bash.probeRequests).toHaveLength(1) + expect(bash.probeRequests[0]?.command).toBe(RG_PROBE_COMMAND) + expect(bash.probeRequests[0]).not.toHaveProperty('workdir') expect(ctx.tools.schemas().map(s => s.name).sort()).toEqual(['glob', 'grep']) const prompt = renderPrompt(await ctx.systemPrompt.assemble()) expect(prompt).toContain('Use the glob tool') expect(prompt).toContain('Use the grep tool') }) + it('does not register glob or grep when the bash executor cannot find rg', async () => { + const { ctx, warnings } = await setup({ probeResult: runResult('', { exitCode: 1 }) }) + expect(ctx.tools.schemas()).toHaveLength(0) + const sections = (await ctx.systemPrompt.assemble()).sections.map(s => s.name) + expect(sections).not.toContain('tool:glob') + expect(sections).not.toContain('tool:grep') + expect(warnings).toEqual([ + 'tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered', + ]) + }) + + it('rejects plugin load when the rg availability probe cannot run', async () => { + await expectSetupRejects({ probeError: new Error('spawn bash ENOENT') }, /spawn bash ENOENT/) + }) + + it('rejects plugin load when the rg availability probe is aborted or killed', async () => { + await expectSetupRejects({ + probeResult: runResult('', { aborted: true, exitCode: null, signal: 'SIGTERM' }), + }, /tool-fs-search: ripgrep availability probe did not complete/) + }) + it('stays pending until ctx.bash exists (inject)', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index e317f2be24..91ac3adfd1 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -39,6 +39,8 @@ import { Context } from 'cordis' import type { ToolSchema } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools' +import { BashExecutor } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskId, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' import LocalBashExecutor from '@deepseek-ai/dsh-bash-local' import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import UserInteractionService from '@deepseek-ai/dsh-user-interaction' @@ -63,6 +65,65 @@ import * as ToolWorkflow from '@deepseek-ai/dsh-tool-workflow' const root = resolve(import.meta.dirname, '..') const OUT = 'docs/tool-catalog.md' +const CATALOG_RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + +/** + * Minimal bash service for harvesting `dsh-tool-fs-search` schemas. The search + * plugin now probes `rg` at registration time, but the generated catalog must + * remain independent of the host PATH and never execute a real search. + */ +class CatalogSearchBashExecutor extends BashExecutor { + override resolve(request: BashExecRequest): BashExecSpec { + return { + command: request.command, + workdir: request.workdir ?? root, + timeoutMs: request.timeoutMs ?? 60_000, + stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, + signal: request.signal, + owner: request.owner, + sandboxMode: request.sandboxMode, + } + } + + override run(spec: BashExecSpec): Promise { + if (spec.command !== CATALOG_RG_PROBE_COMMAND) { + throw new Error(`gen-tool-catalog: unexpected search bash command during schema harvest: ${spec.command}`) + } + return Promise.resolve({ + exitCode: 0, + signal: null, + timedOut: false, + aborted: false, + timeoutMs: spec.timeoutMs, + stdout: { text: '', truncated: false }, + stderr: { text: '', truncated: false }, + }) + } + + override start(): BashTask { + throw new Error('gen-tool-catalog: search schema harvest must not start bash tasks') + } + + override get(): BashTask | undefined { + return undefined + } + + override ownerOf(): OwnerToken | undefined { + return undefined + } + + override list(): BashTask[] { + return [] + } + + override readOutput(id: BashTaskId): BashTaskRead { + throw new Error(`gen-tool-catalog: unknown bash task ${id}`) + } + + override kill(id: BashTaskId): boolean { + throw new Error(`gen-tool-catalog: unknown bash task ${id}`) + } +} /** * One tool-plugin package to boot. `mount` is a per-entry recipe (async): it @@ -192,14 +253,15 @@ const TOOL_PACKAGES: ToolPackage[] = [ writes: ['tool/call', 'tool/result'], async mount(ctx) { // The tools inject `bash` (search executes fixed `rg` commands through - // the executor seam, not ctx.fs); boot the local executor to satisfy it. - // `ctx.spillStore` is optional (read via ctx.get) and does not affect the - // schemas, so no spill backend is mounted. - await ctx.plugin(LocalBashExecutor) + // the executor seam, not ctx.fs). Use a catalog-only executor so the + // registration-time `rg` probe stays deterministic and the generator + // never depends on the host PATH. `ctx.spillStore` is optional (read via + // ctx.get) and does not affect the schemas, so no spill backend is mounted. + await ctx.plugin(CatalogSearchBashExecutor) await ctx.plugin(ToolFsSearch) }, note: - 'glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', + 'glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', }, { pkg: '@deepseek-ai/dsh-tool-skill', From faae2d389b2c2f957fa6e61cc71b1b0e5885ef51 Mon Sep 17 00:00:00 2001 From: Dudu-0223 Date: Mon, 13 Jul 2026 16:43:16 +0800 Subject: [PATCH 02/73] test(fs-search): make loader guard rg-independent --- .../fs/tool-fs-search/tests/load-path.spec.ts | 64 ++++++++++++++++++- 1 file changed, 62 insertions(+), 2 deletions(-) diff --git a/packages/fs/tool-fs-search/tests/load-path.spec.ts b/packages/fs/tool-fs-search/tests/load-path.spec.ts index d3c28619a3..90e4cc16cb 100644 --- a/packages/fs/tool-fs-search/tests/load-path.spec.ts +++ b/packages/fs/tool-fs-search/tests/load-path.spec.ts @@ -18,9 +18,69 @@ import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import { BashExecutor } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskId, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' import * as toolFsSearch from '@deepseek-ai/dsh-tool-fs-search' +const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + +/** + * Deterministic bash service for this Loader guard: the test wants to exercise + * the real unwrap/inject path, not depend on whether the host image has rg. + */ +class ProbeSuccessBashExecutor extends BashExecutor { + override resolve(request: BashExecRequest): BashExecSpec { + return { + command: request.command, + workdir: request.workdir ?? '/work', + timeoutMs: request.timeoutMs ?? 60_000, + stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, + signal: request.signal, + owner: request.owner, + sandboxMode: request.sandboxMode, + } + } + + override run(spec: BashExecSpec): Promise { + if (spec.command !== RG_PROBE_COMMAND) { + throw new Error(`unexpected command in load-path guard: ${spec.command}`) + } + return Promise.resolve({ + exitCode: 0, + signal: null, + timedOut: false, + aborted: false, + timeoutMs: spec.timeoutMs, + stdout: { text: '', truncated: false }, + stderr: { text: '', truncated: false }, + }) + } + + override start(): BashTask { + throw new Error('load-path guard must not start bash tasks') + } + + override get(): BashTask | undefined { + return undefined + } + + override ownerOf(): OwnerToken | undefined { + return undefined + } + + override list(): BashTask[] { + return [] + } + + override readOutput(id: BashTaskId): BashTaskRead { + throw new Error(`unknown bash task ${id}`) + } + + override kill(id: BashTaskId): boolean { + throw new Error(`unknown bash task ${id}`) + } +} + describe('dsh-tool-fs-search real-load-path guard', () => { it('has no default export and keeps name/inject/Config through unwrapExports', () => { expect('default' in toolFsSearch).toBe(false) @@ -38,7 +98,7 @@ describe('dsh-tool-fs-search real-load-path guard', () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(LocalBashExecutor, {}) + await ctx.plugin(ProbeSuccessBashExecutor) const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(toolFsSearch) as Parameters[0] From 0a486f09c91d0e5a3198bb40c22f034b52cea466 Mon Sep 17 00:00:00 2001 From: kingwl Date: Tue, 14 Jul 2026 23:34:47 +0800 Subject: [PATCH 03/73] chore: adopt node-addon-landlock-run source as native/ subtree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bring the node-addon-landlock-run tree (tag v0.0.1, commit 614f7fd) into native/landlock-run as its source of record: launcher development happens here, next to the harness consumers, and the standalone repository becomes the release mirror the tree is exported to for packing and publishing (procedure in native/README.md). The subtree keeps its own pnpm workspace and lockfile and is NOT added to the harness workspace: harness installs, gates, and CI never touch it. The mirror's .github/ stays out of the subtree; a separate manually-dispatched workflow (.github/workflows/landlock-run.yml) runs the subtree's CI legs — the per-architecture native builds, real-kernel launcher proofs, and pack rehearsal — adapted with working-directory/cache paths. eslint ignores the subtree like vendor/; AGENTS.md gains the native/ layout line (+5 words on its budget ceiling). --- .github/workflows/landlock-run.yml | 127 +++++++ AGENTS.md | 1 + eslint.config.mjs | 1 + native/README.md | 20 + native/landlock-run/.gitignore | 13 + native/landlock-run/AGENTS.md | 50 +++ native/landlock-run/LICENSE | 28 ++ native/landlock-run/README.md | 58 +++ native/landlock-run/docs/architecture.md | 34 ++ native/landlock-run/docs/cli-contract.md | 34 ++ native/landlock-run/docs/naming.md | 30 ++ native/landlock-run/docs/packaging.md | 45 +++ native/landlock-run/docs/release.md | 57 +++ native/landlock-run/docs/support-matrix.md | 18 + native/landlock-run/package.json | 30 ++ native/landlock-run/packages/entry/README.md | 16 + .../landlock-run/packages/entry/package.json | 36 ++ .../landlock-run/packages/entry/src/index.ts | 126 +++++++ native/landlock-run/packages/entry/src/main.c | 302 +++++++++++++++ .../landlock-run/packages/entry/tsconfig.json | 11 + .../landlock-run/packages/linux-arm64/LICENSE | 28 ++ .../packages/linux-arm64/README.md | 7 + .../packages/linux-arm64/package.json | 26 ++ .../packages/linux-arm64/prebuilds.json | 10 + .../landlock-run/packages/linux-x64/LICENSE | 28 ++ .../landlock-run/packages/linux-x64/README.md | 7 + .../packages/linux-x64/package.json | 26 ++ .../packages/linux-x64/prebuilds.json | 10 + native/landlock-run/pnpm-lock.yaml | 345 ++++++++++++++++++ native/landlock-run/pnpm-workspace.yaml | 8 + .../scripts/assemble-prebuilds.mjs | 51 +++ native/landlock-run/scripts/build.ts | 86 +++++ native/landlock-run/scripts/bump-release.mjs | 90 +++++ .../landlock-run/scripts/commit-release.mjs | 42 +++ native/landlock-run/scripts/github-matrix.mjs | 66 ++++ native/landlock-run/scripts/pack-release.mjs | 76 ++++ native/landlock-run/scripts/repo.mjs | 88 +++++ .../landlock-run/scripts/verify-entry-lib.mjs | 25 ++ .../scripts/verify-launcher-binary.mjs | 31 ++ .../scripts/verify-packed-install.mjs | 223 +++++++++++ .../landlock-run/scripts/verify-release.mjs | 52 +++ native/landlock-run/test/entry.test.js | 76 ++++ native/landlock-run/test/launcher.test.js | 121 ++++++ native/landlock-run/tsconfig.base.json | 11 + native/landlock-run/tsconfig.json | 11 + scripts/doc-budgets.manifest.json | 2 +- 46 files changed, 2582 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/landlock-run.yml create mode 100644 native/README.md create mode 100644 native/landlock-run/.gitignore create mode 100644 native/landlock-run/AGENTS.md create mode 100644 native/landlock-run/LICENSE create mode 100644 native/landlock-run/README.md create mode 100644 native/landlock-run/docs/architecture.md create mode 100644 native/landlock-run/docs/cli-contract.md create mode 100644 native/landlock-run/docs/naming.md create mode 100644 native/landlock-run/docs/packaging.md create mode 100644 native/landlock-run/docs/release.md create mode 100644 native/landlock-run/docs/support-matrix.md create mode 100644 native/landlock-run/package.json create mode 100644 native/landlock-run/packages/entry/README.md create mode 100644 native/landlock-run/packages/entry/package.json create mode 100644 native/landlock-run/packages/entry/src/index.ts create mode 100644 native/landlock-run/packages/entry/src/main.c create mode 100644 native/landlock-run/packages/entry/tsconfig.json create mode 100644 native/landlock-run/packages/linux-arm64/LICENSE create mode 100644 native/landlock-run/packages/linux-arm64/README.md create mode 100644 native/landlock-run/packages/linux-arm64/package.json create mode 100644 native/landlock-run/packages/linux-arm64/prebuilds.json create mode 100644 native/landlock-run/packages/linux-x64/LICENSE create mode 100644 native/landlock-run/packages/linux-x64/README.md create mode 100644 native/landlock-run/packages/linux-x64/package.json create mode 100644 native/landlock-run/packages/linux-x64/prebuilds.json create mode 100644 native/landlock-run/pnpm-lock.yaml create mode 100644 native/landlock-run/pnpm-workspace.yaml create mode 100644 native/landlock-run/scripts/assemble-prebuilds.mjs create mode 100644 native/landlock-run/scripts/build.ts create mode 100644 native/landlock-run/scripts/bump-release.mjs create mode 100644 native/landlock-run/scripts/commit-release.mjs create mode 100644 native/landlock-run/scripts/github-matrix.mjs create mode 100644 native/landlock-run/scripts/pack-release.mjs create mode 100644 native/landlock-run/scripts/repo.mjs create mode 100644 native/landlock-run/scripts/verify-entry-lib.mjs create mode 100644 native/landlock-run/scripts/verify-launcher-binary.mjs create mode 100644 native/landlock-run/scripts/verify-packed-install.mjs create mode 100644 native/landlock-run/scripts/verify-release.mjs create mode 100644 native/landlock-run/test/entry.test.js create mode 100644 native/landlock-run/test/launcher.test.js create mode 100644 native/landlock-run/tsconfig.base.json create mode 100644 native/landlock-run/tsconfig.json diff --git a/.github/workflows/landlock-run.yml b/.github/workflows/landlock-run.yml new file mode 100644 index 0000000000..8916f59a56 --- /dev/null +++ b/.github/workflows/landlock-run.yml @@ -0,0 +1,127 @@ +# Manually-dispatched CI for the landlock-run source of record +# (native/landlock-run). A separate workflow from ci.yml on purpose: the +# subtree is a self-contained pnpm workspace with its own gates, exercised on +# demand — per-architecture native legs (build + behavioral tests + pack +# rehearsal on real kernels) plus one darwin leg proving the documented +# degradation on hosts without a platform package. Legs derive from the +# subtree's checked-in package matrix (scripts/github-matrix.mjs). Packing +# for npm happens in the release mirror (node-addon-landlock-run) after an +# export — see native/README.md; this workflow never packs for release. +name: Landlock Run + +on: + workflow_dispatch: + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +defaults: + run: + working-directory: native/landlock-run + +jobs: + matrix: + name: Matrix + runs-on: ubuntu-24.04 + outputs: + ci: ${{ steps.matrix.outputs.ci }} + steps: + - uses: actions/checkout@v4 + + - id: matrix + run: echo "ci=$(node ./scripts/github-matrix.mjs ci)" >> "$GITHUB_OUTPUT" + + native: + name: ${{ matrix.platform }} + needs: matrix + runs-on: ${{ matrix.runner }} + strategy: + fail-fast: false + matrix: ${{ fromJson(needs.matrix.outputs.ci) }} + steps: + - uses: actions/checkout@v4 + + - uses: pnpm/action-setup@v4 + with: + package_json_file: native/landlock-run/package.json + + - uses: actions/setup-node@v4 + with: + node-version: 24 + cache: pnpm + cache-dependency-path: native/landlock-run/pnpm-lock.yaml + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + - name: Install musl toolchain + run: | + sudo apt-get update -q + sudo apt-get install -yq musl-tools + + - name: Build TypeScript + run: pnpm build:ts + + - name: Typecheck + run: pnpm typecheck + + - name: Build native binaries (this architecture is the builder of record) + run: pnpm build:native + + - name: Entry tests (keyless) + run: node ./test/entry.test.js + + # NALR_REQUIRE_LANDLOCK: a self-skip on the very platform that exists to + # prove enforcement would be a false green, so an unenforcing kernel + # fails the leg instead of skipping. + - name: Launcher tests (real kernel enforcement) + run: node ./test/launcher.test.js + env: + NALR_REQUIRE_LANDLOCK: 1 + + - name: Pack rehearsal (pack → install → confine, this platform only) + run: | + node ./scripts/pack-release.mjs .release/npm --current-platform-only + node ./scripts/verify-packed-install.mjs .release/npm --current-platform-only + env: + NALR_REQUIRE_LANDLOCK: 1 + + darwin: + name: darwin (no platform package — degradation proof) + runs-on: macos-latest + steps: + - uses: actions/checkout@v4 + + - uses: pnpm/action-setup@v4 + with: + package_json_file: native/landlock-run/package.json + + - uses: actions/setup-node@v4 + with: + node-version: 24 + cache: pnpm + cache-dependency-path: native/landlock-run/pnpm-lock.yaml + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + - name: Build TypeScript + run: pnpm build:ts + + - name: Typecheck + run: pnpm typecheck + + - name: Entry tests (keyless) + run: node ./test/entry.test.js + + - name: Launcher tests (must self-skip cleanly) + run: node ./test/launcher.test.js + + - name: Pack rehearsal (entry only — fallback resolution + unusable probe) + run: | + node ./scripts/pack-release.mjs .release/npm --current-platform-only + node ./scripts/verify-packed-install.mjs .release/npm --current-platform-only diff --git a/AGENTS.md b/AGENTS.md index 44aa15aea5..e5884ddd09 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -30,6 +30,7 @@ packages/ Harness packages at packages///, all named @deepseek-ai support/ dev/test infrastructure packages util/ zero-dependency utilities python/ Python SDK and bundled runtime (see python/README.md) +native/ node-addon-landlock-run source of record (see native/README.md) examples/ Runnable demos: thin cordis.yml leaves over the app packages (see examples/AGENTS.md) docs/ architecture, generated catalogs, RFCs, postmortems, cookbook (see docs/AGENTS.md) scripts/ repo gates and generators diff --git a/eslint.config.mjs b/eslint.config.mjs index 3f0e8ce820..2ec78d51f9 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -13,6 +13,7 @@ export default tseslint.config( '.claude/**', // harness-local state (worktrees, skills) — other checkouts, not this one's sources '**/.doc-typecheck-*/**', 'vendor/**', // vendored source keeps upstream style and idioms + 'native/**', // imported landlock-run subtree: self-contained workspace with its own gates (native/README.md) '**/*.js', '**/*.mjs', '*.config.ts', // root tool configs (vitest, tsdown) — no project service diff --git a/native/README.md b/native/README.md new file mode 100644 index 0000000000..983e67f740 --- /dev/null +++ b/native/README.md @@ -0,0 +1,20 @@ +# native/ + +Source of record for `node-addon-landlock-run`, the Landlock self-restrict-then-exec launcher the harness consumes from npm (`packages/sandbox/sandbox-local`, `packages/bash/bash-sandbox`). Launcher development happens HERE, next to the consumers; the standalone repository is the release mirror that packs and publishes the npm package family. + +## Release mirror + +| Directory | Mirror repo | Last exported release | Commit | +|---|---|---|---| +| `landlock-run/` | https://github.com/deepseek-harness/node-addon-landlock-run | `v0.0.1` | `614f7fd7dc11e6eaceefba9e7ff1fbe28b51ba22` | + +The subtree is a self-contained pnpm workspace with its own `AGENTS.md`, docs, gates, and lockfile; it is NOT part of the harness workspace (`pnpm-workspace.yaml` does not include it), so harness installs, builds, and CI gates never touch it. The mirror's `.github/` stays out of the subtree — [.github/workflows/landlock-run.yml](../.github/workflows/landlock-run.yml) (manual dispatch) runs the subtree's CI legs here, and a change to those legs is mirrored into the mirror's `ci.yml` at the next export. + +## Export procedure (cutting a release) + +1. Land the launcher change here through a normal harness PR; dispatch the `Landlock Run` workflow and get its legs green. +2. In the mirror checkout, replace everything except `.github/`: `git -C rm -rq -- . ':!.github'`, then `git -C archive HEAD:native/landlock-run | tar -x -C `, then `git -C add -A` and commit. +3. In the mirror, follow its release checklist (`docs/release.md`): `pnpm release:commit ` → merge → tag `vX.Y.Z` → two-phase `Release` workflow (`publish=false` rehearsal, then `publish=true` from the tag). +4. Update the manifest table above with the released tag/commit, and bump the harness consumers' dependency range in the same change. + +The mirror must not diverge: a change committed there directly (hotfix during a release) is ported back here before the next export. diff --git a/native/landlock-run/.gitignore b/native/landlock-run/.gitignore new file mode 100644 index 0000000000..0d7597f2db --- /dev/null +++ b/native/landlock-run/.gitignore @@ -0,0 +1,13 @@ +# Built native binaries ride npm tarballs via each package's `files` list, +# never git. Root-level rules on purpose: a package-nested ignore file would +# also steer `pnpm pack` and has silently dropped payload from tarballs before. +packages/*/bin/ +packages/*/lib/ + +/.claude/ +/.release/ +dist/ +node_modules/ +/package-lock.json +*.log +*.tsbuildinfo diff --git a/native/landlock-run/AGENTS.md b/native/landlock-run/AGENTS.md new file mode 100644 index 0000000000..31e12e177c --- /dev/null +++ b/native/landlock-run/AGENTS.md @@ -0,0 +1,50 @@ +# AGENTS.md + +This workspace builds `landlock-run`, a Landlock self-restrict-then-exec launcher: a small, auditable confinement binary distributed as prebuilt per-platform npm packages, plus the thin JS entry package that resolves it and speaks its CLI contract. The source of record is the `deepseek-harness` repository's `native/landlock-run/`; the `node-addon-landlock-run` repository is the release mirror this tree is exported to for packing and publishing (procedure: `native/README.md` in the harness repo). Make changes in the source of record, never only in the mirror. + +## Pre-release stance + +The project is pre-1.0. Prefer the correct public shape over compatibility shims: if a package name, exported field, layout, or contract detail is wrong, rename it and update all references in the same change. Do not add deprecated aliases unless a stable release already needs them. + +## Runtime safety rules + +- Every tool must fail closed. If a ruleset cannot be created or the kernel does not enforce it, exit non-zero WITHOUT exec'ing the wrapped command. Never run unconfined as a fallback. +- Runtime binaries and the entry packages take NO environment-variable overrides: which binary confines a process must never be decidable by the ambient environment. Test injection is by function parameter; the `NALR_*` prefix is for build/test orchestration only. +- Kernel UAPI is self-defined in the C source (verbatim from the kernel headers), keeping builds independent of toolchain header vintage and making the definitions part of the audit record. +- No libraries beyond libc, linked statically against musl. The audit surface of a tool is its C source plus the kernel's stable syscall contract. +- The CLI contract of each tool ([docs/cli-contract.md](docs/cli-contract.md)) is the cross-repo compatibility surface: argv grammar, exit codes, and report lines change only with a version bump and a changelog entry, and consumers parse them only through the entry package. +- There is deliberately NO install-time build fallback: a host without a matching platform package gets a nonexistent launcher path, the consumer's probe fails, and the consumer falls closed — that degradation is part of the design, not a gap to fill with node-gyp. + +## Repository layout + +```text +packages/entry/ Published entry package: JS seam (resolve/probe/grants) + the C source. +packages/linux-*/ Published per-platform packages: one prebuilt static binary, no JavaScript. +scripts/ Build, matrix derivation, prepack gates, and release orchestration. +test/ Plain-node behavioral tests (entry seam + real-kernel launcher proofs). +docs/ Architecture, packaging, CLI contract, release, support matrix, naming. +``` + +## Commands + +```sh +pnpm install +pnpm build:ts # entry packages → lib/ +pnpm build:native # this Linux architecture's binaries (needs musl-tools); fails fast elsewhere +pnpm typecheck +pnpm test # entry tests everywhere; launcher tests need linux + built binary +``` + +## Packaging invariants + +- The package matrix is explicit, checked-in metadata: `packages//package.json` (`os`, `cpu`), `packages//prebuilds.json` (the binaries that may exist there), and [docs/support-matrix.md](docs/support-matrix.md) stay synchronized when the matrix changes. `scripts/github-matrix.mjs` derives CI and release matrices from it; nothing else enumerates platforms. +- Platform package names contain platform only (`-linux-x64`), never tool variants — those stay inside `prebuilds.json`. Static musl linking is why there is no libc suffix: one binary serves glibc and musl distros. +- Platform packages ship no JavaScript; the entry package resolves them to file paths. Backends prove themselves at runtime through the functional probe, never through metadata trust. +- Builds are native-only: each architecture compiles its own binary on its own runner (CI is the builder of record); no cross toolchain enters the repo. +- Every tarball is gated at pack time: platform packages refuse to pack without their declared binaries present, executable, and in the right ELF architecture (`verify-launcher-binary.mjs`), entry packages without built `lib/` (`verify-entry-lib.mjs`), and the release pipeline byte-pins installed binaries against the workspace builds (`verify-packed-install.mjs`). +- Platform tarballs are packed with `npm pack`, never `pnpm pack`: pnpm's pack path strips the executable bit (observed on 11.7.0), shipping a launcher no consumer can spawn. `pack-release.mjs` encodes the split; the rehearsal asserts executability of the installed copy so a regression fails loudly instead of masquerading as a non-enforcing kernel. +- Generated artifacts stay out of git: `packages/*/bin/`, `packages/*/lib/`, `dist/`, `.release/`, `*.tsbuildinfo`. Ignore rules live in the ROOT `.gitignore` only — a package-nested ignore file can silently drop payload from tarballs. + +## Documentation + +User-facing docs are English. Keep the README focused on install, usage, and support status; durable design decisions belong in docs/ alongside the code, and the current implemented shape belongs in [docs/architecture.md](docs/architecture.md). diff --git a/native/landlock-run/LICENSE b/native/landlock-run/LICENSE new file mode 100644 index 0000000000..8187059c9a --- /dev/null +++ b/native/landlock-run/LICENSE @@ -0,0 +1,28 @@ +BSD 3-Clause License + +Copyright (c) 2026, node-addon-landlock-run contributors + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/native/landlock-run/README.md b/native/landlock-run/README.md new file mode 100644 index 0000000000..2bb92843e6 --- /dev/null +++ b/native/landlock-run/README.md @@ -0,0 +1,58 @@ +# node-addon-landlock-run + +A [Landlock](https://landlock.io/) self-restrict-then-exec launcher for confining subprocesses on Linux, distributed as prebuilt per-platform npm packages plus a thin JS entry package that resolves the binary and speaks its CLI contract. Built for agent harnesses and other hosts that need to run untrusted commands under a filesystem allow-list without confining themselves. + +The first tool is **`landlock-run`** — a self-restrict-then-exec [Landlock](https://landlock.io/) launcher (~300 lines of C11 over the raw kernel UAPI, statically linked against musl). It installs a Landlock ruleset on itself and `exec`s the wrapped command; the ruleset is inherited across `execve`, so the command and every process it spawns run confined while the invoking process stays unrestricted. Fail-closed: if the kernel cannot enforce, it exits without running the command. + +## Install + +```sh +npm install node-addon-landlock-run +``` + +Published packages use an entry package plus platform optional packages: + +```text +node-addon-landlock-run +node-addon-landlock-run-linux-x64 +node-addon-landlock-run-linux-arm64 +``` + +npm's `os`/`cpu` fields make installers fetch only the matching platform package. There is no install-time build fallback on purpose: on a host without a platform package the resolved path never exists, the probe reports `unusable`, and the consumer falls closed. + +## Usage + +```js +import { grantArgs, launcherPath, probe } from 'node-addon-landlock-run'; + +const launcher = launcherPath(); +if (probe(launcher) !== 'unusable') { + const argv = [launcher, ...grantArgs({ readOnly: ['/'], readWrite: ['/tmp/work'] }), '--', 'bash', '-c', command]; + // spawn argv with your process runner of choice +} +``` + +The public API is intentionally small: + +- `launcherPath()`: absolute path of this host's launcher (existence deliberately unchecked — the probe is the availability signal). +- `probe(launcher?, { timeoutMs? })`: functional enforcement probe — `'full' | 'partial' | 'unusable'`. +- `grantArgs({ readOnly?, readWrite? })`: the launcher's grant argv; everything not granted is denied. +- `LAUNCHER_BIN`, `LAUNCHER_FAILURE_EXIT` (125): contract constants. + +The full binary contract (argv grammar, exit codes, report lines) is pinned in [docs/cli-contract.md](docs/cli-contract.md). + +## Support + +linux-x64 and linux-arm64, kernel with Landlock enabled (5.13+; ABI level determines `full` vs `partial` enforcement — see [docs/support-matrix.md](docs/support-matrix.md)). Other platforms deliberately have no package: consumers run different confinement backends there. + +## Development + +```sh +corepack enable +pnpm install +pnpm build:ts # entry packages → lib/ +pnpm build:native # this Linux architecture's binaries (apt-get install musl-tools) +pnpm test +``` + +Binaries are git-ignored and built natively per architecture — locally for your own machine, by CI's per-arch runners as the builders of record. Release flow: [docs/release.md](docs/release.md). diff --git a/native/landlock-run/docs/architecture.md b/native/landlock-run/docs/architecture.md new file mode 100644 index 0000000000..e6974f4e50 --- /dev/null +++ b/native/landlock-run/docs/architecture.md @@ -0,0 +1,34 @@ +# Architecture + +This repository owns confinement *mechanism*, not policy: consumers (agent harnesses, sandbox seams) decide which paths a run may read or write; this package family provides the launcher that enforces those grants and the JS seam that resolves and speaks to it. The packaging follows the per-platform-package model of [`node-addon-require-builtin`](https://www.npmjs.com/package/@esplus/node-addon-require-builtin) (and esbuild), adapted from Node addons to standalone static executables. + +## Two-layer package family + +The family is one entry package plus per-platform binary packages: + +- **Entry package** (`node-addon-landlock-run`): ESM JavaScript. Owns the tool's CLI contract — path resolution (`launcherPath`), the functional probe (`probe`), grant-argv construction (`grantArgs`), and the contract constants. Ships the C source in its tarball for auditability. Lists every platform package as an `optionalDependency`. +- **Platform packages** (`node-addon-landlock-run-linux-{x64,arm64}`): one prebuilt static binary under `bin/`, a `prebuilds.json` declaring it, and no JavaScript at all. npm's `os`/`cpu` fields select the matching one at install time; the entry package resolves it to a file path — there is nothing to import. + +Because the contract parser and the binary version together in one family, probe-parsing drift against the binary is structurally impossible — the failure mode the split exists to prevent. + +There is no shared loader package: platform packages have nothing to load. If a second tool ever needs shared JS, extract it then, not preemptively. + +## Resolution and availability + +`launcherPath()` resolves `node-addon-landlock-run--` and returns `/bin/landlock-run`. When the package is not resolvable it returns a deterministic fallback path inside the entry package's own `node_modules` that simply never exists. Existence is deliberately unchecked either way: `probe()` is the single availability signal, and a missing binary probes `unusable` exactly like an unenforcing kernel. Consumers get one degradation path, not two. + +The probe is functional — the launcher builds and enforces a real maximal ruleset in a short-lived child — because version checks would miss a kernel that has the syscalls but refuses enforcement. + +## Fail-closed everywhere + +The launcher exits `125` without exec'ing the command on any launcher-level failure: usage error, unenforcing kernel, unopenable grant root, failed exec. Partial enforcement (an older Landlock ABI governing only a subset of accesses) is accepted, reported on stderr, and surfaced by the probe as `partial` — the consumer decides what its mode vocabulary promises at each level. Neither the binary nor the entry package reads environment variables: which binary confines a process is never decidable by the ambient environment. + +## Build and release model + +Builds are native-only. `scripts/build.ts` compiles the running architecture's binaries with the distro `musl-gcc` (static: no loader or libc expectations on consumers, one binary for glibc and musl distros); CI's per-architecture runners are the builders of record, and no cross toolchain exists in the repo. The audit surface of a tool is its reviewed C source plus CI provenance, enforced by three gates: platform prepack refuses missing/wrong-ELF binaries, entry prepack refuses unbuilt `lib/`, and the release pipeline byte-pins installed binaries against the workspace builds they were packed from. + +The package matrix is checked-in metadata (`prebuilds.json` + `os`/`cpu` fields); `scripts/github-matrix.mjs` derives the CI and Release matrices from it, so adding a platform extends automation without editing workflows. + +## Adding a platform + +A new platform adds one `packages//` package (`package.json` with `os`/`cpu`, `prebuilds.json`, README, LICENSE), a runner entry in `scripts/github-matrix.mjs`, and a row in [support-matrix.md](support-matrix.md) — added only together with a native GitHub runner that builds and proves it (the no-cross-toolchain rule). Sibling launchers for other confinement mechanisms belong in their own repositories on this same template, not as second tools here. diff --git a/native/landlock-run/docs/cli-contract.md b/native/landlock-run/docs/cli-contract.md new file mode 100644 index 0000000000..57ab0f604c --- /dev/null +++ b/native/landlock-run/docs/cli-contract.md @@ -0,0 +1,34 @@ +# CLI contract: landlock-run + +This file pins the launcher's externally observable behavior — the cross-repo compatibility surface between the binaries and every consumer. Consumers interact with it only through the entry package (`launcherPath`/`probe`/`grantArgs`); changing anything below requires a version bump for the whole package family and a note in the release notes. + +## Invocation grammar + +```text +landlock-run [--ro ]... [--rw ]... -- ... +landlock-run --probe +``` + +- `--ro `: grant read + execute beneath ``. +- `--rw `: grant full filesystem access beneath `` (every access the negotiated kernel ABI can govern). +- Everything not granted is denied — Landlock rulesets are allow-lists. +- A grant on a non-directory keeps only its file-compatible access bits (this is how a `--rw /dev/null` grant works). +- `--`: mandatory separator; everything after it is the command argv, exec'd via `execvp` with the launcher's environment unchanged. +- `--probe`: mutually exclusive with grants and a command. +- No other flags, no environment-variable inputs. + +## Exit codes + +- `125` (`LAUNCHER_FAILURE_EXIT`): every launcher-level failure — usage error, kernel that cannot enforce Landlock, unopenable grant root, failed `exec`. The wrapped command was NOT run (fail-closed; the one exception is `exec` itself failing after restriction, which by definition never ran the command either). +- Any other status: the wrapped command's own exit status, passed through unchanged. +- `--probe`: `0` when the kernel enforces (fully or partially), `125` otherwise. + +## Report lines + +- Probe success prints exactly one stdout line: `landlock: fully enforced` or `landlock: partially enforced (older ABI)`. The entry package's `probe()` maps these to `full`/`partial`; a non-zero probe exit maps to `unusable`. +- A confined run under a partial-ABI kernel prints one stderr line `landlock-run: partial enforcement (older Landlock ABI)` and proceeds — still confined for everything the kernel supports. +- Every fatal error prints one stderr line prefixed `landlock-run: ` before exiting `125`. + +## Confinement semantics + +The launcher sets `no_new_privs`, installs the ruleset on itself, and `exec`s the command; the ruleset is inherited across `execve`, so every descendant process is equally confined. The ruleset governs the filesystem accesses of the kernel's negotiated Landlock ABI (up to ABI 5); accesses newer than the running ABI are not governed and are the difference between `full` and `partial`. diff --git a/native/landlock-run/docs/naming.md b/native/landlock-run/docs/naming.md new file mode 100644 index 0000000000..9de9f0b95f --- /dev/null +++ b/native/landlock-run/docs/naming.md @@ -0,0 +1,30 @@ +# Naming + +## npm packages + +The public package family is unscoped, using the `node-addon-landlock-run` package prefix; platform packages append platform information only: + +```text +node-addon-landlock-run +node-addon-landlock-run- +``` + +Platform suffixes carry no libc component (binaries are static musl) and no variant component — variants stay inside `prebuilds.json` and binary filenames. + +## Binaries + +The launcher executable is `landlock-run`, shipped at `bin/landlock-run` inside each platform package. + +## Environment variables + +The `NALR_` prefix (Node Addon Landlock Run) is reserved for build/test orchestration: + +```text +NALR_REQUIRE_LANDLOCK test-only: an unenforcing kernel fails instead of skipping +``` + +Runtime binaries and entry packages read NO environment variables — a runtime safety rule ([AGENTS.md](../AGENTS.md)), not a naming convention. Do not include the npm scope in environment variable names. + +## C symbols + +The launcher is a single C file with static linkage; there is no exported symbol namespace. Kernel UAPI constants keep their kernel names prefixed `LL_` where locally defined. diff --git a/native/landlock-run/docs/packaging.md b/native/landlock-run/docs/packaging.md new file mode 100644 index 0000000000..9a1be47b2a --- /dev/null +++ b/native/landlock-run/docs/packaging.md @@ -0,0 +1,45 @@ +# Packaging + +The package family uses the same broad shape as native packages such as esbuild: one JS entry package plus platform optional packages. Unlike Node addons there is no ABI or backend dimension — each platform package carries exactly the static executables its `prebuilds.json` declares. + +## Published packages + +```text +node-addon-landlock-run +node-addon-landlock-run-linux-x64 +node-addon-landlock-run-linux-arm64 +``` + +Unsupported platforms are intentionally absent from `optionalDependencies` — see [support-matrix.md](support-matrix.md). + +## Package matrix + +The matrix is explicit in checked-in metadata: + +- `packages/entry/package.json` lists the platform packages as `optionalDependencies`. +- `packages//package.json` declares `os` and `cpu`. There is no `libc` field on purpose: the binaries are statically linked against musl and run on glibc and musl distros alike. +- `packages//prebuilds.json` declares the binaries that may exist in that package (`tool`, `kind`, `path`). +- [support-matrix.md](support-matrix.md) explains why unsupported platform packages are not published. + +`scripts/github-matrix.mjs` derives the CI and Release matrices from these files. `scripts/build.ts` builds only the current host's targets, into `packages//bin/`; it is not a matrix generator. When changing the matrix, update package metadata, `prebuilds.json`, the lockfile, and the support/release docs in the same change. + +## Runtime selection + +1. npm's `os`/`cpu` fields make installers fetch only the matching platform package. +2. The entry package's `launcherPath()` resolves it to `/bin/landlock-run`; unresolvable packages yield a deterministic, never-existing fallback path. +3. `probe()` is the single availability signal: missing binary and unenforcing kernel are deliberately indistinguishable (`unusable`), so consumers have one fail-closed path. + +## No install fallback + +The entry package has NO install script and never compiles on the consumer host. A compile fallback would require a musl toolchain everywhere and turn a clean fail-closed degradation into an environment-dependent maybe. The packed-manifest check in `verify-packed-install.mjs` enforces the absence of install lifecycle scripts. + +## Pack gates + +Platform tarballs are produced by `npm pack`, entry tarballs by `pnpm pack` — deliberately split: `pnpm pack` (observed on 11.7.0) normalizes file modes and strips the executable bit, which would ship a launcher no consumer can spawn, while platform packages have no dependencies and so need none of pnpm's workspace-protocol conversion; entry packages need that conversion and carry no executables. `scripts/pack-release.mjs` encodes the split — never hand-pack a platform package with pnpm. + +Both pack paths produce the exact publish bytes behind a `prepack` gate: + +- Platform packages: `scripts/verify-launcher-binary.mjs` — every declared binary present, executable, ELF `e_machine` matching the declared `cpu`, nothing undeclared in `bin/`. +- Entry packages: `scripts/verify-entry-lib.mjs` — built `lib/` present. + +`scripts/verify-packed-install.mjs` then rehearses the consumer path from the packed tarballs: payload checks, a throwaway install, a byte-pin of the installed binary against the workspace build, an executability check on the installed copy, and a real confinement world-proof through the installed launcher. A non-executable or missing binary fails loudly here instead of masquerading as a non-enforcing kernel. diff --git a/native/landlock-run/docs/release.md b/native/landlock-run/docs/release.md new file mode 100644 index 0000000000..e43b2d188c --- /dev/null +++ b/native/landlock-run/docs/release.md @@ -0,0 +1,57 @@ +# Release + +Pre-1.0: treat this as a release checklist, not a stability policy. + +## Versioning + +One version across every package in the repo. Use the bump helper: + +```sh +pnpm release:bump patch # or minor / major / x.y.z +``` + +It updates the root and every `packages/*` manifest, refreshes the lockfile (`--ignore-scripts --lockfile-only`), and runs `release:verify`. Explicit versions accept full semver including prereleases (`pnpm release:bump 0.0.0-test.0`); the publish workflow puts prerelease versions under the `next` dist-tag, so `latest` never points at a test build. Keep `workspace:*` dependencies in source; pnpm converts them to concrete versions during pack. + +Version bumps are normal source changes: open a release PR (or commit) with the manifests and lockfile, merge it, then create the matching `vX.Y.Z` tag from that commit. The publish workflow validates that the tag matches every package version. + +```sh +pnpm release:commit patch # bump + stage + commit in one command +git tag v0.0.2 +``` + +## Preflight + +```sh +pnpm install --frozen-lockfile +pnpm build:ts +pnpm typecheck +pnpm test # launcher half needs a Linux host with the binary built +``` + +On a Linux host, also rehearse the pack path locally: + +```sh +pnpm build:native +node ./scripts/pack-release.mjs .release/npm --current-platform-only +node ./scripts/verify-packed-install.mjs .release/npm --current-platform-only +``` + +## Publish + +Use the `Release` workflow so every binary is built on its matching native runner: + +1. Run it with `publish=false` (from the release commit) to build all platform binaries, assemble and verify the payloads, pack the tarballs in publish order, rehearse the packed install, and upload the `npm-tarballs` artifact for inspection. +2. Create and push the `vX.Y.Z` tag matching the package versions. +3. Run the same workflow from that tag with `publish=true`. + +The workflow publishes only from the final packed tarballs, in `publish-order.txt` order (platform packages before the entry that optionally depends on them). It supports npm trusted publishing through GitHub OIDC; without it, provide an `NPM_TOKEN` secret in the `npm-publish` environment. Packages publish with `--access public`. + +Manual local fallback (current platform's packages only) — always through `pack-release.mjs`, never `pnpm publish` directly (pnpm's pack path strips the launcher's executable bit; see [packaging.md](packaging.md)): + +```sh +node ./scripts/pack-release.mjs dist/npm --current-platform-only +node ./scripts/verify-packed-install.mjs dist/npm --current-platform-only +while IFS= read -r tarball; do npm publish "dist/npm/${tarball}" --access public; done < dist/npm/publish-order.txt +``` + +Do not commit `.npmrc` files with tokens or registry overrides. diff --git a/native/landlock-run/docs/support-matrix.md b/native/landlock-run/docs/support-matrix.md new file mode 100644 index 0000000000..96d02b3cf6 --- /dev/null +++ b/native/landlock-run/docs/support-matrix.md @@ -0,0 +1,18 @@ +# Support matrix + +## Supported + +| Platform package | GitHub runner (builder of record) | Notes | +|---|---|---| +| `node-addon-landlock-run-linux-x64` | `ubuntu-24.04` | static musl — glibc and musl distros alike | +| `node-addon-landlock-run-linux-arm64` | `ubuntu-24.04-arm` | static musl — glibc and musl distros alike | + +Enforcement additionally requires a kernel with Landlock enabled (5.13+). The negotiated ABI level decides the probe verdict: every access this build knows governed → `full`; an older ABI governing a subset → `partial` (still confined for everything it supports); Landlock absent or disabled → `unusable`, and the launcher refuses to run commands at all. The probe — not the kernel version — is the authority: a kernel built without Landlock, or with the LSM disabled, probes `unusable` regardless of its version. + +## Deliberately unsupported + +- **darwin**: macOS consumers typically confine through `sandbox-exec`/Seatbelt, which ships with the OS — there is no binary to distribute. +- **win32**: a Windows confinement launcher would be a different mechanism in its own repository, not a port of this one. +- **Other Linux architectures** (riscv64, s390x, …): no native CI builder of record yet. The no-cross-toolchain rule means a platform package is added only together with a native runner that builds and proves it. + +A consumer on an unsupported platform resolves a nonexistent launcher path, probes `unusable`, and falls closed — the documented degradation, exercised by CI's darwin leg. diff --git a/native/landlock-run/package.json b/native/landlock-run/package.json new file mode 100644 index 0000000000..f516588f17 --- /dev/null +++ b/native/landlock-run/package.json @@ -0,0 +1,30 @@ +{ + "name": "node-addon-landlock-run-workspace", + "version": "0.0.1", + "private": true, + "type": "module", + "license": "BSD-3-Clause", + "packageManager": "pnpm@11.7.0", + "scripts": { + "build": "pnpm build:ts", + "build:ts": "tsc -b", + "build:native": "tsx ./scripts/build.ts", + "typecheck": "tsc --noEmit && tsc -b --dry", + "test": "node ./test/entry.test.js && node ./test/launcher.test.js", + "test:entry": "node ./test/entry.test.js", + "test:launcher": "node ./test/launcher.test.js", + "gha:matrix": "node ./scripts/github-matrix.mjs", + "release:bump": "node ./scripts/bump-release.mjs", + "release:commit": "node ./scripts/commit-release.mjs", + "release:assemble-prebuilds": "node ./scripts/assemble-prebuilds.mjs", + "release:verify": "node ./scripts/verify-release.mjs", + "release:pack": "node ./scripts/pack-release.mjs", + "release:verify-packed-install": "node ./scripts/verify-packed-install.mjs" + }, + "devDependencies": { + "node-addon-landlock-run": "workspace:*", + "@types/node": "^24.10.0", + "tsx": "^4.20.6", + "typescript": "^5.9.3" + } +} diff --git a/native/landlock-run/packages/entry/README.md b/native/landlock-run/packages/entry/README.md new file mode 100644 index 0000000000..789b1ddf6b --- /dev/null +++ b/native/landlock-run/packages/entry/README.md @@ -0,0 +1,16 @@ +# node-addon-landlock-run + +Landlock self-restrict-then-exec launcher for confining subprocesses on Linux: this entry package resolves the per-platform prebuilt binary, runs its functional enforcement probe, and builds its grant argv — consumers never spell launcher flags or parse launcher output themselves. + +```js +import { grantArgs, launcherPath, probe } from 'node-addon-landlock-run'; + +const launcher = launcherPath(); +if (probe(launcher) !== 'unusable') { + const argv = [launcher, ...grantArgs({ readOnly: ['/'], readWrite: ['/tmp/work'] }), '--', 'bash', '-c', command]; +} +``` + +The launcher installs a Landlock ruleset on itself and `exec`s the wrapped command; the ruleset is inherited across `execve`, so the whole process tree runs confined. Everything not granted is denied, and launcher failures exit `125` without running the command — fail-closed, never fail-open. The binary contract is pinned in the repo's `docs/cli-contract.md`; the C source rides this tarball (`src/main.c`) for audit. + +Platform packages (`os`/`cpu`-selected optional dependencies, no JavaScript inside): `node-addon-landlock-run-linux-x64`, `node-addon-landlock-run-linux-arm64`. On hosts without one, `launcherPath()` returns a deterministic nonexistent path and `probe()` reports `'unusable'` — there is deliberately no install-time compile fallback. diff --git a/native/landlock-run/packages/entry/package.json b/native/landlock-run/packages/entry/package.json new file mode 100644 index 0000000000..f05e81f06b --- /dev/null +++ b/native/landlock-run/packages/entry/package.json @@ -0,0 +1,36 @@ +{ + "name": "node-addon-landlock-run", + "version": "0.0.1", + "type": "module", + "description": "Landlock self-restrict-then-exec launcher for sandboxing subprocesses on Linux: per-platform prebuilt static binaries plus the JS seam that resolves, probes, and speaks their CLI contract", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "exports": { + ".": { + "types": "./lib/index.d.ts", + "default": "./lib/index.js" + }, + "./package.json": "./package.json" + }, + "files": [ + "README.md", + "lib/", + "!lib/*.tsbuildinfo", + "src/main.c" + ], + "scripts": { + "build:js": "tsc -b", + "prepack": "node ../../scripts/verify-entry-lib.mjs" + }, + "engines": { + "node": ">=20" + }, + "license": "BSD-3-Clause", + "publishConfig": { + "access": "public" + }, + "optionalDependencies": { + "node-addon-landlock-run-linux-arm64": "workspace:*", + "node-addon-landlock-run-linux-x64": "workspace:*" + } +} diff --git a/native/landlock-run/packages/entry/src/index.ts b/native/landlock-run/packages/entry/src/index.ts new file mode 100644 index 0000000000..53de86122f --- /dev/null +++ b/native/landlock-run/packages/entry/src/index.ts @@ -0,0 +1,126 @@ +/** + * The JS seam over the prebuilt `landlock-run` launcher: resolve the + * binary for this host, build its grant argv, and run its functional probe. + * + * This module owns the launcher's CLI contract (`docs/cli-contract.md`) so + * consumers never parse launcher output or spell launcher flags themselves — + * the contract and the binaries version together in one package family, + * which makes probe-parsing drift against the binary structurally + * impossible. Policy stays with the consumer: this package does not know + * what a "sandbox mode" is, only which paths are granted read or write. + * + * Deliberately no environment-variable overrides anywhere in this module: + * which binary confines a process must never be decidable by the ambient + * environment. Test injection is by function parameter. + */ +import { spawnSync } from 'node:child_process' +import { createRequire } from 'node:module' +import { dirname, join } from 'node:path' +import { fileURLToPath } from 'node:url' + +/** The launcher binary's file name inside each platform package's `bin/`. */ +export const LAUNCHER_BIN = 'landlock-run' + +/** + * The exit code for every launcher-level failure (usage error, unenforcing + * kernel, unopenable grant root, failed exec) — chosen because the wrapped + * command itself is unlikely to use it, so a consumer can tell launcher + * failures from command failures. Part of the CLI contract. + */ +export const LAUNCHER_FAILURE_EXIT = 125 + +/** + * The probe's verdict on this host: `full` when the running kernel enforces + * every access the launcher can govern, `partial` when an older Landlock ABI + * governs only a subset (still confined for everything it supports), and + * `unusable` when nothing can be enforced — a kernel without Landlock, a + * disabled LSM, or a missing binary, all indistinguishable on purpose + * because the consumer's answer is the same: do not trust this launcher. + */ +export type LandlockEnforcement = 'full' | 'partial' | 'unusable' + +/** + * Filesystem grants for one confined run. Everything not granted is denied — + * Landlock rulesets are allow-lists. + */ +export interface LauncherGrants { + /** Roots granted read + execute beneath (the launcher's `--ro`). */ + readonly readOnly?: readonly string[] + /** Roots granted full filesystem access beneath (the launcher's `--rw`). */ + readonly readWrite?: readonly string[] +} + +/** + * Path of the launcher binary for this host: resolved from the per-platform + * npm package `node-addon-landlock-run--` (npm's + * `os`/`cpu` fields make installers fetch only the matching one). When the + * package is not resolvable — a platform without one, or an install that + * skipped the optional dependency — the returned fallback path points inside + * this package's own `node_modules` and simply never exists. Existence is + * deliberately not checked either way: {@link probe} is the single + * availability signal (a missing binary probes `unusable` the same way an + * unenforcing kernel does). + * @param resolvePackageJson - test seam over `require.resolve` (the default + * covers real installs); receives the platform package's `package.json` + * specifier and returns its absolute path, throwing when unresolvable. + * @returns the absolute launcher path to probe and exec. + */ +export function launcherPath( + resolvePackageJson: (specifier: string) => string = createRequire(import.meta.url).resolve, +): string { + const platformPackage = `node-addon-landlock-run-${process.platform}-${process.arch}` + try { + return join(dirname(resolvePackageJson(`${platformPackage}/package.json`)), 'bin', LAUNCHER_BIN) + } catch { + // Unresolvable platform package: no such package exists for this host, or + // it was not installed. Fall back to the path pnpm's layout WOULD use — + // absolute, inside this package's boundary (never cwd-relative: a + // spawnable relative path here would hand cwd control over which binary + // confines), and nonexistent exactly when the package is absent. + return fileURLToPath(new URL(`../node_modules/${platformPackage}/bin/${LAUNCHER_BIN}`, import.meta.url)) + } +} + +/** + * The launcher grant arguments for one set of filesystem grants — everything + * before the `--` argv separator. A caller spawns + * `[launcherPath(), ...grantArgs(grants), '--', ...command]`; the flag + * spellings stay private to this package. + * @param grants - the read-only and read-write roots to allow. + * @returns the `--ro ` / `--rw ` argument list, read-only roots + * first, in the caller's order. + */ +export function grantArgs(grants: LauncherGrants): string[] { + return [ + ...(grants.readOnly ?? []).flatMap(root => ['--ro', root]), + ...(grants.readWrite ?? []).flatMap(root => ['--rw', root]), + ] +} + +/** + * Functional probe: `landlock-run --probe` builds and enforces a maximal + * ruleset in a short-lived child and exits 0 only when the running kernel + * actually enforces it — `--version`-style checks would miss a kernel that + * has the syscalls but refuses enforcement. The probe's one report line is + * part of the CLI contract and distinguishes complete from per-ABI-subset + * enforcement; a zero exit without the partial marker reads as `full`. A + * failed or timed-out spawn (missing binary, wrong architecture, unenforcing + * kernel) probes `unusable`. Synchronous by design: consumers run it once + * and cache the verdict. + * @param launcher - the launcher path to probe; defaults to + * {@link launcherPath}'s resolution for this host. + * @param options - `timeoutMs` bounds the probe child (default 2000). + * @returns the enforcement verdict for this host. + */ +export function probe( + launcher: string = launcherPath(), + options: { timeoutMs?: number } = {}, +): LandlockEnforcement { + const result = spawnSync(launcher, ['--probe'], { + timeout: options.timeoutMs ?? 2000, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'], + }) + if (result.status !== 0) return 'unusable' + return /partially enforced/.test(result.stdout) ? 'partial' : 'full' +} diff --git a/native/landlock-run/packages/entry/src/main.c b/native/landlock-run/packages/entry/src/main.c new file mode 100644 index 0000000000..2535f8bc31 --- /dev/null +++ b/native/landlock-run/packages/entry/src/main.c @@ -0,0 +1,302 @@ +/* + * landlock-run: self-restrict-then-exec Landlock launcher. + * + * The Landlock rung of a consuming sandbox seam, for Linux hosts where + * `bwrap` is + * unusable (not installed, unprivileged user namespaces disabled, or an LSM + * profile that denies mount — Landlock is an independent syscall family and + * needs none of those). The launcher installs a Landlock + * ruleset on itself and `exec`s the wrapped command; the ruleset is inherited + * across `execve`, so the command (and every process it spawns) runs confined + * while the invoking process stays unrestricted. + * + * CLI contract (mirrors the `bwrap` runner argv shape the executor wraps): + * + * landlock-run [--ro ]... [--rw ]... -- ... + * landlock-run --probe + * + * `--ro` grants read+execute beneath the path; `--rw` grants full filesystem + * access beneath the path. Everything else is denied (Landlock is an + * allow-list). `--probe` builds a maximal ruleset and reports whether the + * running kernel actually enforces it — the executor's functional probe. + * + * Fail-closed: if the ruleset cannot be created or is NOT enforced by the + * kernel, the launcher exits non-zero WITHOUT exec'ing the command. A partial + * (best-effort) enforcement on an older ABI is accepted and reported on + * stderr; the consumer's mode vocabulary keeps its file-effect promises + * honest per ABI level (surfaced as `full` vs `partial` by the entry + * package's probe). + * + * Plain C11 over the raw Landlock UAPI — no libraries beyond libc (musl, + * linked statically), so the whole audit surface is this file plus the + * kernel's stable syscall contract. Built natively per architecture by + * `scripts/build.ts` into the per-platform npm packages + * (`node-addon-landlock-run-linux-{x64,arm64}`); the argv grammar, + * exit codes, and report lines are pinned in `docs/cli-contract.md`. + */ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * The Landlock UAPI, defined locally instead of via : the + * kernel's user-space ABI is stable by contract, self-defining it keeps the + * build independent of the toolchain's header vintage, and the definitions + * double as the audit record of exactly which kernel surface this launcher + * touches. Layouts and values are verbatim from the kernel header (the + * path-beneath struct is packed there, so it must be packed here). + */ +struct landlock_ruleset_attr { + uint64_t handled_access_fs; +}; + +struct landlock_path_beneath_attr { + uint64_t allowed_access; + int32_t parent_fd; +} __attribute__((packed)); + +#define LANDLOCK_CREATE_RULESET_VERSION (1U << 0) +#define LANDLOCK_RULE_PATH_BENEATH 1 + +/* Filesystem access bits, grouped by the Landlock ABI that introduced them. */ +#define LL_FS_EXECUTE (UINT64_C(1) << 0) /* ABI 1 */ +#define LL_FS_WRITE_FILE (UINT64_C(1) << 1) +#define LL_FS_READ_FILE (UINT64_C(1) << 2) +#define LL_FS_READ_DIR (UINT64_C(1) << 3) +#define LL_FS_REMOVE_DIR (UINT64_C(1) << 4) +#define LL_FS_REMOVE_FILE (UINT64_C(1) << 5) +#define LL_FS_MAKE_CHAR (UINT64_C(1) << 6) +#define LL_FS_MAKE_DIR (UINT64_C(1) << 7) +#define LL_FS_MAKE_REG (UINT64_C(1) << 8) +#define LL_FS_MAKE_SOCK (UINT64_C(1) << 9) +#define LL_FS_MAKE_FIFO (UINT64_C(1) << 10) +#define LL_FS_MAKE_BLOCK (UINT64_C(1) << 11) +#define LL_FS_MAKE_SYM (UINT64_C(1) << 12) +#define LL_FS_REFER (UINT64_C(1) << 13) /* ABI 2 */ +#define LL_FS_TRUNCATE (UINT64_C(1) << 14) /* ABI 3 (ABI 4 added TCP bits only) */ +#define LL_FS_IOCTL_DEV (UINT64_C(1) << 15) /* ABI 5 */ + +#define LL_ABI1_MASK (LL_FS_REFER - 1) /* bits 0..12: every ABI-1 access, nothing newer */ + +/* + * Newest ABI this build knows; the negotiation below scales the actual + * ruleset down to what the running kernel supports (the best-effort compat + * stance of the previous Rust launcher, made explicit). + */ +#define MAX_ABI 5L + +/* + * Landlock has no libc wrappers; these are the raw syscalls. The numbers are + * identical on every architecture (the post-2011 unified table) — the + * fallbacks only matter to a libc older than the feature. + */ +#ifndef __NR_landlock_create_ruleset +#define __NR_landlock_create_ruleset 444 +#define __NR_landlock_add_rule 445 +#define __NR_landlock_restrict_self 446 +#endif + +/* + * Every fatal launcher error prints `landlock-run: ` to stderr + * and exits 125 — a code the wrapped command itself is unlikely to use, so + * the executor can tell launcher failures from command failures. + */ +#define EXIT_LAUNCHER_FAILURE 125 + +static const char NOT_ENFORCED_MESSAGE[] = + "landlock is not enforced by this kernel (ABI unsupported or disabled)"; + +/* Print one fatal `landlock-run: ...` line; returns the fatal exit code. */ +static int fail(const char *prefix, const char *detail) { + if (detail == NULL) { + fprintf(stderr, "landlock-run: %s\n", prefix); + } else { + fprintf(stderr, "landlock-run: %s: %s\n", prefix, detail); + } + return EXIT_LAUNCHER_FAILURE; +} + +static int fail_usage(const char *message, const char *detail) { + fprintf(stderr, "landlock-run: usage error: %s%s\n", message, detail == NULL ? "" : detail); + return EXIT_LAUNCHER_FAILURE; +} + +/* Parsed CLI: either a probe, or grants plus the command argv after `--`. */ +struct cli { + int probe; + const char **ro; + size_t ro_count; + const char **rw; + size_t rw_count; + char **command; /* NULL-terminated tail of main's argv */ +}; + +/* + * Hand-rolled argv parsing — four flags do not justify a parsing library, + * and the previous Rust launcher made the same call for the same reason. + * Returns 0 on success, else the process exit code (message already printed). + */ +static int parse(int argc, char **argv, struct cli *cli) { + /* argc bounds each grant list; the launcher execs or exits, so no free. */ + cli->ro = calloc(argc > 0 ? (size_t)argc : 1, sizeof *cli->ro); + cli->rw = calloc(argc > 0 ? (size_t)argc : 1, sizeof *cli->rw); + if (cli->ro == NULL || cli->rw == NULL) return fail("out of memory", NULL); + + int index = 1; + while (index < argc) { + const char *arg = argv[index]; + if (strcmp(arg, "--probe") == 0) { + cli->probe = 1; + index += 1; + } else if (strcmp(arg, "--ro") == 0 || strcmp(arg, "--rw") == 0) { + if (index + 1 >= argc) { + return fail_usage(arg, " requires a path"); + } + if (strcmp(arg, "--ro") == 0) { + cli->ro[cli->ro_count++] = argv[index + 1]; + } else { + cli->rw[cli->rw_count++] = argv[index + 1]; + } + index += 2; + } else if (strcmp(arg, "--") == 0) { + cli->command = &argv[index + 1]; + break; + } else { + return fail_usage("unknown argument: ", arg); + } + } + if (cli->probe) { + if (cli->ro_count > 0 || cli->rw_count > 0 || (cli->command != NULL && cli->command[0] != NULL)) { + return fail_usage("--probe takes no other arguments", NULL); + } + } else if (cli->command == NULL || cli->command[0] == NULL) { + return fail_usage("missing `-- ...` command", NULL); + } + return 0; +} + +/* The filesystem accesses the running kernel's ABI can govern. */ +static uint64_t fs_mask_for_abi(long abi) { + uint64_t mask = LL_ABI1_MASK; + if (abi >= 2) mask |= LL_FS_REFER; + if (abi >= 3) mask |= LL_FS_TRUNCATE; + if (abi >= 5) mask |= LL_FS_IOCTL_DEV; + return mask; +} + +/* Add one path-beneath rule; 0 on success, else the exit code. */ +static int add_rule(int ruleset_fd, const char *path, uint64_t access) { + int path_fd = open(path, O_PATH | O_CLOEXEC); + if (path_fd < 0) { + /* Fail closed on an unopenable grant root: silently narrowing the + * granted set would be safe, but running with a profile the caller did + * not get is not worth the ambiguity. */ + fprintf(stderr, "landlock-run: cannot open rule path: %s: %s\n", path, strerror(errno)); + return EXIT_LAUNCHER_FAILURE; + } + /* The kernel rejects directory-only accesses on a non-directory rule + * (EINVAL), so a file grant keeps only the file-compatible bits — how the + * `--rw /dev/null` grant works. Same clamp the Rust crate's + * path_beneath_rules helper applied. */ + struct stat st; + if (fstat(path_fd, &st) == 0 && !S_ISDIR(st.st_mode)) { + access &= LL_FS_EXECUTE | LL_FS_WRITE_FILE | LL_FS_READ_FILE | LL_FS_TRUNCATE | LL_FS_IOCTL_DEV; + } + struct landlock_path_beneath_attr attr = { .allowed_access = access, .parent_fd = path_fd }; + if (syscall(__NR_landlock_add_rule, ruleset_fd, LANDLOCK_RULE_PATH_BENEATH, &attr, 0) != 0) { + int saved = errno; + close(path_fd); + return fail("landlock ruleset error", strerror(saved)); + } + close(path_fd); + return 0; +} + +/* + * Install the ruleset on the current thread, negotiating the kernel's ABI + * down from MAX_ABI. `--ro` paths get the read side of the vocabulary (read + * file/dir + execute — the wrapped `bash` and everything it spawns must + * remain executable); `--rw` paths get every filesystem access the + * negotiated ABI can grant. Sets `no_new_privs` first (mandatory for an + * unprivileged restrict, and it neutralizes setuid/setgid escalation inside + * the sandbox). On success `*partial` reports whether the kernel governs + * only a subset of MAX_ABI's accesses. Returns 0, else the exit code. + */ +static int restrict_self(const struct cli *cli, int *partial) { + long abi = syscall(__NR_landlock_create_ruleset, NULL, 0, LANDLOCK_CREATE_RULESET_VERSION); + if (abi < 0) { + /* ENOSYS: kernel built without Landlock; EOPNOTSUPP: built but disabled. + * Either way: not enforceable — fail CLOSED, never exec unconfined. */ + return fail(NOT_ENFORCED_MESSAGE, NULL); + } + *partial = abi < MAX_ABI; + uint64_t handled = fs_mask_for_abi(abi < MAX_ABI ? abi : MAX_ABI); + + struct landlock_ruleset_attr attr = { .handled_access_fs = handled }; + int ruleset_fd = (int)syscall(__NR_landlock_create_ruleset, &attr, sizeof attr, 0); + if (ruleset_fd < 0) return fail("landlock ruleset error", strerror(errno)); + + const uint64_t read_side = LL_FS_EXECUTE | LL_FS_READ_FILE | LL_FS_READ_DIR; + for (size_t i = 0; i < cli->ro_count; i++) { + int code = add_rule(ruleset_fd, cli->ro[i], read_side & handled); + if (code != 0) return code; + } + for (size_t i = 0; i < cli->rw_count; i++) { + int code = add_rule(ruleset_fd, cli->rw[i], handled); + if (code != 0) return code; + } + + if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0) { + return fail("landlock ruleset error", strerror(errno)); + } + if (syscall(__NR_landlock_restrict_self, ruleset_fd, 0) != 0) { + return fail("landlock ruleset error", strerror(errno)); + } + close(ruleset_fd); + return 0; +} + +int main(int argc, char **argv) { + struct cli cli = { 0 }; + int code = parse(argc, argv, &cli); + if (code != 0) return code; + + if (cli.probe) { + /* The functional probe: build and enforce a maximal ruleset in THIS + * short-lived process (the probe run exits right after). `--version` + * style checks would miss a kernel that has the syscalls but refuses + * enforcement; actually restricting is the only honest signal. The one + * report line is part of the launcher CLI contract — the executor reads + * enforcement completeness from it. */ + static const char *probe_root = "/"; + struct cli probe = { .ro = &probe_root, .ro_count = 1 }; + int partial = 0; + code = restrict_self(&probe, &partial); + if (code != 0) return code; + printf("landlock: %s\n", partial ? "partially enforced (older ABI)" : "fully enforced"); + return 0; + } + + int partial = 0; + code = restrict_self(&cli, &partial); + if (code != 0) return code; + if (partial) { + /* Older ABI: some handled accesses are not governed (e.g. truncate + * before ABI 3). Still confined for everything the kernel supports — + * report, do not refuse. */ + fprintf(stderr, "landlock-run: partial enforcement (older Landlock ABI)\n"); + } + + execvp(cli.command[0], cli.command); + /* exec only returns on failure. */ + return fail("exec failed", strerror(errno)); +} diff --git a/native/landlock-run/packages/entry/tsconfig.json b/native/landlock-run/packages/entry/tsconfig.json new file mode 100644 index 0000000000..bb991d6ceb --- /dev/null +++ b/native/landlock-run/packages/entry/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "declaration": true, + "outDir": "lib", + "rootDir": "src", + "tsBuildInfoFile": "lib/.tsbuildinfo" + }, + "include": ["src/**/*.ts"] +} diff --git a/native/landlock-run/packages/linux-arm64/LICENSE b/native/landlock-run/packages/linux-arm64/LICENSE new file mode 100644 index 0000000000..8187059c9a --- /dev/null +++ b/native/landlock-run/packages/linux-arm64/LICENSE @@ -0,0 +1,28 @@ +BSD 3-Clause License + +Copyright (c) 2026, node-addon-landlock-run contributors + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/native/landlock-run/packages/linux-arm64/README.md b/native/landlock-run/packages/linux-arm64/README.md new file mode 100644 index 0000000000..1921c8f4b5 --- /dev/null +++ b/native/landlock-run/packages/linux-arm64/README.md @@ -0,0 +1,7 @@ +# node-addon-landlock-run-linux-arm64 + +Prebuilt `bin/landlock-run` Landlock launcher for linux-arm64 — a static musl binary compiled natively (no cross toolchain) from the C source shipped in [`node-addon-landlock-run`](https://www.npmjs.com/package/node-addon-landlock-run). npm's `os`/`cpu` fields select this package at install time; the entry package resolves it to a file path — it ships no JavaScript and is never imported. + +The binary is git-ignored and rides the npm tarball via the `files` list; the `prepack` gate refuses to pack when it is missing or has the wrong ELF architecture, and the release pipeline byte-pins the packed binary against the CI build it came from. Static musl linking means one binary for glibc and musl distros alike — hence no libc suffix in the name. + +Sibling: `node-addon-landlock-run-linux-x64`. diff --git a/native/landlock-run/packages/linux-arm64/package.json b/native/landlock-run/packages/linux-arm64/package.json new file mode 100644 index 0000000000..0067f77c8b --- /dev/null +++ b/native/landlock-run/packages/linux-arm64/package.json @@ -0,0 +1,26 @@ +{ + "name": "node-addon-landlock-run-linux-arm64", + "version": "0.0.1", + "description": "Prebuilt landlock-run Landlock launcher binary for linux-arm64 (static musl) — resolved as a file path by node-addon-landlock-run, never imported", + "os": [ + "linux" + ], + "cpu": [ + "arm64" + ], + "files": [ + "README.md", + "bin/", + "prebuilds.json" + ], + "scripts": { + "prepack": "node ../../scripts/verify-launcher-binary.mjs" + }, + "engines": { + "node": ">=20" + }, + "license": "BSD-3-Clause", + "publishConfig": { + "access": "public" + } +} diff --git a/native/landlock-run/packages/linux-arm64/prebuilds.json b/native/landlock-run/packages/linux-arm64/prebuilds.json new file mode 100644 index 0000000000..81e6b429f7 --- /dev/null +++ b/native/landlock-run/packages/linux-arm64/prebuilds.json @@ -0,0 +1,10 @@ +{ + "platform": "linux-arm64", + "binaries": [ + { + "tool": "landlock-run", + "kind": "static-musl", + "path": "bin/landlock-run" + } + ] +} diff --git a/native/landlock-run/packages/linux-x64/LICENSE b/native/landlock-run/packages/linux-x64/LICENSE new file mode 100644 index 0000000000..8187059c9a --- /dev/null +++ b/native/landlock-run/packages/linux-x64/LICENSE @@ -0,0 +1,28 @@ +BSD 3-Clause License + +Copyright (c) 2026, node-addon-landlock-run contributors + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/native/landlock-run/packages/linux-x64/README.md b/native/landlock-run/packages/linux-x64/README.md new file mode 100644 index 0000000000..ce741eb34c --- /dev/null +++ b/native/landlock-run/packages/linux-x64/README.md @@ -0,0 +1,7 @@ +# node-addon-landlock-run-linux-x64 + +Prebuilt `bin/landlock-run` Landlock launcher for linux-x64 — a static musl binary compiled natively (no cross toolchain) from the C source shipped in [`node-addon-landlock-run`](https://www.npmjs.com/package/node-addon-landlock-run). npm's `os`/`cpu` fields select this package at install time; the entry package resolves it to a file path — it ships no JavaScript and is never imported. + +The binary is git-ignored and rides the npm tarball via the `files` list; the `prepack` gate refuses to pack when it is missing or has the wrong ELF architecture, and the release pipeline byte-pins the packed binary against the CI build it came from. Static musl linking means one binary for glibc and musl distros alike — hence no libc suffix in the name. + +Sibling: `node-addon-landlock-run-linux-arm64`. diff --git a/native/landlock-run/packages/linux-x64/package.json b/native/landlock-run/packages/linux-x64/package.json new file mode 100644 index 0000000000..8ea60b636c --- /dev/null +++ b/native/landlock-run/packages/linux-x64/package.json @@ -0,0 +1,26 @@ +{ + "name": "node-addon-landlock-run-linux-x64", + "version": "0.0.1", + "description": "Prebuilt landlock-run Landlock launcher binary for linux-x64 (static musl) — resolved as a file path by node-addon-landlock-run, never imported", + "os": [ + "linux" + ], + "cpu": [ + "x64" + ], + "files": [ + "README.md", + "bin/", + "prebuilds.json" + ], + "scripts": { + "prepack": "node ../../scripts/verify-launcher-binary.mjs" + }, + "engines": { + "node": ">=20" + }, + "license": "BSD-3-Clause", + "publishConfig": { + "access": "public" + } +} diff --git a/native/landlock-run/packages/linux-x64/prebuilds.json b/native/landlock-run/packages/linux-x64/prebuilds.json new file mode 100644 index 0000000000..27b0de360c --- /dev/null +++ b/native/landlock-run/packages/linux-x64/prebuilds.json @@ -0,0 +1,10 @@ +{ + "platform": "linux-x64", + "binaries": [ + { + "tool": "landlock-run", + "kind": "static-musl", + "path": "bin/landlock-run" + } + ] +} diff --git a/native/landlock-run/pnpm-lock.yaml b/native/landlock-run/pnpm-lock.yaml new file mode 100644 index 0000000000..88b1b3df00 --- /dev/null +++ b/native/landlock-run/pnpm-lock.yaml @@ -0,0 +1,345 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + devDependencies: + '@types/node': + specifier: ^24.10.0 + version: 24.13.2 + node-addon-landlock-run: + specifier: workspace:* + version: link:packages/entry + tsx: + specifier: ^4.20.6 + version: 4.23.0 + typescript: + specifier: ^5.9.3 + version: 5.9.3 + + packages/entry: + optionalDependencies: + node-addon-landlock-run-linux-arm64: + specifier: workspace:* + version: link:../linux-arm64 + node-addon-landlock-run-linux-x64: + specifier: workspace:* + version: link:../linux-x64 + + packages/linux-arm64: {} + + packages/linux-x64: {} + +packages: + + '@esbuild/aix-ppc64@0.28.1': + resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==} + engines: {node: '>=18'} + cpu: [ppc64] + os: [aix] + + '@esbuild/android-arm64@0.28.1': + resolution: {integrity: sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==} + engines: {node: '>=18'} + cpu: [arm64] + os: [android] + + '@esbuild/android-arm@0.28.1': + resolution: {integrity: sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==} + engines: {node: '>=18'} + cpu: [arm] + os: [android] + + '@esbuild/android-x64@0.28.1': + resolution: {integrity: sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==} + engines: {node: '>=18'} + cpu: [x64] + os: [android] + + '@esbuild/darwin-arm64@0.28.1': + resolution: {integrity: sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==} + engines: {node: '>=18'} + cpu: [arm64] + os: [darwin] + + '@esbuild/darwin-x64@0.28.1': + resolution: {integrity: sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==} + engines: {node: '>=18'} + cpu: [x64] + os: [darwin] + + '@esbuild/freebsd-arm64@0.28.1': + resolution: {integrity: sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==} + engines: {node: '>=18'} + cpu: [arm64] + os: [freebsd] + + '@esbuild/freebsd-x64@0.28.1': + resolution: {integrity: sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==} + engines: {node: '>=18'} + cpu: [x64] + os: [freebsd] + + '@esbuild/linux-arm64@0.28.1': + resolution: {integrity: sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==} + engines: {node: '>=18'} + cpu: [arm64] + os: [linux] + + '@esbuild/linux-arm@0.28.1': + resolution: {integrity: sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==} + engines: {node: '>=18'} + cpu: [arm] + os: [linux] + + '@esbuild/linux-ia32@0.28.1': + resolution: {integrity: sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==} + engines: {node: '>=18'} + cpu: [ia32] + os: [linux] + + '@esbuild/linux-loong64@0.28.1': + resolution: {integrity: sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==} + engines: {node: '>=18'} + cpu: [loong64] + os: [linux] + + '@esbuild/linux-mips64el@0.28.1': + resolution: {integrity: sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==} + engines: {node: '>=18'} + cpu: [mips64el] + os: [linux] + + '@esbuild/linux-ppc64@0.28.1': + resolution: {integrity: sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==} + engines: {node: '>=18'} + cpu: [ppc64] + os: [linux] + + '@esbuild/linux-riscv64@0.28.1': + resolution: {integrity: sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==} + engines: {node: '>=18'} + cpu: [riscv64] + os: [linux] + + '@esbuild/linux-s390x@0.28.1': + resolution: {integrity: sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==} + engines: {node: '>=18'} + cpu: [s390x] + os: [linux] + + '@esbuild/linux-x64@0.28.1': + resolution: {integrity: sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==} + engines: {node: '>=18'} + cpu: [x64] + os: [linux] + + '@esbuild/netbsd-arm64@0.28.1': + resolution: {integrity: sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==} + engines: {node: '>=18'} + cpu: [arm64] + os: [netbsd] + + '@esbuild/netbsd-x64@0.28.1': + resolution: {integrity: sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==} + engines: {node: '>=18'} + cpu: [x64] + os: [netbsd] + + '@esbuild/openbsd-arm64@0.28.1': + resolution: {integrity: sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==} + engines: {node: '>=18'} + cpu: [arm64] + os: [openbsd] + + '@esbuild/openbsd-x64@0.28.1': + resolution: {integrity: sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==} + engines: {node: '>=18'} + cpu: [x64] + os: [openbsd] + + '@esbuild/openharmony-arm64@0.28.1': + resolution: {integrity: sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==} + engines: {node: '>=18'} + cpu: [arm64] + os: [openharmony] + + '@esbuild/sunos-x64@0.28.1': + resolution: {integrity: sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==} + engines: {node: '>=18'} + cpu: [x64] + os: [sunos] + + '@esbuild/win32-arm64@0.28.1': + resolution: {integrity: sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==} + engines: {node: '>=18'} + cpu: [arm64] + os: [win32] + + '@esbuild/win32-ia32@0.28.1': + resolution: {integrity: sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==} + engines: {node: '>=18'} + cpu: [ia32] + os: [win32] + + '@esbuild/win32-x64@0.28.1': + resolution: {integrity: sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==} + engines: {node: '>=18'} + cpu: [x64] + os: [win32] + + '@types/node@24.13.2': + resolution: {integrity: sha512-fRa09kZTgu8o71KFcDjUFuc7F+dEbZYZmkI0mg5YBTRs0yMKjYHsq/c0urDKeDb+D5qVgXOdFcuu+DZPKOITwA==} + + esbuild@0.28.1: + resolution: {integrity: sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==} + engines: {node: '>=18'} + hasBin: true + + fsevents@2.3.3: + resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==} + engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} + os: [darwin] + + tsx@4.23.0: + resolution: {integrity: sha512-eUdUIaCr963q2h5u3+QwvYp0+eqPvn+egeqZUm0hwERCqqx1E3kK5ehbGCvqSE5MQAULr67ww0cA3jKc3YkM1w==} + engines: {node: '>=18.0.0'} + hasBin: true + + typescript@5.9.3: + resolution: {integrity: sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==} + engines: {node: '>=14.17'} + hasBin: true + + undici-types@7.18.2: + resolution: {integrity: sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==} + +snapshots: + + '@esbuild/aix-ppc64@0.28.1': + optional: true + + '@esbuild/android-arm64@0.28.1': + optional: true + + '@esbuild/android-arm@0.28.1': + optional: true + + '@esbuild/android-x64@0.28.1': + optional: true + + '@esbuild/darwin-arm64@0.28.1': + optional: true + + '@esbuild/darwin-x64@0.28.1': + optional: true + + '@esbuild/freebsd-arm64@0.28.1': + optional: true + + '@esbuild/freebsd-x64@0.28.1': + optional: true + + '@esbuild/linux-arm64@0.28.1': + optional: true + + '@esbuild/linux-arm@0.28.1': + optional: true + + '@esbuild/linux-ia32@0.28.1': + optional: true + + '@esbuild/linux-loong64@0.28.1': + optional: true + + '@esbuild/linux-mips64el@0.28.1': + optional: true + + '@esbuild/linux-ppc64@0.28.1': + optional: true + + '@esbuild/linux-riscv64@0.28.1': + optional: true + + '@esbuild/linux-s390x@0.28.1': + optional: true + + '@esbuild/linux-x64@0.28.1': + optional: true + + '@esbuild/netbsd-arm64@0.28.1': + optional: true + + '@esbuild/netbsd-x64@0.28.1': + optional: true + + '@esbuild/openbsd-arm64@0.28.1': + optional: true + + '@esbuild/openbsd-x64@0.28.1': + optional: true + + '@esbuild/openharmony-arm64@0.28.1': + optional: true + + '@esbuild/sunos-x64@0.28.1': + optional: true + + '@esbuild/win32-arm64@0.28.1': + optional: true + + '@esbuild/win32-ia32@0.28.1': + optional: true + + '@esbuild/win32-x64@0.28.1': + optional: true + + '@types/node@24.13.2': + dependencies: + undici-types: 7.18.2 + + esbuild@0.28.1: + optionalDependencies: + '@esbuild/aix-ppc64': 0.28.1 + '@esbuild/android-arm': 0.28.1 + '@esbuild/android-arm64': 0.28.1 + '@esbuild/android-x64': 0.28.1 + '@esbuild/darwin-arm64': 0.28.1 + '@esbuild/darwin-x64': 0.28.1 + '@esbuild/freebsd-arm64': 0.28.1 + '@esbuild/freebsd-x64': 0.28.1 + '@esbuild/linux-arm': 0.28.1 + '@esbuild/linux-arm64': 0.28.1 + '@esbuild/linux-ia32': 0.28.1 + '@esbuild/linux-loong64': 0.28.1 + '@esbuild/linux-mips64el': 0.28.1 + '@esbuild/linux-ppc64': 0.28.1 + '@esbuild/linux-riscv64': 0.28.1 + '@esbuild/linux-s390x': 0.28.1 + '@esbuild/linux-x64': 0.28.1 + '@esbuild/netbsd-arm64': 0.28.1 + '@esbuild/netbsd-x64': 0.28.1 + '@esbuild/openbsd-arm64': 0.28.1 + '@esbuild/openbsd-x64': 0.28.1 + '@esbuild/openharmony-arm64': 0.28.1 + '@esbuild/sunos-x64': 0.28.1 + '@esbuild/win32-arm64': 0.28.1 + '@esbuild/win32-ia32': 0.28.1 + '@esbuild/win32-x64': 0.28.1 + + fsevents@2.3.3: + optional: true + + tsx@4.23.0: + dependencies: + esbuild: 0.28.1 + optionalDependencies: + fsevents: 2.3.3 + + typescript@5.9.3: {} + + undici-types@7.18.2: {} diff --git a/native/landlock-run/pnpm-workspace.yaml b/native/landlock-run/pnpm-workspace.yaml new file mode 100644 index 0000000000..22299bfea0 --- /dev/null +++ b/native/landlock-run/pnpm-workspace.yaml @@ -0,0 +1,8 @@ +packages: + - packages/* + +# pnpm 10+ blocks any dependency shipping an install/build script until it is +# explicitly reviewed here. Deny by default; esbuild (tsx's bundled native +# binary) genuinely needs its script. +allowBuilds: + esbuild: true diff --git a/native/landlock-run/scripts/assemble-prebuilds.mjs b/native/landlock-run/scripts/assemble-prebuilds.mjs new file mode 100644 index 0000000000..4dcdb23bed --- /dev/null +++ b/native/landlock-run/scripts/assemble-prebuilds.mjs @@ -0,0 +1,51 @@ +#!/usr/bin/env node +/** + * Assemble downloaded release artifacts into the platform packages and + * verify the result. The Release workflow's build legs upload one + * `prebuild-` artifact per platform package (its `bin/` payload); + * this script copies each into `packages//bin/` and then checks + * every declared binary for presence and ELF architecture. + * + * Usage: `node scripts/assemble-prebuilds.mjs `. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { platformDirs, root, verifyPlatformBinaries } from './repo.mjs'; + +const artifactRoot = path.resolve(process.argv[2] || '.release/prebuild-artifacts'); + +if (!fs.existsSync(artifactRoot)) { + throw new Error(`prebuild artifact directory does not exist: ${artifactRoot}`); +} + +const platforms = platformDirs().map((dir) => path.basename(dir)); + +for (const name of platforms) { + const binDir = path.join(root, 'packages', name, 'bin'); + fs.rmSync(binDir, { recursive: true, force: true }); + fs.mkdirSync(binDir, { recursive: true }); +} + +for (const artifactName of fs.readdirSync(artifactRoot)) { + const artifactDir = path.join(artifactRoot, artifactName); + if (!fs.statSync(artifactDir).isDirectory()) continue; + + const name = platforms.find((candidate) => artifactName === `prebuild-${candidate}`); + if (!name) { + throw new Error(`cannot map artifact to a platform package: ${artifactName}`); + } + + for (const file of fs.readdirSync(artifactDir)) { + const source = path.join(artifactDir, file); + const destination = path.join(root, 'packages', name, 'bin', file); + fs.copyFileSync(source, destination); + fs.chmodSync(destination, 0o755); + console.log(`Copied ${path.relative(root, source)} -> ${path.relative(root, destination)}`); + } +} + +for (const dir of platformDirs()) { + const { name, count } = verifyPlatformBinaries(path.join(root, dir)); + console.log(`Verified ${name}: ${count} binaries`); +} diff --git a/native/landlock-run/scripts/build.ts b/native/landlock-run/scripts/build.ts new file mode 100644 index 0000000000..5866cc0dc4 --- /dev/null +++ b/native/landlock-run/scripts/build.ts @@ -0,0 +1,86 @@ +/** + * Build every native tool this host can build, into its per-platform + * package. + * + * Targets are derived from the checked-in matrix: each + * `packages//prebuilds.json` whose `platform` matches this host names + * the binaries to produce; the TOOLS table below maps each `tool` to its C + * source. Builds are NATIVE-ONLY — each Linux architecture compiles its own + * binary with the distro's `musl-gcc` (static musl: runs on glibc and musl + * distros alike, no loader or libc expectations on the consumer host), and + * CI's per-arch runners are the builders of record. No cross toolchain + * exists here on purpose: native runners replace it, and the audit surface + * is the reviewed C source plus CI provenance. + * + * Binaries land in `packages//bin/` — git-ignored (root + * `.gitignore`), packed into the platform package's npm tarball behind its + * `prepack` gate (`scripts/verify-launcher-binary.mjs`). + * + * Run: `pnpm run build:native` (Linux with musl-gcc on PATH: + * `apt-get install musl-tools`). Non-Linux hosts fail fast — no platform + * package exists for them to build. + */ +import { spawnSync } from 'node:child_process' +import { existsSync, mkdirSync, readdirSync, readFileSync } from 'node:fs' +import { basename, dirname, join, resolve } from 'node:path' + +/** Each native tool's C source, keyed by the `tool` field in prebuilds.json. */ +const TOOLS: Record = { + 'landlock-run': { source: 'packages/entry/src/main.c' }, +} + +const repoRoot = resolve(import.meta.dirname, '..') + +if (process.platform !== 'linux') { + console.error(`build: native tools are built natively per Linux architecture (no cross toolchain) — nothing to build on ${process.platform}. CI's per-arch runners build and rehearse every platform package.`) + process.exit(1) +} +const hostPlatform = `linux-${process.arch}` + +/** This host's platform packages, from the checked-in matrix. */ +const targets: { packageDir: string; tool: string; binaryPath: string; kind: string }[] = [] +const packagesRoot = join(repoRoot, 'packages') +for (const name of readdirSync(packagesRoot).sort()) { + const prebuildsFile = join(packagesRoot, name, 'prebuilds.json') + if (!existsSync(prebuildsFile)) continue + const prebuilds = JSON.parse(readFileSync(prebuildsFile, 'utf8')) as { + platform: string + binaries: { tool: string; kind: string; path: string }[] + } + if (prebuilds.platform !== hostPlatform) continue + for (const binary of prebuilds.binaries) { + targets.push({ packageDir: join(packagesRoot, name), tool: binary.tool, binaryPath: binary.path, kind: binary.kind }) + } +} +if (targets.length === 0) { + console.error(`build: no platform package declares binaries for ${hostPlatform} — supported platforms are the packages/*/prebuilds.json "platform" values.`) + process.exit(1) +} + +for (const target of targets) { + const tool = TOOLS[target.tool] + if (tool === undefined) { + console.error(`build: prebuilds.json names unknown tool "${target.tool}" — add it to the TOOLS table in scripts/build.ts.`) + process.exit(1) + } + if (target.kind !== 'static-musl') { + console.error(`build: unknown binary kind "${target.kind}" — the only toolchain here is static musl.`) + process.exit(1) + } + const binary = join(target.packageDir, target.binaryPath) + mkdirSync(dirname(binary), { recursive: true }) + + // -static against musl: self-contained, no loader/libc expectations on the + // consumer host. -Werror is safe to keep hard: CI pins the builder images, + // and a new warning on a toolchain bump deserves a look, not a pass. + const result = spawnSync('musl-gcc', [ + '-std=c11', '-Os', '-Wall', '-Wextra', '-Werror', '-static', '-s', + '-o', binary, join(repoRoot, tool.source), + ], { stdio: ['ignore', 'inherit', 'inherit'] }) + if (result.error !== undefined || result.status !== 0) { + console.error('build: musl-gcc failed' + + (result.error ? ` (${result.error.message} — is musl-tools installed?)` : '')) + process.exit(1) + } + console.log(`build: built ${basename(target.packageDir)}/${target.binaryPath}`) +} diff --git a/native/landlock-run/scripts/bump-release.mjs b/native/landlock-run/scripts/bump-release.mjs new file mode 100644 index 0000000000..29a7777379 --- /dev/null +++ b/native/landlock-run/scripts/bump-release.mjs @@ -0,0 +1,90 @@ +#!/usr/bin/env node +/** + * Bump every package (workspace root + packages/*) to one version, refresh + * the lockfile, and verify. Usage: `pnpm release:bump `. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { packageDirs, readJson, root } from './repo.mjs'; + +const bump = process.argv[2]; +const releaseTypes = new Set(['major', 'minor', 'patch']); + +function writeJson(file, value) { + fs.writeFileSync(file, `${JSON.stringify(value, null, 2)}\n`); +} + +function run(command, args) { + const result = spawnSync(command, args, { + cwd: root, + stdio: 'inherit', + env: { ...process.env, CI: 'true' }, + }); + if (result.error) throw result.error; + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +function packageFiles() { + return ['package.json', ...packageDirs().map((dir) => path.join(dir, 'package.json'))]; +} + +function parseVersion(version) { + const match = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/.exec(version); + if (!match) { + throw new Error(`increment types need a plain x.y.z current version (current: ${version}) — pass an explicit target version instead`); + } + return match.slice(1).map((part) => Number(part)); +} + +/** Explicit target versions accept full semver, prereleases included (test publishes). */ +const EXPLICIT_VERSION = /^\d+\.\d+\.\d+(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$/; + +function nextVersion(current, release) { + if (EXPLICIT_VERSION.test(release)) return release; + + if (!releaseTypes.has(release)) { + throw new Error('Usage: pnpm release:bump '); + } + + const [major, minor, patch] = parseVersion(current); + if (release === 'major') return `${major + 1}.0.0`; + if (release === 'minor') return `${major}.${minor + 1}.0`; + return `${major}.${minor}.${patch + 1}`; +} + +function currentPublishedVersion(files) { + const versions = new Set( + files + .filter((file) => file.startsWith('packages/')) + .map((file) => readJson(path.join(root, file)).version), + ); + if (versions.size !== 1) { + throw new Error(`published package versions differ: ${[...versions].join(', ')}`); + } + return [...versions][0]; +} + +if (!bump) { + console.error('Usage: pnpm release:bump '); + process.exit(1); +} + +const files = packageFiles(); +const targetVersion = nextVersion(currentPublishedVersion(files), bump); + +for (const file of files) { + const fullPath = path.join(root, file); + const json = readJson(fullPath); + json.version = targetVersion; + writeJson(fullPath, json); + console.log(`${file}: ${targetVersion}`); +} + +run('pnpm', ['install', '--ignore-scripts', '--lockfile-only']); +run('node', ['./scripts/verify-release.mjs']); + +console.log(`Release version bumped to ${targetVersion}`); diff --git a/native/landlock-run/scripts/commit-release.mjs b/native/landlock-run/scripts/commit-release.mjs new file mode 100644 index 0000000000..b7bf3e513b --- /dev/null +++ b/native/landlock-run/scripts/commit-release.mjs @@ -0,0 +1,42 @@ +#!/usr/bin/env node +/** + * Bump, stage, and commit a release in one command: + * `pnpm release:commit `. The tag stays manual — + * create it from the merged release commit. + */ + +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { packageDirs, readJson, root } from './repo.mjs'; + +const bump = process.argv[2]; + +function run(command, args) { + const result = spawnSync(command, args, { + cwd: root, + stdio: 'inherit', + env: { ...process.env, CI: 'true' }, + }); + if (result.error) throw result.error; + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +if (!bump) { + console.error('Usage: pnpm release:commit '); + process.exit(1); +} + +run('node', ['./scripts/bump-release.mjs', bump]); + +const version = readJson(path.join(root, packageDirs()[0], 'package.json')).version; +run('git', [ + 'add', + 'package.json', + 'packages/*/package.json', + 'pnpm-lock.yaml', +]); +run('git', ['commit', '-m', `release: ${version}`]); + +console.log(`Committed release ${version}. Create the tag manually: git tag v${version}`); diff --git a/native/landlock-run/scripts/github-matrix.mjs b/native/landlock-run/scripts/github-matrix.mjs new file mode 100644 index 0000000000..9566b89c8a --- /dev/null +++ b/native/landlock-run/scripts/github-matrix.mjs @@ -0,0 +1,66 @@ +#!/usr/bin/env node +/** + * Derive the GitHub Actions matrices from the checked-in package matrix + * (`packages//prebuilds.json`). Single source: adding a platform + * package extends CI and Release without editing a workflow. + * + * node scripts/github-matrix.mjs ci → one leg per distinct platform + * node scripts/github-matrix.mjs release-prebuild → one leg per platform package + */ + +import path from 'node:path'; +import { platformDirs, readJson, root } from './repo.mjs'; + +/** GitHub runner per prebuilds.json `platform` value — native builders only, no cross toolchain. */ +const RUNNERS = { + 'linux-x64': 'ubuntu-24.04', + 'linux-arm64': 'ubuntu-24.04-arm', +}; + +function runnerFor(platform) { + const runner = RUNNERS[platform]; + if (!runner) { + throw new Error(`missing GitHub runner for platform: ${platform}`); + } + return runner; +} + +function platformManifests() { + return platformDirs().map((dir) => ({ + dir, + name: path.basename(dir), + prebuilds: readJson(path.join(root, dir, 'prebuilds.json')), + })); +} + +function ciMatrix() { + const platforms = [...new Set(platformManifests().map(({ prebuilds }) => prebuilds.platform))].sort(); + return { + include: platforms.map((platform) => ({ platform, runner: runnerFor(platform) })), + }; +} + +function releasePrebuildMatrix() { + return { + include: platformManifests().map(({ dir, name, prebuilds }) => ({ + platform: prebuilds.platform, + package: name, + dir, + runner: runnerFor(prebuilds.platform), + artifact: `prebuild-${name}`, + })), + }; +} + +const target = process.argv[2]; +const matrices = { + ci: ciMatrix, + 'release-prebuild': releasePrebuildMatrix, +}; + +if (!target || !matrices[target]) { + console.error(`Usage: node scripts/github-matrix.mjs <${Object.keys(matrices).join('|')}>`); + process.exit(1); +} + +process.stdout.write(JSON.stringify(matrices[target]())); diff --git a/native/landlock-run/scripts/pack-release.mjs b/native/landlock-run/scripts/pack-release.mjs new file mode 100644 index 0000000000..fbec0b610b --- /dev/null +++ b/native/landlock-run/scripts/pack-release.mjs @@ -0,0 +1,76 @@ +#!/usr/bin/env node +/** + * Pack every published package into release tarballs, in publish order + * (platform packages first, then the entries that optionally depend on + * them), and write `publish-order.txt` next to them. `pnpm pack` produces + * the EXACT bytes `pnpm publish` would upload and runs each package's + * `prepack` gate, so a missing binary or unbuilt `lib/` refuses here. + * + * Usage: `node scripts/pack-release.mjs [dest] [--current-platform-only]`. + * The flag packs only THIS host's platform package plus the entries — for + * per-architecture CI legs, where the other architecture's binary does not + * exist (the exact refusal its prepack gate exists for). + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { entryDirs, platformDirs, readJson, root } from './repo.mjs'; + +const args = process.argv.slice(2); +const currentPlatformOnly = args.includes('--current-platform-only'); +const destination = path.resolve(args.find((arg) => !arg.startsWith('--')) || path.join(root, 'dist', 'npm')); + +function hostPlatformDirs() { + const hostPlatform = `${process.platform}-${process.arch}`; + return platformDirs().filter((dir) => readJson(path.join(root, dir, 'prebuilds.json')).platform === hostPlatform); +} + +function run(command, args) { + const result = spawnSync(command, args, { + cwd: root, + stdio: 'inherit', + }); + if (result.error) throw result.error; + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +function tarballName(manifest) { + if (manifest.name.startsWith('@')) { + return `${manifest.name.slice(1).replace('/', '-')}-${manifest.version}.tgz`; + } + return `${manifest.name}-${manifest.version}.tgz`; +} + +fs.rmSync(destination, { recursive: true, force: true }); +fs.mkdirSync(destination, { recursive: true }); + +const dirs = [...(currentPlatformOnly ? hostPlatformDirs() : platformDirs()), ...entryDirs()]; +const platformSet = new Set(platformDirs()); +const publishOrder = []; +for (const dir of dirs) { + const manifest = readJson(path.join(root, dir, 'package.json')); + // Platform packages are packed with npm: pnpm pack (observed on 11.7.0) + // normalizes file modes and STRIPS the executable bit, which ships a + // launcher no consumer can spawn; npm pack preserves it. Platform packages + // have no dependencies by construction, so they need none of pnpm's + // workspace-protocol conversion — the entry packages do, and carry no + // executables, so they keep pnpm pack. + if (platformSet.has(dir)) { + run('npm', ['pack', `./${dir}`, '--pack-destination', destination]); + } else { + run('pnpm', ['--dir', dir, 'pack', '--pack-destination', destination]); + } + + const tarball = tarballName(manifest); + const tarballPath = path.join(destination, tarball); + if (!fs.existsSync(tarballPath)) { + throw new Error(`expected pack output not found: ${tarballPath}`); + } + publishOrder.push(tarball); +} + +fs.writeFileSync(path.join(destination, 'publish-order.txt'), `${publishOrder.join('\n')}\n`); +console.log(`Packed ${publishOrder.length} packages into ${path.relative(root, destination)}`); diff --git a/native/landlock-run/scripts/repo.mjs b/native/landlock-run/scripts/repo.mjs new file mode 100644 index 0000000000..8032d3da37 --- /dev/null +++ b/native/landlock-run/scripts/repo.mjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +/** + * Shared helpers for the repo scripts: package discovery, the checked-in + * prebuild matrix, and binary verification. The package matrix is explicit + * metadata — `packages//prebuilds.json` marks a platform package and + * declares its binaries; everything else under `packages/` is an entry + * package. Scripts derive from these files and never guess. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +export const root = fileURLToPath(new URL('..', import.meta.url)); +export const packagesRoot = path.join(root, 'packages'); + +/** ELF `e_machine` (offset 18, little-endian) per platform-package `cpu` value. */ +export const E_MACHINE = { x64: 62, arm64: 183 }; + +export function readJson(file) { + return JSON.parse(fs.readFileSync(file, 'utf8')); +} + +/** Platform packages: every `packages/` carrying a `prebuilds.json`. */ +export function platformDirs() { + return fs.readdirSync(packagesRoot) + .filter((name) => fs.existsSync(path.join(packagesRoot, name, 'prebuilds.json'))) + .sort() + .map((name) => path.join('packages', name)); +} + +/** Entry packages: every other `packages/` with a `package.json`. */ +export function entryDirs() { + return fs.readdirSync(packagesRoot) + .filter((name) => !fs.existsSync(path.join(packagesRoot, name, 'prebuilds.json'))) + .filter((name) => fs.existsSync(path.join(packagesRoot, name, 'package.json'))) + .sort() + .map((name) => path.join('packages', name)); +} + +/** All published packages in publish order: platform packages before the entries that optionally depend on them. */ +export function packageDirs() { + return [...platformDirs(), ...entryDirs()]; +} + +/** + * Verify one platform package's binaries against its `prebuilds.json`: + * every declared binary exists, nothing undeclared sits in `bin/`, and each + * file's ELF `e_machine` matches the package's declared `cpu`. Throws with + * a remediation message on the first mismatch. + */ +export function verifyPlatformBinaries(packageDir) { + const manifest = readJson(path.join(packageDir, 'package.json')); + const prebuilds = readJson(path.join(packageDir, 'prebuilds.json')); + const cpu = manifest.cpu?.[0]; + if (cpu === undefined || !(cpu in E_MACHINE)) { + throw new Error(`${manifest.name}: unsupported or missing "cpu" in package.json (expected one of: ${Object.keys(E_MACHINE).join(', ')})`); + } + + for (const binary of prebuilds.binaries) { + const file = path.join(packageDir, binary.path); + if (!fs.existsSync(file)) { + throw new Error(`${manifest.name}: missing ${binary.path} — run \`pnpm build:native\` on a ${prebuilds.platform} host (or assemble release artifacts) before packing.`); + } + try { + fs.accessSync(file, fs.constants.X_OK); + } catch { + // Only reachable when the mode was mangled somewhere between build and + // here (e.g. an archive step that normalized permissions) — the build + // itself always produces 755. + throw new Error(`${manifest.name}: ${binary.path} is not executable — a pack/extract step stripped the mode bit.`); + } + const machine = fs.readFileSync(file).readUInt16LE(18); + if (machine !== E_MACHINE[cpu]) { + throw new Error(`${manifest.name}: ${binary.path} has ELF e_machine ${machine}, expected ${E_MACHINE[cpu]} for ${cpu} — the binary was built for a different architecture.`); + } + } + + const declared = prebuilds.binaries.map((binary) => path.basename(binary.path)).sort(); + const binDir = path.join(packageDir, 'bin'); + const actual = fs.existsSync(binDir) ? fs.readdirSync(binDir).sort() : []; + const extra = actual.filter((name) => !declared.includes(name)); + if (extra.length) { + throw new Error(`${manifest.name}: bin/ contains files not declared in prebuilds.json: ${extra.join(', ')}`); + } + + return { name: manifest.name, count: prebuilds.binaries.length }; +} diff --git a/native/landlock-run/scripts/verify-entry-lib.mjs b/native/landlock-run/scripts/verify-entry-lib.mjs new file mode 100644 index 0000000000..214705e2bc --- /dev/null +++ b/native/landlock-run/scripts/verify-entry-lib.mjs @@ -0,0 +1,25 @@ +#!/usr/bin/env node +/** + * Prepack gate for entry packages: refuse to pack a tarball whose built + * `lib/` is missing. Entry `files` lists use globs, and a glob matching + * nothing packs a silently JS-less tarball instead of failing — this gate + * turns that into a loud refusal on a checkout that never ran + * `pnpm build:ts`. + * + * Runs from each entry package's `prepack` hook (pnpm sets the script cwd + * to the package directory). + */ + +import fs from 'node:fs'; +import path from 'node:path'; + +const packageDir = process.cwd(); +const manifest = JSON.parse(fs.readFileSync(path.join(packageDir, 'package.json'), 'utf8')); + +for (const file of ['lib/index.js', 'lib/index.d.ts']) { + if (!fs.existsSync(path.join(packageDir, file))) { + console.error(`verify-entry-lib: ${manifest.name} has no ${file} — run \`pnpm build:ts\` before packing.`); + process.exit(1); + } +} +console.log(`verify-entry-lib: ${manifest.name} built lib/ present.`); diff --git a/native/landlock-run/scripts/verify-launcher-binary.mjs b/native/landlock-run/scripts/verify-launcher-binary.mjs new file mode 100644 index 0000000000..083cd837aa --- /dev/null +++ b/native/landlock-run/scripts/verify-launcher-binary.mjs @@ -0,0 +1,31 @@ +#!/usr/bin/env node +/** + * Prepack gate for platform packages: refuse to pack a tarball whose + * declared binaries are missing or built for the wrong architecture. + * + * Without it, `pnpm pack` on a checkout that never ran + * `pnpm run build:native` would ship an EMPTY platform package — the + * binary's absence surfacing only at runtime as a failed probe on every + * consumer — and a binary copied across packages would advertise an + * architecture it cannot execute. The check is presence + ELF `e_machine` + * against the package's declared `cpu`; byte provenance is + * `verify-packed-install.mjs`'s concern (it pins the installed tarball + * against the workspace build). + * + * Runs from each platform package's `prepack` hook (pnpm sets the script + * cwd to the package directory). Also callable directly with an explicit + * package directory: `node scripts/verify-launcher-binary.mjs packages/`. + */ + +import path from 'node:path'; +import { root, verifyPlatformBinaries } from './repo.mjs'; + +const packageDir = process.argv[2] ? path.resolve(root, process.argv[2]) : process.cwd(); + +try { + const { name, count } = verifyPlatformBinaries(packageDir); + console.log(`verify-launcher-binary: ${name} — ${count} binaries present with the right ELF architecture.`); +} catch (error) { + console.error(`verify-launcher-binary: ${error instanceof Error ? error.message : error}`); + process.exit(1); +} diff --git a/native/landlock-run/scripts/verify-packed-install.mjs b/native/landlock-run/scripts/verify-packed-install.mjs new file mode 100644 index 0000000000..60f225a9d2 --- /dev/null +++ b/native/landlock-run/scripts/verify-packed-install.mjs @@ -0,0 +1,223 @@ +#!/usr/bin/env node +/** + * Publish-path rehearsal without publishing: verify the packed tarballs are + * exactly what a consumer install needs. `pnpm pack` already produced the + * bytes `pnpm publish` would upload; this script checks the payload + * (coverage, concrete dependency versions, NO lifecycle install scripts — + * this family has no install fallback on purpose), unpacks the entry plus + * THIS host's platform tarball into a throwaway consumer OUTSIDE the repo, + * byte-pins the installed binary against the workspace build it was packed + * from, and drives the INSTALLED entry under plain `node` — resolution, + * probe, and a real confinement world-proof through the installed launcher. + * + * On non-Linux hosts (no platform package exists) it instead proves the + * documented degradation: resolution falls back to a nonexistent path and + * the probe reports `unusable`. + * + * Usage: `node scripts/verify-packed-install.mjs [tarball-dir] [--current-platform-only]`. + * The flag skips the all-platforms tarball-presence check for + * per-architecture CI legs. `NALR_REQUIRE_LANDLOCK=1` makes an unenforcing + * kernel a failure instead of a skipped world-proof (set on CI, where the + * kernel is known). + */ + +import crypto from 'node:crypto'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { entryDirs, packageDirs, platformDirs, readJson, root } from './repo.mjs'; + +const args = process.argv.slice(2); +const currentPlatformOnly = args.includes('--current-platform-only'); +const tarballDir = path.resolve(args.find((arg) => !arg.startsWith('--')) || path.join(root, 'dist', 'npm')); +const entryPackageName = 'node-addon-landlock-run'; + +function tarballName(manifest) { + if (manifest.name.startsWith('@')) { + return `${manifest.name.slice(1).replace('/', '-')}-${manifest.version}.tgz`; + } + return `${manifest.name}-${manifest.version}.tgz`; +} + +function tarballPath(manifest) { + const tarball = path.join(tarballDir, tarballName(manifest)); + if (!fs.existsSync(tarball)) { + throw new Error(`missing packed tarball: ${tarball}`); + } + return tarball; +} + +function run(command, commandArgs, options = {}) { + const result = spawnSync(command, commandArgs, { + cwd: options.cwd || root, + stdio: 'inherit', + env: { ...process.env, ...options.env }, + }); + if (result.error) throw result.error; + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +function runCapture(command, commandArgs) { + const result = spawnSync(command, commandArgs, { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 }); + if (result.error) throw result.error; + if (result.status !== 0) { + process.stderr.write(result.stderr); + process.exit(result.status ?? 1); + } + return result.stdout; +} + +function readPackedManifest(manifest) { + return JSON.parse(runCapture('tar', ['-xOf', tarballPath(manifest), 'package/package.json'])); +} + +function verifyPackedManifest(packed) { + const lifecycle = ['preinstall', 'install', 'postinstall', 'prepare']; + for (const script of lifecycle) { + if (packed.scripts?.[script]) { + throw new Error(`${packed.name}: packed manifest carries a "${script}" lifecycle script — this family has no install fallback`); + } + } + for (const field of ['dependencies', 'optionalDependencies', 'peerDependencies']) { + for (const [name, version] of Object.entries(packed[field] ?? {})) { + if (version.includes('workspace:')) { + throw new Error(`${packed.name}: packed ${field} still uses the workspace protocol: ${name}@${version}`); + } + } + } +} + +function sha256(file) { + return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'); +} + +function packageInstallDir(packageName) { + return path.join(tempRoot, 'node_modules', ...packageName.split('/')); +} + +function unpackTarball(manifest) { + const extractRoot = fs.mkdtempSync(path.join(tempRoot, 'extract-')); + run('tar', ['-xzf', tarballPath(manifest), '-C', extractRoot]); + + const source = path.join(extractRoot, 'package'); + const destination = packageInstallDir(manifest.name); + fs.rmSync(destination, { recursive: true, force: true }); + fs.mkdirSync(path.dirname(destination), { recursive: true }); + fs.renameSync(source, destination); + fs.rmSync(extractRoot, { recursive: true, force: true }); + console.log(`Unpacked ${manifest.name} -> ${path.relative(tempRoot, destination)}`); +} + +const manifests = packageDirs().map((dir) => ({ dir, manifest: readJson(path.join(root, dir, 'package.json')) })); +const entryManifest = manifests.find(({ manifest }) => manifest.name === entryPackageName)?.manifest; +if (!entryManifest) throw new Error(`missing source manifest for ${entryPackageName}`); + +const hostPlatform = `${process.platform}-${process.arch}`; +const currentPlatformEntry = manifests.find( + ({ dir, manifest }) => platformDirs().includes(dir) && manifest.name === `${entryPackageName}-${hostPlatform}`, +); + +// Payload checks: every expected tarball exists (full mode), the packed +// entry's optional-dependency set names exactly the platform packages, and +// no packed manifest carries workspace versions or install lifecycle. +const expectedTarballs = currentPlatformOnly + ? manifests.filter(({ dir }) => entryDirs().includes(dir) || dir === currentPlatformEntry?.dir) + : manifests; +for (const { manifest } of expectedTarballs) { + tarballPath(manifest); +} + +const packedEntry = readPackedManifest(entryManifest); +const platformPackageNames = manifests + .filter(({ dir }) => platformDirs().includes(dir)) + .map(({ manifest }) => manifest.name) + .sort(); +const optionalNames = Object.keys(packedEntry.optionalDependencies || {}).sort(); +if (optionalNames.join('\n') !== platformPackageNames.join('\n')) { + throw new Error(`packed entry optionalDependencies mismatch\nactual:\n${optionalNames.join('\n')}\nexpected:\n${platformPackageNames.join('\n')}`); +} +for (const { manifest } of expectedTarballs) { + verifyPackedManifest(readPackedManifest(manifest)); +} + +// Throwaway ESM consumer, built from local tarballs only — no registry. +const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'nalr-packed-install-')); +fs.writeFileSync( + path.join(tempRoot, 'package.json'), + `${JSON.stringify({ name: 'nalr-packed-install-check', version: '0.0.0', private: true, type: 'module' }, null, 2)}\n`, +); +console.log(`Verifying packed install in ${tempRoot}`); + +unpackTarball(entryManifest); +if (currentPlatformEntry) { + unpackTarball(currentPlatformEntry.manifest); + + // Byte-pin: the installed binary must be the workspace build it was packed + // from — any divergence means the tarball did not carry the built bytes. + const prebuilds = readJson(path.join(root, currentPlatformEntry.dir, 'prebuilds.json')); + for (const binary of prebuilds.binaries) { + const workspaceFile = path.join(root, currentPlatformEntry.dir, binary.path); + const installedFile = path.join(packageInstallDir(currentPlatformEntry.manifest.name), binary.path); + if (sha256(workspaceFile) !== sha256(installedFile)) { + throw new Error(`installed ${binary.path} differs from the workspace build it was packed from`); + } + console.log(`Byte-pinned ${binary.path} against the workspace build`); + } +} else if (process.platform === 'linux') { + throw new Error(`linux host without a platform package in the matrix: ${hostPlatform}`); +} + +// Drive the INSTALLED entry under plain node: resolution, probe, and (on an +// enforcing kernel) a real confinement world-proof through the installed +// launcher. +const driver = path.join(tempRoot, 'driver.mjs'); +fs.writeFileSync(driver, ` +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { grantArgs, launcherPath, probe } from 'node-addon-landlock-run'; + +const requireLandlock = process.env.NALR_REQUIRE_LANDLOCK === '1'; +const platformPackage = 'node-addon-landlock-run-' + process.platform + '-' + process.arch; +const resolved = launcherPath(); +assert.ok(path.isAbsolute(resolved), 'launcherPath must be absolute'); +assert.ok(resolved.includes(path.join(...platformPackage.split('/'))), 'launcherPath must point into the platform package: ' + resolved); + +if (process.platform === 'linux') { + assert.ok(fs.existsSync(resolved), 'installed launcher missing at ' + resolved); + try { + fs.accessSync(resolved, fs.constants.X_OK); + } catch { + throw new Error('installed launcher is not executable — the pack path stripped the mode bit: ' + resolved); + } + const enforcement = probe(resolved); + console.log('probe through the installed launcher: ' + enforcement); + if (enforcement === 'unusable') { + if (requireLandlock) throw new Error('NALR_REQUIRE_LANDLOCK=1 but the probe reports unusable'); + console.log('kernel does not enforce Landlock — skipping the confinement world-proof'); + } else { + const work = fs.mkdtempSync(path.join(os.tmpdir(), 'nalr-confine-')); + const denied = path.join(work, 'denied.txt'); + const deniedRun = spawnSync(resolved, [...grantArgs({ readOnly: ['/'] }), '--', '/bin/sh', '-c', 'echo x > ' + denied], { encoding: 'utf8' }); + assert.notEqual(deniedRun.status, 0, 'write outside the grants must fail'); + assert.ok(!fs.existsSync(denied), 'denied write must not land on disk'); + const granted = path.join(work, 'granted.txt'); + const grantedRun = spawnSync(resolved, [...grantArgs({ readOnly: ['/'], readWrite: [work] }), '--', '/bin/sh', '-c', 'echo ok > ' + granted], { encoding: 'utf8' }); + assert.equal(grantedRun.status, 0, 'granted write must succeed: ' + grantedRun.stderr); + assert.equal(fs.readFileSync(granted, 'utf8').trim(), 'ok'); + console.log('confinement world-proof passed through the installed launcher'); + } +} else { + assert.ok(!fs.existsSync(resolved), 'no platform package exists for this host — the fallback path must not exist'); + assert.equal(probe(resolved), 'unusable'); + console.log('non-linux host: fallback resolution and unusable probe verified'); +} +`); +run(process.execPath, [driver], { cwd: tempRoot }); + +console.log('Packed install verification passed.'); diff --git a/native/landlock-run/scripts/verify-release.mjs b/native/landlock-run/scripts/verify-release.mjs new file mode 100644 index 0000000000..e812b34a14 --- /dev/null +++ b/native/landlock-run/scripts/verify-release.mjs @@ -0,0 +1,52 @@ +#!/usr/bin/env node +/** + * Release verification. Always: every published package carries one shared + * version, and — when running from a tag or publishing — the `vX.Y.Z` tag + * matches it. With `--prebuilds`: every platform package's declared + * binaries exist with the right ELF architecture (run after + * `assemble-prebuilds.mjs` or a local `build:native`). + */ + +import path from 'node:path'; +import { packageDirs, platformDirs, readJson, root, verifyPlatformBinaries } from './repo.mjs'; + +function verifyVersions() { + const packages = packageDirs().map((dir) => ({ + dir, + manifest: readJson(path.join(root, dir, 'package.json')), + })); + const versions = new Set(packages.map((pkg) => pkg.manifest.version)); + if (versions.size !== 1) { + throw new Error([ + 'published package versions must match:', + ...packages.map((pkg) => `${pkg.dir}: ${pkg.manifest.version}`), + ].join('\n')); + } + + const version = packages[0].manifest.version; + const ref = process.env.GITHUB_REF || ''; + const publish = process.env.RELEASE_PUBLISH === 'true'; + if (publish && !ref.startsWith('refs/tags/v')) { + throw new Error('publishing requires running the workflow from a v* tag'); + } + if (ref.startsWith('refs/tags/v')) { + const tagVersion = ref.slice('refs/tags/v'.length); + if (tagVersion !== version) { + throw new Error(`tag/version mismatch: tag v${tagVersion}, packages ${version}`); + } + } + + console.log(`Verified release version ${version}`); +} + +function verifyPrebuilds() { + for (const dir of platformDirs()) { + const { name, count } = verifyPlatformBinaries(path.join(root, dir)); + console.log(`Verified ${name}: ${count} binaries`); + } +} + +verifyVersions(); +if (process.argv.includes('--prebuilds')) { + verifyPrebuilds(); +} diff --git a/native/landlock-run/test/entry.test.js b/native/landlock-run/test/entry.test.js new file mode 100644 index 0000000000..2e2cfe8f17 --- /dev/null +++ b/native/landlock-run/test/entry.test.js @@ -0,0 +1,76 @@ +/** + * Keyless entry-package tests — run on every host, no kernel or binary + * required. Cover the JS seam's pure surface: grant-argv construction, the + * resolution contract (platform package → fallback), and probe verdicts over + * fake launchers. Requires built `lib/` (`pnpm build:ts`). + */ + +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { + LAUNCHER_BIN, + LAUNCHER_FAILURE_EXIT, + grantArgs, + launcherPath, + probe, +} from 'node-addon-landlock-run'; + +// --- constants are part of the CLI contract --- +assert.equal(LAUNCHER_BIN, 'landlock-run'); +assert.equal(LAUNCHER_FAILURE_EXIT, 125); + +// --- grantArgs: flag spelling, ordering, and empty grants --- +assert.deepEqual(grantArgs({}), []); +assert.deepEqual(grantArgs({ readOnly: ['/'] }), ['--ro', '/']); +assert.deepEqual( + grantArgs({ readOnly: ['/', '/opt'], readWrite: ['/tmp/work'] }), + ['--ro', '/', '--ro', '/opt', '--rw', '/tmp/work'], +); +assert.deepEqual(grantArgs({ readWrite: ['/a'], readOnly: ['/b'] }), ['--ro', '/b', '--rw', '/a']); + +// --- launcherPath: resolves the platform package next to its package.json --- +const platformPackage = `node-addon-landlock-run-${process.platform}-${process.arch}`; +const resolvedViaSeam = launcherPath((specifier) => { + assert.equal(specifier, `${platformPackage}/package.json`); + return path.join('/fake-install', specifier); +}); +assert.equal(resolvedViaSeam, path.join('/fake-install', platformPackage, 'bin', LAUNCHER_BIN)); + +// --- launcherPath: unresolvable package falls back to an absolute, package-boundary path --- +const fallback = launcherPath(() => { + throw new Error('not installed'); +}); +assert.ok(path.isAbsolute(fallback), 'fallback path must be absolute'); +assert.ok( + fallback.includes(path.join('node_modules', ...platformPackage.split('/'), 'bin', LAUNCHER_BIN)), + `fallback must point at the platform package layout: ${fallback}`, +); + +// --- launcherPath: default resolution agrees with this workspace's layout --- +const defaultPath = launcherPath(); +assert.ok(path.isAbsolute(defaultPath)); +assert.ok(defaultPath.endsWith(path.join('bin', LAUNCHER_BIN)), defaultPath); + +// --- probe: a missing launcher is unusable, indistinguishable from an unenforcing kernel --- +assert.equal(probe(path.join(os.tmpdir(), 'nalr-no-such-launcher')), 'unusable'); + +// --- probe: verdict parsing over fake launchers (POSIX shells only) --- +if (process.platform !== 'win32') { + const fakeDir = fs.mkdtempSync(path.join(os.tmpdir(), 'nalr-fake-launcher-')); + const fake = (name, script) => { + const file = path.join(fakeDir, name); + fs.writeFileSync(file, `#!/bin/sh\n${script}\n`, { mode: 0o755 }); + return file; + }; + + assert.equal(probe(fake('full', 'echo "landlock: fully enforced"; exit 0')), 'full'); + assert.equal(probe(fake('partial', 'echo "landlock: partially enforced (older ABI)"; exit 0')), 'partial'); + assert.equal(probe(fake('failing', `exit ${LAUNCHER_FAILURE_EXIT}`)), 'unusable'); + assert.equal(probe(fake('hanging', 'sleep 10'), { timeoutMs: 200 }), 'unusable'); + + fs.rmSync(fakeDir, { recursive: true, force: true }); +} + +console.log('entry.test: ok'); diff --git a/native/landlock-run/test/launcher.test.js b/native/landlock-run/test/launcher.test.js new file mode 100644 index 0000000000..4f456e2145 --- /dev/null +++ b/native/landlock-run/test/launcher.test.js @@ -0,0 +1,121 @@ +/** + * Behavioral tests against the REAL launcher binary on a real kernel: the + * CLI contract (usage errors, exit codes, argv passthrough) and the + * confinement world-proofs (denied writes stay off disk, grants land). + * + * Preconditions and their skip semantics: + * - Non-Linux host: skips entirely (exit 0) — there is nothing to build here. + * - Linux without the built binary: FAILS — run `pnpm build:native` first. + * - Linux whose kernel does not enforce Landlock: skips the enforcement + * half, unless `NALR_REQUIRE_LANDLOCK=1` (set on CI, where a silent skip on + * the very platform that exists to prove enforcement would be a false + * green). + */ + +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { + LAUNCHER_FAILURE_EXIT, + grantArgs, + launcherPath, + probe, +} from 'node-addon-landlock-run'; + +const requireLandlock = process.env.NALR_REQUIRE_LANDLOCK === '1'; + +if (process.platform !== 'linux') { + console.log(`launcher.test: SKIP — the launcher only exists on linux (host: ${process.platform})`); + process.exit(0); +} + +const launcher = launcherPath(); +assert.ok( + fs.existsSync(launcher), + `launcher.test: no built launcher at ${launcher} — run \`pnpm build:native\` (apt-get install musl-tools) first`, +); + +const run = (args, options = {}) => spawnSync(launcher, args, { encoding: 'utf8', ...options }); + +// --- usage errors: parse failures exit LAUNCHER_FAILURE_EXIT before any restriction --- +{ + const noCommand = run([]); + assert.equal(noCommand.status, LAUNCHER_FAILURE_EXIT); + assert.match(noCommand.stderr, /usage error: missing `-- \.\.\.` command/); + + const unknownFlag = run(['--bogus', '--', 'true']); + assert.equal(unknownFlag.status, LAUNCHER_FAILURE_EXIT); + assert.match(unknownFlag.stderr, /usage error: unknown argument: --bogus/); + + const danglingPath = run(['--ro']); + assert.equal(danglingPath.status, LAUNCHER_FAILURE_EXIT); + assert.match(danglingPath.stderr, /--ro requires a path/); + + const probeWithExtras = run(['--probe', '--ro', '/']); + assert.equal(probeWithExtras.status, LAUNCHER_FAILURE_EXIT); + assert.match(probeWithExtras.stderr, /--probe takes no other arguments/); +} + +// --- probe: the functional availability signal --- +const enforcement = probe(launcher); +console.log(`launcher.test: probe → ${enforcement}`); +if (enforcement === 'unusable') { + if (requireLandlock) { + console.error('launcher.test: NALR_REQUIRE_LANDLOCK=1 but the probe reports unusable — this kernel cannot prove enforcement'); + process.exit(1); + } + console.log('launcher.test: SKIP enforcement half — kernel does not enforce Landlock'); + process.exit(0); +} +{ + const probeRun = run(['--probe']); + assert.equal(probeRun.status, 0); + assert.match(probeRun.stdout, /^landlock: (fully enforced|partially enforced \(older ABI\))\n$/); +} + +// --- confined exec: the command runs, its exit code passes through --- +{ + const echo = run([...grantArgs({ readOnly: ['/'] }), '--', '/bin/sh', '-c', 'echo confined-ok']); + assert.equal(echo.status, 0, echo.stderr); + assert.equal(echo.stdout, 'confined-ok\n'); + + const exitCode = run([...grantArgs({ readOnly: ['/'] }), '--', '/bin/sh', '-c', 'exit 7']); + assert.equal(exitCode.status, 7, 'the wrapped command exit code must pass through unchanged'); +} + +// --- world-proofs: denied writes stay off disk, grants land, inheritance crosses exec --- +{ + const work = fs.mkdtempSync(path.join(os.tmpdir(), 'nalr-launcher-test-')); + + const denied = path.join(work, 'denied.txt'); + const deniedRun = run([...grantArgs({ readOnly: ['/'] }), '--', '/bin/sh', '-c', `echo x > ${denied}`]); + assert.notEqual(deniedRun.status, 0, 'a write outside the grants must fail'); + assert.ok(!fs.existsSync(denied), 'the denied write must not land on disk'); + + const granted = path.join(work, 'granted.txt'); + const grantedRun = run([...grantArgs({ readOnly: ['/'], readWrite: [work] }), '--', '/bin/sh', '-c', `echo ok > ${granted}`]); + assert.equal(grantedRun.status, 0, grantedRun.stderr); + assert.equal(fs.readFileSync(granted, 'utf8'), 'ok\n'); + + // The ruleset is inherited across execve: a CHILD of the wrapped command + // is confined too, not just the direct exec target. + const nested = path.join(work, 'nested.txt'); + const nestedRun = run([...grantArgs({ readOnly: ['/'] }), '--', '/bin/sh', '-c', `/bin/sh -c 'echo x > ${nested}'; true`]); + assert.equal(nestedRun.status, 0, nestedRun.stderr); + assert.ok(!fs.existsSync(nested), 'a denied write from a nested child must not land either'); + + fs.rmSync(work, { recursive: true, force: true }); +} + +// --- fail closed: an unopenable grant root refuses to exec at all --- +{ + const marker = path.join(os.tmpdir(), `nalr-should-not-exist-${process.pid}`); + const badGrant = run(['--ro', '/no/such/grant/root', '--', '/bin/sh', '-c', `echo x > ${marker}`]); + assert.equal(badGrant.status, LAUNCHER_FAILURE_EXIT); + assert.match(badGrant.stderr, /cannot open rule path/); + assert.ok(!fs.existsSync(marker), 'the command must never run when the launcher fails'); +} + +console.log('launcher.test: ok'); diff --git a/native/landlock-run/tsconfig.base.json b/native/landlock-run/tsconfig.base.json new file mode 100644 index 0000000000..a95ca64f6d --- /dev/null +++ b/native/landlock-run/tsconfig.base.json @@ -0,0 +1,11 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "types": ["node"] + } +} diff --git a/native/landlock-run/tsconfig.json b/native/landlock-run/tsconfig.json new file mode 100644 index 0000000000..3813d343cd --- /dev/null +++ b/native/landlock-run/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.base.json", + "compilerOptions": { + "noEmit": true + }, + "files": [], + "include": ["scripts/**/*.ts"], + "references": [ + { "path": "./packages/entry" } + ] +} diff --git a/scripts/doc-budgets.manifest.json b/scripts/doc-budgets.manifest.json index a03f7ca25a..1c75d8e90f 100644 --- a/scripts/doc-budgets.manifest.json +++ b/scripts/doc-budgets.manifest.json @@ -1,5 +1,5 @@ { - "AGENTS.md": 1370, + "AGENTS.md": 1375, "docs/AGENTS.md": 1100, "docs/architecture.md": 1790, "docs/cordis-primer.md": 550, From e8d066f7505afe83c08ba3cb0b8693a7126558fc Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Wed, 15 Jul 2026 16:03:52 +0800 Subject: [PATCH 04/73] feat(core): add post-step request recovery (PR3 phase 1) --- docs/agent-lifecycle.md | 10 +- docs/architecture.md | 58 ++- docs/cordis-catalog/events.md | 52 ++- docs/cordis-catalog/services.md | 2 +- docs/core-data-structures/core.md | 14 + docs/core-data-structures/llm-streaming.md | 3 +- docs/event-producer-consumer.md | 30 +- .../cordis/tool-cordis/src/api-catalog.ts | 12 + packages/core/agent-loop/README.md | 2 +- packages/core/agent-loop/src/loop.ts | 179 +++++-- packages/core/agent-loop/tests/cancel.spec.ts | 57 ++- .../tests/contract-regressions.spec.ts | 10 + .../agent-loop/tests/request-recovery.spec.ts | 442 ++++++++++++++++++ packages/core/agent/README.md | 2 +- packages/core/agent/src/types.ts | 31 ++ packages/llm/llm-deepseek/README.md | 2 +- packages/llm/llm-deepseek/src/adapter.ts | 24 +- .../llm/llm-deepseek/tests/adapter.spec.ts | 28 +- packages/llm/llm-pi-ai/README.md | 2 +- packages/llm/llm-pi-ai/src/convert.ts | 3 +- packages/llm/llm-pi-ai/tests/convert.spec.ts | 14 +- packages/llm/llm/README.md | 5 +- packages/llm/llm/src/adapter-failure.ts | 34 ++ packages/llm/llm/src/error.ts | 41 ++ packages/llm/llm/src/index.ts | 58 ++- packages/llm/llm/tests/service.spec.ts | 198 +++++++- .../invariants/src/scoped-events.generated.ts | 2 + scripts/gen-doc-graphs.ts | 10 +- scripts/type-equiv.manifest.json | 2 + 29 files changed, 1207 insertions(+), 120 deletions(-) create mode 100644 packages/core/agent-loop/tests/request-recovery.spec.ts create mode 100644 packages/llm/llm/src/adapter-failure.ts diff --git a/docs/agent-lifecycle.md b/docs/agent-lifecycle.md index b9292aa80e..e4ed315e5b 100644 --- a/docs/agent-lifecycle.md +++ b/docs/agent-lifecycle.md @@ -32,14 +32,22 @@ sequenceDiagram LLM-->>Driver: StreamChunk* Driver->>Session: assistant/chunk* Session-->>SDK: session/event assistant/chunk* + alt final adapter or terminal in-band request failure + Driver->>Session: step/end + Driver->>Hooks: agent/request-error waterfall + Hooks-->>Driver: retry in a new step or preserve the original error + else model request succeeded Driver->>Hooks: agent/step-result waterfall Driver->>Session: assistant/message Driver->>Session: tool/call Driver->>Tools: execute through pre and post waterfalls Tools-->>Session: tool-owned events when applicable - Driver->>Session: tool/result and step/end + Driver->>Session: tool/result, post-tool context, and steering + Driver->>Hooks: agent/post-step serial checkpoint + Driver->>Session: step/end Driver->>Hooks: agent/turn-continuation waterfall Driver->>Hooks: agent/turn-stop serial terminal checkpoint + end Driver->>Session: turn/end Driver->>Persistence: session/flush parallel checkpoint Driver-->>SDK: agent/status idle diff --git a/docs/architecture.md b/docs/architecture.md index 1fa0ee377c..01988a43df 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -4,7 +4,7 @@ The **DeepSeek Harness SDK** builds agent harnesses on Cordis. The principle is ## Overview -A harness is one [Cordis](cordis-primer.md) context. Packages contribute service keys, typed events, and disposable registrations: services expose stable calls (`ctx.llm`, `ctx.tools`, `ctx.sessions`), events provide interception and notifications (`agent/request`, `tools/pre-execute`, `session/event`), and registrations install prompt sections, tools, providers, adapters, or listeners. +A harness is one [Cordis](cordis-primer.md) context. Packages add services (`ctx.llm`, `ctx.tools`, `ctx.sessions`), typed interception and notification events (`agent/request`, `tools/pre-execute`, `session/event`), and disposable registrations for prompts, tools, providers, adapters, and listeners. `packages/core/` groups the default agent flow; surrounding capabilities are equally first-class Cordis plugins. @@ -43,9 +43,9 @@ Events form the service extension API; see the exhaustive [events catalog](cordi ### Event Domains -- **Session events** are durable, replayable facts. Turn and step boundaries, user input, assistant output, tool calls, tool results, steering, compaction records, and tool-owned durable facts append to the session log and flow through `session/event`. -- **Agent events** carry the live `Agent` handle for status, diagnostics, prompt admission, call-config shaping, result validation, and continuation policy. -- **Capability events** belong to the seam that owns the action. `tools/*`, `llm/*`, `system-prompt/*`, `fs/*`, and `subagent/*` let policy and adapters attach without importing the loop. +- **Session events** are durable replay facts: boundaries, messages, tools, steering, compaction, and tool-owned state flow through `session/event`. +- **Agent events** carry the live `Agent` for status, diagnostics, prompt admission, request shaping, result validation, and continuation. +- **Capability events** belong to their action owner. `tools/*`, `llm/*`, `system-prompt/*`, `fs/*`, and `subagent/*` attach policy and adapters without importing the loop. ### Interception Semantics @@ -53,9 +53,9 @@ Waterfall events behave like around-middleware: a listener delegates by calling ## Default Loop Lifecycle -The shipped loop drains work, assembles requests, streams model answers, executes tools, applies continuation policy, and checkpoints state. Every pause is a service call or event available to plugins. +The shipped loop drains work, assembles requests, streams answers, executes tools, applies continuation policy, and checkpoints state through plugin-visible calls and events. -A **session** is one agent's append-only event log. A **turn** drains one queued batch and runs until the model stops asking for tools and no plugin requests continuation. A **step** is one model request plus the tool executions caused by that response. In the flow below ([sequence companion](agent-lifecycle.md)), quoted names are durable session events and event names are extension points. +A **session** is an agent's append-only log; a **turn** drains one queued batch; a **step** is one model request and its tool executions. Quoted names below are durable events, and unquoted event names are extension points ([sequence companion](agent-lifecycle.md)). ### Turn Flow @@ -76,42 +76,50 @@ forever: assemble system prompt and tool schemas agent/session-prefix (first step) agent/pre-step - 'step/start' snapshot the derived messages (the reconstruction boundary) + 'step/start' agent/request (config only) -> log request/header -> llm/stream (frozen) + on final adapter failure or terminal in-band error/aborted finish: + 'step/end' + agent/request-error(original error, consecutive retry attempt, signal) + retry in the next numbered step or preserve the original error + otherwise: 'assistant/chunk' - agent/step-result - 'assistant/message' (transformed content, or an empty successful-call anchor if step-result rejects) - each tool call: - 'tool/call' - tools/pre-execute -> monotonic guards -> tools/execute -> tools/post-execute -> tools/result - 'tool/result' - append post-tool context and steering - 'step/end' - agent/turn-continuation - agent/turn-stop (terminal policy) - stop unless tools or continuation policy ask for another step + agent/step-result + 'assistant/message' (transformed content, or an empty successful-call anchor if step-result rejects) + each tool call: + 'tool/call' + tools/pre-execute -> monotonic guards -> tools/execute -> tools/post-execute -> tools/result + 'tool/result' + append post-tool context and steering + agent/post-step + 'step/end' + agent/turn-continuation + agent/turn-stop (terminal policy) + stop unless tools or continuation policy ask for another step 'turn/end' checkpoint persistence and notify idle/running status ``` -The loop renders one prompt assembly per step. Plugins contribute ordered sections, tool schemas, and `{{name}}` variables; unknown or valueless references fail the turn instead of shipping a hole. `dsh-system-prompt` owns the harness identity and default deployment persona; an agent-scoped persona may shadow the default. The loop supplies `model` and `cwd`. See the [prompt-ownership RFC](rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md). +Each step renders one prompt assembly. Plugins contribute ordered sections, tool schemas, and `{{name}}` variables; missing values fail the turn. `dsh-system-prompt` owns harness identity and the default persona, which an agent-scoped persona may shadow. The loop supplies `model` and `cwd` ([prompt ownership](rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md)). -Post-tool context lands after all tool results so tool-call/result adjacency stays stable. Steering drains between steps; ordinary leftover steering after a turn is re-queued as input. A terminal `agent/turn-stop` is the explicit exception: it runs after ordinary continuation and steering folding, then remains authoritative through turn close and flush so steering from those later listeners is discarded rather than becoming another step or turn; ordinary queued prompts are preserved. +Post-tool context follows all results, preserving call/result adjacency. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering while the step signal remains open. Leftover steering becomes next-turn input. `agent/turn-stop` is terminal through close and flush: later steering is discarded, while ordinary queued prompts survive. ### Failure Boundaries -The turn is the containment boundary. A throwing listener, adapter error finish, or failed step ends the current turn with an error reason and reports live diagnostics through `agent/error`; it does not kill the driver loop. `cancel()` clears queued and steering work, aborts the active model/tool boundary when possible, and records the appropriate turn end. Disposal stops the loop, awaits quiescence, unregisters the agent, and lets service disposers drain. +The turn is the containment boundary. `LlmService` preserves and privately tags errors from final adapter selection, dispatch, and iteration. Those errors and terminal in-band error/aborted finishes close the failed step before `agent/request-error`; retry reconstructs the next numbered step from the log, while decline or failed recovery preserves the provider error. Attempts count consecutive failures and reset after success. -Every session event is turn-enclosed. Reloading a crashed session preserves the interrupted tail and closes it with a synthetic `interrupted` turn end. A failure after the durable turn has closed reports through `agent/error` only because no safe in-turn position remains. A turn ends with one `TurnEndReason` (`completed`, `aborted`, `error`, `disposed`, `max-tokens`, `rejected`, or `interrupted`); per-variant semantics are in [session.md § TurnEndReasonMap](core-data-structures/session.md#why-a-turn-ended-turnendreasonmap). +Prompt, middleware, result, tool, post-step, and continuation failures remain ordinary `agent/error` failures. Cancellation and disposal beat recovery. Durable undispatched tool calls receive synthetic `ABORTED` results, preventing dangling replay. `cancel()` clears queues and aborts active work; disposal awaits quiescence before unregistering. + +Every session event is turn-enclosed. Reload preserves a crashed tail and closes it with synthetic `interrupted`; post-close failures report only through `agent/error`. A turn has one `TurnEndReason` (`completed`, `aborted`, `error`, `disposed`, `max-tokens`, `rejected`, or `interrupted`), detailed in [session.md](core-data-structures/session.md#why-a-turn-ended-turnendreasonmap). ### Agent Handles -`ctx.agents` owns live agents and returns an `AgentHandle { agent, dispose() }`. `Agent` is the API other plugins drive: `send()` queues work, `steer()` injects mid-turn content, `inject()` appends context and opens a one-shot injection turn when idle, `cancel()` is the public stop primitive, and `whenIdle()` observes quiescence. The caller fiber and concrete factory provider structurally co-own programmatic lifecycles; a consumer handle is the only non-structural teardown capability, and every owner reaches the same awaited disposer. +`ctx.agents` owns live agents and returns `AgentHandle { agent, dispose() }`. Plugins use `send()`, `steer()`, `inject()`, `cancel()`, and `whenIdle()`. The caller fiber and factory provider structurally co-own programmatic lifecycles; the consumer handle is the sole non-structural teardown capability, and all owners share one awaited disposer. ### Agent Scope -Every live agent owns a scoped `agent.ctx`. Its registrations shadow same-named globals, receive only that agent's dispatches, and unwind with the agent. `CreateAgentOptions.setup(agentCtx)` composes the scope before publication. The [semantic-gates RFC](rfc/implemented/process/2026-07-14-typescript-program-backed-semantic-gates.md) defines typed resolvers that derive carrier checks from merged `Events` signatures and `scopeTarget`, eliminating the handwritten event table. See the [agent-scope RFC](rfc/implemented/architecture/2026-07-08-agent-scope-contexts.md); subagent composition controls are documented [separately](rfc/implemented/feature/2026-07-12-subagent-persona-tool-filter-and-depth.md). +Each agent owns `agent.ctx`; its registrations shadow globals, receive only that agent's dispatches, and unwind on disposal. `CreateAgentOptions.setup(agentCtx)` composes it before publication. Typed resolvers derive carrier checks from merged events and `scopeTarget` ([semantic gates](rfc/implemented/process/2026-07-14-typescript-program-backed-semantic-gates.md), [agent scope](rfc/implemented/architecture/2026-07-08-agent-scope-contexts.md), [subagent controls](rfc/implemented/feature/2026-07-12-subagent-persona-tool-filter-and-depth.md)). ## State @@ -152,7 +160,7 @@ New behavior should attach to a documented extension point; changing the shipped | Add command execution | implement and register a `ctx.bash` backend | | Add filesystem access or policy | implement a `ctx.fs` provider or listen on `fs/*` policy events | | Confine spawned processes | a `ctx.sandbox` backend; consumers wrap their argv before spawning | -| Intercept prompts, requests, tool use, or continuation | listen on the relevant `agent/*` or `tools/*` waterfall; use serial `agent/turn-stop` for a monotonic terminal stop | +| Intercept prompts, requests, model completion/failure, tool use, or continuation | listen on the relevant `agent/*` or `tools/*` event; use serial `agent/turn-stop` for a monotonic terminal stop | | Add a session-stable request prefix outside history | compose it on `agent/session-prefix`, once per loop instance; logged on the request header | | Add UI or editor integration | drive `ctx.agents` and render from `session/event` | | Add durable session state | add a `SessionEventMap` member and render/replay from the log | diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 5408a63720..7ce58b6140 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -23,7 +23,7 @@ A fully configured agent and live session were published. Setup is composition-o Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:139`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:145`](../../packages/core/agent/src/types.ts) ### `agent/disposed` — emit @@ -35,7 +35,7 @@ An agent left the registry; AgentLoop emits this after driver quiescence but bef Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:148`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:154`](../../packages/core/agent/src/types.ts) ### `agent/error` — emit @@ -47,7 +47,19 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:283`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:314`](../../packages/core/agent/src/types.ts) + +### `agent/post-step` — serial + +Awaited serial checkpoint after the response, tool results, injected context, and steering are durable but before `step/end`. + +```ts cordis-catalog +'agent/post-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:267`](../../packages/core/agent/src/types.ts) ### `agent/pre-step` — serial @@ -59,7 +71,7 @@ Awaited serial checkpoint for session-surface mutation after prompt assembly and Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:202`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:208`](../../packages/core/agent/src/types.ts) ### `agent/prompt-submit` — waterfall @@ -71,7 +83,7 @@ Allow, rewrite, or block one drained prompt before it becomes a user message. Ca Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:212`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:218`](../../packages/core/agent/src/types.ts) ### `agent/queued` — emit @@ -83,7 +95,7 @@ Detached, frozen content entered the agent's inbox. Source defaults have already Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:167`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:173`](../../packages/core/agent/src/types.ts) ### `agent/request` — waterfall @@ -95,7 +107,19 @@ Replace the frozen call configuration. Model-visible content must use logged cha Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:224`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:230`](../../packages/core/agent/src/types.ts) + +### `agent/request-error` — waterfall + +Recover a model-request failure after its failed step has closed. `retry` opens a new numbered step; `fail` preserves the original request error. Call `next()` to delegate to the next recovery listener or the default. + +```ts cordis-catalog +'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:281`](../../packages/core/agent/src/types.ts) ### `agent/session-prefix` — waterfall @@ -107,7 +131,7 @@ Compose request-only messages placed before derived history. The frozen result i Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:239`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:245`](../../packages/core/agent/src/types.ts) ### `agent/session-start` — emit @@ -119,7 +143,7 @@ The session lifecycle began, once before the first turn. Use `agent.inject()` to Types: [Agent](../core-data-structures/core.md) · [SessionStartSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:180`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:186`](../../packages/core/agent/src/types.ts) ### `agent/status` — emit @@ -131,7 +155,7 @@ Agent status changed (`idle` ⇄ `running`, or → `disposed`). `send()` does no Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:157`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:163`](../../packages/core/agent/src/types.ts) ### `agent/step-result` — waterfall @@ -143,7 +167,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:250`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:256`](../../packages/core/agent/src/types.ts) ### `agent/turn-continuation` — waterfall @@ -155,7 +179,7 @@ Override whether the turn continues. The default continues after tool calls or s Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:260`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:291`](../../packages/core/agent/src/types.ts) ### `agent/turn-stop` — serial @@ -167,7 +191,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:270`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:301`](../../packages/core/agent/src/types.ts) ## `approval/*` @@ -233,7 +257,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:39`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:41`](../../packages/llm/llm/src/index.ts) ## `session/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 150cd35f7f..990a2bc1bc 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -127,7 +127,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:75`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:77`](../../packages/llm/llm/src/index.ts) ## `ctx.permission` — `PermissionService` diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 36d007eb38..1f8dca8abc 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -381,6 +381,20 @@ type ContinuationDecision = | { action: 'continue'; reason?: HookContext } ``` +`agent/request-error` receives the original `RequestError`, whose optional provider-neutral `code` supports stable routing without message parsing: + +```ts type-equiv +type RequestError = Error & { code?: string } +``` + +It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` preserves that error: + +```ts type-equiv +type RequestErrorDecision = { action: 'fail' } | { action: 'retry' } +``` + +`agent/post-step` is the awaited successful-step checkpoint after assistant output, tool results, buffered context, and steering are durable. Its signature is `(agent, turn, step, signal)`; replayable facts remain in the session log rather than a transient payload. + `agent/turn-stop` returns the stop-only `ContinuationStop` subset or `undefined`. The loop calls this serial checkpoint after folding the ordinary decision, its reason, and pending steering; a stop is terminal and discards pending steering. ```ts type-equiv diff --git a/docs/core-data-structures/llm-streaming.md b/docs/core-data-structures/llm-streaming.md index fff329310a..32174f0d03 100644 --- a/docs/core-data-structures/llm-streaming.md +++ b/docs/core-data-structures/llm-streaming.md @@ -26,6 +26,7 @@ Every adapter MUST obey these, and every consumer may rely on them: - **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering. - **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`. - **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop translates a finish-error/aborted into a turn error — it never logs a normal completed assistant message for a failed step. +- **Context overflow has one canonical code.** Both DeepSeek adapters classify explicit provider detail through `isContextWindowExceededError()` and surface `CONTEXT_WINDOW_EXCEEDED`, whether the failure arrives as a thrown HTTP `LlmError` or an in-band finish error. Consumers route on the code, never provider text. - **Every provider HTTP request carries the app-attribution header.** Adapters send `attributionHeaders()` (below) - the `User-Agent` baseline - and prove it with a wire-level test (mock server asserting the received header, or the library's header hook for a library-backed adapter). This contract is why two adapters exist as a deliberate pair: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (the same endpoint through `@earendil-works/pi-ai`). Two independent internals over one contract is what pinned the protocol down — the library-backed adapter can't throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not. @@ -44,7 +45,7 @@ interface AppIdentity { ## `TokenUsage` -Per-call token accounting. Counts are **disjoint**: `inputTokens` is uncached input only; cached input is reported separately, and billed input is the sum of the three. Adapters whose providers fold cache hits into a single prompt total (DeepSeek's `prompt_tokens`) subtract them back out. +Per-call token accounting. Counts are **disjoint**: `inputTokens` is uncached input only; cached input is reported separately, and billed input is the sum of the three. Adapters whose providers fold cache hits into a single prompt total (DeepSeek's `prompt_tokens`) subtract them back out. `reasoningTokens`, when present, is informational detail already included in `outputTokens`; totals must not add it again. ```ts type-equiv interface TokenUsage { diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index e095aa4b7e..e6872669de 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -7,24 +7,26 @@ This matrix shows which packages dispatch each harness-owned event and which pac | Event | Mode | Declared in | Dispatchers | Listeners | | --- | --- | --- | --- | --- | -| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:139`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`jsonrpc`](../packages/ui/jsonrpc), [`stdio`](../packages/ui/stdio) | -| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:148`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:283`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:202`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic), [`user-approval`](../packages/ui/user-approval) | -| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:212`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | -| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:167`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:224`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:239`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill) | -| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:180`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:157`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`stdio`](../packages/ui/stdio) | -| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:250`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:260`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:270`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | +| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:145`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`jsonrpc`](../packages/ui/jsonrpc), [`stdio`](../packages/ui/stdio) | +| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:154`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio) | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:314`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:267`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | - | +| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:208`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic), [`user-approval`](../packages/ui/user-approval) | +| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:218`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | +| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:173`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:230`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:281`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:245`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill) | +| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:186`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:163`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`stdio`](../packages/ui/stdio) | +| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:256`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:291`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:301`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:31`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/ui/acp) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:59`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:68`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:51`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:39`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:41`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | | `session/created` | `emit` | [`packages/core/session/src/index.ts:46`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:56`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | - | | `session/event` | `emit` | [`packages/core/session/src/index.ts:68`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio`](../packages/ui/stdio), [`token-meter`](../packages/llm/token-meter) | diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 5e4f5f0b69..7aafb3ce8a 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -280,6 +280,12 @@ export const EVENT_API: readonly EventApiEntry[] = [ signature: '\'agent/error\'(this: Scoped, agent: Agent, turn: number, step: number, error: Error): void', summary: 'A step or turn errored.', }, + { + name: 'agent/post-step', + mode: 'serial', + signature: '\'agent/post-step\'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void', + summary: 'Awaited serial checkpoint after the response, tool results, injected context, and steering are durable but before `step/end`.', + }, { name: 'agent/pre-step', mode: 'serial', @@ -304,6 +310,12 @@ export const EVENT_API: readonly EventApiEntry[] = [ signature: '\'agent/request\'(this: Scoped, agent: Agent, turn: number, step: number, config: LlmCallConfig, next: () => Promise): Promise', summary: 'Replace the frozen call configuration.', }, + { + name: 'agent/request-error', + mode: 'waterfall', + signature: '\'agent/request-error\'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise', + summary: 'Recover a model-request failure after its failed step has closed.', + }, { name: 'agent/session-prefix', mode: 'waterfall', diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index 803a35fdc5..864ff95e8c 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -52,7 +52,7 @@ The driver owns one agent for its lifetime. It records turn, step, request, stre Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history. -Plugin failure ends the current turn, not the loop. Cancellation clears pending work and aborts the current step without leaking to the next prompt. Terminal continuation stops remain authoritative through turn close and durability flush. +Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and `agent/post-step` remain ordinary turn failures. Recovery observes a closed failed step, and a retry rebuilds the request from the durable log in a new numbered step. Cancellation clears pending work and aborts the current step without leaking to the next prompt; model-requested calls that were already durable receive synthetic aborted results when cancellation prevents dispatch. Terminal continuation stops remain authoritative through turn close and durability flush. ### What belongs to plugins diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 41c5d60646..e749018f55 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -7,37 +7,42 @@ import type { Context } from 'cordis' import type { FinishReason, GenerateOptions, LlmCallConfig, Message, TokenUsage } from '@deepseek-ai/dsh-llm' -import { BlockAssembler, HarnessError, deepFreeze } from '@deepseek-ai/dsh-llm' +import { BlockAssembler, HarnessError, deepFreeze, isLlmAdapterFailure } from '@deepseek-ai/dsh-llm' import { agentEvents, assembleContextFor } from '@deepseek-ai/dsh-agent' -import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision } from '@deepseek-ai/dsh-agent' +import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent' import { canonicalHeader } from '@deepseek-ai/dsh-session' import type { Session, TurnEndReason, TurnTrigger } from '@deepseek-ai/dsh-session' import { createTransmissionLog, recordRequestHeader } from './request-log.ts' import type { TransmissionLog } from './request-log.ts' import { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import type { PromptAssembly } from '@deepseek-ai/dsh-system-prompt' -import type {} from '@deepseek-ai/dsh-tools' +import type { ToolExecutionResult } from '@deepseek-ai/dsh-tools' import type { ReactLoopAgent } from './agent.ts' import type { Inbox } from './inbox.ts' -/** An Error with an optional machine-readable code (e.g., from LlmError or a throwing plugin). */ -type CodedError = Error & { code?: string } - /** Normalize thrown values while preserving an existing error code. */ -function toError(error: unknown): CodedError { +function toError(error: unknown): RequestError { return error instanceof Error ? error : new HarnessError(String(error), 'UNKNOWN', { cause: error }) } +/** Distinguishes a terminal failure finish from failures in later step processing. */ +class TerminalModelRequestFailure extends Error { + constructor(readonly requestError: RequestError) { + super(requestError.message, { cause: requestError }) + this.name = 'TerminalModelRequestFailure' + } +} + /** Convert terminal failure finishes into step errors; unknown extensible finishes remain successful. */ -function finishError(finish: FinishReason): CodedError | undefined { +function finishError(finish: FinishReason): RequestError | undefined { switch (finish.kind) { case 'error': { - const error: CodedError = new Error(finish.message) + const error: RequestError = new Error(finish.message) if (finish.code !== undefined) error.code = finish.code return error } case 'aborted': { - const error: CodedError = new Error('model stream aborted') + const error: RequestError = new Error('model stream aborted') error.code = 'ABORTED' return error } @@ -51,10 +56,19 @@ function finishError(finish: FinishReason): CodedError | undefined { * Build the `{ message, code? }` part of an error payload, omitting the * `code` key entirely when absent (exactOptionalPropertyTypes-correct). */ -function errorData(err: CodedError): { message: string; code?: string } { +function errorData(err: RequestError): { message: string; code?: string } { return { message: err.message, ...typeof err.code === 'string' ? { code: err.code } : {} } } +/** Build the durable result for a model-requested call skipped after cancellation. */ +function skippedToolResult(): ToolExecutionResult { + return { + content: [{ type: 'text', text: 'Error: tool call skipped because the step was aborted before execution' }], + isError: true, + error: { name: 'AbortError', code: 'ABORTED' }, + } +} + /** Map a successful max-token finish onto the turn reason; other successful finishes add nothing. */ function stepFinishReason(finish: FinishReason): TurnEndReason | undefined { switch (finish.kind) { @@ -168,6 +182,7 @@ async function runTurn( let reason: TurnEndReason = { kind: 'completed' } let step = 0 + let requestRetryAttempt = 0 let stepOpen = false let errorReported = false let terminalStopped = false @@ -180,7 +195,7 @@ async function runTurn( } // Record the durable turn failure once and contain the live error notification. - const failTurn = (err: CodedError): void => { + const failTurn = (err: RequestError): void => { if (errorReported) return errorReported = true reason = { kind: 'error', step, ...errorData(err) } @@ -322,14 +337,65 @@ async function runTurn( break } - let stepOutcome: { hadToolCalls: boolean; finish: FinishReason } | { error: Error } + let stepOutcome: + | { hadToolCalls: boolean; finish: FinishReason } + | { requestError: RequestError } + | { error: RequestError } try { stepOutcome = await runStep( ctx, events, agent, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal) } catch (error: unknown) { - stepOutcome = { error: toError(error) } - } finally { + if (isLlmAdapterFailure(error)) { + stepOutcome = { requestError: error } + } else if (error instanceof TerminalModelRequestFailure) { + stepOutcome = { requestError: error.requestError } + } else { + stepOutcome = { error: toError(error) } + } + } + + if ('requestError' in stepOutcome) { + // Recovery observes a balanced failed step and the original provider + // error while the failed step's signal remains the active owner. + closeStep() + if (handle.isDisposed() || abort.signal.aborted) { + handle.setAbort(undefined) + reason = handle.isDisposed() + ? { kind: 'disposed' } + : { kind: 'aborted', reason: String(abort.signal.reason) } + break + } + + const defaultDecision: RequestErrorDecision = { action: 'fail' } + let recoveryDecision: RequestErrorDecision = defaultDecision + try { + recoveryDecision = await events.waterfall( + 'agent/request-error', turn, step, stepOutcome.requestError, + requestRetryAttempt, abort.signal, + () => Promise.resolve(defaultDecision), + ) + } catch (recoveryError: unknown) { + ctx.logger.warn( + `agent "${agent.id}": request recovery failed at turn ${turn}, step ${step}: ${toError(recoveryError).message}`, + ) + } handle.setAbort(undefined) + + // Cancellation and disposal always win over either a recovery decision + // or a recovery-listener failure. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (handle.isDisposed() || abort.signal.aborted) { + reason = handle.isDisposed() + ? { kind: 'disposed' } + : { kind: 'aborted', reason: String(abort.signal.reason) } + break + } + if (recoveryDecision.action === 'retry') { + requestRetryAttempt += 1 + continue + } + failTurn(stepOutcome.requestError) + break } if ('error' in stepOutcome) { @@ -337,7 +403,9 @@ async function runTurn( // runLoop re-enqueues it as a queued message, so an abort-then-steer // starts a fresh turn instead of being silently consumed. closeStep() + handle.setAbort(undefined) const { error } = stepOutcome + /* v8 ignore next -- narrow race: disposal while non-request step work throws. */ if (handle.isDisposed()) { reason = { kind: 'disposed' } } else if (abort.signal.aborted) { @@ -349,6 +417,8 @@ async function runTurn( break } + requestRetryAttempt = 0 + // Preserve max-token completion unless a later disposal, abort, or error wins. const stepReason = stepFinishReason(stepOutcome.finish) if (stepReason) reason = stepReason @@ -356,7 +426,38 @@ async function runTurn( // Steering that arrived during streaming/tool execution. const steered = drainSteering(agent, handle.inbox, turn) + try { + await events.serial('agent/post-step', turn, step, abort.signal) + } catch (error: unknown) { + stepOutcome = { error: toError(error) } + } + + if ('error' in stepOutcome) { + closeStep() + handle.setAbort(undefined) + /* v8 ignore next -- narrow race: disposal while a post-step listener throws. */ + if (handle.isDisposed()) { + reason = { kind: 'disposed' } + } else if (abort.signal.aborted) { + /* v8 ignore next -- signal.reason always set by cancellation or disposal. */ + reason = { kind: 'aborted', reason: String(abort.signal.reason ?? 'aborted') } + } else { + failTurn(stepOutcome.error) + } + break + } + + if (handle.isDisposed() || abort.signal.aborted) { + reason = handle.isDisposed() + ? { kind: 'disposed' } + : { kind: 'aborted', reason: String(abort.signal.reason) } + closeStep() + handle.setAbort(undefined) + break + } + closeStep() + handle.setAbort(undefined) const defaultDecision: ContinuationDecision = { action: stepOutcome.hadToolCalls || steered ? 'continue' : 'stop' } let decision: ContinuationDecision @@ -523,7 +624,7 @@ async function runStep( // Normalize failure finish chunks into the same path as thrown stream errors. const stepError = finishError(assembler.finish) - if (stepError) throw stepError + if (stepError) throw new TerminalModelRequestFailure(stepError) if (assembler.finish.kind === 'max-tokens') { let message: Message = withoutToolCalls(assembler.message()) @@ -553,25 +654,30 @@ async function runStep( const toolCalls = message.content.filter(block => block.type === 'tool-call') // Buffer context until all results are appended to preserve call/result adjacency. const pendingContext: HookContext[] = [] + let aborted = signal.aborted for (const call of toolCalls) { - /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ - if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) const callEvent = session.append('tool/call', { turn, step, callId: call.id, name: call.name, arguments: call.arguments }) - let parsedArguments: unknown - try { - parsedArguments = call.arguments ? JSON.parse(call.arguments) : {} - } catch { - parsedArguments = call.arguments + let result: ToolExecutionResult + if (aborted || signal.aborted) { + aborted = true + result = skippedToolResult() + } else { + let parsedArguments: unknown + try { + parsedArguments = call.arguments ? JSON.parse(call.arguments) : {} + } catch { + parsedArguments = call.arguments + } + // TODO(pre-tool-input-rewrite): Keep logged history and live presentation aligned; + // see docs/rfc/proposed/feature/2026-06-30-pre-tool-input-rewrite.md. + result = await ctx.tools.execute({ + callId: call.id, + name: call.name, + arguments: parsedArguments, + agent, + signal, + }) } - // TODO(pre-tool-input-rewrite): Keep logged history and live presentation aligned; - // see docs/rfc/proposed/feature/2026-06-30-pre-tool-input-rewrite.md. - const result = await ctx.tools.execute({ - callId: call.id, - name: call.name, - arguments: parsedArguments, - agent, - signal, - }) session.append('tool/result', { turn, step, // Correlation comes from the immutable execution input; the result does @@ -584,13 +690,12 @@ async function runStep( ...result.meta !== undefined ? { meta: result.meta } : {}, }, { surfaceOp: 'append', sourceEventSeqs: [callEvent.seq] }) if (result.additionalContext) pendingContext.push(result.additionalContext) - // The signal may flip while the tool is awaited. - /* v8 ignore start -- signal.reason default unreachable: cancel()/disposal always set it */ - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) - /* v8 ignore stop */ + if (signal.aborted) aborted = true } + /* v8 ignore next -- signal.reason always set by cancellation or disposal. */ + if (aborted) throw new Error(String(signal.reason ?? 'aborted')) + // Append buffered context after the complete result batch. for (const context of pendingContext) { agent.inject(context.content, { source: context.source }) diff --git a/packages/core/agent-loop/tests/cancel.spec.ts b/packages/core/agent-loop/tests/cancel.spec.ts index cad830e827..80284b64b5 100644 --- a/packages/core/agent-loop/tests/cancel.spec.ts +++ b/packages/core/agent-loop/tests/cancel.spec.ts @@ -12,10 +12,10 @@ import { Context } from 'cordis' import LlmService, { type Message } from '@deepseek-ai/dsh-llm' import SessionStore, { SessionId, TurnEndReason } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' -import ToolRegistry from '@deepseek-ai/dsh-tools' +import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' -import { MockAdapter, textResponse } from './mock-adapter.ts' +import { MockAdapter, textResponse, toolCallResponse } from './mock-adapter.ts' async function harness(adapter: MockAdapter) { const ctx = new Context() @@ -139,6 +139,59 @@ describe('Agent.cancel()', () => { expect(reasons).toEqual([{ kind: 'aborted', reason: 'cancelled' }]) }) + it('cancel from an assistant/message observer skips execution but balances replay', async () => { + const adapter = new MockAdapter([ + toolCallResponse('c1', 'danger', {}), + textResponse('recovered after cancellation'), + ]) + const ctx = await harness(adapter) + let executions = 0 + ctx.tools.register(defineTool({ + name: 'danger', + description: 'must not run after cancellation', + parameters: {}, + async execute() { + executions += 1 + return [{ type: 'text', text: 'ran' }] + }, + })) + const agent = ctx.agentLoop.create(AgentId('cancel-after-assistant-message'), { model: 'mock' }) + const dispose = ctx.on('session/event', (session, event) => { + if (session === agent.session && event.type === 'assistant/message') { + agent.cancel('cancelled after assistant message') + } + }) + + const reasons: TurnEndReason[] = [] + ctx.on('session/event', (_session, event) => { if (event.type === 'turn/end') reasons.push(event.data.reason) }) + + send(agent, 'go') + await waitForIdle(ctx, agent) + dispose() + + expect(executions).toBe(0) + expect(reasons).toEqual([{ kind: 'aborted', reason: 'cancelled after assistant message' }]) + const call = agent.session.events.find(event => event.type === 'tool/call') + const result = agent.session.events.find(event => event.type === 'tool/result') + expect(call?.type === 'tool/call' ? call.data.callId : undefined).toBe('c1') + expect(result?.type === 'tool/result' ? result.data : undefined).toMatchObject({ + callId: 'c1', + isError: true, + error: { name: 'AbortError', code: 'ABORTED' }, + }) + + send(agent, 'continue safely') + await waitForIdle(ctx, agent) + const replayedResult = adapter.requests[1]!.messages + .flatMap(message => message.content) + .find(block => block.type === 'tool-result') + expect(replayedResult).toMatchObject({ toolCallId: 'c1', isError: true }) + expect(reasons).toEqual([ + { kind: 'aborted', reason: 'cancelled after assistant message' }, + { kind: 'completed' }, + ]) + }) + it('a prompt sent AFTER a cancelled turn settles runs normally (marker reset)', async () => { const adapter = new MockAdapter(['hang', textResponse('second reply')]) const ctx = await harness(adapter) diff --git a/packages/core/agent-loop/tests/contract-regressions.spec.ts b/packages/core/agent-loop/tests/contract-regressions.spec.ts index 3907434395..6ca06fc096 100644 --- a/packages/core/agent-loop/tests/contract-regressions.spec.ts +++ b/packages/core/agent-loop/tests/contract-regressions.spec.ts @@ -204,6 +204,16 @@ describe('abort during tool execution ends the turn', () => { expect(executed).toEqual(['aborter']) // second tool never ran expect(adapter.requests).toHaveLength(1) // no follow-up model call expect(reasons).toEqual([{ kind: 'aborted', reason: 'user interrupt' }]) + const calls = agent.session.events.filter(event => event.type === 'tool/call') + const results = agent.session.events.filter(event => event.type === 'tool/result') + expect(calls.map(event => event.data.callId)).toEqual([CallId('c1'), CallId('c2')]) + expect(results).toHaveLength(2) + expect(results[0]!.data).toMatchObject({ callId: CallId('c1'), isError: false }) + expect(results[1]!.data).toMatchObject({ + callId: CallId('c2'), + isError: true, + error: { name: 'AbortError', code: 'ABORTED' }, + }) }) }) diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts new file mode 100644 index 0000000000..003c678be0 --- /dev/null +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -0,0 +1,442 @@ +/** + * Agent-loop coverage for the successful post-step checkpoint and model-request + * recovery. These tests keep the recovery boundary narrower than the whole + * step and pin retry reconstruction, numbering, cancellation, and identity. + */ + +import { describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import LlmService, { + CallId, + CONTEXT_WINDOW_EXCEEDED_CODE, + LlmAdapter, + LlmError, +} from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import SessionStore from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' +import type { PostToolDecision } from '@deepseek-ai/dsh-tools' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' +import { maxTokensResponse, textResponse, toolCallResponse } from './mock-adapter.ts' + +class FailureScriptAdapter extends LlmAdapter { + requests: GenerateOptions[] = [] + + constructor(private readonly entries: (Error | StreamChunk[])[]) { + super() + } + + async * stream(options: GenerateOptions): AsyncIterable { + this.requests.push(options) + const entry = this.entries.shift() + if (entry === undefined) throw new Error('failure script exhausted') + if (entry instanceof Error) throw entry + yield* entry + } +} + +class IteratorConstructionFailureAdapter extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + throw new LlmError('iterator construction failed', 'ITERATOR_CONSTRUCTION') + }, + } + } +} + +class SynchronousDispatchFailureAdapter extends LlmAdapter { + constructor(private readonly error: Error) { + super() + } + + stream(_options: GenerateOptions): AsyncIterable { + throw this.error + } +} + +class IteratorResultGetterFailureAdapter extends LlmAdapter { + constructor( + private readonly field: 'done' | 'value', + private readonly error: Error, + ) { + super() + } + + stream(_options: GenerateOptions): AsyncIterable { + const result = this.field === 'done' ? {} : { done: false } + Object.defineProperty(result, this.field, { get: () => { throw this.error } }) + return { + [Symbol.asyncIterator](): AsyncIterator { + return { next: () => Promise.resolve(result as unknown as IteratorResult) } + }, + } + } +} + +const streamListenerFailureCases: readonly [string, (ctx: Context) => void][] = [ + ['synchronous listener throw', (ctx) => { + ctx.on('llm/stream', () => { throw new Error('synchronous stream listener failed') }) + }], + ['invalid listener iterable', (ctx) => { + ctx.on('llm/stream', () => ({}) as AsyncIterable) + }], + ['listener wrapper iteration failure', (ctx) => { + ctx.on('llm/stream', (_options, next) => (async function * () { + for await (const chunk of next()) { + yield chunk + throw new Error('stream listener wrapper failed') + } + })()) + }], +] + +async function harness(adapter?: LlmAdapter): Promise { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + if (adapter) ctx.llm.registerAdapter(['mock'], adapter) + return ctx +} + +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { + dispose() + resolve() + } + }) + }) +} + +function send(agent: ReactLoopAgent): void { + agent.send([{ type: 'text', text: 'go' }]) +} + +function contextError(message = 'context too large'): LlmError { + return new LlmError(message, CONTEXT_WINDOW_EXCEEDED_CODE, 400) +} + +describe('agent post-step and request-error lifecycle', () => { + it('fires post-step after results, buffered context, and steering but before step/end', async () => { + const twoCalls: StreamChunk[] = [ + { type: 'block-start', index: 0, blockType: 'tool-call' }, + { type: 'block-end', index: 0, block: { type: 'tool-call', id: CallId('call-1'), name: 'work', arguments: '{}' } }, + { type: 'block-start', index: 1, blockType: 'tool-call' }, + { type: 'block-end', index: 1, block: { type: 'tool-call', id: CallId('call-2'), name: 'work', arguments: '{}' } }, + { type: 'usage', usage: { inputTokens: 10, outputTokens: 5 } }, + { type: 'finish', reason: { kind: 'tool-calls' } }, + ] + const adapter = new FailureScriptAdapter([twoCalls, textResponse('done')]) + const ctx = await harness(adapter) + ctx.tools.register(defineTool({ + name: 'work', + description: 'do work', + parameters: {}, + async execute(_args, exec) { + if (exec.callId === CallId('call-2')) { + exec.agent?.steer([{ type: 'text', text: 'steered' }], { source: { kind: 'plugin', plugin: 'test' } }) + } + return [{ type: 'text', text: 'worked' }] + }, + })) + ctx.on('tools/post-execute', async (exec, _result): Promise => ({ + kind: 'accept', + additionalContext: { + content: [{ type: 'text', text: `context for ${exec.callId}` }], + source: { kind: 'plugin', plugin: 'test' }, + }, + })) + const agent = ctx.agentLoop.create(AgentId('post-step-order'), { model: 'mock' }) + const order: string[] = [] + ctx.on('session/event', (_session, event) => { + if ( + event.type === 'assistant/message' || event.type === 'tool/call' + || event.type === 'tool/result' || event.type === 'context/message' + || event.type === 'steering/message' || event.type === 'step/end' + ) { + if (!('step' in event.data) || event.data.step === 1) order.push(event.type) + } + }) + ctx.on('agent/post-step', (subject, turn, step, signal) => { + if (subject !== agent || step !== 1) return + expect({ turn, step, aborted: signal.aborted }).toEqual({ turn: 1, step: 1, aborted: false }) + subject.inject([{ type: 'text', text: 'listener mutation' }], { source: { kind: 'plugin', plugin: 'post-step' } }) + order.push('agent/post-step') + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(order).toEqual([ + 'assistant/message', + 'tool/call', + 'tool/result', + 'tool/call', + 'tool/result', + 'context/message', + 'context/message', + 'steering/message', + 'context/message', + 'agent/post-step', + 'step/end', + ]) + }) + + it('fires post-step for max-tokens and lets cancellation override that success', async () => { + const adapter = new FailureScriptAdapter([maxTokensResponse('partial')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('cancel-post-step-max-tokens'), { model: 'mock' }) + let entered!: () => void + const postStepEntered = new Promise((resolve) => { entered = resolve }) + ctx.on('agent/post-step', async (_agent, turn, step, signal) => { + expect({ turn, step }).toEqual({ turn: 1, step: 1 }) + entered() + await new Promise((resolve) => { + signal.addEventListener('abort', () => { resolve() }, { once: true }) + }) + }) + + send(agent) + const idle = waitForIdle(ctx, agent) + await postStepEntered + agent.cancel('cancelled during max-tokens post-step') + await idle + + expect(agent.session.events.find(event => event.type === 'assistant/message')).toMatchObject({ + data: { usage: { inputTokens: 10, outputTokens: 7 } }, + }) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'aborted', reason: 'cancelled during max-tokens post-step' } }, + }) + }) + + it.each([ + ['thrown', contextError()], + ['in-band', [{ type: 'finish', reason: { kind: 'error', message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE } }] satisfies StreamChunk[]], + ] as const)('recovers a %s request failure in a new reconstructable step', async (_style, failure) => { + const adapter = new FailureScriptAdapter([failure, textResponse('recovered')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId(`recover-${_style}`), { model: 'mock' }) + const attempts: number[] = [] + ctx.on('agent/request-error', async (subject, turn, step, error, attempt) => { + expect(subject).toBe(agent) + expect({ turn, step, code: error.code }).toEqual({ turn: 1, step: 1, code: CONTEXT_WINDOW_EXCEEDED_CODE }) + attempts.push(attempt) + subject.session.append('context/message', { + content: [{ type: 'text', text: 'RECOVERY SURFACE MUTATION' }], + source: { kind: 'plugin', plugin: 'test-recovery' }, + }, { surfaceOp: 'append' }) + return { action: 'retry' } + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(attempts).toEqual([0]) + expect(adapter.requests).toHaveLength(2) + expect(JSON.stringify(adapter.requests[1]!.messages)).toContain('RECOVERY SURFACE MUTATION') + const starts = agent.session.events.filter(event => event.type === 'step/start') + const ends = agent.session.events.filter(event => event.type === 'step/end') + expect(starts.map(event => event.data.step)).toEqual([1, 2]) + expect(ends.map(event => event.data.step)).toEqual([1, 2]) + const recovery = agent.session.events.find(event => event.type === 'context/message')! + expect(ends[0]!.seq).toBeLessThan(recovery.seq) + expect(recovery.seq).toBeLessThan(starts[1]!.seq) + }) + + it.each(streamListenerFailureCases)('does not offer %s to request recovery', async (_name, install) => { + const ctx = await harness(new FailureScriptAdapter([textResponse('unused')])) + const agent = ctx.agentLoop.create(AgentId(`stream-plugin-${_name.replaceAll(' ', '-')}`), { model: 'mock' }) + let recoveries = 0 + install(ctx) + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + recoveries += 1 + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(recoveries).toBe(0) + expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'error' } } }) + }) + + it.each(['prompt-submit', 'prompt-assembly', 'pre-step', 'request'] as const)( + 'does not offer %s middleware failures to request recovery', + async (boundary) => { + const adapter = new FailureScriptAdapter([textResponse('unused')]) + const ctx = await harness(adapter) + if (boundary === 'prompt-submit') { + ctx.on('agent/prompt-submit', () => { throw new Error('prompt submit failed') }) + } else if (boundary === 'prompt-assembly') { + ctx.on('system-prompt/assemble', () => { throw new Error('prompt assembly failed') }) + } else if (boundary === 'pre-step') { + ctx.on('agent/pre-step', () => { throw new Error('pre-step failed') }) + } else { + ctx.on('agent/request', () => { throw new Error('request middleware failed') }) + } + const agent = ctx.agentLoop.create(AgentId(`${boundary}-not-recoverable`), { model: 'mock' }) + let recoveries = 0 + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + recoveries += 1 + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(recoveries).toBe(0) + expect(adapter.requests).toHaveLength(0) + expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'error' } } }) + }, + ) + + it('does not offer result, tool, or post-step plugin failures to request recovery', async () => { + for (const failure of ['result', 'tool', 'post-step'] as const) { + const adapter = new FailureScriptAdapter([ + failure === 'tool' ? toolCallResponse(`call-${failure}`, 'work', {}) : textResponse('done'), + ]) + const ctx = await harness(adapter) + if (failure === 'result') ctx.on('agent/step-result', () => { throw new Error('result failed') }) + if (failure === 'post-step') ctx.on('agent/post-step', () => { throw new Error('post-step failed') }) + if (failure === 'tool') { + vi.spyOn(ctx.tools, 'execute').mockRejectedValue(new Error('tool service failed')) + } + const agent = ctx.agentLoop.create(AgentId(`${failure}-not-recoverable`), { model: 'mock' }) + let recoveries = 0 + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + recoveries += 1 + return next() + }) + send(agent) + await waitForIdle(ctx, agent) + expect(recoveries, failure).toBe(0) + } + }) + + it.each([ + ['synchronous dispatch', (error: Error) => new SynchronousDispatchFailureAdapter(error)], + ['done getter', (error: Error) => new IteratorResultGetterFailureAdapter('done', error)], + ['value getter', (error: Error) => new IteratorResultGetterFailureAdapter('value', error)], + ] as const)('preserves original Error identity for adapter %s', async (_name, makeAdapter) => { + const original = contextError(`${_name} overflow`) + const ctx = await harness(makeAdapter(original)) + const agent = ctx.agentLoop.create(AgentId(`identity-${_name.replaceAll(' ', '-')}`), { model: 'mock' }) + let seen: Error | undefined + ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => { + seen = error + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(seen).toBe(original) + }) + + it('classifies iterator construction and explicit NO_ADAPTER as model-request failures', async () => { + for (const scenario of ['iterator', 'no-adapter'] as const) { + const ctx = scenario === 'iterator' ? await harness(new IteratorConstructionFailureAdapter()) : await harness() + const agent = ctx.agentLoop.create(AgentId(`request-boundary-${scenario}`), { model: 'mock' }) + let seen = '' + ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => { + seen = error.code ?? '' + return next() + }) + send(agent) + await waitForIdle(ctx, agent) + expect(seen).toBe(scenario === 'iterator' ? 'ITERATOR_CONSTRUCTION' : 'NO_ADAPTER') + } + }) + + it('tracks consecutive retry attempts and resets after a successful request', async () => { + const capped = new FailureScriptAdapter([contextError('first overflow'), contextError('second overflow')]) + const cappedCtx = await harness(capped) + const cappedAgent = cappedCtx.agentLoop.create(AgentId('retry-cap'), { model: 'mock' }) + const cappedAttempts: number[] = [] + cappedCtx.on('agent/request-error', async (_agent, _turn, _step, _error, attempt, _signal, next) => { + cappedAttempts.push(attempt) + return attempt < 1 ? { action: 'retry' } : next() + }) + send(cappedAgent) + await waitForIdle(cappedCtx, cappedAgent) + expect(cappedAttempts).toEqual([0, 1]) + + const reset = new FailureScriptAdapter([ + contextError('first overflow'), + toolCallResponse('retry-reset-call', 'work', {}), + contextError('later overflow'), + ]) + const resetCtx = await harness(reset) + resetCtx.tools.register(defineTool({ + name: 'work', + description: 'continue', + parameters: {}, + async execute() { return [{ type: 'text', text: 'worked' }] }, + })) + const resetAgent = resetCtx.agentLoop.create(AgentId('retry-reset'), { model: 'mock' }) + const resetAttempts: { step: number; attempt: number }[] = [] + resetCtx.on('agent/request-error', async (_agent, _turn, step, _error, attempt, _signal, next) => { + resetAttempts.push({ step, attempt }) + return resetAttempts.length === 1 ? { action: 'retry' } : next() + }) + send(resetAgent) + await waitForIdle(resetCtx, resetAgent) + expect(resetAttempts).toEqual([{ step: 1, attempt: 0 }, { step: 3, attempt: 0 }]) + }) + + it('preserves the original provider error when recovery throws', async () => { + const adapter = new FailureScriptAdapter([contextError('original overflow')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('recovery-throws'), { model: 'mock' }) + ctx.on('agent/request-error', () => { throw new Error('recovery exploded') }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'error', message: 'original overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } }, + }) + }) + + it.each(['cancel', 'dispose'] as const)('keeps %s live through request recovery', async (action) => { + const adapter = new FailureScriptAdapter([contextError()]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId(`${action}-recovery`), { model: 'mock' }) + let entered!: () => void + const recoveryEntered = new Promise((resolve) => { entered = resolve }) + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, signal) => { + entered() + await new Promise((resolve) => { + signal.addEventListener('abort', () => { resolve() }, { once: true }) + }) + return { action: 'retry' } + }) + + send(agent) + const idle = waitForIdle(ctx, agent) + await recoveryEntered + if (action === 'cancel') { + agent.cancel('cancelled during recovery') + await idle + } else { + await ctx.fiber.dispose() + } + + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: action === 'cancel' ? { kind: 'aborted', reason: 'cancelled during recovery' } : { kind: 'disposed' } }, + }) + }) +}) diff --git a/packages/core/agent/README.md b/packages/core/agent/README.md index 5639f30daf..d37851a681 100644 --- a/packages/core/agent/README.md +++ b/packages/core/agent/README.md @@ -31,7 +31,7 @@ The loop plugin registers `AgentFactory`, keeping consumers independent of its c `agent/created` runs after setup and both registry entries; the following `agent/session-start` is the first supported startup injection point. `agent/disposed` means the exact entry left the registry. The loop quiesces its driver first; directly registered custom agents own any stronger ordering. -Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` is a serial surface-mutation checkpoint, while `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale is in the [agent-scope runtime-design RFC](../../../docs/rfc/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way). +Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` and `agent/post-step` are serial checkpoints around a step's durable work, while `agent/request-error` is the failed-model-request recovery waterfall: a retry opens a new numbered step after the failed step closes. `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale for scoped dispatch and terminal settlement is in the [agent-scope runtime-design RFC](../../../docs/rfc/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way). Turn and step boundaries and the model token stream are durable `session/event` facts rather than mirrored `agent/*` notifications. Consumers read `turn/*`, `step/*`, and `assistant/chunk` from the session feed; tool policy and outcome observation belong to the complete pipeline documented by [`dsh-tools`](../tools/README.md). diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index 3aad65a70e..1eda0df2d7 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -70,6 +70,12 @@ export type ContinuationDecision = | { action: 'stop' } | { action: 'continue'; reason?: HookContext } +/** Failed-request recovery decision; `retry` opens another numbered step while listeners delegate by calling `next()`. */ +export type RequestErrorDecision = { action: 'fail' } | { action: 'retry' } + +/** Model-request failure with an optional machine-routable provider code. */ +export type RequestError = Error & { code?: string } + /** * The terminal subset of {@link ContinuationDecision}. A listener on * `agent/turn-stop` returns this to make the already-composed continuation @@ -248,6 +254,31 @@ declare module 'cordis' { * @mode waterfall */ 'agent/step-result'(this: Scoped, agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise + /** + * Awaited serial checkpoint after the response, tool results, injected + * context, and steering are durable but before `step/end`. + * @param agent - the agent that completed the step. + * @param turn - the open turn number. + * @param step - the completed step number. + * @param signal - the turn abort signal. + * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. + * @mode serial + */ + 'agent/post-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void + /** + * Recover a model-request failure after its failed step has closed. `retry` + * opens a new numbered step; `fail` preserves the original request error. + * Call `next()` to delegate to the next recovery listener or the default. + * @param agent - the agent whose request failed. + * @param turn - the open turn number. + * @param step - the failed step number. + * @param error - the original model-request failure. + * @param retryAttempt - zero-based number of prior recovery retries. + * @param signal - the turn abort signal. + * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. + * @mode waterfall + */ + 'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise /** * Override whether the turn continues. The default continues after tool * calls or steering and stops otherwise; a continue reason becomes steering. diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index 1270ad40a5..bd4228165b 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -36,7 +36,7 @@ Every request carries the shared attribution header from dsh-llm's `attributionH ## Errors -Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), `HTTP_` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: }` chunks. +Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: }` chunks. ## Testing diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index 30760a8fbc..113cb6f25e 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -5,7 +5,7 @@ * @module dsh-llm-deepseek/adapter */ -import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { serializeRequest } from './serialize.ts' import type { RequestDefaults } from './serialize.ts' @@ -26,12 +26,17 @@ export interface DeepSeekAdapterOptions { /** * Map an HTTP status to a stable LlmError code. * @param status - status of a non-2xx provider response. - * @returns `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), or `HTTP_` for anything else. + * @param error - parsed provider error body, when available. + * @returns the normalized harness error code. */ -export function httpErrorCode(status: number): string { +export function httpErrorCode(status: number, error?: WireError['error']): string { if (status === 401 || status === 403) return 'AUTH' if (status === 429) return 'RATE_LIMIT' - if (status === 400) return 'INVALID_REQUEST' + if (status === 400) { + const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ') + if (isContextWindowExceededError(detail)) return CONTEXT_WINDOW_EXCEEDED_CODE + return 'INVALID_REQUEST' + } if (status >= 500) return 'SERVER' return `HTTP_${status}` } @@ -67,16 +72,17 @@ export class DeepSeekAdapter extends LlmAdapter { }) if (!response.ok) { - const code = httpErrorCode(response.status) let message = `DeepSeek API error (HTTP ${response.status})` + let providerError: WireError['error'] try { const parsed = await response.json() as WireError - if (parsed.error?.message) message = parsed.error.message + providerError = parsed.error + if (providerError?.message) message = providerError.message } catch { - // Only swallow error-body parsing: status and code are already captured, - // so malformed gateway JSON must not mask the actionable HTTP failure. + // Only swallow error-body parsing: the HTTP status still identifies the + // failure, so malformed gateway JSON must not mask it. } - throw new LlmError(message, code, response.status) + throw new LlmError(message, httpErrorCode(response.status, providerError), response.status) } if (!response.body) { throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE') diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 46f123a1c7..6cca0dde17 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -2,7 +2,7 @@ import { createServer } from 'node:http' import type { IncomingMessage, Server, ServerResponse } from 'node:http' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import LlmService, { LlmError, userAgent } from '@deepseek-ai/dsh-llm' +import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek' import { assemble } from './assemble.ts' @@ -173,6 +173,32 @@ describe('DeepSeekAdapter against a mock server', () => { ).resolves.toBe(status) }) + it('classifies a thrown HTTP context-window rejection with the canonical code', async () => { + const server = await mockServer([{ + kind: 'http-error', + status: 400, + body: JSON.stringify({ + error: { + message: 'This model maximum context length is 128000 tokens; your input exceeds that limit.', + type: 'invalid_request_error', + code: 'context_length_exceeded', + }, + }), + }]) + const ctx = await harness(server.url) + const code = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) + .catch((error: unknown) => (error as LlmError).code) + expect(code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE) + }) + + it('classifies only context-capacity HTTP 400 details as context overflow', () => { + expect(httpErrorCode(400, { message: 'request too large for model context' })) + .toBe(CONTEXT_WINDOW_EXCEEDED_CODE) + expect(httpErrorCode(400, { message: 'invalid input: temperature exceeds maximum allowed value' })) + .toBe('INVALID_REQUEST') + expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413') + }) + it('keeps the status-line message for JSON error bodies without a message', async () => { const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }]) const ctx = await harness(server.url) diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index dd1cb5b48c..6c8fc35104 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -7,7 +7,7 @@ DeepSeek adapter for the harness LLM seam backed by [`@earendil-works/pi-ai`](ht `@deepseek-ai/dsh-llm-deepseek` already talks to the same endpoint. This package is its **design-verification twin**: same models, same wire protocol, completely different internals — a unified LLM library with its own event vocabulary versus hand-rolled fetch/SSE. Anything the harness `StreamChunk` protocol cannot express for BOTH implementations is a core-vocabulary bug. The differences it exercised on purpose: - pi-ai hands tool-call `arguments` around as **parsed objects**; the harness keeps raw JSON strings. The adapter patches replay payloads back to the original raw strings before sending them, and re-stringifies parsed output tool calls at `block-end`. -- pi-ai reports failures as **in-stream error events** (it never throws mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the protocol's other sanctioned error path besides throwing (which llm-deepseek uses). +- pi-ai reports failures as **in-stream error events** (it never throws mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the protocol's other sanctioned error path besides throwing (which llm-deepseek uses). Context-overflow detail maps to the same canonical `CONTEXT_WINDOW_EXCEEDED` code as the hand-rolled adapter. - pi-ai folds reasoning tokens into `usage.output`; there is no separate reasoning count to map. - pi-ai's options omit some DeepSeek/OpenAI-compatible details; the adapter uses its `onPayload` hook to preserve the harness contract (`stop`, scrubbing pi-ai's own per-tool `strict` default — the hand-rolled twin sends no such field — omitted reasoning effort, raw replayed tool arguments). diff --git a/packages/llm/llm-pi-ai/src/convert.ts b/packages/llm/llm-pi-ai/src/convert.ts index 098b93edb3..88f134c8f2 100644 --- a/packages/llm/llm-pi-ai/src/convert.ts +++ b/packages/llm/llm-pi-ai/src/convert.ts @@ -8,7 +8,7 @@ * @module dsh-llm-pi-ai/convert */ -import { CallId, LlmError } from '@deepseek-ai/dsh-llm' +import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm' import type { FinishReason, GenerateOptions, Message, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' import type { AssistantMessage, @@ -165,6 +165,7 @@ export function mapUsage(usage: PiUsage): TokenUsage { function classifyPiAiError(message: string): string { if (/\b(?:401|403)\b/.test(message)) return 'AUTH' if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT' + if (isContextWindowExceededError(message)) return CONTEXT_WINDOW_EXCEEDED_CODE if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' return 'PI_AI_ERROR' diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index 078d2a4d3b..42576f7fed 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest' -import { CallId } from '@deepseek-ai/dsh-llm' +import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm' import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai' import { mapStopReason, mapUsage, toPiContext, toStreamChunks } from '@deepseek-ai/dsh-llm-pi-ai' @@ -300,6 +300,18 @@ describe('mapStopReason / mapUsage', () => { .toMatchObject({ kind: 'error', code: 'RATE_LIMIT' }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' }))) .toMatchObject({ kind: 'error', code: 'SERVER' }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'HTTP 400: input exceeds the model context window limit', + }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'HTTP 400: request too large for model context', + }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value', + }))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' }) }) it('maps cache fields only when nonzero', () => { diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index 296fd0c3d3..b7be205a97 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -12,6 +12,8 @@ An adapter registry plus a single streaming call surface, interceptable via a wa - `ctx.llm.models(): string[]` — model names with a registered adapter. - `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`. +`LlmService` preserves and privately tags errors from final adapter selection, synchronous dispatch, iterator construction, and iteration. `isLlmAdapterFailure(value)` exposes that provenance without classifying `llm/stream` middleware or downstream consumer failures as provider failures, and without replacing the adapter's original coded `Error`. + ### Events | Event | Mode | Purpose | @@ -43,6 +45,7 @@ Every product adapter sends application identity on provider HTTP requests. `att - `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history. - `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams. - `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response. +- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail. ### Real adapters @@ -54,7 +57,7 @@ None, as this adapter registry forwards an already assembled request without add ## Known Limitations and Deferred Work -- **No retry/caching/rate-limit layer ships** — `llm/stream` is the intended wrap seam and has no production listener, so provider 429/5xx failures surface immediately. +- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. - **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)). - **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([RFC](../../../docs/rfc/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)). - **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw. diff --git a/packages/llm/llm/src/adapter-failure.ts b/packages/llm/llm/src/adapter-failure.ts new file mode 100644 index 0000000000..240f934c39 --- /dev/null +++ b/packages/llm/llm/src/adapter-failure.ts @@ -0,0 +1,34 @@ +/** + * Private provider-failure tagging shared by `LlmService` and its consumers. + * + * @module @deepseek-ai/dsh-llm/adapter-failure + */ + +import { HarnessError } from './error.ts' + +/** Errors proven to originate in final adapter dispatch or iteration. */ +const adapterFailures = new WeakSet() + +/** + * Preserve an adapter's Error identity while tagging its provider origin. + * @param value - arbitrary value thrown by adapter dispatch or iteration. + * @returns the original Error, or a coded Error wrapping a non-Error throw. + * @internal + */ +export function markLlmAdapterFailure(value: unknown): Error & { code?: string } { + const error = value instanceof Error + ? value as Error & { code?: string } + : new HarnessError(String(value), 'UNKNOWN', { cause: value }) + adapterFailures.add(error) + return error +} + +/** + * Whether a failure came from final adapter dispatch, iterator construction, + * or iteration rather than from an `llm/stream` waterfall listener. + * @param value - arbitrary failure caught by a model-call consumer. + * @returns true only for errors tagged at the final adapter boundary. + */ +export function isLlmAdapterFailure(value: unknown): value is Error & { code?: string } { + return value instanceof Error && adapterFailures.has(value) +} diff --git a/packages/llm/llm/src/error.ts b/packages/llm/llm/src/error.ts index c1fdbb9ffa..8c1c736492 100644 --- a/packages/llm/llm/src/error.ts +++ b/packages/llm/llm/src/error.ts @@ -21,6 +21,47 @@ export class HarnessError extends Error { } } +/** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */ +export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED' + +/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */ +const STRUCTURED_CONTEXT_OVERFLOW = new RegExp( + String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]` + + String.raw`(?:exceed(?:ed|s)?|overflow(?:ed)?|limit[\s_-]exceeded)(?:$|[^a-z0-9])`, + 'i', +) + +/** Request-size wording that ties "too large" directly to model context capacity. */ +const TOO_LARGE_FOR_CONTEXT = new RegExp( + String.raw`\b(?:request|prompt|input|messages?)\s+(?:is\s+|are\s+)?` + + String.raw`too\s+(?:large|long)\s+for\s+(?:(?:this|the)\s+)?` + + String.raw`(?:model(?:'s)?\s+)?context(?:\s+window)?\b`, + 'i', +) + +/** "Exceeds" wording is safe only when its object is explicitly the model context. */ +const EXCEEDS_MODEL_CONTEXT = new RegExp( + String.raw`\b(?:input|prompt|request|messages?)\b.{0,40}` + + String.raw`\b(?:exceed(?:s|ed)?|overflows?|is\s+larger\s+than)\b.{0,40}` + + String.raw`\b(?:the\s+)?(?:model(?:'s)?\s+)?context(?:\s+(?:length|window))?\b`, + 'i', +) + +/** + * Recognize the context-overflow wording used by OpenAI-compatible providers + * and library adapters. Adapters pass all available provider code, type, and + * message text so both thrown and in-band delivery styles share one classifier. + * @param detail - provider error code/type/message text joined into one string. + * @returns true when the detail identifies a request exceeding the model context window. + */ +export function isContextWindowExceededError(detail: string): boolean { + return STRUCTURED_CONTEXT_OVERFLOW.test(detail) + || /\b(?:maximum|max)(?:\s+(?:allowed|supported))?\s+context\s+(?:length|window)\b/i.test(detail) + || TOO_LARGE_FOR_CONTEXT.test(detail) + || /\b(?:input|prompt|request)\s+(?:is\s+)?too\s+(?:long|large)\s+for\s+(?:this|the)\s+model\b/i.test(detail) + || EXCEEDS_MODEL_CONTEXT.test(detail) +} + /** * Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams). * @param value - the caught value (`unknown` in catch clauses). diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index 08f3f54c51..0b933d12a7 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -9,6 +9,7 @@ import { Context, Service } from 'cordis' import type { GenerateOptions, StreamChunk } from './types.ts' import { HarnessError } from './error.ts' +import { markLlmAdapterFailure } from './adapter-failure.ts' export * from './attribution.ts' export * from './brand.ts' @@ -18,6 +19,7 @@ export * from './types.ts' export { BlockAssembler } from './assembler.ts' export { callConfigEquals, deepFreeze } from './call-config.ts' export type { LlmCallConfig } from './call-config.ts' +export { isLlmAdapterFailure } from './adapter-failure.ts' declare module 'cordis' { interface Context { @@ -118,17 +120,65 @@ export class LlmService extends Service { return adapter } + /** + * Final adapter boundary. It tags only failures from adapter selection, + * synchronous dispatch, iterator construction, or iteration while preserving + * the original Error object. Middleware outside this generator remains + * distinguishable as plugin work. Adapter cleanup is best-effort after an + * earlier failure or downstream close and never masks the winning error. + */ + private async * adapterStream(options: GenerateOptions): AsyncGenerator { + let iterator: AsyncIterator + try { + const stream = this.adapter(options.model).stream(options) + iterator = stream[Symbol.asyncIterator]() + } catch (error: unknown) { + throw markLlmAdapterFailure(error) + } + + let completed = false + try { + while (true) { + let value: StreamChunk + try { + const item = await iterator.next() + if (item.done) { + completed = true + return + } + value = item.value + } catch (error: unknown) { + throw markLlmAdapterFailure(error) + } + // End the adapter-owned try before yielding: consumer/middleware + // failures resumed into this generator must remain untagged. + yield value + } + } finally { + if (!completed) { + try { + const close = iterator.return?.bind(iterator) + if (close) await close() + } catch { + // Lookup and invocation are both adapter-owned cleanup following an + // existing failure/downstream close; neither can replace it. + } + } + } + } + /** * Stream one model call as raw chunks (token-level deltas). Throws * `LlmError` with code `NO_ADAPTER` if no adapter is registered for - * `options.model`. Dispatches through the `llm/stream` waterfall. + * `options.model`. Dispatches through the `llm/stream` waterfall. Final + * adapter dispatch/iteration failures retain their original Error identity + * and are tagged so the agent loop can distinguish them from middleware + * failures without widening request recovery to plugin code. * @param options - the full request; `options.model` selects the adapter. * @returns the chunk stream, possibly wrapped by `llm/stream` listeners. */ stream(options: GenerateOptions): AsyncIterable { - return this.ctx.waterfall(this, 'llm/stream', options, () => { - return this.adapter(options.model).stream(options) - }) + return this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options)) } } diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index f669069c44..8dc46529bd 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -1,6 +1,14 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' -import LlmService, { GenerateOptions, LlmAdapter, LlmError, StreamChunk } from '@deepseek-ai/dsh-llm' +import LlmService, { + GenerateOptions, + HarnessError, + isContextWindowExceededError, + isLlmAdapterFailure, + LlmAdapter, + LlmError, + StreamChunk, +} from '@deepseek-ai/dsh-llm' class ScriptedAdapter extends LlmAdapter { constructor(private script: StreamChunk[]) { @@ -19,6 +27,22 @@ const SCRIPT: StreamChunk[] = [ ] describe('LlmService', () => { + it('recognizes structured and model-capacity context-window overflow details', () => { + expect(isContextWindowExceededError('context_length_exceeded maximum context length')).toBe(true) + expect(isContextWindowExceededError('context-window-overflowed')).toBe(true) + expect(isContextWindowExceededError('This model maximum context length is 128000 tokens')).toBe(true) + expect(isContextWindowExceededError('input is too long for this model')).toBe(true) + expect(isContextWindowExceededError('request too large for model context')).toBe(true) + expect(isContextWindowExceededError('input exceeds the model context window limit')).toBe(true) + }) + + it('does not mistake unrelated input validation for context-window overflow', () => { + expect(isContextWindowExceededError('invalid request: malformed tool arguments')).toBe(false) + expect(isContextWindowExceededError('invalid input: temperature exceeds maximum allowed value')).toBe(false) + expect(isContextWindowExceededError('input exceeds maximum allowed value')).toBe(false) + expect(isContextWindowExceededError('context window size must be positive')).toBe(false) + }) + it('routes stream() to the registered adapter', async () => { const ctx = new Context() await ctx.plugin(LlmService) @@ -32,9 +56,177 @@ describe('LlmService', () => { it('throws NO_ADAPTER for unregistered models', async () => { const ctx = new Context() await ctx.plugin(LlmService) - await expect((async () => { + let caught: unknown + try { for await (const _ of ctx.llm.stream({ model: 'nope', messages: [] })) { /* drain */ } - })()).rejects.toThrow('no adapter registered') + } catch (error: unknown) { + caught = error + } + expect(caught).toBeInstanceOf(LlmError) + expect((caught as LlmError).code).toBe('NO_ADAPTER') + expect((caught as LlmError).message).toContain('no adapter registered') + expect(isLlmAdapterFailure(caught)).toBe(true) + }) + + it.each(['done', 'value'] as const)('tags a throwing IteratorResult.%s getter without replacing its Error', async (field) => { + const original = new LlmError(`${field} getter failed`, 'RESULT_GETTER_FAILED') + const result = field === 'done' ? {} : { done: false } + Object.defineProperty(result, field, { get: () => { throw original } }) + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + return { next: () => Promise.resolve(result as unknown as IteratorResult) } + }, + } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(isLlmAdapterFailure(caught)).toBe(true) + }) + + it.each(['dispatch', 'iterator'] as const)('tags synchronous adapter %s failures without replacing their Error', async (boundary) => { + const original = new LlmError(`${boundary} failed`, 'BOUNDARY_FAILED') + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + if (boundary === 'dispatch') throw original + return { [Symbol.asyncIterator]: () => { throw original } } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(isLlmAdapterFailure(caught)).toBe(true) + }) + + it('tags adapter iteration failures without replacing the original Error or cleanup outcome', async () => { + const original = new LlmError('provider failed', 'PROVIDER_FAILED') + let cleanupCalls = 0 + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + return { + next: () => Promise.reject(original), + return: () => { + cleanupCalls += 1 + return Promise.reject(new Error('cleanup failed')) + }, + } + }, + } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(isLlmAdapterFailure(caught)).toBe(true) + expect(cleanupCalls).toBe(1) + }) + + it('contains a throwing iterator.return getter after next fails without replacing the original Error', async () => { + const original = new LlmError('provider failed', 'PROVIDER_FAILED') + let cleanupLookups = 0 + const iterator: AsyncIterator = { next: () => Promise.reject(original) } + Object.defineProperty(iterator, 'return', { + get: () => { + cleanupLookups += 1 + throw new Error('return getter failed') + }, + }) + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { [Symbol.asyncIterator]: () => iterator } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(isLlmAdapterFailure(caught)).toBe(true) + expect(cleanupLookups).toBe(1) + }) + + it('normalizes and tags non-Error adapter failures once', async () => { + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + // Third-party adapters can reject with arbitrary values; normalization is under test. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors + return { next: () => Promise.reject('plain provider failure') } + }, + } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBeInstanceOf(HarnessError) + expect(caught).toMatchObject({ code: 'UNKNOWN', cause: 'plain provider failure' }) + expect(isLlmAdapterFailure(caught)).toBe(true) + }) + + it('does not tag a failure thrown downstream while consuming adapter output', async () => { + const downstream = new Error('consumer failed') + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT)) + + let caught: unknown + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) throw downstream + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(downstream) + expect(isLlmAdapterFailure(caught)).toBe(false) }) it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => { diff --git a/packages/support/invariants/src/scoped-events.generated.ts b/packages/support/invariants/src/scoped-events.generated.ts index 06cca6bf55..36d1721945 100644 --- a/packages/support/invariants/src/scoped-events.generated.ts +++ b/packages/support/invariants/src/scoped-events.generated.ts @@ -30,10 +30,12 @@ const scopedSubjectResolvers = Object.freeze({ 'agent/created': adapt<'agent/created'>(args => args[0]), 'agent/disposed': adapt<'agent/disposed'>(args => args[0]), 'agent/error': adapt<'agent/error'>(args => args[0]), + 'agent/post-step': adapt<'agent/post-step'>(args => args[0]), 'agent/pre-step': adapt<'agent/pre-step'>(args => args[0]), 'agent/prompt-submit': adapt<'agent/prompt-submit'>(args => args[0]), 'agent/queued': adapt<'agent/queued'>(args => args[0]), 'agent/request': adapt<'agent/request'>(args => args[0]), + 'agent/request-error': adapt<'agent/request-error'>(args => args[0]), 'agent/session-prefix': adapt<'agent/session-prefix'>(args => args[0]), 'agent/session-start': adapt<'agent/session-start'>(args => args[0]), 'agent/status': adapt<'agent/status'>(args => args[0]), diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 71fe28fbe0..11168ba000 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -818,14 +818,22 @@ function renderLifecycle(): string { ' LLM-->>Driver: StreamChunk*', ` Driver->>Session: ${mermaidCode('assistant/chunk')}*`, ` Session-->>SDK: ${mermaidCode('session/event')} ${mermaidCode('assistant/chunk')}*`, + ' alt final adapter or terminal in-band request failure', + ` Driver->>Session: ${mermaidCode('step/end')}`, + ` Driver->>Hooks: ${mermaidCode('agent/request-error')} waterfall`, + ' Hooks-->>Driver: retry in a new step or preserve the original error', + ' else model request succeeded', ` Driver->>Hooks: ${mermaidCode('agent/step-result')} waterfall`, ` Driver->>Session: ${mermaidCode('assistant/message')}`, ` Driver->>Session: ${mermaidCode('tool/call')}`, ' Driver->>Tools: execute through pre and post waterfalls', ' Tools-->>Session: tool-owned events when applicable', - ` Driver->>Session: ${mermaidCode('tool/result')} and ${mermaidCode('step/end')}`, + ` Driver->>Session: ${mermaidCode('tool/result')}, post-tool context, and steering`, + ` Driver->>Hooks: ${mermaidCode('agent/post-step')} serial checkpoint`, + ` Driver->>Session: ${mermaidCode('step/end')}`, ` Driver->>Hooks: ${mermaidCode('agent/turn-continuation')} waterfall`, ` Driver->>Hooks: ${mermaidCode('agent/turn-stop')} serial terminal checkpoint`, + ' end', ` Driver->>Session: ${mermaidCode('turn/end')}`, ` Driver->>Persistence: ${mermaidCode('session/flush')} parallel checkpoint`, ` Driver-->>SDK: ${mermaidCode('agent/status')} idle`, diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 54fe328389..b0cea673ad 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -14,6 +14,8 @@ { "doc": "docs/core-data-structures/core.md", "symbol": "HookContext", "source": "packages/core/agent/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "PromptDecision", "source": "packages/core/agent/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "ContinuationDecision", "source": "packages/core/agent/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "RequestError", "source": "packages/core/agent/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "RequestErrorDecision", "source": "packages/core/agent/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "ContinuationStop", "source": "packages/core/agent/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "SessionStartSource", "source": "packages/core/agent/src/types.ts" }, From 6eddf38a77d1e88c52530c64555124e1fb47788f Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Wed, 15 Jul 2026 16:42:32 +0800 Subject: [PATCH 05/73] feat(acp): enable automatic compaction --- examples/acp-agent/composition.md | 3 +++ examples/acp-agent/cordis.yml | 12 ++++++++++++ 2 files changed, 15 insertions(+) diff --git a/examples/acp-agent/composition.md b/examples/acp-agent/composition.md index b93b4cb1e2..1dc297a5fc 100644 --- a/examples/acp-agent/composition.md +++ b/examples/acp-agent/composition.md @@ -27,6 +27,8 @@ flowchart LR bundle_agent_core --> spine_sessions["ctx.sessions"] bundle_agent_core --> spine_tools["ctx.tools + tool-bash"] bundle_agent_core --> spine_loop["ctx.agents + ctx.agentLoop"] + plugin_acp_compact_basic["compact-basic
@deepseek-ai/dsh-compact-basic"] + cfg --> plugin_acp_compact_basic plugin_acp_subagent["subagent
@deepseek-ai/dsh-subagent"] cfg --> plugin_acp_subagent plugin_acp_subagent_spawn["subagent-spawn
@deepseek-ai/dsh-subagent-spawn"] @@ -59,6 +61,7 @@ flowchart LR | `approval` | `@deepseek-ai/dsh-user-approval` | | `permission` | `@deepseek-ai/dsh-permission` | | `acp-agent` | `@deepseek-ai/dsh-acp-agent` | +| `compact-basic` | `@deepseek-ai/dsh-compact-basic` | | `subagent` | `@deepseek-ai/dsh-subagent` | | `subagent-spawn` | `@deepseek-ai/dsh-subagent-spawn` | | `subagent-fork` | `@deepseek-ai/dsh-subagent-fork` | diff --git a/examples/acp-agent/cordis.yml b/examples/acp-agent/cordis.yml index 201feafe41..17ed222660 100644 --- a/examples/acp-agent/cordis.yml +++ b/examples/acp-agent/cordis.yml @@ -48,6 +48,18 @@ Verify your work by running the code or tests. Keep answers brief and factual. +# Summarize an older range when derived history approaches the context window. +# This leaf consumes `ctx.llm` and the app's `agent/pre-step` seam. +- id: compact-basic + name: '@deepseek-ai/dsh-compact-basic' + config: + contextWindow: 128000 + thresholdRatio: 0.8 + retainTokens: 20480 + summarizationModel: '' + maxTokens: 8192 + compactionRetries: 1 + # Expose fresh-child `spawn` and completed-prefix `fork` through separate tool # names so multi-child scenarios exercise both transports. These leaves follow # the app because it provides `ctx.agents` and `ctx.tools`. From 5ca3eab6ceb522c5a69bb198af7d71316f87a639 Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Wed, 15 Jul 2026 16:46:43 +0800 Subject: [PATCH 06/73] fix(acp): use 256k compaction window --- examples/acp-agent/cordis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/acp-agent/cordis.yml b/examples/acp-agent/cordis.yml index 17ed222660..1e78892113 100644 --- a/examples/acp-agent/cordis.yml +++ b/examples/acp-agent/cordis.yml @@ -53,7 +53,7 @@ - id: compact-basic name: '@deepseek-ai/dsh-compact-basic' config: - contextWindow: 128000 + contextWindow: 256000 thresholdRatio: 0.8 retainTokens: 20480 summarizationModel: '' From 12484104c89fd614a247b1a903f3c9c7821aac3d Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Wed, 15 Jul 2026 16:50:44 +0800 Subject: [PATCH 07/73] feat(compact): recover context overflow (PR3 phase 2) --- docs/agent-lifecycle.md | 2 + docs/architecture.md | 2 + docs/capability-seams.md | 2 +- docs/config-catalog.md | 4 +- docs/cookbook/extension-cookbook.i18n.yaml | 4 +- docs/cookbook/extension-cookbook.md | 2 +- docs/cookbook/extension-cookbook.zh.md | 2 +- docs/cordis-catalog/events.md | 28 +- docs/cordis-catalog/services.md | 6 +- docs/core-data-structures/compaction.md | 10 +- docs/event-producer-consumer.md | 20 +- docs/rfc/INDEX.md | 1 + .../2026-06-11-microkernel-event-taxonomy.md | 4 +- ...t-variables-and-tool-guidance-ownership.md | 4 +- .../2026-07-05-reconstructable-requests.md | 4 +- ...n-pressure-and-overflow-recovery.i18n.yaml | 6 + ...mpaction-pressure-and-overflow-recovery.md | 63 +++ ...ction-pressure-and-overflow-recovery.zh.md | 63 +++ ...07-15-replay-token-meter-service.i18n.yaml | 4 +- .../2026-07-15-replay-token-meter-service.md | 8 +- ...026-07-15-replay-token-meter-service.zh.md | 8 +- .../2026-06-18-compaction-capability-seam.md | 42 +- .../feature/2026-07-07-session-prefix.md | 8 +- examples/coding-agent/cordis.yml | 4 +- packages/compact/compact-basic/README.md | 17 +- .../compact/compact-basic/src/automatic.ts | 58 ++- packages/compact/compact-basic/src/config.ts | 3 + packages/compact/compact-basic/src/index.ts | 64 ++-- .../compact/compact-basic/src/summarizer.ts | 2 +- packages/compact/compact-basic/src/types.ts | 5 +- .../compact-basic/tests/compact-basic.spec.ts | 360 ++++++++++++++++-- .../tests/compact-loop-repro.spec.ts | 176 ++++++++- packages/compact/compact/README.md | 7 +- packages/compact/compact/src/index.ts | 22 +- .../compact/compact/tests/compact.spec.ts | 10 +- .../cordis/tool-cordis/src/api-catalog.ts | 10 +- packages/core/agent-loop/README.md | 2 +- packages/core/agent-loop/src/loop.ts | 12 +- .../agent-loop/tests/interception.spec.ts | 16 +- packages/core/agent-loop/tests/loop.spec.ts | 21 +- .../agent-loop/tests/request-recovery.spec.ts | 42 ++ packages/core/agent/src/types.ts | 18 +- .../invariants/tests/invariants.spec.ts | 2 +- .../ui/user-approval/tests/approval.spec.ts | 2 +- scripts/gen-doc-graphs.ts | 4 +- scripts/type-equiv.manifest.json | 1 + 46 files changed, 913 insertions(+), 242 deletions(-) create mode 100644 docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml create mode 100644 docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md create mode 100644 docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md diff --git a/docs/agent-lifecycle.md b/docs/agent-lifecycle.md index e4ed315e5b..c6010d4139 100644 --- a/docs/agent-lifecycle.md +++ b/docs/agent-lifecycle.md @@ -55,6 +55,8 @@ sequenceDiagram The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set. +`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Recovery compacts between the closed failed step and a fresh retry step, and returns retry only when the surface replacement generation advances; otherwise the original request error remains authoritative. + SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors. Maintenance mode: curated Mermaid sequence; exact event signatures live in the generated Cordis catalog. diff --git a/docs/architecture.md b/docs/architecture.md index 01988a43df..387327a526 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -105,6 +105,8 @@ Each step renders one prompt assembly. Plugins contribute ordered sections, tool Post-tool context follows all results, preserving call/result adjacency. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering while the step signal remains open. Leftover steering becomes next-turn input. `agent/turn-stop` is terminal through close and flush: later steering is discarded, while ordinary queued prompts survive. +When loaded, `dsh-compact-basic` consumes that post-step checkpoint for `ctx.tokenMeter` pressure under the actual routed header. It also consumes canonical context overflow at `agent/request-error`, but authorizes retry only after a tool-balanced compaction advances `surface.replaceGeneration`. The same turn signal owns both summarization paths. + ### Failure Boundaries The turn is the containment boundary. `LlmService` preserves and privately tags errors from final adapter selection, dispatch, and iteration. Those errors and terminal in-band error/aborted finishes close the failed step before `agent/request-error`; retry reconstructs the next numbered step from the log, while decline or failed recovery preserves the provider error. Attempts count consecutive failures and reset after success. diff --git a/docs/capability-seams.md b/docs/capability-seams.md index fae89bfcfb..5f93bf0433 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -203,7 +203,7 @@ flowchart LR | `ctx.permission` | `core` | [`permission`](../packages/ui/permission) | - | [`acp`](../packages/ui/acp) | - | User-facing preset table (`workspace-write`/`danger-full-access`) bundling the sandbox-mode and approval-policy knobs; a switch writes one `permission/preset` event through to both knob events. | | `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | [`tools`](../packages/core/tools) | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the tool registry consumes it for Code Mode). | | `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-policy contributes observed-state checks through the fs/* event gate. | -| `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred. | +| `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred. | | `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-mock`](../packages/support/subagent-mock) | [`tool-subagent`](../packages/subagent/tool-subagent) | - | Providers implement transports; tool-subagent exposes one configured provider as a model-facing tool name. | | `ctx.web` | `seam` | [`web`](../packages/web/web) | [`web-search-exa`](../packages/web/web-search-exa), [`web-search-perplexity`](../packages/web/web-search-perplexity), [`web-search-deepseek`](../packages/web/web-search-deepseek), [`web-fetch-local`](../packages/web/web-fetch-local) | [`tool-web`](../packages/web/tool-web) | - | Search and fetch providers register into one ctx.web seam; tool-web owns the stable model-facing names. | | `ctx.workflows` | `seam` | [`workflow`](../packages/workflow/workflow) | [`workflow-workerthread`](../packages/workflow/workflow-workerthread) | [`tool-workflow`](../packages/workflow/tool-workflow) | - | One engine per context (bash shape, no named-provider registry); the worker-thread engine fans agent() calls out through ctx.subagents. | diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 22fc421f8a..886d7a63a1 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -220,7 +220,9 @@ export interface BasicCompactConfig { maxTokens?: number /** Extra attempts after the first compaction when pressure remains above threshold. Defaults to `1`. */ compactionRetries?: number - /** Enable the automatic `agent/pre-step` pressure listener. Defaults to `true`. */ + /** Maximum retries after canonical context overflow; `0` disables recovery. Defaults to `1`. */ + maxOverflowRetries?: number + /** Enable automatic post-step pressure and overflow-recovery listeners. Defaults to `true`. */ auto?: boolean } diff --git a/docs/cookbook/extension-cookbook.i18n.yaml b/docs/cookbook/extension-cookbook.i18n.yaml index ef1d8d606f..6408aaeea1 100644 --- a/docs/cookbook/extension-cookbook.i18n.yaml +++ b/docs/cookbook/extension-cookbook.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -extension-cookbook.md: 40ee22b352c884d7f295c87726c54ab8e166c844 -extension-cookbook.zh.md: 4e5bc68c973649574bcb2404bea00096eb9ca41f +extension-cookbook.md: 712e4f2f1f98cabfeea1a899c111a39957d16e21 +extension-cookbook.zh.md: 494cb559cf17f13a494e6c695a661c559a0add12 diff --git a/docs/cookbook/extension-cookbook.md b/docs/cookbook/extension-cookbook.md index 40ee22b352..712e4f2f1f 100644 --- a/docs/cookbook/extension-cookbook.md +++ b/docs/cookbook/extension-cookbook.md @@ -102,7 +102,7 @@ Every product feature maps to a listener on a documented extension seam — the | `/loop` | on the `turn/end` session event, `send()` the next iteration; or force-continue | | Dynamic workflow | `ctx.workflows` + the worker-thread engine + the `workflow` tool; structured in-process children enforce output with scoped prompt/tool registrations, a monotonic tool guard, final `tools/result` commit (including enclosing `run_code`), and terminal `agent/turn-stop` | | Queued + steering messages | core `Agent.send()` / `Agent.steer()` | -| Context compaction (auto + manual) | the `ctx.compact` seam + a backend (`dsh-compact-basic`) on the serial `agent/pre-step` seam; auto = token-pressure check before each step; a manual trigger invokes the same `ctx.compact` routine ([compaction RFC](../rfc/implemented/feature/2026-06-18-compaction-capability-seam.md) — the model-facing `/compact` consumer tool is deferred) | +| Context compaction (auto + manual) | the `ctx.compact` seam + `dsh-compact-basic`; automatic pressure runs on serial `agent/post-step`, canonical overflow recovery runs on `agent/request-error`, and manual callers use the same compact service ([compaction RFC](../rfc/implemented/feature/2026-06-18-compaction-capability-seam.md) — the model-facing `/compact` consumer tool is deferred) | | System prompt configurability | `ctx.systemPrompt.section()` with ordering and scope-local shadowing | | AGENTS.md (root) | a section provider reading the file | | AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener | diff --git a/docs/cookbook/extension-cookbook.zh.md b/docs/cookbook/extension-cookbook.zh.md index 4e5bc68c97..494cb559cf 100644 --- a/docs/cookbook/extension-cookbook.zh.md +++ b/docs/cookbook/extension-cookbook.zh.md @@ -102,7 +102,7 @@ export function apply(ctx: Context) { | `/loop` | 在 `turn/end` 会话事件上 `send()` 下一次迭代；或强制继续 | | 动态工作流 | `ctx.workflows` + worker-thread 引擎 + `workflow` 工具；结构化的进程内子任务通过作用域化的 prompt/工具注册、单调工具守卫、最终 `tools/result` 提交（包括外层 `run_code`）和终端 `agent/turn-stop` 来强制输出 | | 排队消息 + steering（中途引导） | 核心 `Agent.send()` / `Agent.steer()` | -| 上下文压缩（context compaction）（自动 + 手动） | `ctx.compact` seam + 串行 `agent/pre-step` seam 上的后端（`dsh-compact-basic`）；自动 = 每步之前的 token 压力检查；手动触发调用同一个 `ctx.compact` 例程（[压缩 RFC](../rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)——面向模型的 `/compact` 消费方工具已推迟） | +| 上下文压缩（context compaction）（自动 + 手动） | `ctx.compact` seam + `dsh-compact-basic`；自动压力检查运行在串行 `agent/post-step`，规范化溢出恢复运行在 `agent/request-error`，手动调用方使用同一个压缩服务（[压缩 RFC](../rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)——面向模型的 `/compact` 消费方工具已推迟） | | 系统提示词可配置性 | `ctx.systemPrompt.section()`，支持排序与作用域局部覆盖 | | AGENTS.md（根目录） | 一个读取该文件的 section provider | | AGENTS.md（子目录，按需触发）+ 文件变更通知 | 从 watcher / tool-result 监听器调用 `agent.inject()` | diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 7ce58b6140..131297b797 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -47,7 +47,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:314`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:308`](../../packages/core/agent/src/types.ts) ### `agent/post-step` — serial @@ -59,19 +59,19 @@ Awaited serial checkpoint after the response, tool results, injected context, an Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:267`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:261`](../../packages/core/agent/src/types.ts) ### `agent/pre-step` — serial -Awaited serial checkpoint for session-surface mutation after prompt assembly and before `step/start`; appends land outside the pending step. The loop derives history once afterward, so compaction records and replacements are included without rewriting an assembled request. The prompt and prefix are the exact pressure inputs for that request, and `signal` cancels listener work. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. +Awaited serial checkpoint before `step/start`; appends land outside the pending step and are included when the loop derives request history. `signal` cancels listener work. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. ```ts cordis-catalog -'agent/pre-step'(this: Scoped, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise | void +'agent/pre-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void ``` -Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:208`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:202`](../../packages/core/agent/src/types.ts) ### `agent/prompt-submit` — waterfall @@ -83,7 +83,7 @@ Allow, rewrite, or block one drained prompt before it becomes a user message. Ca Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:218`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:212`](../../packages/core/agent/src/types.ts) ### `agent/queued` — emit @@ -107,7 +107,7 @@ Replace the frozen call configuration. Model-visible content must use logged cha Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:230`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:224`](../../packages/core/agent/src/types.ts) ### `agent/request-error` — waterfall @@ -119,11 +119,11 @@ Recover a model-request failure after its failed step has closed. `retry` opens Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:281`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:275`](../../packages/core/agent/src/types.ts) ### `agent/session-prefix` — waterfall -Compose request-only messages placed before derived history. The frozen result is computed once per loop instance, logged on its anchoring request header, and reused so the provider prefix remains stable. Interrupted composition is discarded. Composition precedes the first `agent/pre-step` and request boundary, so listener appends join the current request and pressure accounting sees the composed prefix. Changing context belongs in history; contributors should prepend to `await next()` to preserve registration order. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. +Compose request-only messages placed before derived history. The frozen result is computed once per loop instance, logged on its anchoring request header, and reused so the provider prefix remains stable. Interrupted composition is discarded. Composition precedes the first `agent/pre-step` and request boundary, so listener appends join the current request. Changing context belongs in history; contributors should prepend to `await next()` to preserve registration order. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. ```ts cordis-catalog 'agent/session-prefix'(this: Scoped, agent: Agent, prefix: Message[], signal: AbortSignal, next: () => Promise): Promise @@ -131,7 +131,7 @@ Compose request-only messages placed before derived history. The frozen result i Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:245`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:239`](../../packages/core/agent/src/types.ts) ### `agent/session-start` — emit @@ -167,7 +167,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:256`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:250`](../../packages/core/agent/src/types.ts) ### `agent/turn-continuation` — waterfall @@ -179,7 +179,7 @@ Override whether the turn continues. The default continues after tool calls or s Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:291`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:285`](../../packages/core/agent/src/types.ts) ### `agent/turn-stop` — serial @@ -191,7 +191,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:301`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:295`](../../packages/core/agent/src/types.ts) ## `approval/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 990a2bc1bc..02705dee99 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -89,13 +89,11 @@ Source: [`packages/code-runtime/code-runtime/src/index.ts:31`](../../packages/co Abstract compaction service. Implementations own trigger policy, retention, and summarization, and may consume a separate measurement service. A successful run replaces the selected surface span with one summary node and prevents concurrent compaction of the same session. Load one implementation per context as `ctx.compact`. ```ts cordis-catalog -abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise +abstract compactIfNeeded( agent: CompactAgentContext, trigger: CompactionTrigger, signal: AbortSignal, ): Promise abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise ``` -Types: [Message](../core-data-structures/core.md) - -Source: [`packages/compact/compact/src/index.ts:38`](../../packages/compact/compact/src/index.ts) +Source: [`packages/compact/compact/src/index.ts:40`](../../packages/compact/compact/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) diff --git a/docs/core-data-structures/compaction.md b/docs/core-data-structures/compaction.md index 50a80208a9..f60d1e3e08 100644 --- a/docs/core-data-structures/compaction.md +++ b/docs/core-data-structures/compaction.md @@ -50,8 +50,14 @@ interface CompactionResult { ## The service -`CompactService` exposes `compactIfNeeded(...)` for pressure-triggered compaction, returning `null` when no compaction is needed, and `compactRegion(...)` for an explicit inclusive surface range. The pre-step caller supplies the agent, full prompt, session prefix, and abort signal; implementations must forward that signal to summarization. The seam owns no pricing API: `dsh-compact-basic` resolves the routed model through [`ctx.tokenMeter`](token-meter.md), whose model-bound handle owns estimation and replay, while the backend owns retention, event sequencing, and summarization. +Automatic callers state why policy is running; implementations may treat confirmed overflow more aggressively than ordinary pressure. -Auto-compaction runs at serial `agent/pre-step`, before the step and request derivation, so it can replace surface nodes while keeping trace events outside the step. Region boundaries preserve tool-call/result pairing but do not preserve whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns the retention and failure details. +```ts type-equiv +export type CompactionTrigger = 'pressure' | 'context-overflow' +``` + +`CompactService` exposes `compactIfNeeded(agent, trigger, signal)` for automatic `pressure` or `context-overflow` policy, returning `null` when no safe work exists, and `compactRegion(...)` for an explicit inclusive surface range. Implementations must forward the supplied signal to summarization. The seam owns no pricing API: `dsh-compact-basic` resolves the durable routed model through [`ctx.tokenMeter`](token-meter.md), whose model-bound handle owns estimation and replay, while the backend owns retention, event sequencing, and summarization. + +Pressure compaction runs at serial `agent/post-step`, after successful assistant output, tool results, buffered context, and steering are durable but before `step/end`. Failed-request recovery runs through `agent/request-error` after the failed step closes, and authorizes a fresh numbered-step retry only when the surface replacement generation advances. Region boundaries preserve tool-call/result pairing but do not preserve whole turns, allowing early closed steps of one oversized turn to compact. `dsh-compact-basic` owns thresholds, retained-tail policy, overflow caps, and failure handling. The seam exports `toolPairingBalancedBefore(session, node)` and `toolPairingBalancedAfter(session, node)` for those edge checks. Both validate current surface membership, reject stale or missing seqs and orphan results, and ignore a caller-retained `node.next`; the [package contract](../../packages/compact/compact/README.md#tool-pairing-boundaries) owns their cache semantics. diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index e6872669de..1f280d74b1 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -9,19 +9,19 @@ This matrix shows which packages dispatch each harness-owned event and which pac | --- | --- | --- | --- | --- | | `agent/created` | `emit` | [`packages/core/agent/src/types.ts:145`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`jsonrpc`](../packages/ui/jsonrpc), [`stdio`](../packages/ui/stdio) | | `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:154`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:314`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:267`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | - | -| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:208`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic), [`user-approval`](../packages/ui/user-approval) | -| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:218`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:308`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:261`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:202`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`user-approval`](../packages/ui/user-approval) | +| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:212`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | | `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:173`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:230`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:281`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:245`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill) | +| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:224`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:275`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:239`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill) | | `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:186`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `agent/status` | `emit` | [`packages/core/agent/src/types.ts:163`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`stdio`](../packages/ui/stdio) | -| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:256`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:291`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:301`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | +| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:250`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | +| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:285`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:295`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:31`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/ui/acp) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:59`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:68`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | diff --git a/docs/rfc/INDEX.md b/docs/rfc/INDEX.md index 4f41579a5b..0a80128a4e 100644 --- a/docs/rfc/INDEX.md +++ b/docs/rfc/INDEX.md @@ -142,6 +142,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [A shared timeout/deadline primitive, with hard-kill left to each capability](implemented/architecture/2026-07-06-timeout-deadline-library.md) | 2026-07-06 | | [Tool-call timeout policy as a plugin](implemented/architecture/2026-07-07-tool-call-timeout-policy.md) | 2026-07-07 | | [The agent is a registration scope](implemented/architecture/2026-07-08-agent-scope-contexts.md) | 2026-07-08 | +| [After-call compaction pressure and context-overflow recovery](implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) | 2026-07-10 | | [Single-file executable SDK runtime distribution (single-exe)](implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md) | 2026-07-10 | | [Agent-scope runtime design and correctness](implemented/architecture/2026-07-12-agent-scope-runtime-design.md) | 2026-07-12 | | [Replay token meter service](implemented/architecture/2026-07-15-replay-token-meter-service.md) | 2026-07-15 | diff --git a/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md b/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md index 8293924d37..1d10bfa479 100644 --- a/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md +++ b/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md @@ -10,8 +10,8 @@ The product principle is "everything is a plugin": hooks, /goal, /loop, dynamic Pure Cordis event taxonomy. The loop's extension seams are typed events with deliberate dispatch modes: -- **waterfall** (around-middleware) where plugins transform, veto, or wrap: `agent/prompt-submit`, `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/pre-execute`, `tools/execute`, `tools/post-execute`, `llm/stream`, `system-prompt/assemble`. -- **serial** (awaited in listener order; a bail value stops later listeners) for ordered checkpoints: every `agent/pre-step` listener runs when all abstain, while the first stop returned from `agent/turn-stop` makes the terminal decision final. +- **waterfall** (around-middleware) where plugins transform, veto, recover, or wrap: `agent/prompt-submit`, `agent/request`, `agent/request-error`, `agent/step-result`, `agent/turn-continuation`, `tools/pre-execute`, `tools/execute`, `tools/post-execute`, `llm/stream`, `system-prompt/assemble`. +- **serial** (awaited in listener order; a bail value stops later listeners) for ordered checkpoints: every `agent/pre-step` and `agent/post-step` listener runs when all abstain, while the first stop returned from `agent/turn-stop` makes the terminal decision final. - **parallel** (awaited fan-out) where every listener must get an independent chance: the `session/flush` durability checkpoint. - **emit** (synchronous fire-and-forget) for notifications: turn/step boundaries, stream chunks, lifecycle, errors, and the contained immutable `tools/result` observation. diff --git a/docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md b/docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md index 854807c109..e40759a182 100644 --- a/docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md +++ b/docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md @@ -30,7 +30,7 @@ Plugins register `{{name}}` values through `ctx.systemPrompt.variable(name, prov ### Persona as the order-0 section -`dsh-system-prompt` owns `harness:identity` at order `-100` and the configured `deployment:persona` at order 0, so both survive a replacement loop. Prompt rendering has one path, `renderPrompt(assembly)`, and `agent/pre-step` therefore measures the exact prompt used for compaction. An agent-scoped `deployment:persona` shadows the global default and lets subagent providers install a persona before publication. The conventional order bands are identity `-100`, persona `0`, and tool guidance `100–199`. +`dsh-system-prompt` owns `harness:identity` at order `-100` and the configured `deployment:persona` at order 0, so both survive a replacement loop. Prompt rendering has one path, `renderPrompt(assembly)`, and the routed request header therefore records the exact prompt later replayed by `ctx.tokenMeter` for compaction pressure. An agent-scoped `deployment:persona` shadows the global default and lets subagent providers install a persona before publication. The conventional order bands are identity `-100`, persona `0`, and tool guidance `100–199`. ### Tool guidance ownership @@ -43,7 +43,7 @@ Per-tool semantics and selection guidance live in tool descriptions. Prompt sect ## Alternatives considered - **The loop composes an identity line itself** — hardcodes model-facing prose in the one package that must stay thin ("plugins, not loop changes"), and outside the section pipeline it would be a second composition path. (The identity DOES ship as a code literal — but as an ordinary section registered by `dsh-system-prompt`, whose `system-prompt/assemble` waterfall remains the escape valve for a deployment that must drop it.) -- **Inject the model name via the `agent/request` waterfall** — prompt text composed in two places, and `agent/pre-step`'s `fullSystemPrompt` would omit it, so compaction would measure a prompt that is not what the model sees. +- **Inject the model name via the `agent/request` waterfall** — prompt text would be composed in two places and the earlier rendered persona could disagree with the final routed header. The request plugin that owns late routing must also own any earlier prompt claim about that model. - **Hand-write the model name in each persona** — duplicates the `model:` key one line above and silently lies after a config edit; the exact disease this RFC cures. - **Lenient interpolation (leave unknown refs verbatim, or substitute empty)** — a typo ships `{{modle}}` (or a hole) to the model and nobody notices until transcript review. - **Per-instance subagent wording in config** — returns model-facing prose to every deployment × instance, the P2 disease again. **Keying wording off the provider NAME** — `providerName` is itself config, so a renamed provider silently gets the wrong words. diff --git a/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md index d95a709ada..34bae2a2f5 100644 --- a/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md +++ b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md @@ -22,9 +22,9 @@ Prefix-cache stability is corollary #1, not the headline: an append-only log pro `EpochHeader` records the request's non-history state: call config, rendered system prompt, tool schemas, and session prefix, with empty values canonicalized to absence. `request/header` writes a full initial, resume, or fallback snapshot. `request/header-delta` encodes system changes by common-prefix/suffix line trim, tools by name-keyed additions/removals/changes, and config or prefix by full replacement. `foldRequestHeader`, `diffHeader`, and `applyHeaderDelta` are the pure codec. Each loop instance writes a snapshot on its first request to anchor process boundaries. Deltas are only an optimization: the writer verifies round-trip equality and falls back to a full snapshot for unrepresentable changes such as pure tool reordering. -Each step rebuilds prompt assembly. On the instance's first step, `agent/session-prefix` extends a frozen empty seed with request-only opener messages; the result is frozen and cached for that loop instance. `agent/pre-step` then receives the composed prefix before messages are snapshotted immediately ahead of `step/start`. The first call config starts from explicit `AgentOptions`, preserving fork overrides and resume reconfiguration; later calls start from the folded header. `agent/request` may replace only that frozen config seed, while model-visible content enters through logged channels. The loop records the owed header event—the prefix's only durable home—builds `GenerateOptions` from prefix, snapshot, and header, and deep-freezes it while leaving `AbortSignal` live. Per-instance state is only the cached prefix and whether its anchoring snapshot has been written. +Each step rebuilds prompt assembly. On the instance's first step, `agent/session-prefix` extends a frozen empty seed with request-only opener messages; the result is frozen and cached for that loop instance before the generic `agent/pre-step` checkpoint and boundary snapshot. The first call config starts from explicit `AgentOptions`, preserving fork overrides and resume reconfiguration; later calls start from the folded header. `agent/request` may replace only that frozen config seed, while model-visible content enters through logged channels. The loop records the owed header event—the prefix's only durable home—builds `GenerateOptions` from prefix, snapshot, and header, and deep-freezes it while leaving `AbortSignal` live. Per-instance state is only the cached prefix and whether its anchoring snapshot has been written. -**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step` is the seam for content needed by the current request. Header reconstruction folds through the step's own `request/header*` event, or carries the prior fold when no new header is written. +**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step(agent, turn, step, signal)` remains the generic seam for content needed by the current request. Header reconstruction folds through the step's own `request/header*` event, or carries the prior fold when no new header is written. **Enforcement.** In development, `dsh-invariants` independently rebuilds each loop request through a fresh `Session`, so the live cache cannot vouch for itself, then compares messages and folded header fields at `llm/stream`. Loop requests are identified by their frozen shape and session id; direct one-shots are excluded. Correctness depends on sequence-bounded reconstruction rather than listener order. A with-key e2e requires positive cache-read tokens after the first request; per-step usage is the production signal, and a header change or compaction appears as a cache-read drop on the next step. diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml new file mode 100644 index 0000000000..dbb9c76bbf --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 92167dc7d444a3620abfbaab721260ed1c828db9 +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: fe3b6617b25a58ef2c1c311df088f9c805fe9ef4 diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md new file mode 100644 index 0000000000..92167dc7d4 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md @@ -0,0 +1,63 @@ +# RFC: After-call compaction pressure and context-overflow recovery + +Status: implemented + +English | [中文](2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md) + +## Problem + +Automatic compaction originally ran at `agent/pre-step` and received an assembled prompt and session prefix. That boundary was necessarily provisional: `agent/request` could still route another model or change call configuration, tool schemas were not frozen with the compaction inputs, and the next assistant output, tool results, buffered context, and steering did not exist yet. Expanding the pre-step signature could move the stale boundary but could not make it exact. + +Successful calls are not the only pressure signal. A provider can reject a request for exceeding its context window before it returns usage, and some successful calls omit usage. The system therefore needs replayable post-call pressure plus a narrow failure-recovery path that preserves the provider error whenever compaction cannot prove useful progress. + +## Decision + +### Successful pressure moves to a durable post-step checkpoint + +`agent/pre-step` is narrowed to `(agent, turn, step, signal)`. It remains a generic serial checkpoint before `step/start`, but it carries no compaction-only prompt or prefix fields. + +The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after assistant output, every dispatched or synthetic tool result, post-tool context, and steering are durable, but before `step/end`. This placement gives pressure policy the complete successful-call state without splitting an assistant tool call from its result. A listener failure is an ordinary turn failure; it never enters model-request recovery. + +`dsh-compact-basic` resolves the exact latest routed model from the durable request header and asks that model's `ctx.tokenMeter` handle to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work. A durable unknown model throws `TOKEN_METER_MODEL_UNCONFIGURED` with its exact name and fails the otherwise-successful turn; operational selection or summarization failures warn and continue with full history. + +### Request recovery is limited to the final model boundary + +`RequestError`, `RequestErrorDecision`, and the `agent/request-error` waterfall represent failures after the final adapter has been selected. Private `WeakSet` tagging preserves the original thrown error identity across dispatch, iterator construction, and iteration. Terminal in-band `error` or `aborted` finishes enter the same path. Prompt assembly, request middleware, request logging, result processing, tools, post-step listeners, and cleanup remain ordinary failures. + +The failed step closes before recovery runs. A retry opens the next numbered step and rebuilds the request from the durable log; consecutive recovery attempts reset only after a successful provider request. Both DeepSeek adapters normalize recognized provider context-limit failures to `CONTEXT_WINDOW_EXCEEDED`. + +If cancellation lands after assistant tool calls are durable but before all calls dispatch, the loop records a synthetic aborted `tool/result` for every undispatched call before following the normal abort path. The surface therefore never retains orphaned durable tool calls merely because cancellation won the race. + +### CompactService exposes intent, not token accounting + +`CompactService.compactIfNeeded(agent, trigger, signal)` accepts `trigger: 'pressure' | 'context-overflow'`. The interface gains no estimation methods or token types; `ctx.tokenMeter` remains the reusable accounting owner. + +For `pressure`, compact-basic applies the selected meter profile's threshold and retained-tail policy, compares scalar and surface `logRevision`, and uses the same meter for range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`. + +For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It chooses the maximal tool-balanced head range while leaving the newest indivisible unit, then attempts exactly one shrinking compaction under the same signal. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` only when compaction succeeds and the generation increases. A backend returning a result without replacement cannot authorize retry. + +`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, missing or unknown routed models, no safe range, no generation change, and recovery throws all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. Cancellation or disposal remains authoritative even if recovery work completes concurrently. + +The default summarizer still resolves explicit configuration, then the latest logged route, then agent options. Because direct `llm/stream` middleware may reroute that auxiliary call, `compact/summary.model` records the final mutable `GenerateOptions.model` observed after dispatch rather than the pre-waterfall candidate. + +## Testing + +Lifecycle tests pin post-step ordering after durable tool/context/steering work, content-less and max-token successes, final-adapter dispatch/iterator/in-band boundaries, retry numbering, attempt reset, cancellation, disposal, synthetic tool results, and original error identity. + +Compact tests pin low-friction defaults, actual routed-model selection, exact unknown-model behavior, below-threshold forced overflow, newest tool-pair retention, non-shrinking rejection, generation proof, caps, disabled listeners, single downstream delegation, and auxiliary summary routing provenance. Real-loop composition covers both thrown and in-band overflow: the failed step closes, compaction lands between attempts, and the next numbered request is reconstructed from the replacement surface. + +## Alternatives considered + +- **Keep provisional pre-step pressure and add more arguments** — rejected because later routing and request mutation remain outside any earlier snapshot, while generic lifecycle becomes coupled to one plugin. +- **Retry the same numbered step** — rejected because recovery appends durable events after the failed boundary. A new step preserves balanced nesting and reconstructability. +- **Retry whenever `compactIfNeeded` returns a result** — rejected because a custom backend can report success without changing model-visible state. `replaceGeneration` is the authoritative proof. +- **Let compact-basic parse provider wording** — rejected because classification belongs at adapters and must cover both thrown and in-band delivery. +- **Use a universal model/window fallback during recovery** — rejected because destructive policy under the wrong context capacity can hide the original provider failure. Unknown durable routes delegate unchanged. + +## Consequences + +Pressure now describes the actual completed routed request, including durable tool results and request-only prefix fields, rather than a provisional next-call guess. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change. + +The cost is one additional serial checkpoint on successful steps and adapter-maintained overflow classification. Provider wording and heuristic character density remain maintenance risks. Surface compaction still cannot repair an envelope that alone exceeds the window or split one indivisible oversized message/tool unit. + +This RFC supersedes only the pre-step automatic-trigger portion of the [compaction capability-seam RFC](../feature/2026-06-18-compaction-capability-seam.md). The service split, standalone token meter, balanced range contract, log-recorded lock, summary replacement, and sole `summarize()` subclass hook remain unchanged. diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md new file mode 100644 index 0000000000..fe3b6617b2 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md @@ -0,0 +1,63 @@ +# RFC：调用后压缩压力与上下文溢出恢复 + +Status: implemented + +[English](2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) | 中文 + +## 问题 + +自动压缩最初运行在 `agent/pre-step`，并接收已装配提示词与会话前缀。这个边界必然只是临时状态：`agent/request` 仍可能路由到另一个模型或改变调用配置，工具 schema 没有与压缩输入在同一位置冻结，而下一次 assistant 输出、工具结果、缓冲上下文与 steering 此时还不存在。继续扩充 pre-step 签名只能移动陈旧边界，无法让它变得精确。 + +成功调用也不是唯一的压力信号。提供方可能在返回 usage 之前就因上下文窗口超限拒绝请求，一些成功调用也不提供 usage。因此，系统需要可重放的调用后压力，以及一条狭窄的失败恢复路径；当压缩无法证明取得有效进展时，必须保留原始提供方错误。 + +## 决策 + +### 成功压力移动到持久 post-step 检查点 + +`agent/pre-step` 收窄为 `(agent, turn, step, signal)`。它仍是 `step/start` 之前的通用串行检查点，但不再携带压缩专用的提示词或前缀字段。 + +循环在 assistant 输出、所有已分发或合成的工具结果、工具后上下文与 steering 都持久化之后、`step/end` 之前，触发等待式串行 `agent/post-step(agent, turn, step, signal)`。该位置让压力策略看到完整的成功调用状态，同时不会拆开 assistant 工具调用与其结果。监听器失败属于普通 turn 失败，绝不会进入模型请求恢复。 + +`dsh-compact-basic` 从持久请求头解析精确的最新实际路由模型，并让该模型的 `ctx.tokenMeter` handle 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断，因此不执行工作。持久记录的未知模型会携带精确名称抛出 `TOKEN_METER_MODEL_UNCONFIGURED`，使原本成功的 turn 失败；操作性的选择或摘要失败则警告并继续使用完整历史。 + +### 请求恢复只覆盖最终模型边界 + +`RequestError`、`RequestErrorDecision` 与 `agent/request-error` waterfall 表示最终适配器已经选定之后的失败。私有 `WeakSet` 标记在分发、异步迭代器构造与迭代过程中保留原始抛出错误的身份。终止性的带内 `error` 或 `aborted` finish 进入同一路径。提示词装配、请求中间件、请求日志、结果处理、工具、post-step 监听器与清理仍属于普通失败。 + +恢复运行前，失败 step 已经关闭。重试会打开下一个编号 step，并从持久日志重建请求；连续恢复尝试计数只在提供方请求成功后重置。两个 DeepSeek 适配器都把识别出的提供方上下文限制错误规范化为 `CONTEXT_WINDOW_EXCEEDED`。 + +如果取消发生在 assistant 工具调用已经持久化之后、所有调用完成分发之前，循环会为每个尚未分发的调用记录合成的 aborted `tool/result`，随后进入正常中止路径。因此，表层不会仅因取消赢得竞态而留下孤立的持久工具调用。 + +### CompactService 暴露意图，而不拥有 token 核算 + +`CompactService.compactIfNeeded(agent, trigger, signal)` 接收 `trigger: 'pressure' | 'context-overflow'`。接口不增加估算方法或 token 类型；`ctx.tokenMeter` 继续作为可复用的核算所有者。 + +对于 `pressure`，compact-basic 应用所选 meter profile 的阈值与保留尾部策略，比较标量和表层的 `logRevision`，并用同一个 meter 完成范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`。 + +对于规范化溢出，compact-basic 绕过标量压力与普通保留 token 预算。它在保留最新不可分割单元的同时，选择最大的工具配对平衡头部范围，并在同一 signal 下只尝试一次缩小压缩。自动监听器先记录 `session.surface.replaceGeneration`，只有压缩成功且 generation 增加时才返回 `{ action: 'retry' }`。后端若只返回结果但没有替换表层，不能授权重试。 + +`maxOverflowRetries` 可选且默认为 `1`；`0` 只禁用溢出恢复，不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失或未知路由模型、没有安全范围、generation 未变化，以及恢复抛错都会委托给下一个监听器。若没有后续恢复，循环报告原始提供方错误对象与代码。即使恢复工作并发完成，取消或销毁仍具有最终优先级。 + +默认摘要器仍依次解析显式配置、最近记录的路由与 agent options。因为直接 `llm/stream` 中间件可以重新路由该辅助调用，`compact/summary.model` 记录分发后最终可变的 `GenerateOptions.model`，而不是 waterfall 之前的候选值。 + +## 测试 + +生命周期测试固定 post-step 位于持久工具、上下文与 steering 工作之后，覆盖无内容与达到 token 上限的成功、最终适配器分发/迭代器/带内边界、重试编号、尝试重置、取消、销毁、合成工具结果与原始错误身份。 + +压缩测试固定低摩擦默认值、实际路由模型选择、精确未知模型行为、低于阈值的强制溢出、最新工具配对保留、非缩小拒绝、generation 证明、上限、禁用监听器、单次下游委托与辅助摘要路由来源。真实循环组合同时覆盖抛出式和带内溢出：失败 step 关闭，压缩落在两次尝试之间，下一个编号请求从替换表层重建。 + +## 考虑过的替代方案 + +- **保留临时 pre-step 压力并增加更多参数**——不予采纳，因为后续路由与请求变换仍在更早快照之外，同时通用生命周期会耦合到单个插件。 +- **重试相同编号的 step**——不予采纳，因为恢复会在失败边界之后追加持久事件。新 step 保持边界配对与可重建性。 +- **只要 `compactIfNeeded` 返回结果就重试**——不予采纳，因为自定义后端可能报告成功却没有改变模型可见状态。`replaceGeneration` 才是权威证明。 +- **让 compact-basic 解析提供方措辞**——不予采纳，因为分类属于适配器，而且必须同时覆盖抛出式与带内交付。 +- **恢复时使用通用模型/窗口回退**——不予采纳，因为基于错误上下文容量执行破坏性策略可能掩盖原始提供方失败。未知持久路由会原样委托。 + +## 后果 + +压力现在描述实际完成的路由请求，包括持久工具结果与仅请求前缀字段，而不是对下一次调用的临时猜测。当成功 usage 锚点不存在时，规范化溢出提供兜底路径。恢复有上限、受取消所有，并保持单调：只有模型可见的表层 generation 变化后才重试。 + +代价是成功 step 增加一个串行检查点，并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况，也不能拆分单个不可分割的超大消息或工具单元。 + +本 RFC 只取代[压缩能力接缝 RFC](../feature/2026-06-18-compaction-capability-seam.md) 中的 pre-step 自动触发部分。服务拆分、独立 token meter、平衡范围契约、日志记录锁、摘要替换与唯一 `summarize()` 子类 hook 均保持不变。 diff --git a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml index 99c6301f25..dbb2a582f0 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-15-replay-token-meter-service.md: 4452c151e122c4a4ad72e3f0bc2616cd2fa28b9d -2026-07-15-replay-token-meter-service.zh.md: 23edc11ffd19b9cfaeb794f3608e9ced4e7dbb7b +2026-07-15-replay-token-meter-service.md: 079eb58f38a69a40b3f47a23e72d159f7025d285 +2026-07-15-replay-token-meter-service.zh.md: 7c3c9ce47ad81029b03747ddb17b87dc55def8e7 diff --git a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md index 4452c151e1..079eb58f38 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md +++ b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md @@ -32,13 +32,13 @@ Usage sums the disjoint input, cache-read, cache-write, and output buckets. Reas `dsh-compact-basic` requires `ctx.tokenMeter`; `CompactService` gains no token methods or types. The backend is factored into configuration, automatic triggering, region transaction, and summarizer modules, while `summarize()` remains its sole subclass hook. The conversation model's meter consistently prices pressure, retention, shadowed content, provenance, and non-shrinking-summary rejection. -Every metered model receives a compact policy with defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, summarization model `''`, maximum summary output `8192`, one extra compaction attempt, and automatic triggering enabled. Per-model compact overrides merge `thresholdRatio` and `retainTokens`; retention must remain below the resulting threshold. Empty summarization model resolves the latest logged routed model, then `AgentOptions.model`. +Every metered model receives a compact policy with defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, summarization model `''`, maximum summary output `8192`, one extra pressure-compaction attempt, one context-overflow retry, and automatic triggering enabled. Per-model compact overrides merge `thresholdRatio` and `retainTokens`; retention must remain below the resulting threshold. Empty summarization model resolves the latest logged routed model, then `AgentOptions.model`. -The pre-step trigger measures a provisional envelope: the current prompt and prefix override logged values, while the latest logged header supplies model, tools, and other call config. A model-less router-only agent skips that provisional check because `agent/request` can route later; naming an unknown model remains an error. +Automatic pressure runs at `agent/post-step` and measures the canonical durable envelope under the model actually selected by `agent/request`. A headerless session has no completed routed request to assess and produces no work; a durable unknown routed model remains an exact typed error. Canonical overflow recovery uses the same meter for forced range selection, and retries only after a proven surface replacement. ## Testing -Unit coverage pins profiles, field-wise overrides, custom and unknown models, envelope invalidation, model switching, usage and missing-usage paths, seeded append/replace replay, signed deltas, provenance modes, malformed boundaries, immutable snapshots, listener ordering, reload, compact defaults, routing fallback, retention, convergence, and transaction rollback. A real Loader/Include YAML fixture loads the exact zero-config token-meter and compact-basic package names in dependency order. +Unit coverage pins profiles, field-wise overrides, custom and unknown models, envelope invalidation, model switching, usage and missing-usage paths, seeded append/replace replay, signed deltas, provenance modes, malformed boundaries, immutable snapshots, listener ordering, reload, compact defaults, actual routing, retention, convergence, forced overflow, and transaction rollback. A real Loader/Include YAML fixture loads the exact zero-config token-meter and compact-basic package names in dependency order. ## Alternatives considered @@ -54,4 +54,4 @@ Unit coverage pins profiles, field-wise overrides, custom and unknown models, en - Defaults make the bundled DeepSeek composition usable with two zero-config plugin entries, while custom models must state the one fact that cannot be guessed safely: context capacity. - Heuristic density and provider usage remain estimates of provider behavior. Maintainers must update built-in profiles and overflow wording as models evolve. - Measurements fail loudly on malformed durable boundaries. This turns corrupted replay into a named integration failure instead of silently drifting pressure. -- The pre-step compact integration can skip a router-only first check and can miss tool or routing changes applied later in request middleware. +- Post-step pressure reads the exact logged routing/tools/prefix boundary; provider overflow classification remains the adapter-maintained backstop for requests rejected before a successful usage anchor. diff --git a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md index 23edc11ffd..7c3c9ce47a 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md @@ -32,13 +32,13 @@ Usage 会对互不重叠的输入、缓存读取、缓存写入与输出 bucket `dsh-compact-basic` 要求 `ctx.tokenMeter`；`CompactService` 不增加 token 方法或类型。后端拆分为配置、自动触发、区域事务与摘要器模块，而 `summarize()` 仍是唯一的子类 hook。会话模型的 meter 一致用于压力、保留、被遮蔽内容、来源以及非缩小摘要拒绝。 -每个已计量模型都会获得默认压缩策略：阈值比例 `0.8`、保留尾部 `floor(contextWindow × 0.16)`、摘要模型 `''`、摘要最大输出 `8192`、一次额外压缩尝试，以及启用自动触发。逐模型压缩覆盖按字段合并 `thresholdRatio` 与 `retainTokens`；保留值必须小于最终阈值。空摘要模型先解析最近记录的实际路由模型，再使用 `AgentOptions.model`。 +每个已计量模型都会获得默认压缩策略：阈值比例 `0.8`、保留尾部 `floor(contextWindow × 0.16)`、摘要模型 `''`、摘要最大输出 `8192`、一次额外压力压缩尝试、一次上下文溢出重试，以及启用自动触发。逐模型压缩覆盖按字段合并 `thresholdRatio` 与 `retainTokens`；保留值必须小于最终阈值。空摘要模型先解析最近记录的实际路由模型，再使用 `AgentOptions.model`。 -pre-step 触发器计量临时请求信封：当前提示词与前缀覆盖日志值，最近记录的请求头提供模型、工具及其他调用配置。没有模型的纯路由 agent 会跳过该临时检查，因为 `agent/request` 仍可稍后路由；显式命名未知模型仍然报错。 +自动压力检查运行在 `agent/post-step`，并使用 `agent/request` 实际选择的模型计量规范持久信封。没有请求头的会话尚无已完成的路由请求可供判断，因此不执行工作；持久记录的未知路由模型仍抛出带精确名称的类型化错误。规范化溢出恢复使用同一 meter 强制选择范围，并且只有在表层替换得到证明后才重试。 ## 测试 -单元覆盖固定 profile、按字段覆盖、自定义与未知模型、信封失效、模型切换、有无 usage 的路径、种子追加/替换重放、有符号增量、来源模式、畸形边界、不可变快照、监听器顺序、重载、压缩默认值、路由回退、保留、收敛与事务回滚。真实 Loader/Include YAML fixture 按依赖顺序加载精确的零配置 token-meter 与 compact-basic package 名称。 +单元覆盖固定 profile、按字段覆盖、自定义与未知模型、信封失效、模型切换、有无 usage 的路径、种子追加/替换重放、有符号增量、来源模式、畸形边界、不可变快照、监听器顺序、重载、压缩默认值、实际路由、保留、收敛、强制溢出与事务回滚。真实 Loader/Include YAML fixture 按依赖顺序加载精确的零配置 token-meter 与 compact-basic package 名称。 ## 考虑过的替代方案 @@ -54,4 +54,4 @@ pre-step 触发器计量临时请求信封：当前提示词与前缀覆盖日 - 默认值让内置 DeepSeek 组合只需两个零配置插件条目即可使用，而自定义模型必须声明唯一不能安全猜测的事实：上下文容量。 - 启发式密度与提供方 usage 仍然只是提供方行为的估计。随着模型演进，维护者必须更新内置 profile 与溢出措辞。 - 遇到畸形持久边界时，计量会明确失败。这会把损坏的重放转化为具名集成错误，而不是让压力静默漂移。 -- pre-step 压缩集成可能跳过纯路由的首次检查，也可能错过请求中间件稍后应用的工具或路由变化。 +- post-step 压力检查读取精确记录的路由、工具与前缀边界；对于在成功 usage 锚点出现前就被拒绝的请求，提供方溢出分类仍是由适配器维护的兜底路径。 diff --git a/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md b/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md index 42b93f1225..7714db14b7 100644 --- a/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md +++ b/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md @@ -17,7 +17,7 @@ Two forces shape the design. First, compaction policy and reusable token measure Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently: 1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*. -2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that consumes `ctx.tokenMeter` and owns the tail→head retention walk, summarization via `ctx.llm.stream()`, the surface replacement, the lock, and the `agent/pre-step` auto-compaction listener. `summarize()` is its sole subclass hook; pricing and replay stay with the meter. +2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that consumes `ctx.tokenMeter` and owns the tail→head retention walk, summarization via `ctx.llm.stream()`, the surface replacement, the lock, post-step pressure, and canonical context-overflow recovery. `summarize()` is its sole subclass hook; pricing and replay stay with the meter. 3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first. ### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation @@ -30,27 +30,27 @@ This is not a coupling smell — it is the contract's domain. The "only cordis" An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface. That recouples the contract to one strategy: a backend that wants a different retention policy or event sequence would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend and keeps the interface a statement of *what*. Token measurement is not a compaction hook at all; the standalone service lets multiple consumers share one model/session replay fold. -`compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` takes required pressure inputs and cancellation. The session comes from the agent. `compactRegion(session, start, end, agent, signal?)` keeps an optional signal for manual callers and requires `session === agent.session`; implementations reject mismatch before model resolution, lock acquisition, summarization, or log mutation. The pre-step integration resolves a provisional model from the latest logged request header, then `AgentOptions.model`; a model-less router-only first step skips pressure because `agent/request` can route later. The default summarizer resolves its model from explicit config, the latest logged routed model, then agent options. +`compactIfNeeded(agent, trigger, signal)` takes an explicit `'pressure' | 'context-overflow'` trigger and cancellation. It resolves only the latest durable routed request model; no header means no work, while a named unconfigured model produces the token meter's exact typed error. `compactRegion(session, start, end, agent, signal?)` keeps an optional signal for manual callers and requires `session === agent.session`; implementations reject mismatch before model resolution, lock acquisition, summarization, or log mutation. The default summarizer resolves its model from explicit config, the latest logged routed model, then agent options, and records the model after any `llm/stream` routing. -### Auto-compaction runs on `agent/pre-step`, a dedicated surface-mutation seam +### Automatic pressure runs after successful durable step work -Compaction mutates the session surface, so it runs before the step opens and before messages are derived. `agent/request` remains a call-config transform and never needs to rebuild history after a surface change. +The original pre-step placement used a provisional envelope and could not see final `agent/request` routing, tools, provider output, tool results, buffered context, or steering. The corrected lifecycle fires serial `agent/post-step(agent, turn, step, signal)` after those successful facts are durable and before `step/end`. `dsh-compact-basic` measures the canonical logged request through `ctx.tokenMeter`, so the next request sees any replacement without a speculative envelope override. -The fix is a dedicated loop seam, **`agent/pre-step`** (`@mode serial`), fired by the loop *after* system assembly and *before* the step opens (`step/start`): +Canonical provider context overflow takes a separate path. The failed step closes, `agent/request-error` receives the original request error and consecutive retry count, and compact-basic forces one useful balanced reduction. It returns retry only if `session.surface.replaceGeneration` increases; the loop then opens a new numbered step and reconstructs its request from the durable log. No range, no replacement, recovery failure, cancellation, an exhausted cap, or an unrelated error preserves the original provider failure. The complete lifecycle decision is in the [after-call recovery RFC](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md). ``` -assembly = ctx.systemPrompt.assemble() -await ctx.serial('agent/pre-step', agent, turn, step, system, prefix, signal) ⟵ compaction mutates the surface here -session('step/start') ⟵ the step opens AFTER the seam -messages = session.deriveMessages() ⟵ single derive, reflects the compaction -request = waterfall agent/request ⟵ pure request transform (hooks, model switch) -``` +assistant/message → tool/result/context/steering +await serial agent/post-step ⟵ pressure compaction inside the successful step +step/end -The loop derives messages once after `agent/pre-step`. Running before `step/start` keeps compaction records outside any half-open step, simplifying crash repair. The seam is awaited and serial so surface mutations cannot interleave; listeners return `void` and do not use Cordis bail values as vetoes. +provider overflow → step/end +await waterfall agent/request-error ⟵ forced compaction between attempts +retry → next numbered step/start ⟵ derives from the replacement surface +``` ### Retention is turn-agnostic; tool-pairing balance is the only structural guard -Auto-compaction fires before **every** step, not once per turn. This is **load-bearing for runaway-turn survival**: a tool-heavy ReAct turn appends an `assistant/message` + a `tool/result` per step, so the surface grows *within* a turn. A single turn can grow past the window on its own (a "runaway turn") — and the only moment to rescue it before the next model call overflows is the next step's `pre-step` checkpoint. Gating compaction to a turn's first step (or, worse, retaining the whole in-flight turn verbatim) re-opens exactly the hole compaction exists to close: the harness would die when compaction is most needed. +Auto-compaction checks after **every successful** step, not once per turn. This is load-bearing for runaway-turn survival: a tool-heavy ReAct turn appends an `assistant/message` + a `tool/result` per step, so the surface grows within a turn. The post-step check can compact early closed tool pairs before continuation opens the next step, and provider-confirmed overflow remains the backstop when a request crosses the limit first. `compactIfNeeded` retains the smallest tail of whole surface units whose estimated size reaches `retainTokens` and compacts older nodes. A unit is a complete closed step or one no-step message. If the token cutoff lands inside a step, retention expands until the cut is tool-pairing balanced. Balance is checked on surface order, not log sequence, because replacement summaries have new sequence numbers at old surface positions. `dsh-compact` exports the before/after edge helpers; their per-session cache folds only appended surface-tail nodes while `replaceGeneration` is unchanged, does no event reads for log-only growth, and rebuilds current membership, positional successors, and balances after replacement. `compactRegion` rejects boundaries that split a tool call from its result. The in-flight turn receives no special retention. @@ -64,7 +64,7 @@ Auto-compaction always starts at the surface head, merging the prior checkpoint ### Approximate convergence invariant -`resolveConfig` supplies usable common defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, empty summarization-model override, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`. Optional per-model threshold/retention fields merge over those defaults and must name a configured meter profile; retained tokens must be below the resulting threshold. Convergence remains dynamic because provider output caps can be spent on hidden or surfaced reasoning tokens and summary size is unpredictable. If the compacted surface remains over threshold, `compactIfNeeded()` re-compacts the head checkpoint up to the configured retry count, but each committed summary must be smaller than what it shadows. +`resolveConfig` supplies usable common defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, empty summarization-model override, `maxTokens: 8192`, `compactionRetries: 1`, `maxOverflowRetries: 1`, and `auto: true`. Optional per-model threshold/retention fields merge over those defaults and must name a configured meter profile; retained tokens must be below the resulting threshold. Convergence remains dynamic because provider output caps can be spent on hidden or surfaced reasoning tokens and summary size is unpredictable. If pressure remains over threshold, `compactIfNeeded()` re-compacts the head checkpoint up to the configured retry count, but each committed summary must be smaller than what it shadows. Overflow bypasses threshold and retained-tail policy for one maximal balanced head reduction, leaving the newest indivisible unit. ### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary @@ -90,12 +90,12 @@ The basic backend wraps the summary as established checkpoint context and tags i The `compact/start … compact/end` bracket is justified, in order of what now does the work: 1. **Crash-detectable orphan + provenance** (primary). Summarization is a slow model call persisted *after* `compact/start`. A crash mid-summarization leaves a `compact/start` with no matching `compact/end` — a detectable orphan. Releasing the lock last (rather than first) converts the crash window from *silent corruption* into that detectable orphan. -2. **Prevents concurrent compaction.** `compactRegion` refuses to start if the current turn holds an unmatched `compact/start`. (The loop is single-threaded across the awaited `pre-step`, so this is also a re-entry tripwire — a thrown "already in progress" signals a real bug.) +2. **Prevents concurrent compaction.** `compactRegion` refuses to start if the current turn holds an unmatched `compact/start`. (The loop is single-threaded across either awaited automatic seam, so this is also a re-entry tripwire — a thrown "already in progress" signals a real bug.) Two failure paths, both documented: -- **Crash** (the loop dies mid-summarization): a dangling `compact/start`, no closer. Because `compact/*` are **log-only**, the orphan is **inert** — the surface replacement never landed, so the full, uncompacted history derives correctly. Generic turn-repair (`interruptedTurnClosers`) closes the turn with a synthetic `turn/end`; the orphan sits *before* that `turn/end`, so the turn-scoped in-progress check never sees it and a crash can't wedge future compaction. Compaction simply re-attempts at the next `pre-step`. -- **Recoverable** (summarization throws but the loop survives): the backend appends `compact/end` with its **`error`** field set, leaving the surface untouched, and the model call proceeds with full history. +- **Crash** (the loop dies mid-summarization): a dangling `compact/start`, no closer. Because `compact/*` are **log-only**, the orphan is **inert** — the surface replacement never landed, so the full, uncompacted history derives correctly. Generic turn-repair (`interruptedTurnClosers`) closes the turn with a synthetic `turn/end`; the orphan sits *before* that `turn/end`, so the turn-scoped in-progress check never sees it and a crash cannot wedge future compaction. +- **Recoverable** (summarization throws but the loop survives): the backend appends `compact/end` with its **`error`** field set and leaves the surface untouched. Post-step pressure warns and continues; overflow recovery delegates so the original provider error remains authoritative. `compact/end` keeps its `error?` field (mirroring `tool/result`'s self-contained error — one event tells success from failure without correlating a sibling). There is no separate `compact/error` event. @@ -104,14 +104,14 @@ Two failure paths, both documented: ## Alternatives considered - **The full algorithm as concrete interface methods** — rejected because it recouples the contract to one retention strategy. Both core methods are abstract; reusable measurement is a separate LLM-family service and `summarize()` is basic's sole hook. -- **Compaction on the `agent/request` waterfall** — the earlier cut; rejected for the double-derive it forced and for handing the listener context it structurally cannot compact. The dedicated `agent/pre-step` seam makes the layering correct by construction. +- **Compaction on `agent/request` or provisional `agent/pre-step` inputs** — rejected because neither proves the final durable request and both couple generic lifecycle to compaction-specific envelope data. Post-step replay plus canonical overflow recovery covers both successful and rejected calls. - **A separate `compact/error` event** — rejected: `compact/end` keeps an `error?` field, mirroring `tool/result`'s self-contained error — one event tells success from failure without correlating a sibling. - **Teaching core turn-repair about `compact/*`** — rejected: the log-only orphan is inert, and a core module patched for every future `xxx/start … xxx/end` plugin pair is exactly the coupling the capability-seam architecture exists to avoid. ## Consequences - **Packages**: `packages/compact/compact` supplies the interface and `compact-basic` supplies the backend. `packages/llm/token-meter` owns replay-aware measurement independently. The consumer tier is deferred. -- **New loop seam**: `agent/pre-step` (`@mode serial`) declared in `dsh-agent` and emitted by `dsh-agent-loop` after system assembly and before `step/start`. This is a documented change to the loop — `docs/architecture.md` records it and the generated cordis catalog carries its signature. +- **Automatic seams**: `agent/post-step` (`@mode serial`) handles successful-call pressure and `agent/request-error` (`@mode waterfall`) handles final request failures after the failed step closes. Generic `agent/pre-step` remains a four-argument checkpoint with no compaction-only prompt/prefix payload. - **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry. - **`dsh-compact`** owns `toolPairingBalancedBefore(session, node)` and `toolPairingBalancedAfter(session, node)`, the cached surface-edge checks that `compactRegion` and `compactIfNeeded` use to avoid splitting a tool-call/result pair. The cache validates current membership by seq and resolves after-edges from its positional successor map instead of trusting a caller-retained `node.next`; stale or missing seqs and orphan results reject. `dsh-session` continues to own the surface `replace` operation, positional nodes, and rewrite generation. - **`dsh-invariants`** drops its `surface replace: start must be <= end` assertion: a head-anchored compaction lands a high-seq replacement node at an older range's *position*, so `start > end` numerically is normal and valid (the range is positional, validated by the surface's `indexOf` checks that remain). The turn-enclosure invariant is reused unchanged. @@ -119,7 +119,7 @@ Two failure paths, both documented: ## Testing -- **Unit:** Real Loader and invariant plugins cover whole-unit retention, convergence failure, both `compact/end` outcomes, head anchoring, open-tail refusal, inert crash orphans, and compacting closed steps inside one oversized open turn. -- **Loop:** Tests pin one awaited `agent/pre-step` per step between `turn/start` and `step/start`; a surface mutation there lands outside the step and appears in the single derived request. +- **Unit:** Real Loader and invariant plugins cover whole-unit retention, convergence failure, both `compact/end` outcomes, head anchoring, open-tail refusal, inert crash orphans, forced below-threshold overflow, generation proof, caps, and original-error preservation. +- **Loop:** Tests pin post-step after durable tool results and before `step/end`, actual `agent/request` routing, closed failed steps, fresh retry numbering, and complete thrown/in-band overflow → compaction → reconstructed retry composition. - **With-key e2e:** A real model and bash session with lowered limits triggers compaction, records a complete `compact/start…end` pair, shrinks the surface, and finishes the task. - **Snapshot gap:** Runaway-turn compaction cannot yet replay because the summarization call records no `assistant/chunk` events or `sessionId`; interleaved summarization-call replay remains follow-up work. diff --git a/docs/rfc/implemented/feature/2026-07-07-session-prefix.md b/docs/rfc/implemented/feature/2026-07-07-session-prefix.md index f0d458368a..e6ecf69a74 100644 --- a/docs/rfc/implemented/feature/2026-07-07-session-prefix.md +++ b/docs/rfc/implemented/feature/2026-07-07-session-prefix.md @@ -16,13 +16,13 @@ Three properties carry the design: - **Request-only, header-logged.** `deriveMessages()` never returns the prefix; its one durable record is `EpochHeader.messagePrefix` on the instance's anchoring `request/header` snapshot — the channel the reconstructable-requests RFC already owns for the request's non-history half, so no new session event exists. The dev invariant ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)) recomputes `messagePrefix + boundary derivation` against every loop-built request; an unlogged prefix cannot reach the wire. - **Frozen per instance.** Reuse is structural, not disciplined: the cached product cannot change mid-session, so the provider's prompt cache holds by construction and the prefix extends the cacheable region at zero marginal cost per step. A process restart or `ctx.agents.resume()` is a new instance: it recomposes, and any drift lands attributably on the `'resume'` header snapshot. This is the routing rule the seam creates: session-frozen openers ride the prefix; content that changes mid-session rides the append-only history channels (`agent.inject()`, a `tools/post-execute` decision's `additionalContext`, prompt-submit `additionalContext` — [the interception-seams RFC](2026-06-30-interception-seams.md)), each a durable `context/message` paid once and prefix-cached thereafter. -- **Composed before the pressure gate.** Composition precedes the instance's first `agent/pre-step`, and the seam hands the composed value through: `agent/pre-step` carries a `sessionPrefix` parameter and `CompactService.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` counts it in its token-pressure estimate — a gate reading the previous instance's folded prefix instead would under-gate a resumed or forked instance whose contributor grew, skipping compaction and shipping an over-window first request. A composition interrupted by a cancel/dispose landing inside the waterfall is discarded, never cached: an abort-aware listener's degraded fallback cannot leak into later requests, and the next turn recomposes under a live signal. +- **Exact in the durable request envelope.** Composition precedes the instance's first `agent/pre-step` and request boundary. The first routed request logs the current prefix on its header, so post-step token pressure reads the exact prefix together with the actual prompt, tools, and routed model; no compaction-only parameter is carried through the generic pre-step seam. A composition interrupted by cancel/dispose is discarded, never cached: an abort-aware listener's degraded fallback cannot leak into later requests, and the next turn recomposes under a live signal. Because composition runs before the boundary snapshot, a composing listener's session append joins the CURRENT request's derived history. Compaction structurally cannot touch the prefix (or the system prompt): it rewrites surface nodes, and header state never enters the surface. ## Testing -[Interception tests](../../../../packages/core/agent-loop/tests/interception.spec.ts) pin compose-once reuse with no header deltas, prepend order, empty-prefix omission, immutability, and composition before pre-step; [cancellation tests](../../../../packages/core/agent-loop/tests/cancel.spec.ts) pin discard and recomposition. Session codec, invariant, and compaction tests cover header round trips, request reconstruction, and prefix-aware pressure accounting. Snapshot normalization preserves prefix counts, while the [pinned-header scenario](../testing/2026-07-06-pin-request-header-content-in-one-scenario.md) owns content and the default example remains prefix-free. No prefix-specific e2e is needed because the seam is deterministic and provider-independent; the with-key [request-cache e2e](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts) covers its cache economics. +[Interception tests](../../../../packages/core/agent-loop/tests/interception.spec.ts) pin compose-once reuse with no header deltas, prepend order, empty-prefix omission, immutability, composition before pre-step, and the prefix on the routed header; [cancellation tests](../../../../packages/core/agent-loop/tests/cancel.spec.ts) pin discard and recomposition. Session codec, invariant, token-meter, and compaction tests cover header round trips, request reconstruction, and durable prefix-aware pressure accounting. Snapshot normalization preserves prefix counts, while the [pinned-header scenario](../testing/2026-07-06-pin-request-header-content-in-one-scenario.md) owns content and the default example remains prefix-free. No prefix-specific e2e is needed because the seam is deterministic and provider-independent; the with-key [request-cache e2e](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts) covers its cache economics. ## Alternatives considered @@ -30,12 +30,12 @@ Because composition runs before the boundary snapshot, a composing listener's se - **A system-prompt section** (`system-prompt/assemble`) — rejected for this content: the assembly renders to the single `system` string, so message-shaped openers do not fit, and the system prompt is deliberately re-assembled per step (with header deltas when it changes) while the opener wants instance-frozen semantics. - **A durable history opener** (`inject()` at session start) — rejected: permanent history is the failure mode in the problem statement — replayed everywhere, compactable, stale across resumes. - **Compose per turn instead of per instance** — rejected: a turn-boundary recompose either desyncs silently from the log or forces a header delta per change, and it busts the provider cache exactly as often as it fires; the legitimate refresh point is the instance boundary, where the `'resume'` snapshot already records drift attributably. -- **Compose lazily at the first request and let compaction read the folded header** (the shape as first merged) — superseded in review: the fold matches the live prefix only from the instance's second request on, so on a resumed/forked instance's first step the pressure gate read the PREVIOUS instance's prefix and could under-gate. Composing before the first pre-step and handing the live value through the seam makes the estimate exact at every step. +- **Carry prompt/prefix through `agent/pre-step` for provisional pressure** — superseded by post-step replay. It coupled a generic lifecycle seam to one consumer and still missed later request routing/tools; the routed header is the exact durable home for all request-envelope fields. - **A dedicated session event carrying the prefix** — rejected: the header events are the request's non-history record by design; a second event would be a second home for the same fact and another codec to keep total. ## Consequences -- `agent/pre-step` and `CompactService.compactIfNeeded` carry a `sessionPrefix` parameter: every pre-step listener and compaction backend sees the real per-instance value (all in-repo implementations updated in the same change, per the pre-release stance). +- `agent/pre-step` stays a generic `(agent, turn, step, signal)` checkpoint. Compaction receives no prefix parameter; `ctx.tokenMeter` folds the prefix from the canonical routed header at post-step. - A contributor whose content changes mid-session is not re-read until the next instance — by design. A deployment needing mid-session catalog updates routes the change notice through the append-only history channels and pays one durable `context/message`. - The dropped `after` slot leaves no request-only channel near the request tail; nothing in the repo needs one, and adding it back would re-open the every-step re-pay cost the design exists to avoid. - The `request/header-delta` `messagePrefix` arm (whole-array replacement, empty array encoding transition to absence) exists for codec totality; the loop never exercises it, because the cached prefix cannot change within an instance. diff --git a/examples/coding-agent/cordis.yml b/examples/coding-agent/cordis.yml index 1208480388..281ae3e9d1 100644 --- a/examples/coding-agent/cordis.yml +++ b/examples/coding-agent/cordis.yml @@ -49,8 +49,8 @@ - id: token-meter name: '@deepseek-ai/dsh-token-meter' -# Summarize an older range when measured history approaches the context window. -# Built-in model policies provide the ordinary threshold and retained-tail defaults. +# Summarize an older range after measured pressure or a canonical provider overflow. +# Built-in policies provide pressure, retention, and one overflow-retry default. - id: compact-basic name: '@deepseek-ai/dsh-compact-basic' diff --git a/packages/compact/compact-basic/README.md b/packages/compact/compact-basic/README.md index c9264e2e99..b3b910cfb7 100644 --- a/packages/compact/compact-basic/README.md +++ b/packages/compact/compact-basic/README.md @@ -8,13 +8,14 @@ This is the implementation tier of the compaction capability — see the [interf This backend owns the compaction policy: -- **Measurement** — the effective conversation model's `ModelTokenMeter` prices the provisional request envelope and current surface at one consumed-log revision. The current prompt and prefix override their logged values; the pre-step boundary reuses logged tools and call config. +- **Measurement** — the latest durable routed request model's `ModelTokenMeter` prices the canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering. - **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope. - **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold. - **Summarization** — a direct `llm/stream` call uses the configured model and cap without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call. - **Framing** — the replacement user message marks established checkpoint context with `` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint. -- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step, outside an open step, so a tool-heavy turn remains compactable and the loop derives history once after mutation. -- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Recoverable failure records an error end and leaves the surface unchanged. +- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes. +- **Overflow recovery** — below-threshold overflow bypasses normal retention and attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized only when `surface.replaceGeneration` advances; no range, no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure. +- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Operational post-step failures warn and continue, while an actually routed model without a meter profile fails the otherwise-successful turn with the typed meter error. `summarize()` is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on the conversation model's meter. The hook returns the summary blocks together with the call envelope it used (`{ summary, model, maxTokens? }`), which is logged on `compact/summary`. @@ -29,7 +30,8 @@ Every common setting is optional. Every model known to `ctx.tokenMeter` receives | `summarizationModel` | no (default `''`) | Empty resolves the latest logged routed model, then `AgentOptions.model`. | | `maxTokens` | no (default `8192`) | Provider generation cap for the summarization call; may include reasoning tokens. | | `compactionRetries` | no (default `1`) | Extra attempts after the first when pressure remains above threshold. | -| `auto` | no (default `true`) | Register the `agent/pre-step` automatic listener. Set `false` for manual-only. | +| `maxOverflowRetries` | no (default `1`) | Maximum retries after canonical context-window overflow; `0` disables recovery only. | +| `auto` | no (default `true`) | Register post-step pressure and overflow-recovery listeners. Set `false` for manual-only. | ## Usage @@ -39,7 +41,7 @@ import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic' import TokenMeterService from '@deepseek-ai/dsh-token-meter' export const name = 'compact-basic' -export const inject = ['llm'] +export const inject = ['llm', 'tokenMeter'] export function apply(ctx: Context): void { ctx.plugin(TokenMeterService) @@ -53,7 +55,7 @@ Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it c ### Conversation history -**What the model sees**: Before a step whose estimated envelope and history exceed the threshold, the conversation model receives the checkpoint preamble below, a blank line, ``, the data-dependent summary, and ``. This one checkpoint replaces the selected older range and is followed by the retained recent units. +**What the model sees**: After a successful step crosses the threshold, the next request receives the checkpoint preamble below, a blank line, ``, the data-dependent summary, and ``. Overflow recovery rebuilds the immediate retry from that replacement. This one checkpoint replaces the selected older range and is followed by the retained recent units. **Token effect**: The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget. @@ -115,8 +117,9 @@ Rules: ## Known Limitations and Deferred Work -- **Pre-step sees a provisional request envelope** — the current prompt and prefix are exact, but routing and tool changes made later in `agent/request` are not logged yet. A router-only agent with no provisional model skips that check. - **Meter accuracy follows the selected profile** — missing provider usage falls back to the token meter's configured character density and structural overhead. +- **Overflow classification is adapter-maintained** — provider wording can change; both DeepSeek adapters normalize currently recognized context-limit failures to `CONTEXT_WINDOW_EXCEEDED`. +- **Single-unit and envelope-only overflow remain outside surface compaction** — recovery cannot split one indivisible message/tool unit or shrink system/tools/prefix. - **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting. - **Summarization failure fails closed with full, over-budget history** — including truncation at the summarization `maxTokens`, which hidden reasoning tokens can consume; the auto path logs a warning and proceeds. - **The summarization call has no transcript-snapshot coverage** — `dsh-llm-replay` derives calls from `assistant/chunk` events, so this chunk-less direct `ctx.llm.stream()` call cannot replay (named deferred replay infrastructure in [the seam RFC](../../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)). diff --git a/packages/compact/compact-basic/src/automatic.ts b/packages/compact/compact-basic/src/automatic.ts index 504b0d8a9f..16e71c081a 100644 --- a/packages/compact/compact-basic/src/automatic.ts +++ b/packages/compact/compact-basic/src/automatic.ts @@ -1,12 +1,12 @@ /** - * Automatic pre-step pressure listener for compact-basic. + * Automatic post-step pressure and context-overflow recovery listeners. * * @module @deepseek-ai/dsh-compact-basic/automatic */ import type { Context } from 'cordis' -import type { CompactionResult } from '@deepseek-ai/dsh-compact' -import type { Message } from '@deepseek-ai/dsh-llm' +import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' +import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' import { TOKEN_METER_MODEL_UNCONFIGURED, TokenMeterError, @@ -14,10 +14,10 @@ import { import type { Agent } from '@deepseek-ai/dsh-agent' interface AutomaticCompactor { + readonly config: { readonly maxOverflowRetries: number } compactIfNeeded( agent: Agent, - fullSystemPrompt: string, - sessionPrefix: readonly Message[], + trigger: CompactionTrigger, signal: AbortSignal, ): Promise } @@ -31,30 +31,54 @@ export function registerAutomaticCompaction( ctx: Context, service: AutomaticCompactor, ): void { - ctx.on('agent/pre-step', async ( + const logResult = (result: CompactionResult, trigger: string): void => { + ctx.logger.info( + `compaction (${trigger}): shadowed ${result.shadowedSeqs.length} surface nodes ` + + `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, ` + + `~${result.shadowedTokenCount} tokens)`, + ) + } + + ctx.on('agent/post-step', async ( agent: Agent, _turn: number, _step: number, - fullSystemPrompt: string, - sessionPrefix: readonly Message[], signal: AbortSignal, ) => { try { - const result = await service.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal) - if (result !== null) { - ctx.logger.info( - `compaction: shadowed ${result.shadowedSeqs.length} surface nodes ` - + `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, ` - + `~${result.shadowedTokenCount} tokens)`, - ) - } + const result = await service.compactIfNeeded(agent, 'pressure', signal) + if (result !== null) logResult(result, 'post-step pressure') } catch (error: unknown) { // A named routed model without a meter profile is configuration failure, // not an optional operational compaction miss. if (error instanceof TokenMeterError && error.code === TOKEN_METER_MODEL_UNCONFIGURED) throw error const message = error instanceof Error ? error.message : String(error) - ctx.logger.warn(`compaction failed: ${message}; proceeding with full history`) + ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`) } }) + + ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => { + if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE + || retryAttempt >= service.config.maxOverflowRetries + || signal.aborted) return next() + + let generation: number + let result: CompactionResult | null + try { + generation = agent.session.surface.replaceGeneration + result = await service.compactIfNeeded(agent, 'context-overflow', signal) + } catch (recoveryError: unknown) { + const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError) + ctx.logger.warn( + `context-overflow compaction failed: ${message}; preserving the original request error`, + ) + return next() + } + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited. + if (signal.aborted || result === null + || agent.session.surface.replaceGeneration <= generation) return next() + logResult(result, 'context overflow recovery') + return { action: 'retry' } + }) } diff --git a/packages/compact/compact-basic/src/config.ts b/packages/compact/compact-basic/src/config.ts index 3169d7f5aa..e4d567da3b 100644 --- a/packages/compact/compact-basic/src/config.ts +++ b/packages/compact/compact-basic/src/config.ts @@ -47,6 +47,7 @@ export function resolveConfig( summarizationModel: '', maxTokens: 8192, compactionRetries: 1, + maxOverflowRetries: 1, auto: true, }, meter) } @@ -56,10 +57,12 @@ export function resolveConfig( summarizationModel: config.summarizationModel ?? '', maxTokens: config.maxTokens ?? 8192, compactionRetries: config.compactionRetries ?? 1, + maxOverflowRetries: config.maxOverflowRetries ?? 1, auto: config.auto ?? true, } assertPositiveInteger('maxTokens', resolved.maxTokens) assertNonNegativeInteger('compactionRetries', resolved.compactionRetries) + assertNonNegativeInteger('maxOverflowRetries', resolved.maxOverflowRetries) if (typeof resolved.summarizationModel !== 'string') { throw new Error('BasicCompactConfig: summarizationModel must be a string') } diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index cd4ea5d1d5..d420851166 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -7,10 +7,9 @@ import { Context } from 'cordis' import z from 'schemastery' import { CompactService } from '@deepseek-ai/dsh-compact' -import type { CompactionResult } from '@deepseek-ai/dsh-compact' -import { canonicalHeader } from '@deepseek-ai/dsh-session' -import type { EpochHeader, Session } from '@deepseek-ai/dsh-session' -import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm' +import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' +import type { Session } from '@deepseek-ai/dsh-session' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { ModelTokenMeter } from '@deepseek-ai/dsh-token-meter' import type { Agent } from '@deepseek-ai/dsh-agent' import { registerAutomaticCompaction } from './automatic.ts' @@ -36,24 +35,10 @@ function effectiveModel(agent: Agent): string | undefined { return agent.session.requestHeader()?.config.model ?? agent.options.model } -/** - * Build the provisional pre-step request envelope. Prompt and prefix are exact; - * tools and non-model call config come from the latest logged request because - * later request middleware has not run yet. - */ -function provisionalHeader( - model: string, - session: Session, - fullSystemPrompt: string, - sessionPrefix: readonly Message[], -): EpochHeader { - const latest = session.requestHeader() - return canonicalHeader({ - config: latest === undefined ? { model } : { ...latest.config, model }, - ...fullSystemPrompt.length === 0 ? {} : { system: fullSystemPrompt }, - ...latest?.tools === undefined ? {} : { tools: latest.tools }, - ...sessionPrefix.length === 0 ? {} : { messagePrefix: [...sessionPrefix] }, - }) +/** Resolve the exact model durably routed for the latest provider request. */ +function routedModel(session: Session): string | undefined { + const model = session.requestHeader()?.config.model + return model === undefined || model.length === 0 ? undefined : model } /** @@ -75,6 +60,7 @@ export class BasicCompactService extends CompactService { summarizationModel: z.string().default(''), maxTokens: z.number().step(1).min(1).default(8192), compactionRetries: z.number().step(1).min(0).default(1), + maxOverflowRetries: z.number().step(1).min(0).default(1), auto: z.boolean().default(true), }) @@ -106,29 +92,33 @@ export class BasicCompactService extends CompactService { } /** - * Check replayed pressure for the provisional pre-step envelope and compact - * a tool-balanced head until it falls below the effective model threshold. - * A genuinely model-less router-first step skips this provisional check; - * naming an unconfigured model throws the token meter's typed error. - * @param agent - agent whose session and provisional model are measured. - * @param fullSystemPrompt - current assembled system prompt override. - * @param sessionPrefix - current request-only prefix override. - * @param signal - live step cancellation signal forwarded to summarization. + * Compact for replayed post-step pressure or one provider-confirmed context + * overflow. Both triggers price the latest durable routed request model; + * overflow bypasses the normal threshold and retained-tail policy so it can + * force one useful balanced reduction. + * @param agent - agent whose latest durable routed request is measured. + * @param trigger - normal post-step pressure or context-overflow recovery. + * @param signal - live turn cancellation signal forwarded to summarization. * @returns the latest compaction result, or `null` when no check/work applies. */ override async compactIfNeeded( agent: Agent, - fullSystemPrompt: string, - sessionPrefix: readonly Message[], + trigger: CompactionTrigger, signal: AbortSignal, ): Promise { - const model = effectiveModel(agent) - if (model === undefined || model.length === 0) return null + const model = routedModel(agent.session) + if (model === undefined) return null const meter = this.ctx.tokenMeter.resolve(model) const policy = this._modelConfig(meter) - const requestHeader = provisionalHeader(model, agent.session, fullSystemPrompt, sessionPrefix) + if (trigger === 'context-overflow') { + const surface = meter.measureSurface(agent.session) + const range = selectCompactableRange(agent.session, surface, 0) + if (range === null) return null + return this.compactRegion(agent.session, range.start, range.end, agent, signal) + } + const threshold = Math.floor(policy.contextWindow * policy.thresholdRatio) - let measurement = meter.measure(agent.session, requestHeader) + let measurement = meter.measure(agent.session) if (measurement.totalTokens < threshold) return null let result: CompactionResult | null = null @@ -147,7 +137,7 @@ export class BasicCompactService extends CompactService { break } result = await this.compactRegion(agent.session, range.start, range.end, agent, signal) - measurement = meter.measure(agent.session, requestHeader) + measurement = meter.measure(agent.session) if (measurement.totalTokens < threshold) return result } diff --git a/packages/compact/compact-basic/src/summarizer.ts b/packages/compact/compact-basic/src/summarizer.ts index 359421f0f5..78730cc1a6 100644 --- a/packages/compact/compact-basic/src/summarizer.ts +++ b/packages/compact/compact-basic/src/summarizer.ts @@ -106,7 +106,7 @@ export async function summarizeWithLlm( if (!summary.some(block => block.text.trim().length > 0)) { throw new Error('summarization produced no text summary content') } - return { summary, model, maxTokens: config.maxTokens } + return { summary, model: options.model, maxTokens: config.maxTokens } } /** diff --git a/packages/compact/compact-basic/src/types.ts b/packages/compact/compact-basic/src/types.ts index 44ff06435d..4a3f372e08 100644 --- a/packages/compact/compact-basic/src/types.ts +++ b/packages/compact/compact-basic/src/types.ts @@ -22,7 +22,9 @@ export interface BasicCompactConfig { maxTokens?: number /** Extra attempts after the first compaction when pressure remains above threshold. Defaults to `1`. */ compactionRetries?: number - /** Enable the automatic `agent/pre-step` pressure listener. Defaults to `true`. */ + /** Maximum retries after canonical context overflow; `0` disables recovery. Defaults to `1`. */ + maxOverflowRetries?: number + /** Enable automatic post-step pressure and overflow-recovery listeners. Defaults to `true`. */ auto?: boolean } @@ -32,6 +34,7 @@ export interface ResolvedConfig { readonly summarizationModel: string readonly maxTokens: number readonly compactionRetries: number + readonly maxOverflowRetries: number readonly auto: boolean } diff --git a/packages/compact/compact-basic/tests/compact-basic.spec.ts b/packages/compact/compact-basic/tests/compact-basic.spec.ts index 98b4b2f15b..163c2b6b45 100644 --- a/packages/compact/compact-basic/tests/compact-basic.spec.ts +++ b/packages/compact/compact-basic/tests/compact-basic.spec.ts @@ -6,9 +6,10 @@ import BasicCompactService, { } from '@deepseek-ai/dsh-compact-basic' import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic' import { selectCompactableRange } from '@deepseek-ai/dsh-compact-basic/src/region.ts' +import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact' import type { CompactionResult } from '@deepseek-ai/dsh-compact' -import LlmService, { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm' -import type { ContentBlock, GenerateOptions, Message, StreamChunk } from '@deepseek-ai/dsh-llm' +import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { Session, SessionId } from '@deepseek-ai/dsh-session' import TokenMeterService, { TOKEN_METER_MODEL_UNCONFIGURED, @@ -43,6 +44,12 @@ function conversation(turns = 4, text = 'fixture'): Session { source: { kind: 'user' }, }, { surfaceOp: 'append' }) session.append('step/start', { turn, step: 1 }) + if (turn === 1) { + session.append('request/header', { + header: { config: { model: MODEL } }, + reason: 'initial', + }) + } session.append('assistant/message', { turn, step: 1, @@ -68,6 +75,12 @@ function toolConversation(): Session { source: { kind: 'user' }, }, { surfaceOp: 'append' }) session.append('step/start', { turn, step: 1 }) + if (turn === 1) { + session.append('request/header', { + header: { config: { model: MODEL } }, + reason: 'initial', + }) + } session.append('assistant/message', { turn, step: 1, @@ -120,11 +133,10 @@ function service( async function compactIfNeeded( compact: BasicCompactService, session: Session, + trigger: 'pressure' | 'context-overflow' = 'pressure', model: string | undefined = MODEL, - system = '', - prefix: readonly Message[] = [], ): Promise { - return compact.compactIfNeeded(agent(session, model), system, prefix, SIGNAL) + return compact.compactIfNeeded(agent(session, model), trigger, SIGNAL) } describe('compact configuration and defaults', () => { @@ -140,6 +152,7 @@ describe('compact configuration and defaults', () => { summarizationModel: '', maxTokens: 8192, compactionRetries: 1, + maxOverflowRetries: 1, auto: true, }) expect(resolveModelConfig(resolved, ctx.tokenMeter.resolve(MODEL))).toEqual({ @@ -176,6 +189,7 @@ describe('compact configuration and defaults', () => { const bad = [ [{ maxTokens: 0 }, /maxTokens/], [{ compactionRetries: -1 }, /compactionRetries/], + [{ maxOverflowRetries: -1 }, /maxOverflowRetries/], [{ auto: 'yes' }, /auto must be a boolean/], [{ summarizationModel: 1 }, /summarizationModel must be a string/], [{ models: null }, /models must be an object/], @@ -207,19 +221,57 @@ describe('pressure measurement and retention', () => { models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } }, } - it('skips the provisional check only when no routed or fallback model exists', async () => { + it('skips when no durable routed model exists instead of using AgentOptions fallback', async () => { const compact = service(compactConfig) - const session = conversation() - expect(await compact.compactIfNeeded(agent(session), '', [], SIGNAL)).toBeNull() + const session = new Session(SessionId('headerless')) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + await expect(compact.compactIfNeeded(agent(session, MODEL), 'pressure', SIGNAL)) + .resolves.toBeNull() expect(compact.calls).toHaveLength(0) }) it('throws for a named unconfigured model instead of swallowing it', async () => { const compact = service(compactConfig) - await expect(compactIfNeeded(compact, conversation(), 'missing')) + const session = conversation() + session.append('request/header', { + header: { config: { model: 'missing' } }, + reason: 'resume', + }) + await expect(compactIfNeeded(compact, session)) .rejects.toMatchObject({ code: TOKEN_METER_MODEL_UNCONFIGURED, model: 'missing' }) }) + it('declines forced overflow when the whole surface is one indivisible tool pair', async () => { + const compact = service(compactConfig) + const session = new Session(SessionId('single-tool-pair')) + const callId = CallId('single-call') + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + session.append('request/header', { + header: { config: { model: MODEL } }, + reason: 'initial', + }) + session.append('assistant/message', { + turn: 1, + step: 1, + content: [{ type: 'tool-call', id: callId, name: 'read', arguments: '{}' }], + }, { surfaceOp: 'append' }) + session.append('tool/call', { turn: 1, step: 1, callId, name: 'read', arguments: '{}' }) + session.append('tool/result', { + turn: 1, + step: 1, + callId, + content: [{ type: 'text', text: 'result' }], + isError: false, + }, { surfaceOp: 'append' }) + session.append('step/end', { turn: 1, step: 1 }) + const generation = session.surface.replaceGeneration + + await expect(compactIfNeeded(compact, session, 'context-overflow')).resolves.toBeNull() + expect(session.surface.replaceGeneration).toBe(generation) + expect(session.events.some(event => event.type === 'compact/start')).toBe(false) + }) + it('does nothing below threshold and compacts a priced head above threshold', async () => { const compact = service(compactConfig) expect(await compactIfNeeded(compact, conversation(2))).toBeNull() @@ -231,7 +283,7 @@ describe('pressure measurement and retention', () => { expect(session.surface.nodes.length).toBeLessThan(8) }) - it('counts the current prompt and request prefix without putting either on the surface', async () => { + it('counts the durable routed request envelope without putting its prefix on the surface', async () => { const compact = service({ auto: false, models: { [MODEL]: { thresholdRatio: 0.7, retainTokens: 9 } }, @@ -239,11 +291,16 @@ describe('pressure measurement and retention', () => { const session = conversation(2, 'x'.repeat(2_000)) expect(await compactIfNeeded(compact, session)).toBeNull() - const prefix: Message[] = [{ - role: 'user', - content: [{ type: 'text', text: 'p'.repeat(10_000) }], - }] - const result = await compactIfNeeded(compact, session, MODEL, 's'.repeat(5_000), prefix) + const prefix = [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'p'.repeat(10_000) }] }] + session.append('request/header', { + header: { + config: { model: MODEL }, + system: 's'.repeat(5_000), + messagePrefix: prefix, + }, + reason: 'resume', + }) + const result = await compactIfNeeded(compact, session) expect(result).not.toBeNull() expect(prefix).toHaveLength(1) expect(session.events.some(event => event.type === 'context/message')).toBe(false) @@ -264,17 +321,26 @@ describe('pressure measurement and retention', () => { reason: 'initial', }) - const result = await compactIfNeeded(compact, session, 'fallback') + const result = await compactIfNeeded(compact, session, 'pressure', 'fallback') expect(result).not.toBeNull() }) it('declines when envelope pressure is high but the surface has no compactable range', async () => { const compact = service(compactConfig) const empty = new Session(SessionId('empty')) - expect(await compactIfNeeded(compact, empty, MODEL, 'x'.repeat(100_000))).toBeNull() + empty.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + empty.append('request/header', { + header: { config: { model: MODEL }, system: 'x'.repeat(100_000) }, + reason: 'initial', + }) + expect(await compactIfNeeded(compact, empty)).toBeNull() const retained = conversation(1) - expect(await compactIfNeeded(compact, retained, MODEL, 'x'.repeat(100_000))).toBeNull() + retained.append('request/header', { + header: { config: { model: MODEL }, system: 'x'.repeat(100_000) }, + reason: 'resume', + }) + expect(await compactIfNeeded(compact, retained)).toBeNull() }) it('detects scalar/surface revision disagreement', async () => { @@ -587,7 +653,19 @@ describe('compaction region transaction', () => { it('requires a conversation model for pricing', async () => { const compact = service() - const session = conversation(1) + const session = new Session(SessionId('model-less-region')) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { + content: [{ type: 'text', text: 'history' }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) + session.append('step/start', { turn: 1, step: 1 }) + session.append('assistant/message', { + turn: 1, + step: 1, + content: [{ type: 'text', text: 'answer' }], + }, { surfaceOp: 'append' }) + session.append('step/end', { turn: 1, step: 1 }) const nodes = session.surface.nodes await expect(compact.compactRegion( session, @@ -679,6 +757,25 @@ describe('default one-shot summarizer', () => { expect(adapter.lastOptions?.model).toBe('routed') }) + it('records the model actually dispatched after one-shot stream routing', async () => { + const { ctx, compact } = await summarizerHarness([{ type: 'text', text: 'unused' }]) + const routedAdapter = new ScriptedAdapter([{ type: 'text', text: 'routed summary' }]) + ctx.llm.registerAdapter(['routed-summary-model'], routedAdapter) + ctx.on('llm/stream', (options, next) => { + options.model = 'routed-summary-model' + return next() + }) + + const session = conversation(3, 'large history '.repeat(500)) + const nodes = session.surface.nodes + await compact.compactRegion(session, nodes[0]!.seq, nodes[3]!.seq, agent(session, MODEL), SIGNAL) + expect(session.events.findLast(event => event.type === 'compact/summary')?.data).toMatchObject({ + summary: [{ type: 'text', text: 'routed summary' }], + model: 'routed-summary-model', + }) + expect(routedAdapter.lastOptions?.model).toBe('routed-summary-model') + }) + it('fails clearly when no summarization model can be resolved', async () => { const ctx = new Context() await ctx.plugin(LlmService) @@ -717,21 +814,36 @@ describe('default one-shot summarizer', () => { }) describe('automatic listener and loader composition', () => { - function preStep(ctx: Context, owner: Agent): Promise { - return ctx.serial('agent/pre-step', owner, 1, 1, '', [], SIGNAL) + function postStep(ctx: Context, owner: Agent, signal = SIGNAL): Promise { + return ctx.serial('agent/post-step', owner, 1, 1, signal) } - it('compacts above threshold and remains idle below it', async () => { + function recover( + ctx: Context, + owner: Agent, + error: Error & { code?: string }, + retryAttempt = 0, + signal = SIGNAL, + next: () => Promise<{ action: 'fail' | 'retry' }> = () => Promise.resolve({ action: 'fail' }), + ): Promise<{ action: 'fail' | 'retry' }> { + return ctx.waterfall('agent/request-error', owner, 1, 1, error, retryAttempt, signal, next) + } + + function overflow(message = 'provider overflow'): Error & { code: string } { + return Object.assign(new Error(message), { code: CONTEXT_WINDOW_EXCEEDED_CODE }) + } + + it('compacts post-step above threshold using the durable routed model and remains idle below it', async () => { const ctx = createContext() const compact = new TestCompactService(ctx, { models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } }, }) const pressured = conversation(4) - await preStep(ctx, agent(pressured, MODEL)) + await postStep(ctx, agent(pressured, 'unconfigured-agent-fallback')) expect(pressured.events.some(event => event.type === 'compact/summary')).toBe(true) const small = conversation(1) - await preStep(ctx, agent(small, MODEL)) + await postStep(ctx, agent(small, MODEL)) expect(small.events.some(event => event.type === 'compact/start')).toBe(false) expect(compact.calls).toHaveLength(1) }) @@ -746,7 +858,7 @@ describe('automatic listener and loader composition', () => { compact.error = 'temporary failure' const session = conversation(4) - await expect(preStep(ctx, agent(session, MODEL))).resolves.toBeUndefined() + await expect(postStep(ctx, agent(session, MODEL))).resolves.toBeUndefined() expect(warnings).toContainEqual(expect.stringContaining('temporary failure')) expect(session.events.some(event => event.type === 'compact/summary')).toBe(false) }) @@ -754,21 +866,210 @@ describe('automatic listener and loader composition', () => { it('propagates a named unknown-model configuration failure', async () => { const ctx = createContext() void new TestCompactService(ctx) - await expect(preStep(ctx, agent(conversation(4), 'missing'))).rejects.toMatchObject({ + const session = conversation(4) + session.append('request/header', { + header: { config: { model: 'missing' } }, + reason: 'resume', + }) + await expect(postStep(ctx, agent(session, MODEL))).rejects.toMatchObject({ code: TOKEN_METER_MODEL_UNCONFIGURED, model: 'missing', }) }) - it('auto:false installs no listener', async () => { + it('force-compacts below normal pressure for canonical overflow and retries only after replacement', async () => { + const ctx = createContext() + void new TestCompactService(ctx, { + models: { [MODEL]: { thresholdRatio: 1, retainTokens: 90 } }, + }) + const session = conversation(3) + const beforeGeneration = session.surface.replaceGeneration + const retainedSeq = session.surface.nodes.at(-1)!.seq + const threshold = 100 + expect(ctx.tokenMeter.resolve(MODEL).measure(session).totalTokens).toBeLessThan(threshold) + const decision = await recover(ctx, agent(session, 'unconfigured-agent-fallback'), overflow()) + + expect(decision).toEqual({ action: 'retry' }) + expect(session.surface.replaceGeneration).toBe(beforeGeneration + 1) + expect(session.events.some(event => event.type === 'compact/summary')).toBe(true) + expect(session.surface.nodes.some(node => node.seq === retainedSeq)).toBe(true) + }) + + it('preserves the newest whole tool-call/result pair during forced overflow compaction', async () => { + const ctx = createContext() + void new TestCompactService(ctx, { + models: { [MODEL]: { thresholdRatio: 1, retainTokens: 90 } }, + }) + const session = toolConversation() + const newestAssistant = session.surface.nodes.at(-2)! + const newestResult = session.surface.nodes.at(-1)! + + expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' }) + const currentAssistant = session.surface.nodes.find(node => node.seq === newestAssistant.seq) + const currentResult = session.surface.nodes.find(node => node.seq === newestResult.seq) + expect(currentAssistant).toBeDefined() + expect(currentResult).toBeDefined() + expect(toolPairingBalancedBefore(session, currentAssistant!)).toBe(true) + expect(toolPairingBalancedAfter(session, currentResult!)).toBe(true) + }) + + it('does not retry when a backend reports success without replacing the surface', async () => { + const ctx = createContext() + const compact = new TestCompactService(ctx) + const session = conversation(2) + const fakeResult: CompactionResult = { + startSeq: 1, + summarySeq: 2, + endSeq: 3, + summary: [{ type: 'text', text: 'fake' }], + shadowedRange: { start: 1, end: 2 }, + shadowedSeqs: [1, 2], + shadowedTokenCount: 10, + } + vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(fakeResult) + + expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' }) + expect(session.surface.replaceGeneration).toBe(0) + }) + + it('delegates downstream exactly once when no replacement is available', async () => { + const ctx = createContext() + const compact = new TestCompactService(ctx) + vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(null) + const downstream = new Error('downstream recovery failed') + let calls = 0 + + await expect(recover( + ctx, + agent(conversation(2), MODEL), + overflow(), + 0, + SIGNAL, + () => { + calls += 1 + return Promise.reject(downstream) + }, + )).rejects.toBe(downstream) + expect(calls).toBe(1) + }) + + it('preserves the original provider error when recovery throws', async () => { + const ctx = createContext() + const warnings: string[] = [] + ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn + const compact = new TestCompactService(ctx) + compact.error = new Error('summary unavailable') + const original = overflow('original provider overflow') + + expect(await recover(ctx, agent(conversation(3), MODEL), original)).toEqual({ action: 'fail' }) + expect(original).toMatchObject({ + message: 'original provider overflow', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + expect(warnings).toContainEqual(expect.stringContaining('preserving the original request error')) + }) + + it('delegates once when overflow recovery throws a non-Error value', async () => { + const ctx = createContext() + const warnings: string[] = [] + ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn + const compact = new TestCompactService(ctx) + compact.error = 'non-error recovery failure' + const session = conversation(3) + const generation = session.surface.replaceGeneration + const original = overflow('original provider failure') + let delegations = 0 + + const decision = await recover(ctx, agent(session, MODEL), original, 0, SIGNAL, () => { + delegations += 1 + return Promise.resolve({ action: 'fail' }) + }) + + expect(decision).toEqual({ action: 'fail' }) + expect(delegations).toBe(1) + expect(session.surface.replaceGeneration).toBe(generation) + expect(original).toMatchObject({ + message: 'original provider failure', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + expect(warnings).toContainEqual(expect.stringContaining('non-error recovery failure')) + }) + + it('delegates once and preserves the original overflow for an unknown routed meter model', async () => { + const ctx = createContext() + void new TestCompactService(ctx) + const session = conversation(2) + session.append('request/header', { + header: { config: { model: 'unknown-routed-model' } }, + reason: 'resume', + }) + const original = overflow('original unknown-model overflow') + let delegations = 0 + + const decision = await recover(ctx, agent(session, MODEL), original, 0, SIGNAL, () => { + delegations += 1 + return Promise.resolve({ action: 'fail' }) + }) + expect(decision).toEqual({ action: 'fail' }) + expect(delegations).toBe(1) + expect(original).toMatchObject({ + message: 'original unknown-model overflow', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + }) + + it('honors retry caps, non-context failures, and cancellation', async () => { + const ctx = createContext() + const compact = new TestCompactService(ctx, { maxOverflowRetries: 1 }) + const compactSpy = vi.spyOn(compact, 'compactIfNeeded') + const owner = agent(conversation(3), MODEL) + expect(await recover(ctx, owner, Object.assign(new Error('rate limit'), { code: 'RATE_LIMIT' }))) + .toEqual({ action: 'fail' }) + expect(await recover(ctx, owner, overflow(), 1)).toEqual({ action: 'fail' }) + + const controller = new AbortController() + controller.abort('cancelled') + expect(await recover(ctx, owner, overflow(), 0, controller.signal)).toEqual({ action: 'fail' }) + expect(compactSpy).not.toHaveBeenCalled() + }) + + it('does not retry when cancellation lands during an awaited compaction', async () => { + const ctx = createContext() + const compact = new TestCompactService(ctx) + const controller = new AbortController() + compact.mutateDuringSummary = () => { controller.abort('cancelled during summary') } + const session = conversation(3) + const generation = session.surface.replaceGeneration + + expect(await recover(ctx, agent(session, MODEL), overflow(), 0, controller.signal)) + .toEqual({ action: 'fail' }) + expect(session.surface.replaceGeneration).toBe(generation + 1) + }) + + it('maxOverflowRetries:0 disables recovery without disabling post-step pressure', async () => { + const ctx = createContext() + void new TestCompactService(ctx, { + maxOverflowRetries: 0, + models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } }, + }) + const session = conversation(4) + await postStep(ctx, agent(session, MODEL)) + const summaries = session.events.filter(event => event.type === 'compact/summary').length + expect(summaries).toBe(1) + expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' }) + expect(session.events.filter(event => event.type === 'compact/summary')).toHaveLength(summaries) + }) + + it('auto:false installs neither automatic listener', async () => { const ctx = createContext() void new TestCompactService(ctx, { auto: false, models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } }, }) const session = conversation(4) - await preStep(ctx, agent(session, MODEL)) + await postStep(ctx, agent(session, MODEL)) expect(session.events.some(event => event.type === 'compact/start')).toBe(false) + expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' }) }) it('loads and disposes the real zero-config service stack', async () => { @@ -797,8 +1098,9 @@ describe('automatic listener and loader composition', () => { await fiber.dispose() const session = conversation(4) - await preStep(ctx, agent(session, MODEL)) + await postStep(ctx, agent(session, MODEL)) expect(session.events.some(event => event.type === 'compact/start')).toBe(false) + expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' }) }) }) diff --git a/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts b/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts index cc1111c5f1..2fe9725d77 100644 --- a/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts +++ b/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact' -import LlmService from '@deepseek-ai/dsh-llm' +import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm' import SessionStore from '@deepseek-ai/dsh-session' @@ -53,6 +53,45 @@ class StepwiseToolAdapter extends LlmAdapter { } } +/** First conversation request overflows, then the rebuilt retry succeeds. */ +class OverflowRecoveryAdapter extends LlmAdapter { + readonly conversationRequests: GenerateOptions[] = [] + readonly summaryRequests: GenerateOptions[] = [] + + constructor(private readonly delivery: 'thrown' | 'in-band') { + super() + } + + override async * stream(options: GenerateOptions): AsyncIterable { + if (options.system?.includes('You are a compaction engine')) { + this.summaryRequests.push(options) + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'block-end', index: 0, block: { type: 'text', text: 'RECOVERY CHECKPOINT' } } + yield { type: 'finish', reason: { kind: 'stop' } } + return + } + + this.conversationRequests.push(options) + if (this.conversationRequests.length === 1) { + if (this.delivery === 'thrown') { + throw new LlmError('request too large for model context', CONTEXT_WINDOW_EXCEEDED_CODE, 400) + } + yield { + type: 'finish', + reason: { + kind: 'error', + message: 'request too large for model context', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }, + } + return + } + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } } + yield { type: 'finish', reason: { kind: 'stop' } } + } +} + async function harness(toolSteps: number): Promise<{ ctx: Context; compact: ReproCompactService }> { const ctx = new Context() await ctx.plugin(LlmService) @@ -98,6 +137,53 @@ function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { } describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => { + it('uses the model actually routed by agent/request for post-step pressure', async () => { + const { ctx } = await harness(8) + ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, model: 'mock' })) + try { + const agent = ctx.agentLoop.create(AgentId('routed-pressure'), { + model: 'unconfigured-agent-fallback', + }) + agent.send([{ type: 'text', text: 'do a routed multi-step task' }]) + await waitForIdle(ctx, agent) + + expect(agent.session.requestHeader()?.config.model).toBe('mock') + expect(agent.session.events.some(event => event.type === 'compact/summary')).toBe(true) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'completed' } }, + }) + } finally { + await ctx.fiber.dispose() + } + }) + + it('runs automatic pressure after the current tool result and before step/end', async () => { + const { ctx } = await harness(4) + try { + const agent = ctx.agentLoop.create(AgentId('post-step-order'), { model: 'mock' }) + agent.send([{ type: 'text', text: 'do tool work' }]) + await waitForIdle(ctx, agent) + + const events = [...agent.session.events] + const compactStart = events.find(event => event.type === 'compact/start') + expect(compactStart).toBeDefined() + const precedingResult = events.findLast(event => + event.type === 'tool/result' && event.seq < compactStart!.seq, + ) + if (precedingResult?.type !== 'tool/result') throw new Error('expected a durable tool result before compaction') + const stepEnd = events.find(event => + event.type === 'step/end' + && event.data.step === precedingResult.data.step + && event.seq > compactStart!.seq, + ) + expect(precedingResult.seq).toBeLessThan(compactStart!.seq) + expect(compactStart!.seq).toBeLessThan(stepEnd!.seq) + } finally { + await ctx.fiber.dispose() + } + }) + it('the head checkpoint the loop lands is a balanced cut on both sides', async () => { const { ctx } = await harness(8) try { @@ -130,3 +216,91 @@ describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () } }) }) + +describe('context-overflow recovery across the real loop and compact-basic', () => { + it.each(['thrown', 'in-band'] as const)( + 'force-compacts a %s overflow between failed and retry steps', + async (delivery) => { + const ctx = new Context() + const adapter = new OverflowRecoveryAdapter(delivery) + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(Invariants) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(TokenMeterService, { + models: { mock: { contextWindow: 128, charsPerToken: 4 } }, + }) + ctx.llm.registerAdapter(['mock'], adapter) + ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, model: 'mock' })) + await ctx.plugin(BasicCompactService, { + models: { mock: { thresholdRatio: 1, retainTokens: 100 } }, + maxTokens: 64, + compactionRetries: 0, + maxOverflowRetries: 1, + }) + + try { + const agent = ctx.agentLoop.create(AgentId(`overflow-${delivery}`), { + model: 'unconfigured-agent-fallback', + }) + for (let turn = 1; turn <= 2; turn += 1) { + const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY' + agent.session.append('turn/start', { + turn, + trigger: { kind: 'message', source: { kind: 'user' } }, + }) + agent.session.append('user/message', { + content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) + agent.session.append('step/start', { turn, step: 1 }) + agent.session.append('assistant/message', { + turn, + step: 1, + content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }], + }, { surfaceOp: 'append' }) + agent.session.append('step/end', { turn, step: 1 }) + agent.session.append('turn/end', { turn, reason: { kind: 'completed' } }) + } + + agent.send([{ type: 'text', text: 'continue from history' }]) + await agent.whenIdle() + + expect(adapter.conversationRequests).toHaveLength(2) + expect(adapter.summaryRequests).toHaveLength(1) + expect(JSON.stringify(adapter.conversationRequests[0]!.messages)).toContain('OLD HISTORY SENTINEL') + const retry = JSON.stringify(adapter.conversationRequests[1]!.messages) + expect(retry).toContain('RECOVERY CHECKPOINT') + expect(retry).not.toContain('OLD HISTORY SENTINEL') + + const events = [...agent.session.events] + const failedEnd = events.find(event => + event.type === 'step/end' && event.data.turn === 3 && event.data.step === 1, + )! + const retryStart = events.find(event => + event.type === 'step/start' && event.data.turn === 3 && event.data.step === 2, + )! + const compaction = events.filter(event => + event.type === 'compact/start' + || event.type === 'compact/summary' + || event.type === 'compact/end', + ) + expect(compaction.map(event => event.type)).toEqual([ + 'compact/start', + 'compact/summary', + 'compact/end', + ]) + expect(compaction.every(event => event.seq > failedEnd.seq && event.seq < retryStart.seq)).toBe(true) + expect(events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'completed' } }, + }) + } finally { + await ctx.fiber.dispose() + } + }, + ) +}) diff --git a/packages/compact/compact/README.md b/packages/compact/compact/README.md index 89aacbb09b..a4c94f6c3a 100644 --- a/packages/compact/compact/README.md +++ b/packages/compact/compact/README.md @@ -18,7 +18,7 @@ Both methods are **abstract** — the backend owns trigger policy, retention, ev | Member | Semantics | |---|---| -| `compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` | Estimate the surface-derived history size; if over the backend's threshold, compact an older range via `compactRegion`, keeping recent context intact. Returns the `CompactionResult`, or `null` if nothing needed compacting. All parameters required — the loop's `agent/pre-step` checkpoint supplies the agent, assembled `fullSystemPrompt`, composed `sessionPrefix` (request-only messages every request carries but the derived history omits — the pressure estimate must count them), and turn `signal`. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. | +| `compactIfNeeded(agent, trigger, signal)` | Consider automatic compaction for `trigger: 'pressure' \| 'context-overflow'`. A pressure trigger may apply the backend's threshold and retained-tail policy; a confirmed overflow may force a useful balanced reduction. Returns the `CompactionResult`, or `null` when no safe range exists. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. | | `compactRegion(session, start, end, agent, signal?)` | Forcibly summarize surface nodes `[start, end]` (inclusive seqs) into a single replacement node. The agent must own the exact target (`session === agent.session`); a backend rejects mismatch before model resolution, lock acquisition, summarization, or log mutation. **Throws** if a compaction is already in progress, if `start`/`end` aren't surface nodes, or if `start` is positioned after `end` on the surface. The range is a SURFACE-POSITION span, not a numeric seq interval — after a prior replace lands a fresh high-seq summary node at the shadowed range's position, surface order no longer tracks seq order. | `compactIfNeeded` takes a required `signal`; `compactRegion`'s is optional. A backend that summarizes via `ctx.llm.stream()` **must** forward it into the call's `GenerateOptions.signal`, so an abort or fiber dispose tears down the in-flight summarization instead of leaving an orphaned model call running past the cancellation. The turn that the `compact/*` events belong to is recoverable from the owned session's log (the currently-open turn), so the backend stamps it from the log rather than trusting a caller-supplied value. @@ -72,6 +72,5 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and ## Known Limitations and Deferred Work - **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener. -- **Single-unit overflow is out of contract** — one retained unit (a closed step or a large pasted `user/message`) alone exceeding the budget cannot be compacted; the call may go out over-budget. -- **A session prefix that alone approaches the window is a configuration error no backend fixes** — compaction shrinks derived history, never the prefix. -- **Request context injected by downstream `agent/request` listeners sits outside pressure accounting** — `compactIfNeeded` counts prefix, derived history, and system prompt only. +- **Single-unit overflow is out of contract** — one indivisible unit (a closed tool pair or a large pasted `user/message`) alone exceeding the budget cannot be compacted. +- **An envelope that alone approaches the window is not surface-compaction work** — compaction shrinks derived history, never the system prompt, tools, or session prefix. diff --git a/packages/compact/compact/src/index.ts b/packages/compact/compact/src/index.ts index 7361d38ef3..80058f6145 100644 --- a/packages/compact/compact/src/index.ts +++ b/packages/compact/compact/src/index.ts @@ -8,7 +8,6 @@ */ import { Context, Service } from 'cordis' -import type { Message } from '@deepseek-ai/dsh-llm' import type { Session } from '@deepseek-ai/dsh-session' import type { CompactionResult } from './types.ts' @@ -16,6 +15,9 @@ export type { CompactionResult } from './types.ts' export { renderContentBlocks, renderTranscript } from './render.ts' export { toolPairingBalancedAfter, toolPairingBalancedBefore } from './tool-pairing.ts' +/** Why automatic policy is asking a backend to consider compaction. */ +export type CompactionTrigger = 'pressure' | 'context-overflow' + /** Minimal agent context compaction needs without depending on the agent package. */ export interface CompactAgentContext { session: Session @@ -41,24 +43,20 @@ export abstract class CompactService extends Service { } /** - * Check token pressure and compact if the conversation is too large. - * Estimate the next request, including its session prefix, derived history, - * and system prompt. Above threshold, compact a head-anchored range ending at - * a balanced tool boundary and reconsolidate any prior automatic checkpoint. - * Return `null` when no compaction is needed or an open tail leaves no safe - * cutoff. A single oversized retained unit or prefix cannot be repaired here. + * Consider automatic compaction for one explicit trigger. Pressure policy + * uses the latest durable routed request, while context-overflow policy may + * force a useful balanced reduction even below the normal threshold. Return + * `null` when no safe range can be compacted. A single oversized retained + * unit or request envelope cannot be repaired through surface compaction. * * @param agent - agent context owning the session surface and model options. - * @param fullSystemPrompt - assembled system prompt, counted toward the estimate. - * @param sessionPrefix - the instance's composed session prefix, counted toward the - * estimate. + * @param trigger - normal pressure or provider-confirmed context overflow. * @param signal - cancellation signal; model-backed implementations must forward it. * @returns the compaction result, or `null` if no compaction was needed. */ abstract compactIfNeeded( agent: CompactAgentContext, - fullSystemPrompt: string, - sessionPrefix: readonly Message[], + trigger: CompactionTrigger, signal: AbortSignal, ): Promise diff --git a/packages/compact/compact/tests/compact.spec.ts b/packages/compact/compact/tests/compact.spec.ts index c4daa8cc5a..946b2e8b83 100644 --- a/packages/compact/compact/tests/compact.spec.ts +++ b/packages/compact/compact/tests/compact.spec.ts @@ -1,8 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import { CompactService } from '@deepseek-ai/dsh-compact' -import type { CompactionResult } from '@deepseek-ai/dsh-compact' -import type { Message } from '@deepseek-ai/dsh-llm' +import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' import { Session, SessionId } from '@deepseek-ai/dsh-session' import type { CompactAgentContext } from '@deepseek-ai/dsh-compact' @@ -18,8 +17,7 @@ class StubCompactService extends CompactService { override async compactIfNeeded( _agent: CompactAgentContext, - _fullSystemPrompt: string, - _sessionPrefix: readonly Message[], + _trigger: CompactionTrigger, signal: AbortSignal, ): Promise { this.lastSignal = signal @@ -80,7 +78,7 @@ describe('CompactService seam', () => { const ctx = new Context() const svc = new StubCompactService(ctx) const session = new Session(SessionId('s')) - expect(await svc.compactIfNeeded(stubAgent(session), '', [], new AbortController().signal)).toBeNull() + expect(await svc.compactIfNeeded(stubAgent(session), 'pressure', new AbortController().signal)).toBeNull() }) it('compact/* events merge into SessionEventMap and are log-only', async () => { @@ -109,7 +107,7 @@ describe('CompactService seam', () => { await svc.compactRegion(session, 0, 0, stubAgent(session, 'm'), controller.signal) expect(svc.lastSignal).toBe(controller.signal) - await svc.compactIfNeeded(stubAgent(session), '', [], controller.signal) + await svc.compactIfNeeded(stubAgent(session), 'context-overflow', controller.signal) expect(svc.lastSignal).toBe(controller.signal) }) }) diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 7aafb3ce8a..fad2094fa9 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -108,7 +108,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ key: 'compact', summary: 'Abstract compaction service.', methods: [ - 'abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise', + 'abstract compactIfNeeded( agent: CompactAgentContext, trigger: CompactionTrigger, signal: AbortSignal, ): Promise', 'abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise', ], }, @@ -289,8 +289,8 @@ export const EVENT_API: readonly EventApiEntry[] = [ { name: 'agent/pre-step', mode: 'serial', - signature: '\'agent/pre-step\'(this: Scoped, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise | void', - summary: 'Awaited serial checkpoint for session-surface mutation after prompt assembly and before `step/start`; appends land outside the pending step.', + signature: '\'agent/pre-step\'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void', + summary: 'Awaited serial checkpoint before `step/start`; appends land outside the pending step and are included when the loop derives request history.', }, { name: 'agent/prompt-submit', @@ -656,6 +656,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'CompactionResult', declaration: 'export interface CompactionResult {\n startSeq: number;\n summarySeq: number;\n endSeq: number;\n summary: ContentBlock[];\n shadowedRange: {\n start: number;\n end: number;\n };\n shadowedSeqs: number[];\n shadowedTokenCount: number;\n}', }, + { + name: 'CompactionTrigger', + declaration: 'export type CompactionTrigger = \'pressure\' | \'context-overflow\';', + }, { name: 'ConfinedArgv', declaration: 'export interface ConfinedArgv {\n argv: string[];\n enforcement: SandboxEnforcement;\n denialSignatures: readonly string[];\n runnerFailureSignatures: readonly string[];\n}', diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index 864ff95e8c..70d7c16cf0 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -58,7 +58,7 @@ Plugin failure ends the current turn, not the loop. Only final adapter dispatch/ Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy: - Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute` → `tools/execute` → `tools/post-execute` → `tools/result` pipeline; exact signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md) -- Compaction: `agent/pre-step` +- Compaction: pressure on `agent/post-step`; canonical context overflow on `agent/request-error` - Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation - Sub-agents: implemented outside the loop as `ctx.subagents` providers; in-process providers use `ctx.agents.create()` and owned `AgentHandle` teardown, while child streaming/progress and background/poll collection remain deferred. - Persistence: `session/event` + `session/flush` diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index e749018f55..620cdd9fbc 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -276,7 +276,7 @@ async function runTurn( const abort = new AbortController() handle.setAbort(abort) - // Assemble once before pre-step so pressure checks and the request share the same prompt. + // Assemble once before pre-step so listener work and the request share one prompt value. const assembly = await ctx.systemPrompt.assemble(assembleContextFor(agent)) const fullSystemPrompt = renderPrompt(assembly) @@ -287,9 +287,9 @@ async function runTurn( break } - // Compose the request-only prefix once per loop instance before pressure - // checks. It precedes all derived history and is recorded only in the - // request header, not as session history. + // Compose the request-only prefix once per loop instance before the first + // request boundary. It precedes all derived history and is recorded only + // in the request header, not as session history. if (transmission.sessionPrefix === undefined) { const emptyPrefix: Message[] = deepFreeze([]) const composed = await events.waterfall( @@ -306,8 +306,8 @@ async function runTurn( transmission.sessionPrefix = deepFreeze(structuredClone(composed)) } - // Await surface mutations outside the step; pressure checks receive the pending prefix. - await events.serial('agent/pre-step', turn, step, fullSystemPrompt, transmission.sessionPrefix, abort.signal) + // Await surface mutations outside the step before snapshotting history. + await events.serial('agent/pre-step', turn, step, abort.signal) // Interruption landing during the pre-step seam: do not open an empty step. if (handle.isCancelled() || handle.isDisposed()) { diff --git a/packages/core/agent-loop/tests/interception.spec.ts b/packages/core/agent-loop/tests/interception.spec.ts index 4b37210993..d89e5b0b0e 100644 --- a/packages/core/agent-loop/tests/interception.spec.ts +++ b/packages/core/agent-loop/tests/interception.spec.ts @@ -118,7 +118,7 @@ describe('agent/prompt-submit', () => { it('a prompt-submit rewrite + additionalContext is VISIBLE to the agent/pre-step seam (merged ordering)', async () => { // Prompt rewrites and injected context land before `agent/pre-step`, so a - // compaction listener measures the current surface before the single derive. + // surface listener sees the current state before the single derive. const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) @@ -130,8 +130,7 @@ describe('agent/prompt-submit', () => { additionalContext: { content: [{ type: 'text', text: 'injected ctx' }], source: { kind: 'plugin', plugin: 'test' } }, })) - // The pre-step seam (where compaction lives) derives the surface it would act - // on. Capture what it sees on the first step. + // Capture the surface visible at the generic pre-step seam on the first step. let preStepDerived: string | undefined ctx.on('agent/pre-step', (subject, _turn, step) => { if (subject === agent && step === 1) preStepDerived = JSON.stringify(subject.session.deriveMessages()) @@ -375,7 +374,7 @@ describe('agent/session-prefix', () => { expect(agent.session.deriveMessages()[0]).toEqual({ role: 'user', content: [{ type: 'text', text: 'go' }] }) }) - it('composes before the first pre-step and hands the prefix to the seam (pressure gates see the real value)', async () => { + it('composes before the first pre-step and records the prefix on the request header', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) @@ -386,20 +385,15 @@ describe('agent/session-prefix', () => { order.push('compose') return [reminder, ...await next()] }) - const seen: (readonly Message[])[] = [] - ctx.on('agent/pre-step', (_agent, _turn, _step, _system, sessionPrefix) => { + ctx.on('agent/pre-step', () => { order.push('pre-step') - seen.push(sessionPrefix) }) send(agent, 'hi') await waitForIdle(ctx, agent) - // Composition precedes the pre-step seam, and the seam receives THIS - // instance's composed prefix — a token-pressure gate (compaction) counts - // what the request will actually carry, never a stale logged prefix. expect(order).toEqual(['compose', 'pre-step']) - expect(seen[0]).toEqual([reminder]) + expect(agent.session.requestHeader()?.messagePrefix).toEqual([reminder]) }) it('the canonical prepend pattern composes contributions in registration order', async () => { diff --git a/packages/core/agent-loop/tests/loop.spec.ts b/packages/core/agent-loop/tests/loop.spec.ts index f282301a2b..89d8cc306c 100644 --- a/packages/core/agent-loop/tests/loop.spec.ts +++ b/packages/core/agent-loop/tests/loop.spec.ts @@ -486,9 +486,8 @@ describe('agent loop', () => { it('agent/pre-step fires once per step before the step is opened', async () => { // Two steps (a tool call, then a final text turn) → two model calls → two - // pre-step fires, each carrying the assembled full system prompt, BEFORE - // the step is opened and its request is derived (the request the adapter - // sees reflects any surface state at fire time). + // pre-step fires BEFORE the step is opened and its request is derived (the + // request the adapter sees reflects any surface state at fire time). const adapter = new MockAdapter([ toolCallResponse('c1', 'echo', {}, 'calling echo'), textResponse('done'), @@ -500,21 +499,19 @@ describe('agent loop', () => { })) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) - const fires: { turn: number; step: number; fullSystemPrompt: string }[] = [] - ctx.on('agent/pre-step', (subject, turn, step, fullSystemPrompt) => { - if (subject === agent) fires.push({ turn, step, fullSystemPrompt }) + const fires: { turn: number; step: number; signal: AbortSignal }[] = [] + ctx.on('agent/pre-step', (subject, turn, step, signal) => { + if (subject === agent) fires.push({ turn, step, signal }) }) send(agent, 'go') await waitForIdle(ctx, agent) - // One fire per step, in order, each with the assembled system prompt - // (here just the loop's own harness-identity section — no persona set). - const HARNESS = 'You are an AI agent powered by the DeepSeek Harness SDK.' - expect(fires).toEqual([ - { turn: 1, step: 1, fullSystemPrompt: HARNESS }, - { turn: 1, step: 2, fullSystemPrompt: HARNESS }, + expect(fires.map(({ turn, step }) => ({ turn, step }))).toEqual([ + { turn: 1, step: 1 }, + { turn: 1, step: 2 }, ]) + expect(fires.every(({ signal }) => signal instanceof AbortSignal)).toBe(true) }) it('agent/pre-step fires BEFORE the step it precedes opens (events land outside the step)', async () => { diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts index 003c678be0..416ede21b3 100644 --- a/packages/core/agent-loop/tests/request-recovery.spec.ts +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -219,6 +219,48 @@ describe('agent post-step and request-error lifecycle', () => { }) }) + it('closes the successful step as disposed when disposal lands during post-step', async () => { + const adapter = new FailureScriptAdapter([ + toolCallResponse('dispose-call', 'work', {}), + textResponse('must not continue'), + ]) + const ctx = await harness(adapter) + ctx.tools.register(defineTool({ + name: 'work', + description: 'do work', + parameters: {}, + async execute() { return [{ type: 'text', text: 'worked' }] }, + })) + const agent = ctx.agentLoop.create(AgentId('dispose-post-step'), { model: 'mock' }) + let entered!: () => void + const postStepEntered = new Promise((resolve) => { entered = resolve }) + ctx.on('agent/post-step', async (_agent, turn, step, signal) => { + expect({ turn, step }).toEqual({ turn: 1, step: 1 }) + entered() + await new Promise((resolve) => { + signal.addEventListener('abort', () => { resolve() }, { once: true }) + }) + }) + + send(agent) + await postStepEntered + await ctx.fiber.dispose() + + expect(adapter.requests).toHaveLength(1) + const boundaries = agent.session.events.filter(event => + event.type === 'step/start' || event.type === 'step/end', + ) + expect(boundaries.map(event => event.type)).toEqual(['step/start', 'step/end']) + expect(boundaries.map(event => event.data)).toEqual([ + { turn: 1, step: 1 }, + { turn: 1, step: 1 }, + ]) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'disposed' } }, + }) + }) + it.each([ ['thrown', contextError()], ['in-band', [{ type: 'finish', reason: { kind: 'error', message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE } }] satisfies StreamChunk[]], diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index 1eda0df2d7..05f473e07c 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -189,23 +189,17 @@ declare module 'cordis' { // ---- step/request extension seams (serial + waterfall) ---- /** - * Awaited serial checkpoint for session-surface mutation after prompt - * assembly and before `step/start`; appends land outside the pending step. - * The loop derives history once afterward, so compaction records and - * replacements are included without rewriting an assembled request. The - * prompt and prefix are the exact pressure inputs for that request, and + * Awaited serial checkpoint before `step/start`; appends land outside the + * pending step and are included when the loop derives request history. * `signal` cancels listener work. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @param agent - the agent opening the step. * @param turn - the open turn number. * @param step - the pending step number. - * @param fullSystemPrompt - the assembled prompt. - * @param sessionPrefix - the frozen request prefix. * @param signal - the turn abort signal. * @mode serial */ - // TODO: Move prompt-pressure inputs behind a compaction-specific seam if no second consumer appears. - 'agent/pre-step'(this: Scoped, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise | void + 'agent/pre-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void /** * Allow, rewrite, or block one drained prompt before it becomes a user * message. Call `next()` for the unchanged default. @@ -233,9 +227,9 @@ declare module 'cordis' { * result is computed once per loop instance, logged on its anchoring request * header, and reused so the provider prefix remains stable. Interrupted * composition is discarded. Composition precedes the first `agent/pre-step` - * and request boundary, so listener appends join the current request and - * pressure accounting sees the composed prefix. Changing context belongs in - * history; contributors should prepend to `await next()` to preserve registration order. + * and request boundary, so listener appends join the current request. + * Changing context belongs in history; contributors should prepend to + * `await next()` to preserve registration order. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @param agent - the agent whose session prefix is being composed. * @param prefix - the frozen seed; return an extended replacement. diff --git a/packages/support/invariants/tests/invariants.spec.ts b/packages/support/invariants/tests/invariants.spec.ts index 7af8d97848..af84c569b9 100644 --- a/packages/support/invariants/tests/invariants.spec.ts +++ b/packages/support/invariants/tests/invariants.spec.ts @@ -849,7 +849,7 @@ describe('scoped-dispatch invariants', () => { ['agent/status', [agent, 'idle']], ['agent/queued', [agent, [], { source: { kind: 'user' }, steering: false }]], ['agent/session-start', [agent, 'startup']], - ['agent/pre-step', [agent, 1, 1, '', new AbortController().signal]], + ['agent/pre-step', [agent, 1, 1, new AbortController().signal]], ['agent/prompt-submit', [agent, [], { kind: 'user' }, () => Promise.resolve({ kind: 'allow' })]], ['agent/request', [agent, 1, 1, { model: 'm' }, () => Promise.resolve({ model: 'm' })]], ['agent/session-prefix', [agent, [], new AbortController().signal, () => Promise.resolve([])]], diff --git a/packages/ui/user-approval/tests/approval.spec.ts b/packages/ui/user-approval/tests/approval.spec.ts index f248e5c468..0a2874499a 100644 --- a/packages/ui/user-approval/tests/approval.spec.ts +++ b/packages/ui/user-approval/tests/approval.spec.ts @@ -371,7 +371,7 @@ describe('approval policy (the approval/policy fold)', () => { } const preStep = (ctx: Context, agent: Agent): Promise => - ctx.serial('agent/pre-step', agent, 1, 1, '', [], new AbortController().signal) + ctx.serial('agent/pre-step', agent, 1, 1, new AbortController().signal) /** Append a `request/header` snapshot whose system text is exactly `system`. */ function appendHeader(session: Session, system: string): void { diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 11168ba000..05547c9b5b 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -228,7 +228,7 @@ const SERVICE_ROLES: ServiceRole[] = [ mode: 'seam', implementations: ['compact-basic'], consumers: ['compact-basic'], - note: 'The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred.', + note: 'The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred.', }, { key: 'subagents', @@ -841,6 +841,8 @@ function renderLifecycle(): string { '', 'The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set.', '', + '`dsh-compact-basic` uses `agent/post-step` for pressure after those durable facts and `agent/request-error` only for canonical context overflow. Recovery compacts between the closed failed step and a fresh retry step, and returns retry only when the surface replacement generation advances; otherwise the original request error remains authoritative.', + '', 'SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.', '', ...maintenanceFooter(maintenance), diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index b0cea673ad..17bbb787dc 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -134,6 +134,7 @@ { "doc": "docs/core-data-structures/skills.md", "symbol": "Config", "source": "packages/skill/skill/src/index.ts" }, { "doc": "docs/core-data-structures/compaction.md", "symbol": "CompactionResult", "source": "packages/compact/compact/src/types.ts" }, + { "doc": "docs/core-data-structures/compaction.md", "symbol": "CompactionTrigger", "source": "packages/compact/compact/src/index.ts" }, { "doc": "docs/core-data-structures/subagent.md", "symbol": "SubagentCapabilities", "source": "packages/subagent/subagent/src/types.ts" }, { "doc": "docs/core-data-structures/subagent.md", "symbol": "SubagentStartRequest", "source": "packages/subagent/subagent/src/types.ts" }, From d66c926d7a63d5416f766f2de90b16760ffb5176 Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Wed, 15 Jul 2026 17:43:33 +0800 Subject: [PATCH 08/73] fix(agent): preserve lifecycle recovery boundaries (PR3 round 2) --- .../compact/compact-basic/src/automatic.ts | 1 + .../compact-basic/tests/compact-basic.spec.ts | 15 +++ packages/core/agent-loop/src/loop.ts | 6 +- .../tests/contract-regressions.spec.ts | 57 ++++++++++- packages/llm/llm/src/index.ts | 19 ++-- packages/llm/llm/tests/service.spec.ts | 95 ++++++++++++++----- 6 files changed, 151 insertions(+), 42 deletions(-) diff --git a/packages/compact/compact-basic/src/automatic.ts b/packages/compact/compact-basic/src/automatic.ts index 16e71c081a..e837e076cc 100644 --- a/packages/compact/compact-basic/src/automatic.ts +++ b/packages/compact/compact-basic/src/automatic.ts @@ -45,6 +45,7 @@ export function registerAutomaticCompaction( _step: number, signal: AbortSignal, ) => { + if (signal.aborted) return try { const result = await service.compactIfNeeded(agent, 'pressure', signal) if (result !== null) logResult(result, 'post-step pressure') diff --git a/packages/compact/compact-basic/tests/compact-basic.spec.ts b/packages/compact/compact-basic/tests/compact-basic.spec.ts index 163c2b6b45..d5e3246777 100644 --- a/packages/compact/compact-basic/tests/compact-basic.spec.ts +++ b/packages/compact/compact-basic/tests/compact-basic.spec.ts @@ -848,6 +848,21 @@ describe('automatic listener and loader composition', () => { expect(compact.calls).toHaveLength(1) }) + it('skips post-step pressure when the step signal is already aborted', async () => { + const ctx = createContext() + const compact = new TestCompactService(ctx, { + models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } }, + }) + const pressured = conversation(4) + const compactIfNeeded = vi.spyOn(compact, 'compactIfNeeded') + + await expect(postStep(ctx, agent(pressured, MODEL), AbortSignal.abort('step aborted'))) + .resolves.toBeUndefined() + + expect(compactIfNeeded).not.toHaveBeenCalled() + expect(pressured.events.some(event => event.type === 'compact/start')).toBe(false) + }) + it('warns and continues after operational failures, including non-Errors', async () => { const ctx = createContext() const warnings: string[] = [] diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 620cdd9fbc..59b0d26088 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -650,7 +650,8 @@ async function runStep( session, turn, step, message.content, assembler.usage, chunkSeqs, ) - // Tool execution stays sequential; recheck abort around each normalized result. + // Tool execution stays sequential; cancellation latches synthetic results for + // every remaining call while preserving one complete result batch. const toolCalls = message.content.filter(block => block.type === 'tool-call') // Buffer context until all results are appended to preserve call/result adjacency. const pendingContext: HookContext[] = [] @@ -693,9 +694,6 @@ async function runStep( if (signal.aborted) aborted = true } - /* v8 ignore next -- signal.reason always set by cancellation or disposal. */ - if (aborted) throw new Error(String(signal.reason ?? 'aborted')) - // Append buffered context after the complete result batch. for (const context of pendingContext) { agent.inject(context.content, { source: context.source }) diff --git a/packages/core/agent-loop/tests/contract-regressions.spec.ts b/packages/core/agent-loop/tests/contract-regressions.spec.ts index 6ca06fc096..75afb08b0c 100644 --- a/packages/core/agent-loop/tests/contract-regressions.spec.ts +++ b/packages/core/agent-loop/tests/contract-regressions.spec.ts @@ -156,7 +156,7 @@ describe('successful provider completion survives agent/step-result failure', () }) describe('abort during tool execution ends the turn', () => { - it('aborting the in-flight step inside a tool prevents both remaining tools and the next model step', async () => { + it('balances an aborted tool batch through context, steering, and post-step before closing', async () => { const adapter = new MockAdapter([ // model asks for two tool calls in one step [ @@ -175,8 +175,12 @@ describe('abort during tool execution ends the turn', () => { name: 'aborter', description: '', parameters: {}, - async execute() { + async execute(_args, exec) { executed.push('aborter') + exec.agent?.steer( + [{ type: 'text', text: 'steering before abort' }], + { source: { kind: 'plugin', plugin: 'abort-test' } }, + ) // Fire the in-flight step's AbortController directly (the loop registers // it on the agent). This is the bare step-abort path — distinct from // cancel(), which would also clear the inbox; here the subject is the @@ -185,6 +189,13 @@ describe('abort during tool execution ends the turn', () => { return [{ type: 'text', text: 'done' }] }, })) + ctx.on('tools/post-execute', async exec => ({ + kind: 'accept', + additionalContext: { + content: [{ type: 'text', text: `context for ${exec.callId}` }], + source: { kind: 'plugin', plugin: 'abort-test' }, + }, + })) ctx.tools.register(defineTool({ name: 'second', description: '', @@ -196,13 +207,53 @@ describe('abort during tool execution ends the turn', () => { })) const reasons: TurnEndReason[] = [] - ctx.on('session/event', (_s, event) => { if (event.type === 'turn/end') reasons.push(event.data.reason) }) + const order: string[] = [] + ctx.on('session/event', (session, event) => { + if (session !== agent.session) return + switch (event.type) { + case 'assistant/message': order.push('assistant/message'); break + case 'tool/call': order.push(`tool/call:${event.data.callId}`); break + case 'tool/result': { + const outcome = event.data.error?.code === 'ABORTED' ? 'synthetic-aborted' : 'real' + order.push(`tool/result:${event.data.callId}:${outcome}`) + break + } + case 'context/message': order.push('context/message'); break + case 'steering/message': order.push('steering/message'); break + case 'step/end': order.push('step/end'); break + case 'turn/end': { + reasons.push(event.data.reason) + order.push(`turn/end:${event.data.reason.kind}`) + break + } + } + }) + let postSteps = 0 + ctx.on('agent/post-step', (subject, turn, step, signal) => { + if (subject !== agent) return + postSteps += 1 + expect({ turn, step, aborted: signal.aborted }).toEqual({ turn: 1, step: 1, aborted: true }) + order.push('agent/post-step') + }) send(agent, 'go') await waitForIdle(ctx, agent) expect(executed).toEqual(['aborter']) // second tool never ran expect(adapter.requests).toHaveLength(1) // no follow-up model call + expect(postSteps).toBe(1) + expect(order).toEqual([ + 'assistant/message', + 'tool/call:c1', + 'tool/result:c1:real', + 'tool/call:c2', + 'tool/result:c2:synthetic-aborted', + 'context/message', + 'steering/message', + 'agent/post-step', + 'step/end', + 'turn/end:aborted', + ]) expect(reasons).toEqual([{ kind: 'aborted', reason: 'user interrupt' }]) const calls = agent.session.events.filter(event => event.type === 'tool/call') const results = agent.session.events.filter(event => event.type === 'tool/result') diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index 0b933d12a7..1be716cbfb 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -124,8 +124,9 @@ export class LlmService extends Service { * Final adapter boundary. It tags only failures from adapter selection, * synchronous dispatch, iterator construction, or iteration while preserving * the original Error object. Middleware outside this generator remains - * distinguishable as plugin work. Adapter cleanup is best-effort after an - * earlier failure or downstream close and never masks the winning error. + * distinguishable as plugin work. An iteration failure skips adapter cleanup + * so it cannot suppress the primary provider error. A downstream close awaits + * adapter cleanup, whose failures remain ordinary untagged work. */ private async * adapterStream(options: GenerateOptions): AsyncGenerator { let iterator: AsyncIterator @@ -137,6 +138,7 @@ export class LlmService extends Service { } let completed = false + let iterationFailed = false try { while (true) { let value: StreamChunk @@ -148,6 +150,7 @@ export class LlmService extends Service { } value = item.value } catch (error: unknown) { + iterationFailed = true throw markLlmAdapterFailure(error) } // End the adapter-owned try before yielding: consumer/middleware @@ -155,14 +158,10 @@ export class LlmService extends Service { yield value } } finally { - if (!completed) { - try { - const close = iterator.return?.bind(iterator) - if (close) await close() - } catch { - // Lookup and invocation are both adapter-owned cleanup following an - // existing failure/downstream close; neither can replace it. - } + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- the iteration catch sets its latch before entering finally. + if (!completed && !iterationFailed) { + const close = iterator.return?.bind(iterator) + if (close) await close() } } } diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index 8dc46529bd..b488002666 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -72,11 +72,21 @@ describe('LlmService', () => { const original = new LlmError(`${field} getter failed`, 'RESULT_GETTER_FAILED') const result = field === 'done' ? {} : { done: false } Object.defineProperty(result, field, { get: () => { throw original } }) + let cleanupLookups = 0 + const iterator: AsyncIterator = { + next: () => Promise.resolve(result as unknown as IteratorResult), + } + Object.defineProperty(iterator, 'return', { + get: () => { + cleanupLookups += 1 + throw new Error('return getter must not run after iteration fails') + }, + }) const adapter = new class extends LlmAdapter { stream(_options: GenerateOptions): AsyncIterable { return { [Symbol.asyncIterator](): AsyncIterator { - return { next: () => Promise.resolve(result as unknown as IteratorResult) } + return iterator }, } } @@ -94,6 +104,7 @@ describe('LlmService', () => { expect(caught).toBe(original) expect(isLlmAdapterFailure(caught)).toBe(true) + expect(cleanupLookups).toBe(0) }) it.each(['dispatch', 'iterator'] as const)('tags synchronous adapter %s failures without replacing their Error', async (boundary) => { @@ -119,7 +130,7 @@ describe('LlmService', () => { expect(isLlmAdapterFailure(caught)).toBe(true) }) - it('tags adapter iteration failures without replacing the original Error or cleanup outcome', async () => { + it('propagates a rejected next promptly without awaiting a non-settling return', async () => { const original = new LlmError('provider failed', 'PROVIDER_FAILED') let cleanupCalls = 0 const adapter = new class extends LlmAdapter { @@ -130,7 +141,49 @@ describe('LlmService', () => { next: () => Promise.reject(original), return: () => { cleanupCalls += 1 - return Promise.reject(new Error('cleanup failed')) + return new Promise>(() => {}) + }, + } + }, + } + } + }() + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-model'], adapter) + + const failure = (async (): Promise => { + try { + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + } catch (error: unknown) { + return error + } + return new Error('expected adapter iteration to fail') + })() + let timer: ReturnType | undefined + const timeout = new Promise((resolve) => { + timer = setTimeout(() => { resolve(new Error('adapter failure did not settle promptly')) }, 100) + }) + const caught = await Promise.race([failure, timeout]) + if (timer !== undefined) clearTimeout(timer) + + expect(caught).toBe(original) + expect(isLlmAdapterFailure(caught)).toBe(true) + expect(cleanupCalls).toBe(0) + }) + + it('awaits one adapter return on downstream close and leaves its rejection unclassified', async () => { + const cleanup = new Error('cleanup failed') + let cleanupCalls = 0 + const adapter = new class extends LlmAdapter { + stream(_options: GenerateOptions): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + return { + next: () => Promise.resolve({ done: false, value: SCRIPT[0]! }), + return: () => { + cleanupCalls += 1 + return Promise.reject(cleanup) }, } }, @@ -143,45 +196,37 @@ describe('LlmService', () => { let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) break } catch (error: unknown) { caught = error } - expect(caught).toBe(original) - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(caught).toBe(cleanup) + expect(isLlmAdapterFailure(caught)).toBe(false) expect(cleanupCalls).toBe(1) }) - it('contains a throwing iterator.return getter after next fails without replacing the original Error', async () => { - const original = new LlmError('provider failed', 'PROVIDER_FAILED') - let cleanupLookups = 0 - const iterator: AsyncIterator = { next: () => Promise.reject(original) } - Object.defineProperty(iterator, 'return', { - get: () => { - cleanupLookups += 1 - throw new Error('return getter failed') - }, - }) + it('allows downstream close when the adapter iterator has no return method', async () => { const adapter = new class extends LlmAdapter { stream(_options: GenerateOptions): AsyncIterable { - return { [Symbol.asyncIterator]: () => iterator } + return { + [Symbol.asyncIterator](): AsyncIterator { + return { next: () => Promise.resolve({ done: false, value: SCRIPT[0]! }) } + }, + } } }() const ctx = new Context() await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) - let caught: unknown - try { - for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ } - } catch (error: unknown) { - caught = error + let chunks = 0 + for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { + chunks += 1 + break } - expect(caught).toBe(original) - expect(isLlmAdapterFailure(caught)).toBe(true) - expect(cleanupLookups).toBe(1) + expect(chunks).toBe(1) }) it('normalizes and tags non-Error adapter failures once', async () => { From d027ea0d1003b22f36c389bf126c7dd47a598981 Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Thu, 16 Jul 2026 15:10:45 +0800 Subject: [PATCH 09/73] refactor(compact-basic): inline automatic listeners into the service Fold automatic.ts into BasicCompactService as a private _registerAutomaticCompaction method, removing the AutomaticCompactor structural interface the standalone module needed to avoid an import cycle. Listener behavior is unchanged; compactIfNeeded stays dynamically dispatched so subclass overrides are honored at event time. --- .../compact/compact-basic/src/automatic.ts | 85 ------------------- packages/compact/compact-basic/src/index.ts | 68 ++++++++++++++- 2 files changed, 66 insertions(+), 87 deletions(-) delete mode 100644 packages/compact/compact-basic/src/automatic.ts diff --git a/packages/compact/compact-basic/src/automatic.ts b/packages/compact/compact-basic/src/automatic.ts deleted file mode 100644 index e837e076cc..0000000000 --- a/packages/compact/compact-basic/src/automatic.ts +++ /dev/null @@ -1,85 +0,0 @@ -/** - * Automatic post-step pressure and context-overflow recovery listeners. - * - * @module @deepseek-ai/dsh-compact-basic/automatic - */ - -import type { Context } from 'cordis' -import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' -import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' -import { - TOKEN_METER_MODEL_UNCONFIGURED, - TokenMeterError, -} from '@deepseek-ai/dsh-token-meter' -import type { Agent } from '@deepseek-ai/dsh-agent' - -interface AutomaticCompactor { - readonly config: { readonly maxOverflowRetries: number } - compactIfNeeded( - agent: Agent, - trigger: CompactionTrigger, - signal: AbortSignal, - ): Promise -} - -/** - * Register the implementation-owned automatic compaction listener. - * @param ctx - context owning the listener effect and logger. - * @param service - compactor whose public methods remain dynamically dispatched. - */ -export function registerAutomaticCompaction( - ctx: Context, - service: AutomaticCompactor, -): void { - const logResult = (result: CompactionResult, trigger: string): void => { - ctx.logger.info( - `compaction (${trigger}): shadowed ${result.shadowedSeqs.length} surface nodes ` - + `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, ` - + `~${result.shadowedTokenCount} tokens)`, - ) - } - - ctx.on('agent/post-step', async ( - agent: Agent, - _turn: number, - _step: number, - signal: AbortSignal, - ) => { - if (signal.aborted) return - try { - const result = await service.compactIfNeeded(agent, 'pressure', signal) - if (result !== null) logResult(result, 'post-step pressure') - } catch (error: unknown) { - // A named routed model without a meter profile is configuration failure, - // not an optional operational compaction miss. - if (error instanceof TokenMeterError - && error.code === TOKEN_METER_MODEL_UNCONFIGURED) throw error - const message = error instanceof Error ? error.message : String(error) - ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`) - } - }) - - ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => { - if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE - || retryAttempt >= service.config.maxOverflowRetries - || signal.aborted) return next() - - let generation: number - let result: CompactionResult | null - try { - generation = agent.session.surface.replaceGeneration - result = await service.compactIfNeeded(agent, 'context-overflow', signal) - } catch (recoveryError: unknown) { - const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError) - ctx.logger.warn( - `context-overflow compaction failed: ${message}; preserving the original request error`, - ) - return next() - } - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited. - if (signal.aborted || result === null - || agent.session.surface.replaceGeneration <= generation) return next() - logResult(result, 'context overflow recovery') - return { action: 'retry' } - }) -} diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index d420851166..64f8586bac 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -9,10 +9,14 @@ import z from 'schemastery' import { CompactService } from '@deepseek-ai/dsh-compact' import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' import type { Session } from '@deepseek-ai/dsh-session' +import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import { + TOKEN_METER_MODEL_UNCONFIGURED, + TokenMeterError, +} from '@deepseek-ai/dsh-token-meter' import type { ModelTokenMeter } from '@deepseek-ai/dsh-token-meter' import type { Agent } from '@deepseek-ai/dsh-agent' -import { registerAutomaticCompaction } from './automatic.ts' import { resolveConfig, resolveModelConfig } from './config.ts' import { compactSurfaceRegion, selectCompactableRange } from './region.ts' import { summarizeWithLlm } from './summarizer.ts' @@ -72,7 +76,67 @@ export class BasicCompactService extends CompactService { constructor(ctx: Context, config: BasicCompactConfig = {}) { super(ctx) this.config = resolveConfig(config, ctx.tokenMeter) - if (this.config.auto) registerAutomaticCompaction(ctx, this) + if (this.config.auto) this._registerAutomaticCompaction() + } + + /** + * Register the automatic post-step pressure and context-overflow recovery + * listeners. `compactIfNeeded` stays dynamically dispatched so subclass + * overrides are honored at event time. + */ + private _registerAutomaticCompaction(): void { + const { ctx } = this + const logResult = (result: CompactionResult, trigger: string): void => { + ctx.logger.info( + `compaction (${trigger}): shadowed ${result.shadowedSeqs.length} surface nodes ` + + `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, ` + + `~${result.shadowedTokenCount} tokens)`, + ) + } + + ctx.on('agent/post-step', async ( + agent: Agent, + _turn: number, + _step: number, + signal: AbortSignal, + ) => { + if (signal.aborted) return + try { + const result = await this.compactIfNeeded(agent, 'pressure', signal) + if (result !== null) logResult(result, 'post-step pressure') + } catch (error: unknown) { + // A named routed model without a meter profile is configuration failure, + // not an optional operational compaction miss. + if (error instanceof TokenMeterError + && error.code === TOKEN_METER_MODEL_UNCONFIGURED) throw error + const message = error instanceof Error ? error.message : String(error) + ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`) + } + }) + + ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => { + if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE + || retryAttempt >= this.config.maxOverflowRetries + || signal.aborted) return next() + + let generation: number + let result: CompactionResult | null + try { + generation = agent.session.surface.replaceGeneration + result = await this.compactIfNeeded(agent, 'context-overflow', signal) + } catch (recoveryError: unknown) { + const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError) + ctx.logger.warn( + `context-overflow compaction failed: ${message}; preserving the original request error`, + ) + return next() + } + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited. + if (signal.aborted || result === null + || agent.session.surface.replaceGeneration <= generation) return next() + logResult(result, 'context overflow recovery') + return { action: 'retry' } + }) } /** From 231aeabe55f3d58b5f02cd0f9a1495c529613db9 Mon Sep 17 00:00:00 2001 From: Hypatia May Date: Thu, 16 Jul 2026 17:11:41 +0800 Subject: [PATCH 10/73] docs(compact): align recovery RFC with singleton meter --- ...compaction-pressure-and-overflow-recovery.i18n.yaml | 4 ++-- ...r-call-compaction-pressure-and-overflow-recovery.md | 10 +++++----- ...all-compaction-pressure-and-overflow-recovery.zh.md | 10 +++++----- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml index dbb9c76bbf..7cfb11d29d 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 92167dc7d444a3620abfbaab721260ed1c828db9 -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: fe3b6617b25a58ef2c1c311df088f9c805fe9ef4 +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: 7d68bc32d3860bf5edd94c4eda76922c91ae6af2 +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 2315bd4d9ca9b93eb9a8d4850f917e6aa1bc6476 diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md index 92167dc7d4..7d68bc32d3 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md @@ -18,7 +18,7 @@ Successful calls are not the only pressure signal. A provider can reject a reque The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after assistant output, every dispatched or synthetic tool result, post-tool context, and steering are durable, but before `step/end`. This placement gives pressure policy the complete successful-call state without splitting an assistant tool call from its result. A listener failure is an ordinary turn failure; it never enters model-request recovery. -`dsh-compact-basic` resolves the exact latest routed model from the durable request header and asks that model's `ctx.tokenMeter` handle to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work. A durable unknown model throws `TOKEN_METER_MODEL_UNCONFIGURED` with its exact name and fails the otherwise-successful turn; operational selection or summarization failures warn and continue with full history. +`dsh-compact-basic` reads the exact latest routed model from the durable request header only to establish that a completed route exists, then asks the singleton `ctx.tokenMeter` to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work; any durable non-empty model name uses the same estimator. Operational measurement or summarization failures warn and continue with full history. ### Request recovery is limited to the final model boundary @@ -32,11 +32,11 @@ If cancellation lands after assistant tool calls are durable but before all call `CompactService.compactIfNeeded(agent, trigger, signal)` accepts `trigger: 'pressure' | 'context-overflow'`. The interface gains no estimation methods or token types; `ctx.tokenMeter` remains the reusable accounting owner. -For `pressure`, compact-basic applies the selected meter profile's threshold and retained-tail policy, compares scalar and surface `logRevision`, and uses the same meter for range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`. +For `pressure`, compact-basic applies the service-wide threshold and retained-tail policy to one unified `ctx.tokenMeter.measure()` result. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`. For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It chooses the maximal tool-balanced head range while leaving the newest indivisible unit, then attempts exactly one shrinking compaction under the same signal. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` only when compaction succeeds and the generation increases. A backend returning a result without replacement cannot authorize retry. -`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, missing or unknown routed models, no safe range, no generation change, and recovery throws all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. Cancellation or disposal remains authoritative even if recovery work completes concurrently. +`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, a missing routed model, no safe range, no generation change, and recovery throws all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. Cancellation or disposal remains authoritative even if recovery work completes concurrently. The default summarizer still resolves explicit configuration, then the latest logged route, then agent options. Because direct `llm/stream` middleware may reroute that auxiliary call, `compact/summary.model` records the final mutable `GenerateOptions.model` observed after dispatch rather than the pre-waterfall candidate. @@ -44,7 +44,7 @@ The default summarizer still resolves explicit configuration, then the latest lo Lifecycle tests pin post-step ordering after durable tool/context/steering work, content-less and max-token successes, final-adapter dispatch/iterator/in-band boundaries, retry numbering, attempt reset, cancellation, disposal, synthetic tool results, and original error identity. -Compact tests pin low-friction defaults, actual routed-model selection, exact unknown-model behavior, below-threshold forced overflow, newest tool-pair retention, non-shrinking rejection, generation proof, caps, disabled listeners, single downstream delegation, and auxiliary summary routing provenance. Real-loop composition covers both thrown and in-band overflow: the failed step closes, compaction lands between attempts, and the next numbered request is reconstructed from the replacement surface. +Compact tests pin low-friction service-wide defaults, actual routed-model selection, unlisted-model measurement, unified pressure-and-retention decisions, below-threshold forced overflow, newest tool-pair retention, non-shrinking rejection, generation proof, caps, disabled listeners, single downstream delegation, and auxiliary summary routing provenance. Real-loop composition covers both thrown and in-band overflow: the failed step closes, compaction lands between attempts, and the next numbered request is reconstructed from the replacement surface. ## Alternatives considered @@ -52,7 +52,7 @@ Compact tests pin low-friction defaults, actual routed-model selection, exact un - **Retry the same numbered step** — rejected because recovery appends durable events after the failed boundary. A new step preserves balanced nesting and reconstructability. - **Retry whenever `compactIfNeeded` returns a result** — rejected because a custom backend can report success without changing model-visible state. `replaceGeneration` is the authoritative proof. - **Let compact-basic parse provider wording** — rejected because classification belongs at adapters and must cover both thrown and in-band delivery. -- **Use a universal model/window fallback during recovery** — rejected because destructive policy under the wrong context capacity can hide the original provider failure. Unknown durable routes delegate unchanged. +- **Fall back to `AgentOptions.model` when no durable route exists** — rejected because automatic policy must describe a completed logged request. Headerless pressure and recovery delegate unchanged. ## Consequences diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md index fe3b6617b2..2315bd4d9c 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md @@ -18,7 +18,7 @@ Status: implemented 循环在 assistant 输出、所有已分发或合成的工具结果、工具后上下文与 steering 都持久化之后、`step/end` 之前，触发等待式串行 `agent/post-step(agent, turn, step, signal)`。该位置让压力策略看到完整的成功调用状态，同时不会拆开 assistant 工具调用与其结果。监听器失败属于普通 turn 失败，绝不会进入模型请求恢复。 -`dsh-compact-basic` 从持久请求头解析精确的最新实际路由模型，并让该模型的 `ctx.tokenMeter` handle 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断，因此不执行工作。持久记录的未知模型会携带精确名称抛出 `TOKEN_METER_MODEL_UNCONFIGURED`，使原本成功的 turn 失败；操作性的选择或摘要失败则警告并继续使用完整历史。 +`dsh-compact-basic` 从持久请求头读取精确的最新实际路由模型，只用它确认已经存在完整路由，随后让单例 `ctx.tokenMeter` 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断，因此不执行工作；任意持久记录的非空模型名都使用同一个估算器。操作性的计量或摘要失败会发出警告，并继续使用完整历史。 ### 请求恢复只覆盖最终模型边界 @@ -32,11 +32,11 @@ Status: implemented `CompactService.compactIfNeeded(agent, trigger, signal)` 接收 `trigger: 'pressure' | 'context-overflow'`。接口不增加估算方法或 token 类型；`ctx.tokenMeter` 继续作为可复用的核算所有者。 -对于 `pressure`，compact-basic 应用所选 meter profile 的阈值与保留尾部策略，比较标量和表层的 `logRevision`，并用同一个 meter 完成范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`。 +对于 `pressure`，compact-basic 把服务级阈值与保留尾部策略应用到一次统一的 `ctx.tokenMeter.measure()` 结果。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`。 对于规范化溢出，compact-basic 绕过标量压力与普通保留 token 预算。它在保留最新不可分割单元的同时，选择最大的工具配对平衡头部范围，并在同一 signal 下只尝试一次缩小压缩。自动监听器先记录 `session.surface.replaceGeneration`，只有压缩成功且 generation 增加时才返回 `{ action: 'retry' }`。后端若只返回结果但没有替换表层，不能授权重试。 -`maxOverflowRetries` 可选且默认为 `1`；`0` 只禁用溢出恢复，不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失或未知路由模型、没有安全范围、generation 未变化，以及恢复抛错都会委托给下一个监听器。若没有后续恢复，循环报告原始提供方错误对象与代码。即使恢复工作并发完成，取消或销毁仍具有最终优先级。 +`maxOverflowRetries` 可选且默认为 `1`；`0` 只禁用溢出恢复，不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失路由模型、没有安全范围、generation 未变化，以及恢复抛错都会委托给下一个监听器。若没有后续恢复，循环报告原始提供方错误对象与代码。即使恢复工作并发完成，取消或销毁仍具有最终优先级。 默认摘要器仍依次解析显式配置、最近记录的路由与 agent options。因为直接 `llm/stream` 中间件可以重新路由该辅助调用，`compact/summary.model` 记录分发后最终可变的 `GenerateOptions.model`，而不是 waterfall 之前的候选值。 @@ -44,7 +44,7 @@ Status: implemented 生命周期测试固定 post-step 位于持久工具、上下文与 steering 工作之后，覆盖无内容与达到 token 上限的成功、最终适配器分发/迭代器/带内边界、重试编号、尝试重置、取消、销毁、合成工具结果与原始错误身份。 -压缩测试固定低摩擦默认值、实际路由模型选择、精确未知模型行为、低于阈值的强制溢出、最新工具配对保留、非缩小拒绝、generation 证明、上限、禁用监听器、单次下游委托与辅助摘要路由来源。真实循环组合同时覆盖抛出式和带内溢出：失败 step 关闭，压缩落在两次尝试之间，下一个编号请求从替换表层重建。 +压缩测试固定低摩擦服务级默认值、实际路由模型选择、未列出模型计量、统一压力与保留决策、低于阈值的强制溢出、最新工具配对保留、非缩小拒绝、generation 证明、上限、禁用监听器、单次下游委托与辅助摘要路由来源。真实循环组合同时覆盖抛出式和带内溢出：失败 step 关闭，压缩落在两次尝试之间，下一个编号请求从替换表层重建。 ## 考虑过的替代方案 @@ -52,7 +52,7 @@ Status: implemented - **重试相同编号的 step**——不予采纳，因为恢复会在失败边界之后追加持久事件。新 step 保持边界配对与可重建性。 - **只要 `compactIfNeeded` 返回结果就重试**——不予采纳，因为自定义后端可能报告成功却没有改变模型可见状态。`replaceGeneration` 才是权威证明。 - **让 compact-basic 解析提供方措辞**——不予采纳，因为分类属于适配器，而且必须同时覆盖抛出式与带内交付。 -- **恢复时使用通用模型/窗口回退**——不予采纳，因为基于错误上下文容量执行破坏性策略可能掩盖原始提供方失败。未知持久路由会原样委托。 +- **没有持久路由时回退到 `AgentOptions.model`**——不予采纳，因为自动策略必须描述已完成且已记录的请求。没有请求头的压力检查与恢复会原样委托。 ## 后果 From a6915745e068209142e68564b967b1d2a2c35e03 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 17 Jul 2026 17:21:10 +0800 Subject: [PATCH 11/73] fix(bash-local): bound inherited pipe drain and spill files --- docs/config-catalog.md | 4 +- packages/bash/bash-local/README.md | 11 +-- packages/bash/bash-local/src/index.ts | 10 ++- packages/bash/bash-local/src/run.ts | 75 ++++++++++++++---- .../bash/bash-local/tests/executor.spec.ts | 1 + packages/bash/bash-local/tests/run.spec.ts | 77 +++++++++++++++++-- 6 files changed, 151 insertions(+), 27 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index c94b5d571e..9ba1f73cf5 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -152,7 +152,9 @@ export interface Config { maxTimeoutMs?: number /** Per-stream in-memory output cap; overflow spills to a temp file. */ maxOutputBytes?: number - /** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */ + /** Per-stream spill-file cap; larger streams retain only their in-memory tail. */ + maxSpillBytes?: number + /** Grace period for kill escalation and for inherited pipes after shell exit. */ graceMs?: number } ``` diff --git a/packages/bash/bash-local/README.md b/packages/bash/bash-local/README.md index 6048f886a3..c6fc6e863d 100644 --- a/packages/bash/bash-local/README.md +++ b/packages/bash/bash-local/README.md @@ -1,6 +1,6 @@ # @deepseek-ai/dsh-bash-local -Local-subprocess implementation of the `@deepseek-ai/dsh-bash` executor seam: `LocalBashExecutor` spawns `bash -c ` per call in its own process group, collects bounded output with full-stream spill files, and escalates kills SIGTERM→SIGKILL across the whole group. +Local-subprocess implementation of the `@deepseek-ai/dsh-bash` executor seam: `LocalBashExecutor` spawns `bash -c ` per call in its own process group, collects bounded output with size-limited full-stream spill files, and escalates kills SIGTERM→SIGKILL across the whole group. The package root exports the default and named `LocalBashExecutor` plugin plus its `Config`; subprocess plumbing stays internal to the implementation package. @@ -14,7 +14,8 @@ The package root exports the default and named `LocalBashExecutor` plugin plus i timeoutMs: 120000 # default foreground timeout maxTimeoutMs: 600000 # cap for per-call overrides maxOutputBytes: 64000 # per-stream in-memory cap; overflow spills to disk - graceMs: 3000 # SIGTERM→SIGKILL escalation grace on kills + maxSpillBytes: 67108864 # per-stream full-output spill cap + graceMs: 3000 # kill escalation and post-exit pipe-drain grace ``` ## Behavior (and where it came from) @@ -22,8 +23,8 @@ The package root exports the default and named `LocalBashExecutor` plugin plus i Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi; the notable choices: - **Spawn per call, no shell state** — every call is a fresh non-login `bash -c` (deterministic; no rc files). All four surveyed tools spawn per call. `XXX(stateful-shell)` in `src/run.ts` records the two proven stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec sessions) for when real workflows demand them. -- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the `graceMs` grace (default 3s — OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools. -- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file. +- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the `graceMs` grace (default 3s — OpenCode's escalation; pipelines and subshells die with the parent). After the main shell exits, inherited stdout/stderr pipes receive the same bounded drain grace so a surviving descendant cannot hold the command open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools. +- **Tail-keep truncation + bounded spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. A stream larger than `maxSpillBytes` discards its now-incomplete spill and returns only the marked truncated tail. If the final spill close reports a delayed writeback failure, the executor likewise withholds the path rather than advertising an incomplete file. - **Model-friendly env + credential scrub** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`), then `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` (Codex's hardcoded set) so pagers and ANSI color don't garble results. This scrub is the security control that keeps the harness's *ambient* credentials out of a spawned command. A spec's `env` is merged LAST (after the scrub), so a caller's explicit entry — a value it already holds — wins even on a credential-shaped name. The spec's `stdin`, when supplied, is written to the child and closed; with none supplied, fd 0 is `/dev/null` — the exact pre-seam default, so a command that probes stdin's file type is unaffected. Both `env`/`stdin` are set by in-process plugins (the hooks bridges); the model-facing tool doesn't expose them. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md). - **Background processes** — `start()` returns a live `BashProcess` handle immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), the handle's `readOutput()` is incremental with whole-stream byte offsets, and disposal kills every running process and awaits its exit. Everything task-shaped (ids, ownership, polling, notices) lives in the generic [`ctx.tasks` runtime](../../tasks/tasks/README.md), which the tool layer registers the handle with — this executor never sees a session or a registry. @@ -37,6 +38,6 @@ Indirectly, through `dsh-tool-bash`, which renders this executor's bounded stdou - **No persistent shell or PTY** — every call starts a fresh non-login `bash -c`; cwd-only persistence and interactive terminal sessions remain deferred until a real workflow requires them. - **POSIX-only** — the `bash` binary, detached process groups, group kills, and SIGTERM→SIGKILL escalation are hardcoded; Windows is unsupported. - **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. -- **Spill files are never deleted** — full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them. +- **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are deleted immediately. The raw process handling lives in `src/run.ts`; `src/index.ts` is the service wiring. diff --git a/packages/bash/bash-local/src/index.ts b/packages/bash/bash-local/src/index.ts index 9c2b0b4511..4e942abb06 100644 --- a/packages/bash/bash-local/src/index.ts +++ b/packages/bash/bash-local/src/index.ts @@ -10,7 +10,7 @@ import z from 'schemastery' import { BashExecutor } from '@deepseek-ai/dsh-bash' import type { BashExecRequest, BashExecSpec, BashProcess, BashProcessRead, BashRunResult } from '@deepseek-ai/dsh-bash' import { clampTimeout, deadline, timeoutOf } from '@deepseek-ai/dsh-timeout' -import { DEFAULT_GRACE_MS, runBash } from './run.ts' +import { DEFAULT_GRACE_MS, DEFAULT_MAX_SPILL_BYTES, runBash } from './run.ts' import type { RunInternals, RunningBash } from './run.ts' /** Plugin config (all optional — `static Config` supplies the defaults). */ @@ -23,7 +23,9 @@ export interface Config { maxTimeoutMs?: number /** Per-stream in-memory output cap; overflow spills to a temp file. */ maxOutputBytes?: number - /** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */ + /** Per-stream spill-file cap; larger streams retain only their in-memory tail. */ + maxSpillBytes?: number + /** Grace period for kill escalation and for inherited pipes after shell exit. */ graceMs?: number } @@ -46,6 +48,7 @@ export class LocalBashExecutor extends BashExecutor { timeoutMs: z.number().default(120_000), maxTimeoutMs: z.number().default(600_000), maxOutputBytes: z.number().default(64_000), + maxSpillBytes: z.number().default(DEFAULT_MAX_SPILL_BYTES), graceMs: z.number().default(DEFAULT_GRACE_MS), }) @@ -64,6 +67,7 @@ export class LocalBashExecutor extends BashExecutor { assertPositiveFinite('timeoutMs', this.config.timeoutMs) assertPositiveFinite('maxTimeoutMs', this.config.maxTimeoutMs) assertPositiveFinite('maxOutputBytes', this.config.maxOutputBytes) + assertPositiveFinite('maxSpillBytes', this.config.maxSpillBytes) assertPositiveFinite('graceMs', this.config.graceMs) ctx.effect(() => async () => { // Await closure so even a TERM-trapping child cannot outlive the fiber. @@ -112,6 +116,7 @@ export class LocalBashExecutor extends BashExecutor { command: spec.command, cwd: spec.workdir, maxOutputBytes: this.config.maxOutputBytes, + maxSpillBytes: this.config.maxSpillBytes, graceMs: this.config.graceMs, signal: d.signal, stdin: spec.stdin, @@ -129,6 +134,7 @@ export class LocalBashExecutor extends BashExecutor { command: spec.command, cwd: spec.workdir, maxOutputBytes: this.config.maxOutputBytes, + maxSpillBytes: this.config.maxSpillBytes, graceMs: this.config.graceMs, signal: spec.signal, stdin: spec.stdin, diff --git a/packages/bash/bash-local/src/run.ts b/packages/bash/bash-local/src/run.ts index 02e7a963be..d475bcb538 100644 --- a/packages/bash/bash-local/src/run.ts +++ b/packages/bash/bash-local/src/run.ts @@ -8,7 +8,7 @@ import { type ChildProcessByStdio, spawn } from 'node:child_process' import type { Readable, Writable } from 'node:stream' import { randomBytes } from 'node:crypto' -import { closeSync, mkdtempSync, openSync, writeSync } from 'node:fs' +import { closeSync, mkdtempSync, openSync, unlinkSync, writeSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import type { CollectedOutput } from '@deepseek-ai/dsh-bash' @@ -54,7 +54,9 @@ export interface SpawnSpec { cwd: string /** Per-stream in-memory cap; overflow spills to disk (tail kept in memory). */ maxOutputBytes: number - /** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */ + /** Per-stream spill-file cap; larger streams retain only their in-memory tail. */ + maxSpillBytes: number + /** Grace period for kill escalation and for inherited pipes after shell exit. */ graceMs: number /** * Abort signal — kills the process group when it fires. The executor owns @@ -101,6 +103,9 @@ export interface RunInternals { /** Default SIGTERM→SIGKILL grace period (the `graceMs` config; matches OpenCode's 3s). */ export const DEFAULT_GRACE_MS = 3_000 +/** Default per-stream spill cap (the `maxSpillBytes` config). */ +export const DEFAULT_MAX_SPILL_BYTES = 64 * 1024 * 1024 + let spillCounter = 0 let defaultSpillDir: string | undefined @@ -115,9 +120,9 @@ function privateSpillDir(): string { } /** - * Collects one stream with a bounded in-memory tail. The FULL stream is - * always recoverable: on first overflow a spill file is created and every - * chunk (including those already collected) is appended there. + * Collects one stream with a bounded in-memory tail. On first overflow a + * spill file is created and every chunk (including those already collected) + * is appended there while the full stream remains within `maxSpillBytes`. * * Tail-keep rationale (pi/OpenCode): errors and final results cluster at the * end of command output; the spill file covers the head. @@ -128,11 +133,13 @@ export class OutputCollector { private dropped = false private spillFd: number | undefined private spillFile: string | undefined + private spillDisabled = false /** Total bytes ever pushed (not just retained). */ private total = 0 constructor( private readonly maxBytes: number, + private readonly maxSpillBytes: number, private readonly label: string, private readonly spillDir: string, ) {} @@ -148,7 +155,7 @@ export class OutputCollector { push(chunk: Buffer): void { this.total += chunk.length const overflows = this.bytes + chunk.length > this.maxBytes - if (overflows || this.spillFd !== undefined) this.spillAll(chunk) + if (!this.spillDisabled && (overflows || this.spillFd !== undefined)) this.spillAll(chunk) this.chunks.push(chunk) this.bytes += chunk.length while (this.bytes > this.maxBytes && this.chunks.length > 1) { @@ -170,6 +177,10 @@ export class OutputCollector { /** Open the spill file lazily and append `chunk` (and any prior chunks once). */ private spillAll(chunk: Buffer): void { + if (this.total > this.maxSpillBytes) { + this.discardSpill() + return + } if (this.spillFd === undefined) { // Random suffix + O_EXCL + no-follow-equivalent ('wx' fails on any // existing path, symlink or not) + owner-only mode: defeats spill-path @@ -184,6 +195,30 @@ export class OutputCollector { writeSync(this.spillFd, chunk) } + /** Stop spilling and remove the file once it can no longer hold the complete stream. */ + private discardSpill(): void { + const fd = this.spillFd + const file = this.spillFile + this.spillFd = undefined + this.spillFile = undefined + this.spillDisabled = true + if (fd !== undefined) { + try { + closeSync(fd) + } catch { + // Retain the descriptor so finalize can retry the failed close. + this.spillFd = fd + } + } + if (file !== undefined) { + try { + unlinkSync(file) + } catch { + // A failed unlink leaves at most maxSpillBytes behind, never an unbounded file. + } + } + } + /** * Incremental read in whole-stream byte coordinates: returns everything * pushed since `fromByte`. When `fromByte` has already slid out of the @@ -283,8 +318,8 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB ? spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['pipe', 'pipe', 'pipe'], detached: true }) : spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['ignore', 'pipe', 'pipe'], detached: true }) - const stdout = new OutputCollector(spec.maxOutputBytes, 'stdout', spillDir) - const stderr = new OutputCollector(spec.maxOutputBytes, 'stderr', spillDir) + const stdout = new OutputCollector(spec.maxOutputBytes, spec.maxSpillBytes, 'stdout', spillDir) + const stderr = new OutputCollector(spec.maxOutputBytes, spec.maxSpillBytes, 'stderr', spillDir) child.stdout.on('data', (chunk: Buffer) => { stdout.push(chunk) }) child.stderr.on('data', (chunk: Buffer) => { stderr.push(chunk) }) @@ -310,12 +345,13 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB } const done = new Promise((resolve, reject) => { - child.on('error', (error) => { - // No meaningful close outcome follows a spawn failure. - cleanup() - reject(error) - }) - child.on('close', (exitCode, signal) => { + let settled = false + let pipeDrainTimer: NodeJS.Timeout | undefined + const settle = (exitCode: number | null, signal: NodeJS.Signals | null): void => { + if (settled) return + settled = true + child.stdout.destroy() + child.stderr.destroy() cleanup() resolve({ exitCode, @@ -323,9 +359,20 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB stdout: stdout.finalize(), stderr: stderr.finalize(), }) + } + child.on('error', (error) => { + // No meaningful close outcome follows a spawn failure. + settled = true + cleanup() + reject(error) }) + child.on('exit', (exitCode, signal) => { + pipeDrainTimer = setTimeout(() => { settle(exitCode, signal) }, spec.graceMs) + }) + child.on('close', settle) function cleanup(): void { if (graceTimer !== undefined) clearTimeout(graceTimer) + if (pipeDrainTimer !== undefined) clearTimeout(pipeDrainTimer) spec.signal?.removeEventListener('abort', onAbort) } }) diff --git a/packages/bash/bash-local/tests/executor.spec.ts b/packages/bash/bash-local/tests/executor.spec.ts index b176cbaf86..4aa4f50ca1 100644 --- a/packages/bash/bash-local/tests/executor.spec.ts +++ b/packages/bash/bash-local/tests/executor.spec.ts @@ -66,6 +66,7 @@ describe('LocalBashExecutor.run', () => { await expect(setup({ timeoutMs: Number.NaN })).rejects.toThrow(/timeoutMs/) await expect(setup({ maxTimeoutMs: 0 })).rejects.toThrow(/maxTimeoutMs/) await expect(setup({ maxOutputBytes: -1 })).rejects.toThrow(/maxOutputBytes/) + await expect(setup({ maxSpillBytes: 0 })).rejects.toThrow(/maxSpillBytes/) await expect(setup({ graceMs: 0 })).rejects.toThrow(/graceMs/) const { bash } = await setup() diff --git a/packages/bash/bash-local/tests/run.spec.ts b/packages/bash/bash-local/tests/run.spec.ts index 923b3adf7b..b182575f1a 100644 --- a/packages/bash/bash-local/tests/run.spec.ts +++ b/packages/bash/bash-local/tests/run.spec.ts @@ -1,11 +1,14 @@ -import { mkdtempSync, readFileSync, statSync } from 'node:fs' +import { mkdtempSync, readFileSync, statSync, unlinkSync } from 'node:fs' import { tmpdir } from 'node:os' import { dirname, join } from 'node:path' import { describe, expect, it, vi } from 'vitest' import { killGroup, OutputCollector, runBash } from '../src/run.ts' import type { RunningBash } from '../src/run.ts' -const { failNextClose } = vi.hoisted(() => ({ failNextClose: { value: false } })) +const { failNextClose, failNextUnlink } = vi.hoisted(() => ({ + failNextClose: { value: false }, + failNextUnlink: { value: false }, +})) vi.mock('node:fs', async (importOriginal) => { const actual = await importOriginal() return { @@ -17,6 +20,13 @@ vi.mock('node:fs', async (importOriginal) => { } actual.closeSync(fd) }, + unlinkSync(path: Parameters[0]): void { + if (failNextUnlink.value) { + failNextUnlink.value = false + throw Object.assign(new Error('simulated EIO on unlink'), { code: 'EIO' }) + } + actual.unlinkSync(path) + }, } }) @@ -27,6 +37,7 @@ function spec(command: string, overrides: Partial[0]> command, cwd: process.cwd(), maxOutputBytes: 64_000, + maxSpillBytes: 64 * 1024 * 1024, graceMs: 3_000, ...overrides, } @@ -171,6 +182,22 @@ describe('runBash', () => { const result = await running.done expect(result.signal).toBe('SIGTERM') }) + + it('bounds inherited-pipe draining after the shell exits', async () => { + const pidFile = join(spillDir, `pipe-holder-${Date.now()}.pid`) + const started = Date.now() + const running = runBash(spec(`sleep 60 & echo $! > ${pidFile}; echo shell-done`, { graceMs: 100 })) + const descendant = await waitForPidFile(pidFile) + try { + const result = await running.done + expect(Date.now() - started).toBeLessThan(1_000) + expect(result.exitCode).toBe(0) + expect(result.stdout.text).toBe('shell-done\n') + } finally { + process.kill(descendant, 'SIGKILL') + await waitGone(descendant) + } + }) }) describe('stdin and extra env (set by in-process plugins)', () => { @@ -266,7 +293,7 @@ describe('output truncation and spill', () => { describe('OutputCollector', () => { it('keeps the tail of a single oversized chunk', () => { - const collector = new OutputCollector(10, 'test', spillDir) + const collector = new OutputCollector(10, 100, 'test', spillDir) collector.push(Buffer.from('0123456789abcdef')) const out = collector.finalize() expect(out.text).toBe('6789abcdef') @@ -275,7 +302,7 @@ describe('OutputCollector', () => { }) it('readFrom returns increments and flags lossy reads', () => { - const collector = new OutputCollector(10, 'test', spillDir) + const collector = new OutputCollector(10, 100, 'test', spillDir) collector.push(Buffer.from('aaaaa')) const first = collector.readFrom(0) expect(first.text).toBe('aaaaa') @@ -296,7 +323,7 @@ describe('OutputCollector', () => { }) it('contains close failures and drops the spill path', () => { - const collector = new OutputCollector(4, 'closefail', spillDir) + const collector = new OutputCollector(4, 100, 'closefail', spillDir) collector.push(Buffer.from('aaaa')) collector.push(Buffer.from('bbbb')) expect(collector.readFrom(0).spillPath).toBeDefined() @@ -310,6 +337,46 @@ describe('OutputCollector', () => { expect(out!.truncated).toBe(true) expect(out!.spillPath).toBeUndefined() }) + + it('discards a spill that exceeds its configured cap', () => { + const collector = new OutputCollector(4, 8, 'bounded', spillDir) + collector.push(Buffer.from('aaaa')) + collector.push(Buffer.from('bbbb')) + const spillPath = collector.readFrom(0).spillPath! + expect(readFileSync(spillPath, 'utf8')).toBe('aaaabbbb') + + collector.push(Buffer.from('c')) + collector.push(Buffer.from('dddd')) + const out = collector.finalize() + expect(out.text).toBe('dddd') + expect(out.truncated).toBe(true) + expect(out.spillPath).toBeUndefined() + expect(() => readFileSync(spillPath)).toThrow() + }) + + it('does not create a spill when the first overflowing chunk exceeds the cap', () => { + const collector = new OutputCollector(4, 4, 'no-spill', spillDir) + collector.push(Buffer.from('abcdefgh')) + const out = collector.finalize() + expect(out.text).toBe('efgh') + expect(out.truncated).toBe(true) + expect(out.spillPath).toBeUndefined() + }) + + it('contains cleanup failures while disabling an oversize spill', () => { + const collector = new OutputCollector(4, 8, 'cleanup-fail', spillDir) + collector.push(Buffer.from('aaaa')) + collector.push(Buffer.from('bbbb')) + const spillPath = collector.readFrom(0).spillPath! + + failNextClose.value = true + failNextUnlink.value = true + expect(() => { collector.push(Buffer.from('c')) }).not.toThrow() + expect(failNextClose.value).toBe(false) + expect(failNextUnlink.value).toBe(false) + expect(collector.finalize().spillPath).toBeUndefined() + unlinkSync(spillPath) + }) }) describe('killGroup', () => { From a38ff125a7a9c4d3e9fc3b0e6e86ade46e9b3876 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 17 Jul 2026 17:29:38 +0800 Subject: [PATCH 12/73] feat(dsbench): add SDK evaluation composition --- docs/config-catalog.md | 16 +- examples/README.md | 4 + examples/dsbench-coding-agent/README.md | 24 +++ examples/dsbench-coding-agent/cordis.yml | 77 +++++++++ examples/dsbench-coding-agent/package.json | 7 + .../tests/keyless-smoke.e2e.ts | 146 ++++++++++++++++++ packages/examples/acp-demo/src/index.ts | 4 +- packages/examples/agent-spine-demo/README.md | 2 +- .../examples/agent-spine-demo/src/index.ts | 20 ++- .../agent-spine-demo/tests/agent-core.spec.ts | 21 ++- packages/examples/stdio-demo/src/index.ts | 4 +- packages/llm/llm-deepseek/src/adapter.ts | 3 + .../llm/llm-deepseek/tests/adapter.spec.ts | 14 ++ packages/ui/jsonrpc/README.md | 2 +- packages/ui/jsonrpc/src/index.ts | 12 +- packages/ui/jsonrpc/src/server.ts | 16 +- packages/ui/jsonrpc/tests/server.spec.ts | 22 ++- 17 files changed, 365 insertions(+), 29 deletions(-) create mode 100644 examples/dsbench-coding-agent/README.md create mode 100644 examples/dsbench-coding-agent/cordis.yml create mode 100644 examples/dsbench-coding-agent/package.json create mode 100644 examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 9ba1f73cf5..c888731228 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -56,7 +56,7 @@ export interface Config { skills?: agentCore.SkillConfig /** Model-facing bash tool config forwarded through agent-core. */ toolBash?: NonNullable - /** Generic background-task control-tool config forwarded through agent-core. */ + /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable } ``` @@ -120,12 +120,14 @@ export interface Config { skills?: SkillConfig /** Model-facing bash tool config, including this producer's background opt-in. */ toolBash?: toolBash.Config - /** Generic background-task control-tool wait bounds. */ - toolTasks?: toolTasks.Config + /** Generic background-task controls; set false to keep the task service without model-facing task tools. */ + toolTasks?: toolTasks.Config | false } /** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */ export interface SkillConfig { + /** Mount the bundled local skill provider and model-facing skill tool (default true). */ + enabled?: boolean /** Registry-level discovery cache settings. */ registry?: SkillRegistryConfig /** Local filesystem skill provider settings. */ @@ -137,7 +139,7 @@ export interface SkillConfig { Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts) -Source: [`packages/examples/agent-spine-demo/src/index.ts:55`](../packages/examples/agent-spine-demo/src/index.ts) +Source: [`packages/examples/agent-spine-demo/src/index.ts:57`](../packages/examples/agent-spine-demo/src/index.ts) ## `@deepseek-ai/dsh-bash-local` @@ -343,8 +345,10 @@ Source: [`packages/hooks/hooks-codex/src/index.ts:41`](../packages/hooks/hooks-c Requires: `agents` ```ts config-catalog -/** Runtime-only test seams; no field is configurable from `cordis.yml`. */ +/** JSON-RPC deployment config plus runtime-only test seams. */ export interface JsonRpcConfig { + /** Report max-token turn/subagent termination as a successful SDK result. */ + maxTokensAsSuccess?: boolean /** Transport input override; production uses `process.stdin`. */ input?: Readable /** Transport output override; production uses `process.stdout`. */ @@ -724,7 +728,7 @@ export interface Config { skills?: agentCore.SkillConfig /** Model-facing bash tool config forwarded through agent-core. */ toolBash?: NonNullable - /** Generic background-task control-tool config forwarded through agent-core. */ + /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable /** * If set, the `main` agent RESUMES this persisted session id instead of diff --git a/examples/README.md b/examples/README.md index 5db1e18372..556a3bf41e 100644 --- a/examples/README.md +++ b/examples/README.md @@ -21,6 +21,10 @@ Run with: `pnpm run demo:repl` (needs `DEEPSEEK_API_KEY` in the environment or a Run the Code Mode overlay with `pnpm run demo:code-mode`, or pass `acp` for the ACP example. See the [Code Mode example](coding-agent/README.md#code-mode) for its composition and a sample task. +## dsbench-coding-agent + +The unattended SDK composition used by DSBench: JSON-RPC stdio, foreground-only `bash`, `read` / `write` / `edit`, one foreground `subagent`, `todo_write`, JSONL persistence, and compaction. It excludes terminal UI, stdout logging, approvals, skills, and background task controls. See [dsbench-coding-agent/README.md](dsbench-coding-agent/README.md). + ## cordis-agent The **self-referential** demo: the coding spine plus [`@deepseek-ai/dsh-tool-cordis`](../packages/cordis/tool-cordis), whose three tools (`cordis_inspect` / `cordis_mount` / `cordis_unmount`) let the agent inspect the live cordis runtime it runs inside, mount model-written plugins into it (an event listener, a brand-new tool for itself, or a service another mount injects), and dispose them again — all dynamic mounts grouped under one `cordis-dynamic` fiber subtree. The `ctx.fs`/`ctx.web` services ride along provider-only, as the capabilities those plugins build on. diff --git a/examples/dsbench-coding-agent/README.md b/examples/dsbench-coding-agent/README.md new file mode 100644 index 0000000000..ff45ba36cf --- /dev/null +++ b/examples/dsbench-coding-agent/README.md @@ -0,0 +1,24 @@ +# dsbench-coding-agent + +The DSBench deployment composition for the Python SDK's bundled JSON-RPC runtime. It intentionally loads no terminal UI, console logger, approval surface, or user-interaction tool because stdout belongs to the SDK protocol and benchmark turns are unattended. + +The model-facing tools are: + +- `bash`, foreground only +- `read`, `write`, and `edit` +- `subagent`, using one foreground in-process spawn provider +- `todo_write` + +The surrounding runtime also loads JSONL session persistence and automatic context compaction. `maxTokensAsSuccess` keeps a token-limited model turn as an accepted benchmark result while preserving its `max-tokens` reason. + +## Runtime environment + +| Variable | Purpose | +|---|---| +| `DEEPSEEK_API_KEY` | Credential passed to the OpenAI-compatible host endpoint | +| `DEEPSEEK_BASE_URL` | Host endpoint used by `dsh-llm-deepseek` | +| `DSH_CWD` | Benchmark workspace for bash and filesystem tools | +| `DSH_SESSION_ROOT` | JSONL trajectory directory | +| `DSH_SYSTEM_PROMPT` | DSBench-provided coding persona | + +Pass the config path through the Python SDK's `cordis` option or `DSH_CORDIS_CONFIG`. The bundled executable already carries every plugin named by this file; the target machine does not need Node.js. diff --git a/examples/dsbench-coding-agent/cordis.yml b/examples/dsbench-coding-agent/cordis.yml new file mode 100644 index 0000000000..67e1604b07 --- /dev/null +++ b/examples/dsbench-coding-agent/cordis.yml @@ -0,0 +1,77 @@ +# DSBench deployment for the bundled dsh-jsonrpc-agent runtime. +# stdout is reserved for JSON-RPC; do not add a console logger or terminal UI. + +- id: jsonrpc + name: '@deepseek-ai/dsh-jsonrpc' + config: + maxTokensAsSuccess: true + +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + config: + apiKey: !!js process.env.DEEPSEEK_API_KEY + baseURL: !!js process.env.DEEPSEEK_BASE_URL + models: + - deepseek-v4-flash + - deepseek-v4-pro + +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: !!js process.env.DSH_CWD ?? process.cwd() + timeoutMs: 60000 + +- id: agent-spine + name: '@deepseek-ai/dsh-agent-spine-demo' + config: + persona: !!js process.env.DSH_SYSTEM_PROMPT ?? 'You are a coding agent driven by DSBench.' + workspaceContext: false + skills: + enabled: false + toolBash: + enableRunInBackground: false + toolTasks: false + +- id: sessions + name: '@deepseek-ai/dsh-session-persistence-jsonl' + config: + root: !!js process.env.DSH_SESSION_ROOT ?? './.sessions' + +- id: subagent + name: '@deepseek-ai/dsh-subagent' + +- id: subagent-spawn + name: '@deepseek-ai/dsh-subagent-spawn' + config: + providerName: spawn + +- id: tool-subagent + name: '@deepseek-ai/dsh-tool-subagent' + config: + provider: spawn + toolName: subagent + enableRunInBackground: false + +- id: tool-todo + name: '@deepseek-ai/dsh-tool-todo' + +- id: fs-local + name: '@deepseek-ai/dsh-fs-local' + config: + cwd: !!js process.env.DSH_CWD ?? process.cwd() + +- id: fs-policy + name: '@deepseek-ai/dsh-fs-policy' + +- id: tool-fs + name: '@deepseek-ai/dsh-tool-fs' + +- id: compact-basic + name: '@deepseek-ai/dsh-compact-basic' + config: + contextWindow: 128000 + thresholdRatio: 0.8 + retainTokens: 20480 + summarizationModel: '' + maxTokens: 8192 + compactionRetries: 1 diff --git a/examples/dsbench-coding-agent/package.json b/examples/dsbench-coding-agent/package.json new file mode 100644 index 0000000000..900c88d69b --- /dev/null +++ b/examples/dsbench-coding-agent/package.json @@ -0,0 +1,7 @@ +{ + "name": "dsbench-coding-agent-example", + "private": true, + "version": "0.0.1", + "type": "module", + "description": "DSBench JSON-RPC coding-agent composition" +} diff --git a/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts b/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts new file mode 100644 index 0000000000..9bbc07a829 --- /dev/null +++ b/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts @@ -0,0 +1,146 @@ +import { spawn } from 'node:child_process' +import { createServer } from 'node:http' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { describe, expect, it } from 'vitest' + +const binScript = fileURLToPath(new URL('../../../packages/examples/jsonrpc-demo/src/bin.ts', import.meta.url)) +const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) +const repoRoot = fileURLToPath(new URL('../../..', import.meta.url)) + +function waitForLine( + lines: string[], + predicate: (value: Record) => boolean, + stderr: () => string, +): Promise> { + return new Promise((resolve, reject) => { + const deadline = Date.now() + 30_000 + const poll = (): void => { + while (lines.length > 0) { + const line = lines.shift()! + if (!line.trim()) continue + try { + const value = JSON.parse(line) as Record + if (predicate(value)) { + resolve(value) + return + } + } catch { + reject(new Error(`non-JSON stdout from DSBench runtime: ${line}`)) + return + } + } + if (Date.now() >= deadline) { + reject(new Error(`timed out waiting for JSON-RPC response; stderr=${stderr()}`)) + return + } + setTimeout(poll, 10) + } + poll() + }) +} + +describe('dsbench-coding-agent keyless smoke', () => { + it('boots the real Cordis tree and serves initialize/shutdown over clean stdout', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-dsbench-smoke-')) + const modelRequests: Record[] = [] + const modelServer = createServer((request, response) => { + let body = '' + request.setEncoding('utf8') + request.on('data', (chunk: string) => { body += chunk }) + request.on('end', () => { + modelRequests.push(JSON.parse(body) as Record) + response.writeHead(200, { 'content-type': 'text/event-stream' }) + response.write('data: {"choices":[{"delta":{"role":"assistant","content":null}}]}\n\n') + response.write('data: {"choices":[{"delta":{"content":"done"}}]}\n\n') + response.write('data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}\n\n') + response.end('data: [DONE]\n\n') + }) + }) + await new Promise(resolve => modelServer.listen(0, '127.0.0.1', resolve)) + const address = modelServer.address() + if (address === null || typeof address === 'string') throw new Error('model server did not bind a TCP port') + const child = spawn(process.execPath, [ + '--expose-internals', + '--import', + 'tsx', + binScript, + configPath, + ], { + cwd: repoRoot, + env: { + ...process.env, + DEEPSEEK_API_KEY: 'keyless-smoke-no-call', + DEEPSEEK_BASE_URL: `http://127.0.0.1:${address.port}`, + DSH_CWD: root, + DSH_SESSION_ROOT: join(root, '.sessions'), + }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + const lines: string[] = [] + let stdoutBuffer = '' + let stderr = '' + child.stdout.setEncoding('utf8') + child.stdout.on('data', (chunk: string) => { + stdoutBuffer += chunk + const parts = stdoutBuffer.split('\n') + stdoutBuffer = parts.pop() ?? '' + lines.push(...parts) + }) + child.stderr.setEncoding('utf8') + child.stderr.on('data', (chunk: string) => { stderr += chunk }) + + try { + child.stdin.write(`${JSON.stringify({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { cwd: root, model: 'deepseek-v4-pro' }, + })}\n`) + const initialized = await waitForLine(lines, value => value.id === 1, () => stderr) + expect(initialized).toMatchObject({ + jsonrpc: '2.0', + id: 1, + result: { serverInfo: { name: 'deepseek-harness-sdk-runtime' } }, + }) + + child.stdin.write(`${JSON.stringify({ + jsonrpc: '2.0', + id: 2, + method: 'session/prompt', + params: { sessionId: 'main', contentBlocks: [{ type: 'text', text: 'inspect tools' }] }, + })}\n`) + const prompt = await waitForLine(lines, value => value.id === 2, () => stderr) + expect(prompt).toMatchObject({ jsonrpc: '2.0', id: 2, result: { accepted: true } }) + const tools = modelRequests[0]?.tools as { function?: { name?: string } }[] + expect(tools.map(tool => tool.function?.name).sort()).toEqual([ + 'bash', + 'edit', + 'read', + 'subagent', + 'todo_write', + 'write', + ]) + + child.stdin.write(`${JSON.stringify({ jsonrpc: '2.0', id: 3, method: 'shutdown' })}\n`) + const shutdown = await waitForLine(lines, value => value.id === 3, () => stderr) + expect(shutdown).toMatchObject({ jsonrpc: '2.0', id: 3, result: {} }) + if (child.exitCode === null) { + await new Promise((resolve, reject) => { + child.once('exit', (code) => { + if (code === 0) resolve() + else reject(new Error(`runtime exited ${code}; stderr=${stderr}`)) + }) + }) + } else { + expect(child.exitCode, stderr).toBe(0) + } + } finally { + if (child.exitCode === null) child.kill('SIGKILL') + await new Promise(resolve => modelServer.close(() => { resolve() })) + await rm(root, { recursive: true, force: true }) + } + }, 40_000) +}) diff --git a/packages/examples/acp-demo/src/index.ts b/packages/examples/acp-demo/src/index.ts index e52b21b9f6..ca94315093 100644 --- a/packages/examples/acp-demo/src/index.ts +++ b/packages/examples/acp-demo/src/index.ts @@ -46,7 +46,7 @@ export interface Config { skills?: agentCore.SkillConfig /** Model-facing bash tool config forwarded through agent-core. */ toolBash?: NonNullable - /** Generic background-task control-tool config forwarded through agent-core. */ + /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable } @@ -67,7 +67,7 @@ export const Config: z = z.object({ workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), skills: agentCore.SkillConfigSchema, toolBash: agentCore.ToolBashConfigSchema, - toolTasks: agentCore.ToolTasksConfigSchema, + toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), }) /* jscpd:ignore-end */ diff --git a/packages/examples/agent-spine-demo/README.md b/packages/examples/agent-spine-demo/README.md index 0d02d5f597..a59c741994 100644 --- a/packages/examples/agent-spine-demo/README.md +++ b/packages/examples/agent-spine-demo/README.md @@ -46,7 +46,7 @@ import type { Config } from '@deepseek-ai/dsh-agent-spine-demo' // workspaceContext requires { maxBytes } or false; the other owner schemas supply defaults. ``` -The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. `toolBash.enableRunInBackground` controls only the bash producer, while `toolTasks` controls generic `task_output` wait bounds; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields. +The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. Set `skills.enabled: false` to omit both the local provider and model-facing skill tool, and set `toolTasks: false` to retain the task service for foreground producers without exposing `task_output` / `task_list` / `task_kill`. `toolBash.enableRunInBackground` controls only the bash producer; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields. ## Why a code bundle, not a shared YAML include diff --git a/packages/examples/agent-spine-demo/src/index.ts b/packages/examples/agent-spine-demo/src/index.ts index 5652078a0f..deb433eb85 100644 --- a/packages/examples/agent-spine-demo/src/index.ts +++ b/packages/examples/agent-spine-demo/src/index.ts @@ -30,6 +30,8 @@ export const name = 'agent-spine-demo' /** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */ export interface SkillConfig { + /** Mount the bundled local skill provider and model-facing skill tool (default true). */ + enabled?: boolean /** Registry-level discovery cache settings. */ registry?: SkillRegistryConfig /** Local filesystem skill provider settings. */ @@ -67,12 +69,13 @@ export interface Config { skills?: SkillConfig /** Model-facing bash tool config, including this producer's background opt-in. */ toolBash?: toolBash.Config - /** Generic background-task control-tool wait bounds. */ - toolTasks?: toolTasks.Config + /** Generic background-task controls; set false to keep the task service without model-facing task tools. */ + toolTasks?: toolTasks.Config | false } /** The skill config schema exported for app packages that forward `skills`. */ export const SkillConfigSchema: z = z.object({ + enabled: z.boolean().default(true), registry: SkillService.Config, local: SkillLocal.Config, tool: toolSkill.Config, @@ -93,7 +96,7 @@ export const Config = z.intersect([ skills: SkillConfigSchema, workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), toolBash: ToolBashConfigSchema, - toolTasks: ToolTasksConfigSchema, + toolTasks: z.union([z.const(false), ToolTasksConfigSchema]), }) as unknown as z>, ]) as unknown as z @@ -134,8 +137,11 @@ export function apply(ctx: Context, config: Config): void { ...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {}, }) ctx.plugin(ToolRegistry, config.tools ?? {}) - ctx.plugin(SkillService, config.skills?.registry ?? {}) - ctx.plugin(SkillLocal, config.skills?.local ?? {}) + const skillsEnabled = config.skills?.enabled ?? true + if (skillsEnabled) { + ctx.plugin(SkillService, config.skills?.registry ?? {}) + ctx.plugin(SkillLocal, config.skills?.local ?? {}) + } ctx.plugin(AgentRegistry) ctx.plugin(TaskService) ctx.plugin(invariants) @@ -145,7 +151,7 @@ export function apply(ctx: Context, config: Config): void { } // Both plugins prepend session-prefix messages. Registration order is the // rendered order, so workspace instructions must precede the skill catalog. - ctx.plugin(toolSkill, config.skills?.tool ?? {}) - ctx.plugin(toolTasks, config.toolTasks ?? {}) + if (skillsEnabled) ctx.plugin(toolSkill, config.skills?.tool ?? {}) + if (config.toolTasks !== false) ctx.plugin(toolTasks, config.toolTasks ?? {}) ctx.plugin(AgentLoop, { agents: config.agents ?? [] }) } diff --git a/packages/examples/agent-spine-demo/tests/agent-core.spec.ts b/packages/examples/agent-spine-demo/tests/agent-core.spec.ts index 51562f547d..e9763b3488 100644 --- a/packages/examples/agent-spine-demo/tests/agent-core.spec.ts +++ b/packages/examples/agent-spine-demo/tests/agent-core.spec.ts @@ -301,6 +301,21 @@ describe('dsh-agent-spine-demo bundle', () => { await ctx.fiber.dispose() }) + it('can omit skills and model-facing task controls for a foreground-only deployment', async () => { + const ctx = await mount({ + workspaceContext: false, + skills: { enabled: false }, + toolBash: { enableRunInBackground: false }, + toolTasks: false, + }, true) + + expect(ctx.tools.schemas().map(tool => tool.name)).toEqual(['bash']) + expect(ctx.get('skills')).toBeUndefined() + expect(ctx.get('tasks')).toBeDefined() + + await ctx.fiber.dispose() + }) + it('picks shared spine config without leaking front-door fields', () => { const appConfig = { model: 'front-door-only', @@ -308,9 +323,9 @@ describe('dsh-agent-spine-demo bundle', () => { toolOrder: ['zulu'], tools: { mode: 'native' as const }, workspaceContext: false as const, - skills: {}, + skills: { enabled: false }, toolBash: { enableRunInBackground: false }, - toolTasks: { waitTimeoutMs: 7, maxWaitTimeoutMs: 11 }, + toolTasks: false as const, } expect(agentCore.pickSpineConfig(appConfig)).toEqual({ @@ -318,7 +333,7 @@ describe('dsh-agent-spine-demo bundle', () => { toolOrder: appConfig.toolOrder, tools: appConfig.tools, workspaceContext: false, - skills: {}, + skills: appConfig.skills, toolBash: appConfig.toolBash, toolTasks: appConfig.toolTasks, }) diff --git a/packages/examples/stdio-demo/src/index.ts b/packages/examples/stdio-demo/src/index.ts index d7ae866b53..754dd7d8de 100644 --- a/packages/examples/stdio-demo/src/index.ts +++ b/packages/examples/stdio-demo/src/index.ts @@ -51,7 +51,7 @@ export interface Config { skills?: agentCore.SkillConfig /** Model-facing bash tool config forwarded through agent-core. */ toolBash?: NonNullable - /** Generic background-task control-tool config forwarded through agent-core. */ + /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable /** * If set, the `main` agent RESUMES this persisted session id instead of @@ -77,7 +77,7 @@ export const Config: z = z.object({ welcome: z.string().default('ready.'), skills: agentCore.SkillConfigSchema, toolBash: agentCore.ToolBashConfigSchema, - toolTasks: agentCore.ToolTasksConfigSchema, + toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), resumeSessionId: z.string(), workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), }) diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index 30760a8fbc..aa169f320e 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -61,6 +61,9 @@ export class DeepSeekAdapter extends LlmAdapter { 'content-type': 'application/json', 'accept': 'text/event-stream', ...attributionHeaders(), + ...options.sessionId !== undefined + ? { 'x-deepseek-harness-session-id': String(options.sessionId) } + : {}, }, body: JSON.stringify(body), ...options.signal ? { signal: options.signal } : {}, diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 46f123a1c7..7898c62131 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -3,6 +3,7 @@ import type { IncomingMessage, Server, ServerResponse } from 'node:http' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import LlmService, { LlmError, userAgent } from '@deepseek-ai/dsh-llm' +import { SessionId } from '@deepseek-ai/dsh-session' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek' import { assemble } from './assemble.ts' @@ -131,6 +132,19 @@ describe('DeepSeekAdapter against a mock server', () => { expect(kinds).toEqual(['block-start', 'text-delta', 'block-end', 'usage', 'finish']) }) + it('forwards the harness session id for host-side trajectory routing', async () => { + const server = await mockServer([{ kind: 'sse', events: textEvents }]) + const ctx = await harness(server.url) + + await assemble(ctx, { + model: 'deepseek-v4-flash', + messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + sessionId: SessionId('child-session'), + }) + + expect(server.headers[0]?.['x-deepseek-harness-session-id']).toBe('child-session') + }) + it('forwards thinking config onto the wire', async () => { const server = await mockServer([{ kind: 'sse', events: textEvents }]) const ctx = await harness(server.url, { thinking: 'disabled', reasoningEffort: 'high' }) diff --git a/packages/ui/jsonrpc/README.md b/packages/ui/jsonrpc/README.md index bf686ea800..fdec9920f7 100644 --- a/packages/ui/jsonrpc/README.md +++ b/packages/ui/jsonrpc/README.md @@ -8,7 +8,7 @@ Stdio JSON-RPC plugin for out-of-process SDK clients such as Python `deepseek_ha ## Config -No `cordis.yml` keys. `JsonRpcConfig.input`, `output`, and `exit` are test-only runtime seams; production uses process stdio and `process.exit`. +`maxTokensAsSuccess` defaults to `false`. Set it to `true` for evaluation hosts that distinguish an accepted, token-limited agent result from an infrastructure failure. `JsonRpcConfig.input`, `output`, and `exit` are test-only runtime seams; production uses process stdio and `process.exit`. ## stdout is the protocol diff --git a/packages/ui/jsonrpc/src/index.ts b/packages/ui/jsonrpc/src/index.ts index 29fb1aeff8..fe782c934f 100644 --- a/packages/ui/jsonrpc/src/index.ts +++ b/packages/ui/jsonrpc/src/index.ts @@ -22,8 +22,10 @@ export const name = 'jsonrpc' // Only the agent factory is required; initialize reads the optional LLM seam with ctx.get(). export const inject = ['agents'] -/** Runtime-only test seams; no field is configurable from `cordis.yml`. */ +/** JSON-RPC deployment config plus runtime-only test seams. */ export interface JsonRpcConfig { + /** Report max-token turn/subagent termination as a successful SDK result. */ + maxTokensAsSuccess?: boolean /** Transport input override; production uses `process.stdin`. */ input?: Readable /** Transport output override; production uses `process.stdout`. */ @@ -32,7 +34,9 @@ export interface JsonRpcConfig { exit?: (code: number) => void } -export const Config: Schema = Schema.object({}) +export const Config: Schema = Schema.object({ + maxTokensAsSuccess: Schema.boolean().default(false), +}) /** * Serve SDK requests over the configured streams. Effect disposal shuts down @@ -51,7 +55,9 @@ export function apply(ctx: Context, config: JsonRpcConfig): void { const exit = config.exit ?? ((code: number): void => { process.exit(code) }) const transport = new JsonRpcLineTransport(input, output) - const server = new HarnessSdkServer(ctx, transport) + const server = new HarnessSdkServer(ctx, transport, { + maxTokensAsSuccess: config.maxTokensAsSuccess ?? false, + }) // Share one exit task and attempt flush and disposal independently before exiting. let exitTask: Promise | undefined diff --git a/packages/ui/jsonrpc/src/server.ts b/packages/ui/jsonrpc/src/server.ts index 91ef655c0c..44b746adce 100644 --- a/packages/ui/jsonrpc/src/server.ts +++ b/packages/ui/jsonrpc/src/server.ts @@ -59,6 +59,12 @@ interface SubagentRecord { parentSessionId: string | undefined } +/** Deployment-specific status mapping for SDK turn and subagent outcomes. */ +export interface HarnessSdkServerOptions { + /** Report max-token termination as an accepted result instead of an infrastructure error. */ + maxTokensAsSuccess?: boolean +} + /** * SDK server over one booted harness context and transport peer. Construction * subscribes to session, agent, and subagent lifecycle events until shutdown; @@ -78,6 +84,7 @@ export class HarnessSdkServer { constructor( private readonly ctx: Context, private readonly transport: JsonRpcTransportPeer, + private readonly options: HarnessSdkServerOptions = {}, ) { this.disposers.push(ctx.on('session/event', (session, event) => { if (event.type === 'turn/end') { @@ -116,7 +123,7 @@ export class HarnessSdkServer { agentId: String(info.id), ...(parentSessionId === undefined ? {} : { parentSessionId }), childSessionId, - status: info.stopReason === 'completed' ? 'ok' : 'error', + status: this.successStatus(info.stopReason), stopReason: info.stopReason, ...(info.lastAssistantMessage === undefined ? {} : { lastAssistantMessage: info.lastAssistantMessage }), }) @@ -253,7 +260,12 @@ export class HarnessSdkServer { private finishedStatus(reason: TurnEndReason | undefined): 'ok' | 'error' { if (!reason) return 'error' - return reason.kind === 'completed' ? 'ok' : 'error' + return this.successStatus(reason.kind) + } + + private successStatus(reason: string): 'ok' | 'error' { + if (reason === 'completed') return 'ok' + return reason === 'max-tokens' && this.options.maxTokensAsSuccess === true ? 'ok' : 'error' } private hasAdapterFor(model: string): boolean { diff --git a/packages/ui/jsonrpc/tests/server.spec.ts b/packages/ui/jsonrpc/tests/server.spec.ts index c993bd4e2c..c8a6bbb217 100644 --- a/packages/ui/jsonrpc/tests/server.spec.ts +++ b/packages/ui/jsonrpc/tests/server.spec.ts @@ -329,7 +329,7 @@ describe('HarnessSdkServer', () => { agentOptions: { model: 'deepseek' }, }) const transport = new FakeTransport() - const server = new HarnessSdkServer(ctx, transport) + const server = new HarnessSdkServer(ctx, transport, { maxTokensAsSuccess: true }) await settleSubagent(ctx, parentHandle.agent, { provider: 'fork', @@ -355,7 +355,7 @@ describe('HarnessSdkServer', () => { agentId: 'fallback-child-agent', parentSessionId: 'fallback-parent', childSessionId: 'fallback-child-session', - status: 'error', + status: 'ok', stopReason: 'max-tokens', lastAssistantMessage: [], }, @@ -443,6 +443,24 @@ describe('HarnessSdkServer', () => { } }) + it('can report max-token turn termination as an accepted evaluation result', async () => { + const storageDir = await mkdtemp(join(tmpdir(), 'dsh-jsonrpc-max-tokens-success-')) + const ctx = await makeHarness(storageDir) + try { + const server = new HarnessSdkServer(ctx, new FakeTransport(), { maxTokensAsSuccess: true }) as unknown as { + finishedStatus(reason: unknown): 'ok' | 'error' + shutdown(): Promise> + } + + expect(server.finishedStatus({ kind: 'max-tokens' })).toBe('ok') + expect(server.finishedStatus({ kind: 'error' })).toBe('error') + await server.shutdown() + } finally { + await ctx.fiber.dispose() + await rm(storageDir, { recursive: true, force: true }) + } + }) + it('reports no adapter when the LLM service is absent', async () => { const ctx = new Context() try { From b50e02a2b33eae201e83b016b49aae4e50ab68ea Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Fri, 17 Jul 2026 23:33:30 +0800 Subject: [PATCH 13/73] ci: add manual pi-ai provider e2e --- .github/workflows/pi-ai-provider-e2e.yml | 77 +++++++++ .../llm/llm-pi-ai/tests/provider-apis.e2e.ts | 146 ++++++++++++++++++ vitest.e2e.config.ts | 5 +- 3 files changed, 226 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/pi-ai-provider-e2e.yml create mode 100644 packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts diff --git a/.github/workflows/pi-ai-provider-e2e.yml b/.github/workflows/pi-ai-provider-e2e.yml new file mode 100644 index 0000000000..7a273c52f3 --- /dev/null +++ b/.github/workflows/pi-ai-provider-e2e.yml @@ -0,0 +1,77 @@ +name: E2E (pi-ai OpenAI and Anthropic) + +# This suite spends tokens against two external providers and is intentionally +# opt-in. It has no push, pull_request, schedule, or workflow_call trigger. +on: + workflow_dispatch: + inputs: + openai_model: + description: OpenAI model from pi-ai's installed catalog + required: true + default: gpt-5.5 + type: string + anthropic_model: + description: Anthropic model from pi-ai's installed catalog + required: true + default: claude-opus-4-8 + type: string + +permissions: + contents: read + +jobs: + e2e: + runs-on: ubuntu-latest + name: OpenAI Responses + Anthropic Messages + timeout-minutes: 20 + steps: + - uses: actions/checkout@v6 + + - uses: actions/setup-node@v6 + with: + node-version: 24 + + - name: Enable corepack (pnpm) + run: corepack enable + + - name: Resolve pnpm store path + id: pnpm-store + run: echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT" + + - uses: actions/cache@v4 + with: + path: ${{ steps.pnpm-store.outputs.path }} + key: ${{ runner.os }}-node-24-pnpm-${{ hashFiles('pnpm-lock.yaml') }} + restore-keys: | + ${{ runner.os }}-node-24-pnpm- + + - name: Install (immutable) + run: pnpm install --frozen-lockfile + + # The tests self-skip locally when a credential is absent. A manually + # dispatched CI run must fail instead of reporting an all-skipped green. + - name: Preflight (require provider API keys) + env: + OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY_EXTERNAL }} + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }} + run: | + set -euo pipefail + missing=0 + for name in OPENAI_API_KEY ANTHROPIC_API_KEY; do + if [ -z "${!name:-}" ]; then + echo "::error::${name} is empty. Configure the corresponding *_EXTERNAL repository secret." + missing=1 + fi + done + exit "$missing" + + - name: E2E tests (real OpenAI and Anthropic APIs) + env: + OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY_EXTERNAL }} + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }} + DSH_PI_AI_OPENAI_MODEL: ${{ inputs.openai_model }} + DSH_PI_AI_ANTHROPIC_MODEL: ${{ inputs.anthropic_model }} + DSH_E2E_MAX_WORKERS: 2 + run: >- + pnpm exec vitest run --config vitest.e2e.config.ts + packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts new file mode 100644 index 0000000000..7fffba4618 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -0,0 +1,146 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService, { CallId } from '@deepseek-ai/dsh-llm' +import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' +import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' +import type { PiAiReplayState } from '@deepseek-ai/dsh-llm-pi-ai' +import { assemble, type AssembledResult } from './assemble.ts' + +interface ProviderCase { + provider: 'openai' | 'anthropic' + api: 'openai-responses' | 'anthropic-messages' + model: string + apiKey?: string +} + +const providerCases: ProviderCase[] = [ + { + provider: 'openai', + api: 'openai-responses', + model: process.env.DSH_PI_AI_OPENAI_MODEL ?? 'gpt-5.5', + ...process.env.OPENAI_API_KEY ? { apiKey: process.env.OPENAI_API_KEY } : {}, + }, + { + provider: 'anthropic', + api: 'anthropic-messages', + model: process.env.DSH_PI_AI_ANTHROPIC_MODEL ?? 'claude-opus-4-8', + ...process.env.ANTHROPIC_API_KEY ? { apiKey: process.env.ANTHROPIC_API_KEY } : {}, + }, +] + +const contexts: Context[] = [] + +async function harness(): Promise { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(LlmService) + await ctx.plugin(LlmPiAi, { + providers: providerCases.map(profile => ({ + provider: profile.provider, + ...profile.apiKey === undefined ? {} : { apiKey: profile.apiKey }, + })), + }) + return ctx +} + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose())) +}) + +function ask(text: string): Message[] { + return [{ role: 'user', content: [{ type: 'text', text }] }] +} + +function textOf(result: AssembledResult): string { + return result.message.content + .filter(block => block.type === 'text') + .map(block => block.text) + .join('') +} + +function expectNativeReplay(result: AssembledResult, profile: ProviderCase): PiAiReplayState { + const replayState = result.message.provenance?.replayState + expect(replayState).toMatchObject({ + kind: 'pi-ai', + version: 1, + api: profile.api, + provider: profile.provider, + model: profile.model, + }) + return replayState as PiAiReplayState +} + +const lookupTool: ToolSchema = { + name: 'lookup_code', + description: 'Look up the word represented by a short code.', + parameters: { + type: 'object', + properties: { code: { type: 'string', description: 'The code to look up.' } }, + required: ['code'], + }, +} + +for (const profile of providerCases) { + describe.skipIf(profile.apiKey === undefined)( + `llm-pi-ai ${profile.provider} e2e (${profile.api})`, + () => { + it('streams text with usage and native replay metadata', async () => { + const ctx = await harness() + const result = await assemble(ctx, { + provider: profile.provider, + model: profile.model, + messages: ask('Reply with exactly the word: pong'), + maxTokens: 64, + }) + + expect(result.finish.kind).toBe('stop') + expect(textOf(result).toLowerCase()).toContain('pong') + expect(result.usage?.inputTokens).toBeGreaterThan(0) + expect(result.usage?.outputTokens).toBeGreaterThan(0) + expect(expectNativeReplay(result, profile).stopReason).toBe('stop') + }) + + it('round-trips a tool call with provider-native replay metadata', async () => { + const ctx = await harness() + const prompt = ask('Use lookup_code with code "blue". Do not answer without calling the tool.') + const first = await assemble(ctx, { + provider: profile.provider, + model: profile.model, + messages: prompt, + tools: [lookupTool], + maxTokens: 256, + }) + + expect(first.finish.kind).toBe('tool-calls') + const call = first.message.content.find(block => block.type === 'tool-call') + expect(call).toBeDefined() + expect(call!.name).toBe('lookup_code') + expect(JSON.parse(call!.arguments)).toMatchObject({ code: 'blue' }) + expect(expectNativeReplay(first, profile).stopReason).toBe('toolUse') + + const second = await assemble(ctx, { + provider: profile.provider, + model: profile.model, + messages: [ + ...prompt, + first.message, + { + role: 'user', + content: [{ + type: 'tool-result', + toolCallId: CallId(call!.id), + content: [{ type: 'text', text: 'The code blue means ocean.' }], + }], + }, + ], + tools: [lookupTool], + maxTokens: 256, + }) + + expect(second.finish.kind).toBe('stop') + expect(textOf(second).toLowerCase()).toContain('ocean') + expect(expectNativeReplay(second, profile).stopReason).toBe('stop') + }) + }, + ) +} diff --git a/vitest.e2e.config.ts b/vitest.e2e.config.ts index adc8b3f3b2..6a84b13daf 100644 --- a/vitest.e2e.config.ts +++ b/vitest.e2e.config.ts @@ -2,8 +2,9 @@ import tsconfigPaths from 'vite-tsconfig-paths' import { defineConfig } from 'vitest/config' // Real-API suite, separate because it spends tokens. Each test self-skips without -// DEEPSEEK_API_KEY for keyless CI; the credentialed workflow preflights the secret. Values may come -// from the environment or gitignored root `.env`, with optional DEEPSEEK_BASE_URL. +// its provider credential for keyless CI; credentialed workflows preflight the +// secrets they require. Values may come from the environment or gitignored root +// `.env`, with provider-specific endpoint overrides where supported. try { // Node >= 21.7 native; throws when the file does not exist. process.loadEnvFile(new URL('.env', import.meta.url).pathname) From 923b04606eea08202b31eed132513b6d49628009 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 14:09:14 +0800 Subject: [PATCH 14/73] docs: DeepSeek Harness SDK follow-up work design draft --- docs/sdk-后续工作-设计.md | 197 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 docs/sdk-后续工作-设计.md diff --git a/docs/sdk-后续工作-设计.md b/docs/sdk-后续工作-设计.md new file mode 100644 index 0000000000..a2096cf42f --- /dev/null +++ b/docs/sdk-后续工作-设计.md @@ -0,0 +1,197 @@ +# DeepSeek Harness SDK 后续工作设计 + +> 状态：设计成稿，供通读与评审。定案后由 ccyu 转正式 RFC 并双语化。本文件为临时设计文档，不走 doc-sync / 文档预算门禁。 +> 一句话：**SDK 初版已合并；本轮把"创建项目""创建插件""遥测""交互测试"四块补齐，核心是抽出一个既撑交互又撑 headless 的创建内核，其余三块围绕它扩展。** + +## 0. 总览（一屏读完） + +抽一个既撑交互又撑 headless 的**创建内核**（`Prompter` seam + 复用已有 `ProjectEditSession` + `CreationDriver` 发 NDJSON），四块活围绕它扩展。 + +| 块 | 做什么 | 核心对象 | 结论要点 | +|---|---|---|---| +| **#1 headless + skill** | create/config 非交互化，agent 端到端建项目 | `HeadlessPrompter` + `CreationDriver`(NDJSON) | 无 spec 文件、传结构化对象；薄 SKILL.md 入口；beyond-eve | +| **#2 建插件** | `dsh-sdk create ` 拉插件并接线 | `PluginSource` + `PluginFetcher`(giget/pacote) | 只解压不执行、锁版本、经 `ProjectEditSession` 显式接线 | +| **#3 遥测** | 每个 `dsh-sdk` 命令上报 | `TelemetryReporter` / `ConsentResolver` / `SecretRedactor` | 发 cordis.yml+package.json 全文；不发 `.env`、疑似密钥脱敏；关闭 = cordis.yml 有明确 disabled 的遥测条目（甲）| +| **#4 交互测试** | 覆盖 wizard 各分支、快照 cordis.yml | `WizardHarness` + clack mock 注入 | 注入流为主、真 PTY 仅 1–2 个可选 smoke | + +**节奏**：先做地基（`Prompter` 重构，同时解锁 #1+#4，不并行）→ 再 fan out 四棵 teammate worktree。 + +细节见下文；不想通读的话，读完本节 + §5（砍掉的路线）+ §6（节奏）即可。 + +## 1. 背景与现状 + +SDK 初版（`packages/sdk/*`）已经落地三个包： + +| 包 | 职责 | 当前局限 | +|---|---|---| +| `@deepseek-ai/create-sdk` | `npm create @deepseek-ai/sdk` 交互式建项目 | **TTY-only**：flag 只能预填问题，创建仍须交互终端；feature 配置面窄，复杂 feature 难在命令行表达 | +| `@deepseek-ai/dsh-helper` | 项目领域模型：feature 催表、蓝图、`ProjectEditSession`（唯一的改写/提交边界） | create-sdk 与 `dsh-sdk config` 共用同一套催表与 configurator，但目前只被交互式 wizard 驱动 | +| `@deepseek-ai/dsh-scripts` | `dsh-sdk` launcher：`start` / `dev` / `build` / `config` | `config` 同样 **TTY-only**；`build` 只跑 tsdown、`create` 时项目尚不存在——两者都不 boot cordis | + +四块后续工作： + +1. **非交互（headless）创建 + 通过 skill 创建**（#1）：去掉 create/config 的 TTY-only 限制，让 agent 能端到端把项目建出来。 +2. **`dsh-sdk create ` 从 github/npm 建插件**（#2）：把外部插件拉进现有项目并接线。 +3. **遥测**（#3）：每次 `dsh-sdk `（含 create/首次初始化）上报开发者周期数据。 +4. **终端交互测试**（#4）：给 clack wizard 加可回归、可核对的交互测试。 + +## 2. 边界澄清：三件事分开 + +调研 `vercel/eve` 及其引用的 `skills` CLI 后，确认三件事必须分开、不能混： + +| 轨 | 命令入口 | 产物 | +|---|---|---| +| **建项目** | `npm create @deepseek-ai/sdk`（现有 wizard）+ 其 headless 形态（本轮新增） | 一个新的 SDK 项目 | +| **建插件** | `dsh-sdk create `（现有 create-plugin 扩展） | 现有项目里多一个接好线的插件 | +| **分发 skill** | 发 SKILL.md → agent `npx skills add deepseek-ai/` | 任意 agent 拉到我们的 skill playbook | + +事实依据：`skills`（vercel-labs/skills）是 markdown SKILL.md 的包管理器，来源只认 github/git/本地、不认 npm scope，只往 agent 的 skills 目录丢文件，不建项目、不 install；eve 建项目走的是独立的 `npx eve init`。因此 `npx skills add @deepseek-ai/sdk`（把"装 SDK"和 `skills add` 揉在一起）不是真实用法，本设计据此拆开。 + +## 3. 总体架构 + +### 3.1 核心洞见：一个创建内核，四条轨围绕它 + +四块活看似独立，实则都咬合在同一组对象上。**建项目和改配置的本质是"问答 → 改项目文件"**，而 `dsh-helper` 已经有唯一的改写边界 `ProjectEditSession`。本轮把"问答"这一侧也抽成 seam，就能让交互与 headless 共用一套逻辑，测试和 skill 顺势接上去。 + +``` + ┌───────────────────────────────┐ + │ CreationDriver / SetupRunner │ 编排：问答序列 → 组装 spec → 驱动改写 + │ (emits NDJSON) │ + └───────────────┬───────────────┘ + 借助 │ 驱动 + ┌─────────────────────┐ │ ┌────────────────────────────┐ + │ Prompter (seam) │◄──────────┘ │ ProjectEditSession (已有) │ 唯一改写/提交边界 + ├─────────────────────┤ └────────────────────────────┘ + │ InteractivePrompter │ clack + 注入 input/output（解锁 #4 测试） + │ HeadlessPrompter │ 永不阻塞：答案取自结构化 spec，缺必答项→抛错/发 action-required（解锁 #1 + skill） + └─────────────────────┘ +``` + +- **`Prompter`（新 seam）**：把"向用户要一个答案"抽象出来。两个实现： + - `InteractivePrompter`：clack 实现，接受注入的 `input`/`output` 流（不再硬绑 `process.stdin/stdout`）。这正是 #4 交互测试的前置。 + - `HeadlessPrompter`：永不阻塞——答案取自调用方传入的结构化 spec；遇到未提供的必答项，直接**响亮失败**（抛错 / 发 `action-required` 事件），不猜默认。这是 #1 headless 与 skill 驱动的地基。 +- **`ProjectEditSession`（复用已有）**：唯一的改写/提交边界。create、config、以及 #2 建插件改 cordis.yml，全部经它落盘。 +- **`CreationDriver`（新，或改造现有 wizard 编排）**：跑问答序列、组装项目 spec、驱动 `ProjectEditSession`；headless 模式下向外发 **NDJSON 生命周期事件**（`action-required` / `done` / `error` / 进度）。 +- **skill 路径**：agent import 这个内核、传结构化 config 对象、读 NDJSON 事件；附一层薄 SKILL.md 教 agent 怎么驱动。 + +**扩展点**：新增一个 feature 只改 `dsh-helper` 的催表/催配置器；交互与 headless 两条路都自动获得它，不需各改一遍。 + +> **读码修正（重要，落地以此为准）**：上文 `Prompter`/`InteractivePrompter`/`CreationDriver` 是概念名，对应现有代码： +> - **seam 已存在**：`dsh-helper` 的 `PromptPort`（`questions/prompt-port.ts`）即 `Prompter`；问答走 `Question.resolve(port, prefilled?)`——给了 prefill 就不碰 port。 +> - **交互实现 + 注入已存在**：`ClackPromptPort` 构造函数已接受注入 `input`/`output`（源码注释即 "for snapshots and tests"）；`CreateWizard` 与 `ConfigWorkflow` 都已接受注入的 `PromptPort` + `output`。 +> - **⇒ #4 交互测试不被地基阻塞**：注入点今天就有，已单独开 teammate 并行做（覆盖 create + config 两个 wizard）。 +> - **地基真正要做的（比原设想小）**：新增 `HeadlessPromptPort implements PromptPort`（缺项 fail-fast + 发 NDJSON）+ 补全 prefill 覆盖——目前 feature 选择走原始 `nestedMultiselect`、`FeatureConfigurator` 的 valueInputs 无 prefill、suggests 确认无 prefill；headless 要让结构化 spec 喂满这些点。 + +### 3.2 四块如何咬合到这组对象 + +| 块 | 落在哪个对象 | 关系 | +|---|---|---| +| #1 headless + skill | `HeadlessPromptPort`（实现已有 `PromptPort`）+ prefill 补全 + NDJSON | 地基（缩小版）| +| #4 交互测试 | 注入 `ClackPromptPort(mockIn, mockOut)` 进已有 `CreateWizard`/`ConfigWorkflow` + `WizardHarness` | **注入点已存在，不阻塞，已并行开工** | +| #2 建插件 | `PluginSource` + `PluginFetcher` + 经 `ProjectEditSession` 接线 | 复用改写边界 | +| #3 遥测 | launcher 侧 `TelemetryReporter` + `ConsentResolver` + `SecretRedactor` | 独立于内核，挂在 launcher 命令生命周期 | + +## 4. 详细设计 + +### 4.1 headless 创建 + skill（#1） + +**目标**：`create-sdk`（建新项目）与 `dsh-sdk config`（改现有项目）都能非交互运行；agent 能端到端把项目建完。 + +**设计**： + +- **Prompter seam**：如 §3。交互走 `InteractivePrompter`，headless 走 `HeadlessPrompter`（fail-fast + NDJSON），二者背后是同一套 `dsh-helper` 催表和同一个 `ProjectEditSession`。 +- **输入编码**（次要、可解耦）： + - **agent 路径**：传结构化 config 对象（程序化，或 `--config-json '{...}'`）+ 读 NDJSON。**不需要 spec 文件**——与 eve 一致；我们 feature 比 eve 重（嵌套有限选项 + 密钥），结构化对象比一长串扁平 flag 干净。 + - **人 / CI 路径**（可选）：`--config `（yaml/json）或 flags，只是喂给同一内核的另一种编码。 +- **skill**：核心是 headless 内核；agent 传参直接建完，缺必答项就响亮失败让 agent 补答。附一层**薄 SKILL.md**（指向内核、教 agent 驱动），让"通过 skill 创建"字面落地。 +- **比 eve 更进一步**：eve 把 headless 原语（`runHeadless` + 非阻塞 Prompter + NDJSON）造好了，却没接到它的 skill——它的 SKILL.md 只指向半交互 CLI，且 agent 跑 `eve init` 时只打印指引、打回给人。我们把 **skill → headless 内核接通**，才真正做到"headless 为 skill 服务"。 + +### 4.2 `dsh-sdk create ` 建插件（#2） + +**目标**：从 github repo 或 npm 包拉一个插件进现有项目并接线；安全第一。 + +**设计（只解压不执行 + 锁版本 + 显式接线）**： + +- **`PluginSource`（判别联合）**：`GithubSource`（`owner/repo[/subdir]#ref`）| `NpmSource`（`pkg@version`）。由 spec 字符串解析而来。 +- **`PluginFetcher`（seam）**：把源抓进 temp 目录，**绝不执行被拉代码的生命周期脚本**。 + - `GigetFetcher`（github/git）：giget；`#ref` 先解析成 commit SHA 再下、记进 lock。 + - `PacoteFetcher`（npm）：pacote `extract`（只解包不跑 postinstall），带 `integrity` 校验。来源类型限定放在**上游 `resolvePluginSource`**（只产出 `name@version`）作为主保证，不依赖 pacote 的 `allowRegistry`（`@types/pacote` 无此选项，且 registry tarball extract 本就不跑脚本）。 +- **接线（显式可审）**： + 1. `package.json` 精确锁版本（npm：exact + integrity；github：`github:owner/repo#`）。 + 2. 经 `ProjectEditSession` 改 `cordis.yml` 挂插件——**给 diff、要确认**再写。 + 3. `install --ignore-scripts`（pnpm v10 默认亦拦依赖 build 脚本）。 + 4. 打印清单（dep spec + 锁的 ref/integrity + cordis.yml diff）。 +- **信任模型**：学 `npm create` 的手感，但把信任反过来——**confirm-before-run，而非 run-on-fetch**。 +- **repo 初始化模式**（从模板仓库整体建项目）同走 giget（优于 degit——degit 的 `degit.json` 会自动跑动作）；建远程新仓可用 `gh repo create --template`。注：eve 不支持 template-repo init，这是我们的自有取舍。 + +### 4.3 遥测（#3） + +**目标**：每次 `dsh-sdk `（create / dev / build / config / start / 首次初始化）上报当前 `cordis.yml` + `package.json` 内容。 + +**设计**： + +- **上报器位置**：在**我们自己的代码执行时机**里——`create-sdk` 进程 + `dsh-scripts` launcher 进程，包住命令生命周期。 + - 理由：`build` 只跑 tsdown、`create` 时项目还不存在，都不 boot cordis；写在 cordis.yml 里的 cordis 插件抓不到它们。调研的全部工具（Next/Astro/Nuxt/Vite/Angular/Gatsby/Turbo/Homebrew）无一例外把上报器放 CLI/launcher，从不放 app 运行时。 +- **`TelemetryReporter`（launcher 侧）**：包住命令，收集 `{command, 时长, 成败, cordis.yml 内容, package.json 内容}`。 +- **`ConsentResolver`**：在每个命令**解析（非 boot）`cordis.yml`**，读遥测插件状态当 consent。**关闭（甲，ccyu 拍板）= cordis.yml 里有一条明确 `disabled` 的遥测条目**；其余一切（无 cordis.yml / 有文件但无遥测条目 / 有 enabled 条目）都上报——唯一的关只有"存在且 disabled"，无不对称。（可选、近零成本补充，留到实现定：额外认 `DO_NOT_TRACK` / CI 自动关。） +- **`SecretRedactor`（安全硬线）**：绝不发 `.env`；cordis.yml / package.json 里若出现类似密钥的值，**脱敏替换**（redact 占位，不整段丢）。依赖 SDK 约定——密钥只进 `.env`、cordis.yml 只引用 env 不内联——脱敏是兜底。 +- **匿名 id**：全局配置里的随机 UUID；**绝不从 git remote / repo URL 派生**。 +- **endpoint**：内置在代码里。 +- **consent 承载**：遥测作为 `create` 时默认打开的 feature 写进 cordis.yml（对用户可见、随项目）。 + +**在案取舍**：发全文会把第三方（含私有 scoped）包名、cordis 配置值（base-url/路径）暴露给 endpoint 持有方；主流工具都不发这些（Turbo 排除包名、Angular 禁模块名）。ccyu 作为本 SDK 维护者接受此暴露——目的即掌握开发者用了哪些 plugin/依赖/配置。 + +### 4.4 交互测试（#4） + +**目标**：CI 仅 mac/linux；覆盖 create wizard 主流程 + config wizard 各选择分支；产出不同选择下的 `cordis.yml` 快照便于核对。 + +**设计（clack mock 注入，不上真 PTY 打头阵）**： + +- **主力**：进程内注入 mock stdin/stdout。`@clack/prompts` 官方支持 `input`/`output` 注入，`isTTY=false` 时自动跳过 raw mode——零原生依赖、mac/linux 天然确定。**前置已就绪**：`ClackPromptPort` 已接受注入流，`CreateWizard`/`ConfigWorkflow` 已接受注入 `PromptPort`。 +- **`WizardHarness`（测试工具）**：用脚本化 keypress 序列（`input.emit('keypress', …)` 走 down/space/return）驱动 wizard、写到 temp 目录、返回生成的 `cordis.yml`。 +- **断言**：`test.each(选择组合)` → `toMatchFileSnapshot('.cordis.yml')`；**快照生成的 cordis.yml**，不快照交互 transcript（transcript 脆、且是在测 clack 自己）。 +- **可选 1–2 个真 PTY smoke**：仅覆盖"真二进制 + interactive-vs-CI TTY gate"这条注入测不到的分支；node-pty 在我们 Node（`^22.19 || >=24`）上有原生编译风险，**挪出关键路径**，缺工具链 self-skip。 + +## 5. 砍掉的路线（调研依据） + +| 砍掉的路线 | 理由 | +|---|---| +| #4 用真 PTY（node-pty）打头阵 | clack 官方支持注入流，无需真 TTY；node-pty 在我们 Node 版本上有原生编译风险。真 PTY 降级为 1–2 个可选 smoke | +| #4 快照交互 transcript | transcript 受重绘/spinner/ANSI 影响脆弱，且主要在测 clack 渲染而非我们的生成逻辑 | +| #3 上报器做成 cordis 插件 | `build`/`create` 不 boot cordis，插件抓不到；无一主流工具用 app 内插件做遥测 | +| #3 匿名 id 从 git remote 派生 | 会让"匿名"变假（Next 因此挨批） | +| #1 headless 以 spec 文件为主 | eve 没有 spec 文件；agent 路径传结构化对象更干净，spec 文件退成人/CI 可选 | +| #2 从 npm/github 拉完自动 install+build | 会执行被拉代码的 postinstall，供应链风险；改为只解压 + 显式接线 + `--ignore-scripts` | +| #1 用 `npx skills add` 创建项目 | `skills` 是 markdown SKILL.md 包管理器、不建项目、不认 npm scope，属概念混淆 | + +## 6. 推进节奏：地基缩小，已并发开工 + +**读码后修正**：`PromptPort` seam + 可注入的 `ClackPromptPort` + 可注入的 `CreateWizard`/`ConfigWorkflow` 都已存在，所以地基比原设想小，且 #4 不再被它阻塞。当前并发结构： + +1. **地基（主线程，我做）**：新增 `HeadlessPromptPort implements PromptPort`（缺项 fail-fast + 发 NDJSON）+ 把 prefill 覆盖补全（feature 选择、valueInputs、suggests 确认），让结构化 spec 能喂满 create/config；顺带把 launcher 命令注册、helper 催表扩展点留成清晰 seam。这是 #1 headless 与 #2/#3 接线步的公共前置。 +2. **已并行开工的 teammate worktree**（与地基低冲突，只碰各自新模块）： + - **树 B 建插件**：`PluginSource` + `GigetFetcher`/`PacoteFetcher`（greenfield；`dsh-sdk create` 命令注册 + cordis.yml 接线等地基后再做）。 + - **树 A 遥测**：`SecretRedactor` / `ConsentResolver` / `TelemetryPayload` / `TelemetryReporter`（greenfield；launcher 接线 + 催表加 feature 等地基后再做）。 + - **树 C 交互测试**：`WizardHarness` + create/config 的 `test.each` cordis.yml 快照（注入点已就绪，可全量做）。 +3. **地基落地后的收尾**（接线，碰共享文件）：#2 的 `dsh-sdk create` 命令注册 + cordis.yml 接线；#3 的 launcher 上报接线 + 催表遥测 feature；#1 的 skill 薄封装 + `--config-json`/`--config` 入口。 +4. **合并**：stacked PR，地基在底，其余 rebase 到地基上，逐层向上同步。 + - **树 D 薄 SKILL.md**：SKILL.md + 相关文档（并入第 3 步收尾）。 + +> 冲突面：树 A/B 的接线步都会碰 `dsh-scripts` 命令注册与 `dsh-helper` 催表——地基里须把命令注册做成可加式（各命令自注册）、催表扩展点清晰，接线才能真正独立、rebase 顺滑。greenfield 模块阶段（当前）不碰这些共享文件，所以能安全并行。 + +## 7. 各轨改动清单 + +| 轨 | 主要包 | 关键改动 | +|---|---|---| +| 地基 | dsh-helper, create-sdk, dsh-scripts | `HeadlessPromptPort`（实现已有 `PromptPort`）+ prefill 补全 + NDJSON + 命令注册/催表扩展点整理 | +| #1 headless+skill | create-sdk, dsh-scripts, （新）skill 包 | 结构化 spec 入口 `--config-json`/`--config`、薄 SKILL.md | +| #2 建插件 | （新）fetcher 包, dsh-scripts, dsh-helper | `PluginSource`、`GigetFetcher`/`PacoteFetcher`、`dsh-sdk create ` 注册、经 `ProjectEditSession` 接线+diff、新依赖 giget/pacote | +| #3 遥测 | （新）telemetry 包, dsh-scripts, dsh-helper | `TelemetryReporter`/`ConsentResolver`/`SecretRedactor`、launcher 接线、催表遥测 feature、内置 endpoint、全局 UUID | +| #4 测试 | packages/support, （已可注入）create-sdk/dsh-scripts | `WizardHarness`、create/config 的 `test.each` cordis.yml 快照、可选 PTY smoke | + +## 8. 调研来源 + +- eve / skills：https://github.com/vercel/eve · https://github.com/vercel-labs/skills +- 从源拉取：https://github.com/unjs/giget · https://github.com/npm/pacote · https://github.com/Rich-Harris/degit +- 遥测规范：https://nextjs.org/telemetry · https://astro.build/telemetry/ · https://github.com/nuxt/telemetry · https://angular.dev/cli/analytics · https://turborepo.dev/docs/telemetry · https://consoledonottrack.com +- PTY / clack 测试：https://github.com/bombshell-dev/clack · https://vitest.dev/guide/snapshot · https://github.com/microsoft/node-pty From dcd886798ff93bc6f4945ad4aea6e3c8658969ef Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 14:11:07 +0800 Subject: [PATCH 15/73] feat(sdk): headless PromptPort + create/config headless plan Add HeadlessPromptPort (fail-loud non-interactive PromptPort), thread prefilled feature values through FeatureConfigurator, and let CreateWizard and ConfigWorkflow accept a headless feature plan that skips the interactive tree/suggests prompts. Per-file 100% coverage on all changed files. --- packages/sdk/create-sdk/src/create-wizard.ts | 75 ++++++------ packages/sdk/create-sdk/tests/create.spec.ts | 50 ++++++++ .../src/features/feature-configurator.ts | 9 +- packages/sdk/helper/src/index.ts | 1 + .../src/questions/headless-prompt-port.ts | 97 ++++++++++++++++ .../helper/tests/headless-prompt-port.spec.ts | 95 +++++++++++++++ packages/sdk/helper/tests/questions.spec.ts | 31 +++++ .../sdk/scripts/src/config/config-workflow.ts | 108 +++++++++++------- packages/sdk/scripts/tests/scripts.spec.ts | 25 +++- 9 files changed, 415 insertions(+), 76 deletions(-) create mode 100644 packages/sdk/helper/src/questions/headless-prompt-port.ts create mode 100644 packages/sdk/helper/tests/headless-prompt-port.spec.ts diff --git a/packages/sdk/create-sdk/src/create-wizard.ts b/packages/sdk/create-sdk/src/create-wizard.ts index 8391fae1e4..fb6849ba2a 100644 --- a/packages/sdk/create-sdk/src/create-wizard.ts +++ b/packages/sdk/create-sdk/src/create-wizard.ts @@ -48,6 +48,7 @@ export class CreateWizard { private readonly versionProbe: PackageManagerVersionProbe private readonly userAgent: string private readonly linkWorkspaceRoot: string | undefined + private readonly featurePlan: readonly FeatureSelection[] | undefined /** Bind parsed args and infrastructure to one wizard run. */ constructor(options: { @@ -57,6 +58,7 @@ export class CreateWizard { releaseVersion: string versionProbe?: PackageManagerVersionProbe userAgent?: string + features?: readonly FeatureSelection[] }) { this.args = options.args this.port = options.port @@ -68,6 +70,7 @@ export class CreateWizard { this.linkWorkspaceRoot = options.args.linkWorkspace ? fileURLToPath(new URL('../../../../', import.meta.url)) : undefined + this.featurePlan = options.features } /** Collect all answers before constructing any project files. */ @@ -129,39 +132,43 @@ export class CreateWizard { const configurable = registry.all().filter(feature => feature.id === 'bash' || feature.id === 'persistence' || (!feature.required && feature.isApplicable(profile))) - const selected = [...requireAnswer(await this.port.nestedMultiselect({ - message: 'Select features', - options: configurable.map((feature) => { - const nested = feature.mode !== 'single' - const defaults = new Set(feature.defaultOptions(profile)) - return { - value: feature.id, - label: feature.summary, - required: feature.required, - default: feature.required || feature.id === 'hmr' || feature.id === 'fs' || feature.id === 'todo' - || feature.id === 'skill', - ...nested ? { - choiceMode: feature.mode === 'multiple' ? 'multiple' as const : 'exclusive' as const, - choices: feature.options.map(option => ({ - value: option.id, - label: option.label, - default: defaults.has(option.id), - })), - } : {}, + const selected = this.featurePlan + ? this.featurePlan.map(feature => ({ value: feature.id, choices: feature.options })) + : [...requireAnswer(await this.port.nestedMultiselect({ + message: 'Select features', + options: configurable.map((feature) => { + const nested = feature.mode !== 'single' + const defaults = new Set(feature.defaultOptions(profile)) + return { + value: feature.id, + label: feature.summary, + required: feature.required, + default: feature.required || feature.id === 'hmr' || feature.id === 'fs' || feature.id === 'todo' + || feature.id === 'skill', + ...nested ? { + choiceMode: feature.mode === 'multiple' ? 'multiple' as const : 'exclusive' as const, + choices: feature.options.map(option => ({ + value: option.id, + label: option.label, + default: defaults.has(option.id), + })), + } : {}, + } + }), + }))] + if (!this.featurePlan) { + for (const { value: id } of [...selected]) { + const feature = registry.get(id) + for (const suggestedId of feature.suggests) { + if (selected.some(item => item.value === suggestedId)) continue + const suggested = registry.get(suggestedId) + const add = requireAnswer(await new ConfirmQuestion({ + id: `${feature.id}.${suggested.id}`, + message: `Add the recommended ${suggested.summary.toLowerCase()} for ${feature.summary.toLowerCase()}?`, + initialValue: true, + }).resolve(this.port)) + if (add) selected.push({ value: suggested.id, choices: suggested.defaultOptions(profile) }) } - }), - }))] - for (const { value: id } of [...selected]) { - const feature = registry.get(id) - for (const suggestedId of feature.suggests) { - if (selected.some(item => item.value === suggestedId)) continue - const suggested = registry.get(suggestedId) - const add = requireAnswer(await new ConfirmQuestion({ - id: `${feature.id}.${suggested.id}`, - message: `Add the recommended ${suggested.summary.toLowerCase()} for ${feature.summary.toLowerCase()}?`, - initialValue: true, - }).resolve(this.port)) - if (add) selected.push({ value: suggested.id, choices: suggested.defaultOptions(profile) }) } } const fixed = new Set(selections.map(selection => selection.id)) @@ -174,12 +181,16 @@ export class CreateWizard { for (const choice of selected) { choices.set(choice.value, choice.choices.length > 0 ? choice.choices : undefined) } + const plannedById = new Map((this.featurePlan ?? []).map(feature => [feature.id, feature])) for (const [id, options] of choices) { + const planned = plannedById.get(id) selections.push(await configurator.configure( registry.get(id), profile, undefined, options, + planned?.secrets ?? {}, + planned?.values ?? {}, )) } return selections diff --git a/packages/sdk/create-sdk/tests/create.spec.ts b/packages/sdk/create-sdk/tests/create.spec.ts index 9d75700bb9..f4b66dc8ff 100644 --- a/packages/sdk/create-sdk/tests/create.spec.ts +++ b/packages/sdk/create-sdk/tests/create.spec.ts @@ -5,9 +5,11 @@ import { PassThrough, Writable } from 'node:stream' import { fileURLToPath } from 'node:url' import { afterEach, describe, expect, it, vi } from 'vitest' import { + HeadlessPromptPort, LocalPluginBlueprint, featureId, NpmPackageManager, + type FeatureSelection, type NestedMultiSelectValue, type PromptPort, } from '@deepseek-ai/dsh-helper' @@ -233,6 +235,54 @@ describe('CreateWizard and scaffolder', () => { expect(resolved.request.features.find(item => item.id === 'hmr')).toMatchObject({ options: ['default'] }) }) + it('runs headlessly from a feature plan without reaching the terminal', async () => { + const cwd = await mkdtemp(join(tmpdir(), 'create-headless-')) + temporary.push(cwd) + const features: FeatureSelection[] = [ + { id: featureId('persistence'), options: ['sqlite'], values: { region: 'us' } }, + { id: featureId('web'), options: ['exa'], secrets: { apiKey: 'exa-key' } }, + ] + const resolved = await new CreateWizard({ + args: parseCreateArgs([ + 'my-agent', '--description=demo', '--provider=deepseek', '--api-key=deepseek-key', + '--model=deepseek-v4-flash', '--interface=stdio', '--pm=npm', '--no-install', + ]), + port: new HeadlessPromptPort(), + cwd, + releaseVersion: '0.0.1', + versionProbe: async () => '10.0.0', + features, + }).run() + expect(resolved.install).toBe(false) + expect(resolved.request.localPlugins).toEqual([]) + expect(resolved.request.features.find(item => item.id === 'web')).toMatchObject({ + options: ['exa'], secrets: { apiKey: 'exa-key' }, + }) + expect(resolved.request.features.find(item => item.id === 'persistence')).toMatchObject({ options: ['sqlite'] }) + expect(resolved.request.features.find(item => item.id === 'provider')).toMatchObject({ + secrets: { apiKey: 'deepseek-key' }, + }) + }) + + it('rejects a non-string feature value in a headless plan', async () => { + const cwd = await mkdtemp(join(tmpdir(), 'create-headless-bad-')) + temporary.push(cwd) + const features = [ + { id: featureId('persistence'), options: ['sqlite'], values: { bad: 1 } }, + ] as unknown as FeatureSelection[] + await expect(new CreateWizard({ + args: parseCreateArgs([ + 'my-agent', '--description=demo', '--provider=deepseek', '--api-key=k', + '--model=m', '--interface=stdio', '--pm=npm', '--no-install', + ]), + port: new HeadlessPromptPort(), + cwd, + releaseVersion: '0.0.1', + versionProbe: async () => '10.0.0', + features, + }).run()).rejects.toThrow('must be a string') + }) + it('writes the project once and refuses every existing target', async () => { const root = await mkdtemp(join(tmpdir(), 'create-scaffold-')) temporary.push(root) diff --git a/packages/sdk/helper/src/features/feature-configurator.ts b/packages/sdk/helper/src/features/feature-configurator.ts index 47cc011d16..e6e14030f9 100644 --- a/packages/sdk/helper/src/features/feature-configurator.ts +++ b/packages/sdk/helper/src/features/feature-configurator.ts @@ -26,6 +26,7 @@ export class FeatureConfigurator { * @param current - currently installed selection, when configuring. * @param prefilledOptions - options already chosen by a tree picker. * @param prefilledSecrets - non-interactive secret values supplied by creation. + * @param prefilledValues - non-interactive value inputs supplied by a headless spec. * @returns normalized selection with captured values and secrets. */ async configure( @@ -34,6 +35,7 @@ export class FeatureConfigurator { current?: FeatureSelection, prefilledOptions?: readonly string[], prefilledSecrets: Readonly> = {}, + prefilledValues: Readonly> = {}, ): Promise { let options: readonly string[] switch (feature.mode) { @@ -69,6 +71,11 @@ export class FeatureConfigurator { id: feature.id, options, } + const coercedPrefilled: Record = {} + for (const [key, value] of Object.entries(prefilledValues)) { + if (typeof value !== 'string') throw new Error(`${feature.id}.${key} value must be a string`) + coercedPrefilled[key] = value + } const values: Record = {} for (const input of feature.valueInputs(selected, profile)) { const existing = current?.values?.[input.id] @@ -81,7 +88,7 @@ export class FeatureConfigurator { ...existing === undefined ? {} : { initialValue: existing }, validate: value => value.trim().length === 0 ? 'A value is required' : undefined, }) - values[input.id] = requireAnswer(await question.resolve(this.port)) + values[input.id] = requireAnswer(await question.resolve(this.port, coercedPrefilled[input.id])) } const base: FeatureSelection = Object.keys(values).length === 0 ? selected diff --git a/packages/sdk/helper/src/index.ts b/packages/sdk/helper/src/index.ts index 98c55c3f8b..85aba58a99 100644 --- a/packages/sdk/helper/src/index.ts +++ b/packages/sdk/helper/src/index.ts @@ -43,3 +43,4 @@ export { } from './questions/question.ts' export type { Question } from './questions/question.ts' export { ClackPromptPort } from './questions/clack-prompt-port.ts' +export { HeadlessPromptError, HeadlessPromptPort } from './questions/headless-prompt-port.ts' diff --git a/packages/sdk/helper/src/questions/headless-prompt-port.ts b/packages/sdk/helper/src/questions/headless-prompt-port.ts new file mode 100644 index 0000000000..658500aed0 --- /dev/null +++ b/packages/sdk/helper/src/questions/headless-prompt-port.ts @@ -0,0 +1,97 @@ +/** + * Non-interactive prompt port for headless create/config and skill-driven runs. + * + * @module @deepseek-ai/dsh-helper/questions/headless-prompt-port + */ + +import type { + ConfirmPromptRequest, + MultiSelectPromptRequest, + NestedMultiSelectRequest, + NestedMultiSelectValue, + PromptOutcome, + PromptPort, + SecretPromptRequest, + SelectPromptRequest, + TextPromptRequest, +} from './prompt-port.ts' + +/** + * Raised when a headless run reaches a decision that was neither prefilled nor + * carries a usable default. The message names the unanswered prompt so an agent + * or CI caller can see exactly which input the spec must supply. + */ +export class HeadlessPromptError extends Error { + /** The unanswered prompt's user-facing message. */ + readonly prompt: string + + /** Build an error naming the unanswered prompt. */ + constructor(prompt: string) { + super(`headless run needs an answer for: ${prompt}`) + this.name = 'HeadlessPromptError' + this.prompt = prompt + } +} + +/** Resolve an answered outcome. */ +function answered(value: T): Promise> { + return Promise.resolve({ status: 'answered', value }) +} + +/** Reject with a named unanswered-prompt error. */ +function unanswered(message: string): Promise> { + return Promise.reject(new HeadlessPromptError(message)) +} + +/** + * A {@link PromptPort} that never blocks on a terminal. + * + * Answers are expected to arrive as prefilled values through the `Question` / + * `FeatureConfigurator` layers, so in a fully specified run this port is never + * reached. When it *is* reached, it takes the prompt's own declared default + * (`defaultValue` / `initialValue`) if one exists; otherwise it fails loud with + * {@link HeadlessPromptError}. Nested feature selection has no scalar default, + * so it always fails loud — headless callers must supply the feature set through + * the spec rather than the tree picker. + */ +export class HeadlessPromptPort implements PromptPort { + /** Answer visible text from its default, or fail loud. */ + text(request: TextPromptRequest): Promise> { + const fallback = request.initialValue ?? request.defaultValue + if (fallback === undefined) return unanswered(request.message) + const diagnostic = request.validate?.(fallback) + if (diagnostic) return unanswered(`${request.message} (${diagnostic})`) + return answered(fallback) + } + + /** A secret has no safe default: always fail loud. */ + secret(request: SecretPromptRequest): Promise> { + return unanswered(request.message) + } + + /** Answer a single choice from its initial value, or fail loud. */ + select(request: SelectPromptRequest): Promise> { + if (request.initialValue === undefined) return unanswered(request.message) + return answered(request.initialValue) + } + + /** Answer a multi-choice from its initial values, or fail loud when required. */ + multiselect(request: MultiSelectPromptRequest): Promise> { + const initial = request.initialValues ?? [] + if (request.required && initial.length === 0) return unanswered(request.message) + return answered(initial) + } + + /** Answer a confirmation from its initial value, or fail loud. */ + confirm(request: ConfirmPromptRequest): Promise> { + if (request.initialValue === undefined) return unanswered(request.message) + return answered(request.initialValue) + } + + /** Nested feature selection has no scalar default: always fail loud. */ + nestedMultiselect( + request: NestedMultiSelectRequest, + ): Promise[]>> { + return unanswered(request.message) + } +} diff --git a/packages/sdk/helper/tests/headless-prompt-port.spec.ts b/packages/sdk/helper/tests/headless-prompt-port.spec.ts new file mode 100644 index 0000000000..12febcfeff --- /dev/null +++ b/packages/sdk/helper/tests/headless-prompt-port.spec.ts @@ -0,0 +1,95 @@ +import { describe, expect, it } from 'vitest' +import { HeadlessPromptError, HeadlessPromptPort } from '../src/questions/headless-prompt-port.ts' + +/** Unwrap an answered outcome or fail the test. */ +async function answered(promise: Promise<{ status: 'answered'; value: T } | { status: 'cancelled' }>): Promise { + const outcome = await promise + if (outcome.status !== 'answered') throw new Error('expected an answered outcome') + return outcome.value +} + +describe('HeadlessPromptError', () => { + it('names the unanswered prompt', () => { + const error = new HeadlessPromptError('DeepSeek API key') + expect(error).toBeInstanceOf(Error) + expect(error.name).toBe('HeadlessPromptError') + expect(error.prompt).toBe('DeepSeek API key') + expect(error.message).toContain('DeepSeek API key') + }) +}) + +describe('HeadlessPromptPort', () => { + const port = new HeadlessPromptPort() + + describe('text', () => { + it('takes the initial value when present', async () => { + expect(await answered(port.text({ message: 'name', initialValue: 'agent' }))).toBe('agent') + }) + + it('falls back to the default value', async () => { + expect(await answered(port.text({ message: 'dir', defaultValue: 'my-agent' }))).toBe('my-agent') + }) + + it('prefers the initial value over the default value', async () => { + expect(await answered(port.text({ message: 'dir', initialValue: 'given', defaultValue: 'my-agent' }))).toBe('given') + }) + + it('fails loud when no default exists', async () => { + await expect(port.text({ message: 'base URL' })).rejects.toThrow(HeadlessPromptError) + }) + + it('fails loud when the default is invalid', async () => { + await expect(port.text({ + message: 'name', + defaultValue: '', + validate: value => value.length === 0 ? 'required' : undefined, + })).rejects.toThrow(/required/) + }) + }) + + describe('secret', () => { + it('always fails loud', async () => { + await expect(port.secret({ message: 'API key' })).rejects.toThrow(HeadlessPromptError) + }) + }) + + describe('select', () => { + it('takes the initial value when present', async () => { + expect(await answered(port.select({ message: 'pm', options: [{ value: 'npm', label: 'npm' }], initialValue: 'npm' }))).toBe('npm') + }) + + it('fails loud without an initial value', async () => { + await expect(port.select({ message: 'pm', options: [{ value: 'npm', label: 'npm' }] })).rejects.toThrow(HeadlessPromptError) + }) + }) + + describe('multiselect', () => { + it('returns the initial values', async () => { + expect(await answered(port.multiselect({ message: 'x', options: [], initialValues: ['a', 'b'] }))).toEqual(['a', 'b']) + }) + + it('returns an empty selection when none are supplied and none are required', async () => { + expect(await answered(port.multiselect({ message: 'x', options: [] }))).toEqual([]) + }) + + it('fails loud when required and nothing is preselected', async () => { + await expect(port.multiselect({ message: 'x', options: [], required: true })).rejects.toThrow(HeadlessPromptError) + }) + }) + + describe('confirm', () => { + it('takes the initial value when present', async () => { + expect(await answered(port.confirm({ message: 'install?', initialValue: false }))).toBe(false) + }) + + it('fails loud without an initial value', async () => { + await expect(port.confirm({ message: 'apply?' })).rejects.toThrow(HeadlessPromptError) + }) + }) + + describe('nestedMultiselect', () => { + it('always fails loud', async () => { + await expect(port.nestedMultiselect({ message: 'Select features', options: [] })).rejects.toThrow(HeadlessPromptError) + }) + }) +}) diff --git a/packages/sdk/helper/tests/questions.spec.ts b/packages/sdk/helper/tests/questions.spec.ts index 463f3b979f..5eb205075f 100644 --- a/packages/sdk/helper/tests/questions.spec.ts +++ b/packages/sdk/helper/tests/questions.spec.ts @@ -450,4 +450,35 @@ describe('feature configurator', () => { await expect(new FeatureConfigurator(new QueuePromptPort([])).configure(new EmptyExclusive(), profile)) .rejects.toThrow('has no default option') }) + + it('configures fully from prefilled options, values, and secrets without prompting', async () => { + const registry = createBuiltinRegistry(profile) + const port = new QueuePromptPort([]) + const result = await new FeatureConfigurator(port).configure( + registry.get(featureId('provider')), + profile, + undefined, + ['custom'], + { apiKey: 'prefilled-key' }, + { baseURL: 'https://prefilled' }, + ) + expect(result).toMatchObject({ + options: ['custom'], + values: { baseURL: 'https://prefilled' }, + secrets: { apiKey: 'prefilled-key' }, + }) + expect(port.requests).toEqual([]) + }) + + it('rejects a non-string prefilled feature value', async () => { + const registry = createBuiltinRegistry(profile) + await expect(new FeatureConfigurator(new QueuePromptPort([])).configure( + registry.get(featureId('provider')), + profile, + undefined, + ['custom'], + { apiKey: 'k' }, + { baseURL: 123 }, + )).rejects.toThrow('must be a string') + }) }) diff --git a/packages/sdk/scripts/src/config/config-workflow.ts b/packages/sdk/scripts/src/config/config-workflow.ts index 7d3d7ee43a..016d9d09b8 100644 --- a/packages/sdk/scripts/src/config/config-workflow.ts +++ b/packages/sdk/scripts/src/config/config-workflow.ts @@ -28,6 +28,17 @@ export interface ConfigWorkflowResult { installError?: Error } +/** + * Non-interactive desired end-state for a config run: the complete set of enabled + * features, with options and any secrets/values a newly installed feature needs. + * Features not listed are reconciled to disabled, exactly as an interactive tree + * selection would be. Custom (non-feature) cordis plugins keep their current state; + * toggling them headlessly is not yet supported. + */ +export interface ConfigPlan { + features: readonly FeatureSelection[] +} + function featureTarget(feature: Feature): string { return `feature:${feature.id}` } @@ -66,48 +77,58 @@ export class ConfigWorkflow { } /** Select desired state, reconcile the working copy, review, and apply. */ - async run(project: SdkProject, registry: FeatureRegistry): Promise { + async run(project: SdkProject, registry: FeatureRegistry, plan?: ConfigPlan): Promise { const edit = project.edit(registry) const configurator = new FeatureConfigurator(this.port) const features = registry.all().filter(feature => feature.isApplicable(project.profile)) const inspections = new Map(edit.inspections().map(item => [item.id, item])) const custom = edit.cordisConfigEntries().filter(entry => !registry.ownerOfPackage(entry.name, project.profile)) - const desired = requireAnswer(await this.port.nestedMultiselect({ - message: 'Configure the project', - showChanges: true, - options: [ - ...features.map((feature) => { - const installation = inspections.get(feature.id) - /* v8 ignore next -- inspections() is built from this exact feature registry */ - if (!installation) throw new Error(`feature inspection is missing: ${feature.id}`) - const inconsistent = installation.state === 'inconsistent' - const selectedOptions = new Set(installation.options.length > 0 - ? installation.options - : feature.defaultOptions(project.profile)) - return { - value: featureTarget(feature), - label: feature.summary, - required: feature.required, - default: feature.required || installation.state === 'enabled' || inconsistent, - disabled: inconsistent, - ...inconsistent ? { warning: installation.diagnostics.join('; ') } : {}, - ...feature.mode === 'single' ? {} : { - choiceMode: feature.mode, - choices: feature.options.map(option => ({ - value: option.id, - label: option.label, - default: selectedOptions.has(option.id), - })), - }, - } - }), - ...custom.map(entry => ({ - value: pluginTarget(entry.id), - label: `${entry.name} [custom]`, - default: !entry.disabled, + const desired = plan + ? [ + ...plan.features.map(selection => ({ + value: featureTarget(registry.get(selection.id)), + choices: selection.options, })), - ], - })) + ...custom + .filter(entry => !entry.disabled) + .map(entry => ({ value: pluginTarget(entry.id), choices: [] as readonly string[] })), + ] + : requireAnswer(await this.port.nestedMultiselect({ + message: 'Configure the project', + showChanges: true, + options: [ + ...features.map((feature) => { + const installation = inspections.get(feature.id) + /* v8 ignore next -- inspections() is built from this exact feature registry */ + if (!installation) throw new Error(`feature inspection is missing: ${feature.id}`) + const inconsistent = installation.state === 'inconsistent' + const selectedOptions = new Set(installation.options.length > 0 + ? installation.options + : feature.defaultOptions(project.profile)) + return { + value: featureTarget(feature), + label: feature.summary, + required: feature.required, + default: feature.required || installation.state === 'enabled' || inconsistent, + disabled: inconsistent, + ...inconsistent ? { warning: installation.diagnostics.join('; ') } : {}, + ...feature.mode === 'single' ? {} : { + choiceMode: feature.mode, + choices: feature.options.map(option => ({ + value: option.id, + label: option.label, + default: selectedOptions.has(option.id), + })), + }, + } + }), + ...custom.map(entry => ({ + value: pluginTarget(entry.id), + label: `${entry.name} [custom]`, + default: !entry.disabled, + })), + ], + })) const desiredByTarget = new Map(desired.map(item => [item.value, item])) const targetProfile = { ...project.profile, @@ -117,6 +138,9 @@ export class ConfigWorkflow { if (!feature.isApplicable(targetProfile)) desiredByTarget.delete(featureTarget(feature)) } + const plannedById = new Map( + (plan?.features ?? []).map(selection => [selection.id, selection]), + ) for (const feature of features) { const installation = inspections.get(feature.id) /* v8 ignore next -- inspections() is built from this exact feature registry */ @@ -124,7 +148,7 @@ export class ConfigWorkflow { if (installation.state === 'inconsistent') continue const choice = desiredByTarget.get(featureTarget(feature)) if (!choice && !feature.required) continue - await this.enableOrConfigure(feature, installation, choice, project, edit, configurator) + await this.enableOrConfigure(feature, installation, choice, project, edit, configurator, plannedById.get(feature.id)) } for (const feature of [...features].reverse()) { @@ -176,6 +200,7 @@ export class ConfigWorkflow { project: SdkProject, edit: ReturnType, configurator: FeatureConfigurator, + planned?: FeatureSelection, ): Promise { const options = choice?.choices.length ? choice.choices @@ -183,7 +208,9 @@ export class ConfigWorkflow { ? installation.options : feature.defaultOptions(project.profile) if (installation.state === 'absent') { - const selection = await configurator.configure(feature, project.profile, undefined, options) + const selection = await configurator.configure( + feature, project.profile, undefined, options, planned?.secrets ?? {}, planned?.values ?? {}, + ) edit.installFeature(feature, selection) return } @@ -191,10 +218,7 @@ export class ConfigWorkflow { if (!installation.selection) throw new Error(`feature ${feature.id} has no readable selection`) if (!sameOptions(installation.options, options)) { const selection: FeatureSelection = await configurator.configure( - feature, - project.profile, - installation.selection, - options, + feature, project.profile, installation.selection, options, planned?.secrets ?? {}, planned?.values ?? {}, ) edit.configureFeature(feature, selection) } diff --git a/packages/sdk/scripts/tests/scripts.spec.ts b/packages/sdk/scripts/tests/scripts.spec.ts index 8d110799ea..42c74f5631 100644 --- a/packages/sdk/scripts/tests/scripts.spec.ts +++ b/packages/sdk/scripts/tests/scripts.spec.ts @@ -5,6 +5,7 @@ import { PassThrough, Writable } from 'node:stream' import { fileURLToPath, pathToFileURL } from 'node:url' import { afterEach, describe, expect, expectTypeOf, it, vi } from 'vitest' import { + HeadlessPromptPort, LocalPluginBlueprint, NpmPackageManager, SdkProject, @@ -29,7 +30,7 @@ import { parseDshSdkArgs, parseSdkBootArgs } from '../src/args.ts' import { PluginBuild, ProjectBuild, runProjectBuild } from '../src/build.ts' import { runDshSdkCommand, type DshSdkCommandContext } from '../src/command.ts' import { runConfigCommand } from '../src/config.ts' -import { ConfigWorkflow } from '../src/config/config-workflow.ts' +import { ConfigWorkflow, type ConfigPlan } from '../src/config/config-workflow.ts' import { initialize, resolve as resolveLocalPlugin } from '../src/local-plugin-loader-hooks.ts' const temporary: string[] = [] @@ -399,6 +400,28 @@ describe('ConfigWorkflow', () => { expect(output.read()).toContain('Disable feature: todo') }) + it('reconciles a headless plan without prompting and preserves custom plugins', async () => { + const project = await committedProject([], [new LocalPluginBlueprint('plugin', 'plugin')]) + const registry = createBuiltinRegistry(project.profile) + const output = outputBuffer() + let installs = 0 + const plan: ConfigPlan = { + features: [ + { id: featureId('bash'), options: ['local'] }, + { id: featureId('persistence'), options: ['jsonl'] }, + { id: featureId('todo'), options: ['default'] }, + { id: featureId('web'), options: ['exa'], secrets: { apiKey: 'exa-key' } }, + ], + } + const result = await new ConfigWorkflow( + new HeadlessPromptPort(), output.stream, async () => { installs += 1 }, + ).run(project, registry, plan) + expect(result.commit?.project.cordis.entry('tool-todo')).toBeDefined() + // the unlisted custom local plugin keeps its enabled state (not nuked by the plan) + expect(result.commit?.project.cordis.entry('plugin')?.disabled).toBeFalsy() + expect(installs).toBe(1) + }) + it('installs once after NPM dependency changes and keeps committed files on install failure', async () => { const project = await committedProject() const registry = createBuiltinRegistry(project.profile) From 825a63ab012671864a8d831b6d180a19538a5691 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 13:20:34 +0800 Subject: [PATCH 16/73] feat(sdk): add dsh-plugin-fetch source + fetcher seam MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greenfield #2 "建插件" modules for the forthcoming `dsh-sdk create ` command, in a new foundation-independent package so it touches none of the dsh-scripts / dsh-helper / create-sdk hotspots the foundation refactor edits. - `resolvePluginSource(spec)` parses `owner/repo[/subdir]#ref` (github) or `pkg@version` (npm) into a `PluginSource` discriminated union, failing loud on an ambiguous or malformed spec. - `PluginFetcher` seam + `fetchPlugin` tag dispatch returning a common `FetchedPlugin` (temp dir + immutable provenance). - `GigetFetcher` (github) over @bluwy/giget-core: resolve `#ref` to a commit SHA first, download that SHA; provenance pins the SHA. Chosen over unjs/giget for its single runtime dep and absent install/action surface. - `PacoteFetcher` (npm) over pacote: resolve the manifest, then extract the tarball verified against its registry integrity. Registry-only is enforced by the source resolver; extract runs no lifecycle scripts. - Branded `CommitSha`/`Integrity`; network + temp-dir boundaries are injected so the logic is unit-tested at 100% per-file coverage without network. Wiring (package.json pin, cordis.yml via ProjectEditSession with a confirmed diff, install --ignore-scripts) and the launcher command registration land later with the foundation. --- docs/module-graph.md | 3 + packages/sdk/README.md | 1 + packages/sdk/plugin-fetch/README.md | 32 + packages/sdk/plugin-fetch/package.json | 37 + packages/sdk/plugin-fetch/src/fetcher.ts | 90 ++ .../sdk/plugin-fetch/src/giget-fetcher.ts | 116 +++ packages/sdk/plugin-fetch/src/ids.ts | 39 + packages/sdk/plugin-fetch/src/index.ts | 48 + packages/sdk/plugin-fetch/src/never.ts | 19 + .../sdk/plugin-fetch/src/pacote-fetcher.ts | 129 +++ packages/sdk/plugin-fetch/src/source.ts | 126 +++ .../sdk/plugin-fetch/tests/fetcher.spec.ts | 64 ++ .../plugin-fetch/tests/giget-fetcher.spec.ts | 145 +++ packages/sdk/plugin-fetch/tests/ids.spec.ts | 39 + packages/sdk/plugin-fetch/tests/never.spec.ts | 19 + .../plugin-fetch/tests/pacote-fetcher.spec.ts | 108 +++ .../sdk/plugin-fetch/tests/source.spec.ts | 107 +++ packages/sdk/plugin-fetch/tsconfig.json | 15 + pnpm-lock.yaml | 833 +++++++++++++++++- .../verify-package-readme-model-experience.ts | 1 + tsconfig.build.json | 3 +- tsconfig.json | 3 +- 22 files changed, 1974 insertions(+), 3 deletions(-) create mode 100644 packages/sdk/plugin-fetch/README.md create mode 100644 packages/sdk/plugin-fetch/package.json create mode 100644 packages/sdk/plugin-fetch/src/fetcher.ts create mode 100644 packages/sdk/plugin-fetch/src/giget-fetcher.ts create mode 100644 packages/sdk/plugin-fetch/src/ids.ts create mode 100644 packages/sdk/plugin-fetch/src/index.ts create mode 100644 packages/sdk/plugin-fetch/src/never.ts create mode 100644 packages/sdk/plugin-fetch/src/pacote-fetcher.ts create mode 100644 packages/sdk/plugin-fetch/src/source.ts create mode 100644 packages/sdk/plugin-fetch/tests/fetcher.spec.ts create mode 100644 packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts create mode 100644 packages/sdk/plugin-fetch/tests/ids.spec.ts create mode 100644 packages/sdk/plugin-fetch/tests/never.spec.ts create mode 100644 packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts create mode 100644 packages/sdk/plugin-fetch/tests/source.spec.ts create mode 100644 packages/sdk/plugin-fetch/tsconfig.json diff --git a/docs/module-graph.md b/docs/module-graph.md index 8b67c73d27..c9439d5501 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -138,6 +138,7 @@ flowchart TD end subgraph group_sdk["packages/sdk"] pkg_helper["helper"] + pkg_plugin_fetch["plugin-fetch"] pkg_scripts["scripts"] end subgraph group_tasks["packages/tasks"] @@ -152,6 +153,7 @@ flowchart TD pkg_llm --> pkg_brand pkg_code_runtime_worker --> pkg_code_runtime pkg_helper --> pkg_brand + pkg_plugin_fetch --> pkg_brand pkg_scripts --> pkg_app_boot pkg_llm_deepseek --> pkg_llm pkg_llm_pi_ai --> pkg_llm @@ -442,6 +444,7 @@ flowchart TD | [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand) | | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime) | | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) | +| [`plugin-fetch`](../packages/sdk/plugin-fetch) | `sdk` | [`brand`](../packages/util/brand) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) | | [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | | [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) | diff --git a/packages/sdk/README.md b/packages/sdk/README.md index 953d52ea76..92c8794e64 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -9,6 +9,7 @@ The [feature RFC](../../docs/rfc/proposed/feature/2026-07-14-sdk-developer-proje | [`helper`](helper/README.md) | Project aggregate, edit session, builtin features, project documents, templates, package managers, and prompt abstraction | | [`scripts`](scripts/README.md) | The `dsh-sdk` launcher: `start`, `dev`, `build`, and interactive `config` | | [`create-sdk`](create-sdk/README.md) | The `npm create @deepseek-ai/sdk` initializer | +| [`plugin-fetch`](plugin-fetch/README.md) | Fetch an external plugin (github/npm) into a temp dir — pinned and un-executed — for `dsh-sdk create` | `@deepseek-ai/create-sdk` is the one package-name exception to the repository's `@deepseek-ai/dsh-*` rule: npm's scoped initializer convention requires that name for `npm create @deepseek-ai/sdk`. diff --git a/packages/sdk/plugin-fetch/README.md b/packages/sdk/plugin-fetch/README.md new file mode 100644 index 0000000000..801dc2a2a6 --- /dev/null +++ b/packages/sdk/plugin-fetch/README.md @@ -0,0 +1,32 @@ +# `@deepseek-ai/dsh-plugin-fetch` + +Fetch an external Cordis plugin into a temp directory — pinned to an immutable commit or integrity and never executed — for the forthcoming `dsh-sdk create ` command. + +The package parses a source spec into a `PluginSource`, dispatches to the matching `PluginFetcher`, and returns a common `FetchedPlugin` (temp dir + immutable provenance) that the wiring step pins into `package.json`, mounts in `cordis.yml`, and installs with `--ignore-scripts`. + +| Export | Role | +|---|---| +| `resolvePluginSource(spec)` → `PluginSource` | Parse `owner/repo[/subdir]#ref` (github) or `pkg@version` (npm); fail loud on an ambiguous or malformed spec | +| `PluginFetcher` | The fetch seam: resolve the pin BEFORE download, extract without executing pulled code | +| `GigetFetcher` / `createGigetFetcher()` | Github fetcher over `@bluwy/giget-core`: resolve `#ref` to a commit SHA, download that SHA | +| `PacoteFetcher` / `createPacoteFetcher()` | Npm fetcher over `pacote`: resolve the manifest, then extract the tarball verified against its integrity | +| `fetchPlugin(source, fetchers)` → `FetchedPlugin` | Dispatch one source to its fetcher by discriminant tag | + +## Safety model — confirm-before-run, not run-on-fetch + +A fetch only downloads and unpacks; it runs no install, no `postinstall`/`prepare`, and no degit-style template actions. + +- **github** uses `@bluwy/giget-core` (one runtime dependency, `modern-tar`; no CLI, install, or JSON-registry surface) so a fetch can only download and untar a tarball. The commit is pinned first: `GigetFetcher` resolves `#ref` — or the default branch when absent — to an immutable SHA via the GitHub commits API, then downloads that SHA. Provenance carries the SHA so wiring pins `github:owner/repo#`. +- **npm** uses `pacote`. Registry-only is enforced upstream: `resolvePluginSource` produces only a `name@version` registry spec, so pacote never sees a git/file/dir spec whose lifecycle scripts would run, and a registry tarball extract is a plain untar. The manifest is resolved first so extract verifies the artifact against the registry-published integrity (a mismatch raises `EINTEGRITY`). Provenance carries the exact version, resolved URL, and integrity. + +Both network boundaries (giget download, GitHub ref resolution, pacote, temp-dir allocation) are constructor-injected, so the fetch logic is unit-tested without network; the `create*Fetcher()` factories wire the real libraries. + +## Model Experience + +None, as this developer tooling acquires plugin sources for the SDK launcher and registers no live agent or model surface. + +## Known Limitations and Deferred Work + +- **Wiring is not here yet** — pinning `package.json`, mounting `cordis.yml` through `ProjectEditSession` with a confirmed diff, and `install --ignore-scripts` land with the `dsh-sdk create` command. This package stops at a fetched, pinned temp directory. +- **npm registry authentication** — `PacoteFetcher` targets a public or default-configured registry; private-registry auth beyond pacote's ambient npm config is deferred. +- **Template-repo initialization** — the whole-project init mode (`dsh-sdk create` from a template repository) is out of scope; this package fetches a single plugin into an existing project. diff --git a/packages/sdk/plugin-fetch/package.json b/packages/sdk/plugin-fetch/package.json new file mode 100644 index 0000000000..e8e38090e6 --- /dev/null +++ b/packages/sdk/plugin-fetch/package.json @@ -0,0 +1,37 @@ +{ + "name": "@deepseek-ai/dsh-plugin-fetch", + "description": "Fetch an external Cordis plugin (github or npm) into a temp dir, pinned and un-executed, for dsh-sdk create", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "dependencies": { + "@bluwy/giget-core": "^0.1.7", + "pacote": "^22.0.0" + }, + "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", + "@types/pacote": "^11.1.8", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/sdk/plugin-fetch/src/fetcher.ts b/packages/sdk/plugin-fetch/src/fetcher.ts new file mode 100644 index 0000000000..7c3d3a763c --- /dev/null +++ b/packages/sdk/plugin-fetch/src/fetcher.ts @@ -0,0 +1,90 @@ +/** + * The `PluginFetcher` seam, its common `FetchedPlugin` result, and the + * tag-dispatched entry point. A fetcher acquires one plugin source into a fresh + * temp directory WITHOUT executing any pulled code, and reports immutable + * provenance the wiring step pins the dependency to. + * + * @module @deepseek-ai/dsh-plugin-fetch/fetcher + */ + +import { mkdtemp } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { assertNever } from './never.ts' +import type { CommitSha, Integrity } from './ids.ts' +import type { GithubSource, NpmSource, PluginSource } from './source.ts' + +/** Immutable pin for a github fetch: the resolved commit the tarball came from. */ +export interface GithubProvenance { + readonly kind: 'github' + readonly sha: CommitSha +} + +/** Immutable pin for an npm fetch: exact version, tarball URL, and integrity. */ +export interface NpmProvenance { + readonly kind: 'npm' + /** Concrete resolved version (e.g. `1.2.3`), never the requested range/tag. */ + readonly version: string + /** Tarball URL the artifact resolved to. */ + readonly resolved: string + /** Subresource integrity the artifact was verified against. */ + readonly integrity: Integrity +} + +/** Provenance a fetch records so wiring can pin an immutable dependency. */ +export type PluginProvenance = GithubProvenance | NpmProvenance + +/** The common result of fetching any plugin source. */ +export interface FetchedPlugin { + /** Absolute temp directory holding the extracted, UN-executed source. */ + readonly dir: string + /** The source that produced this fetch, echoed for the wiring step. */ + readonly source: PluginSource + /** Immutable provenance to pin the dependency during wiring. */ + readonly provenance: PluginProvenance +} + +/** + * A fetcher for one source kind. Implementations resolve the immutable pin + * BEFORE download and must never run lifecycle scripts or template actions. + */ +export interface PluginFetcher { + /** The single source kind this fetcher handles. */ + readonly kind: S['kind'] + /** + * Fetch one source into a fresh temp directory. + * @param source - the resolved source to fetch. + * @returns the temp dir plus immutable provenance. + */ + fetch(source: S): Promise +} + +/** The per-kind fetchers {@link fetchPlugin} dispatches across. */ +export interface PluginFetchers { + readonly github: PluginFetcher + readonly npm: PluginFetcher +} + +/** + * Dispatch one source to its fetcher by discriminant tag. + * @param source - the resolved plugin source. + * @param fetchers - the per-kind fetchers to route across. + * @returns the fetch result from the matching fetcher. + */ +export function fetchPlugin(source: PluginSource, fetchers: PluginFetchers): Promise { + switch (source.kind) { + case 'github': return fetchers.github.fetch(source) + case 'npm': return fetchers.npm.fetch(source) + default: return assertNever(source, 'fetchPlugin') + } +} + +/** + * Create a fresh, empty temp directory for one fetch — the default temp-dir + * seam shared by the concrete fetchers. + * @param prefix - a `mkdtemp` name prefix identifying the fetch kind. + * @returns the absolute path of the created directory. + */ +export function createTempDir(prefix: string): Promise { + return mkdtemp(join(tmpdir(), prefix)) +} diff --git a/packages/sdk/plugin-fetch/src/giget-fetcher.ts b/packages/sdk/plugin-fetch/src/giget-fetcher.ts new file mode 100644 index 0000000000..55e001063c --- /dev/null +++ b/packages/sdk/plugin-fetch/src/giget-fetcher.ts @@ -0,0 +1,116 @@ +/** + * The github {@link PluginFetcher}, backed by `@bluwy/giget-core`. + * + * `@bluwy/giget-core` is chosen over unjs `giget`: it carries a single runtime + * dependency (`modern-tar`) versus giget's CLI/registry stack, and it dropped + * the `install` and JSON-registry options entirely, so a fetch can only ever + * download and untar a tarball — never run install or degit-style actions. That + * is exactly the "extract, never execute" guarantee this feature needs. + * + * The commit is pinned BEFORE download: {@link GigetFetcher} resolves `#ref` to + * an immutable SHA (default via the GitHub commits API), then downloads that + * SHA. Provenance carries the SHA so wiring pins `github:owner/repo#`. + * + * @module @deepseek-ai/dsh-plugin-fetch/giget-fetcher + */ + +import { downloadTemplate } from '@bluwy/giget-core' +import { createTempDir, type FetchedPlugin, type PluginFetcher } from './fetcher.ts' +import { commitSha, type CommitSha } from './ids.ts' +import type { GithubSource } from './source.ts' + +/** Temp-dir name prefix for github fetches. */ +export const GITHUB_TEMP_PREFIX = 'dsh-plugin-github-' + +/** Downloads a giget input string into `dir`; the tarball-extraction seam. */ +export type DownloadTemplate = ( + input: string, + options: { dir: string; force: 'clean' }, +) => Promise<{ dir: string }> + +/** Resolves a github source's ref to an immutable commit SHA before download. */ +export type ResolveRef = (source: GithubSource) => Promise + +/** The injected collaborators a {@link GigetFetcher} needs. */ +export interface GigetFetcherDeps { + /** Downloads a pinned giget input into a directory. */ + download: DownloadTemplate + /** Resolves `source.ref` (or the default branch) to a commit SHA. */ + resolveRef: ResolveRef + /** Allocates the fresh temp directory to download into. */ + createTempDir: (prefix: string) => Promise +} + +/** Build the giget input string that pins a github source to a commit SHA. */ +function gigetInput(source: GithubSource, sha: CommitSha): string { + const path = source.subdir ? `${source.owner}/${source.repo}/${source.subdir}` : `${source.owner}/${source.repo}` + return `${path}#${sha}` +} + +/** A human-readable label for one github source, for error messages. */ +function githubLabel(source: GithubSource): string { + return `${source.owner}/${source.repo}#${source.ref ?? 'HEAD'}` +} + +/** + * Resolve a github source's ref to an immutable SHA via the GitHub commits API. + * Uses the `application/vnd.github.sha` media type, which returns the resolved + * commit id as plain text. + * @param source - the github source; an absent `ref` resolves the default branch (`HEAD`). + * @param token - optional bearer token for private repositories. + * @returns the resolved immutable commit SHA. + * @throws if the GitHub API rejects the request. + */ +export async function defaultResolveRef(source: GithubSource, token?: string): Promise { + const ref = source.ref ?? 'HEAD' + const url = `https://api.github.com/repos/${source.owner}/${source.repo}/commits/${ref}` + const headers: Record = { Accept: 'application/vnd.github.sha' } + if (token !== undefined) headers.Authorization = `Bearer ${token}` + const response = await fetch(url, { headers }) + if (!response.ok) { + throw new Error(`cannot resolve github ref ${githubLabel(source)}: HTTP ${response.status}`) + } + return commitSha((await response.text()).trim()) +} + +/** Fetches a github plugin source by pinning `#ref` to a commit SHA, then downloading it. */ +export class GigetFetcher implements PluginFetcher { + readonly kind = 'github' as const + private readonly deps: GigetFetcherDeps + + /** Construct with injected download, ref-resolution, and temp-dir seams. */ + constructor(deps: GigetFetcherDeps) { + this.deps = deps + } + + async fetch(source: GithubSource): Promise { + const sha = await this.deps.resolveRef(source) + const dir = await this.deps.createTempDir(GITHUB_TEMP_PREFIX) + await this.deps.download(gigetInput(source, sha), { dir, force: 'clean' }) + return { dir, source, provenance: { kind: 'github', sha } } + } +} + +/** Options for the production github fetcher. */ +export interface GithubFetchOptions { + /** Bearer token for private repositories; defaults to `GITHUB_TOKEN`. */ + token?: string +} + +/** + * Build the production github fetcher wired to `@bluwy/giget-core` and the + * GitHub commits API. + * @param options - optional token override (else `process.env.GITHUB_TOKEN`). + * @returns a {@link GigetFetcher} using the real download and ref-resolution seams. + */ +export function createGigetFetcher(options: GithubFetchOptions = {}): GigetFetcher { + const token = options.token ?? process.env.GITHUB_TOKEN + return new GigetFetcher({ + download: (input, downloadOptions) => downloadTemplate(input, { + ...downloadOptions, + ...token !== undefined ? { providerOptions: { auth: token } } : {}, + }), + resolveRef: source => defaultResolveRef(source, token), + createTempDir, + }) +} diff --git a/packages/sdk/plugin-fetch/src/ids.ts b/packages/sdk/plugin-fetch/src/ids.ts new file mode 100644 index 0000000000..a3a2ffa575 --- /dev/null +++ b/packages/sdk/plugin-fetch/src/ids.ts @@ -0,0 +1,39 @@ +/** + * Branded provenance identities owned by the plugin-fetch layer. Both cross the + * fetch → wiring boundary and are opaque tokens that must not be confused with + * ordinary strings (a package name, a URL) at that seam. + * + * @module @deepseek-ai/dsh-plugin-fetch/ids + */ + +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** An immutable git commit object id a github fetch pins to. */ +export type CommitSha = Branded<'CommitSha'> + +/** + * Construct a {@link CommitSha}, validating the hexadecimal object-id shape. + * @param value - lowercase hex of an abbreviated or full commit id (7–64 chars, covering SHA-1 and SHA-256). + * @returns the branded commit id. + */ +export function commitSha(value: string): CommitSha { + if (!/^[0-9a-f]{7,64}$/.test(value)) { + throw new Error(`invalid commit sha: ${JSON.stringify(value)}`) + } + return value as CommitSha +} + +/** A Subresource Integrity string an npm fetch pins to. */ +export type Integrity = Branded<'Integrity'> + +/** + * Construct an {@link Integrity}, validating the SRI `-` shape. + * @param value - a single SRI entry using sha256, sha384, or sha512. + * @returns the branded integrity string. + */ +export function integrity(value: string): Integrity { + if (!/^sha(256|384|512)-[A-Za-z0-9+/]+={0,2}$/.test(value)) { + throw new Error(`invalid subresource integrity: ${JSON.stringify(value)}`) + } + return value as Integrity +} diff --git a/packages/sdk/plugin-fetch/src/index.ts b/packages/sdk/plugin-fetch/src/index.ts new file mode 100644 index 0000000000..d4b9682ff5 --- /dev/null +++ b/packages/sdk/plugin-fetch/src/index.ts @@ -0,0 +1,48 @@ +/** + * Fetch an external Cordis plugin (github or npm) into a temp directory — + * pinned to an immutable commit/integrity and never executed — for the + * `dsh-sdk create ` command. Parses a source spec, dispatches to the + * matching fetcher, and returns a common {@link FetchedPlugin} the wiring step + * pins and mounts. + * + * @module @deepseek-ai/dsh-plugin-fetch + */ + +export { resolvePluginSource } from './source.ts' +export type { GithubSource, NpmSource, PluginSource } from './source.ts' +export { commitSha, integrity } from './ids.ts' +export type { CommitSha, Integrity } from './ids.ts' +export { createTempDir, fetchPlugin } from './fetcher.ts' +export type { + FetchedPlugin, + GithubProvenance, + NpmProvenance, + PluginFetcher, + PluginFetchers, + PluginProvenance, +} from './fetcher.ts' +export { + createGigetFetcher, + defaultResolveRef, + GigetFetcher, + GITHUB_TEMP_PREFIX, +} from './giget-fetcher.ts' +export type { + DownloadTemplate, + GigetFetcherDeps, + GithubFetchOptions, + ResolveRef, +} from './giget-fetcher.ts' +export { + createPacoteFetcher, + NPM_TEMP_PREFIX, + PacoteFetcher, +} from './pacote-fetcher.ts' +export type { + NpmFetchOptions, + PacoteApi, + PacoteExtractResult, + PacoteFetcherDeps, + PacoteFetchOptions, + PacoteResolution, +} from './pacote-fetcher.ts' diff --git a/packages/sdk/plugin-fetch/src/never.ts b/packages/sdk/plugin-fetch/src/never.ts new file mode 100644 index 0000000000..664ab3293d --- /dev/null +++ b/packages/sdk/plugin-fetch/src/never.ts @@ -0,0 +1,19 @@ +/** + * Exhaustiveness helper for this package's closed unions. Kept local so the + * SDK plugin-fetch tooling stays free of the model-runtime `dsh-llm` dependency + * that owns the shared `assertNever`. + * + * @module @deepseek-ai/dsh-plugin-fetch/never + */ + +/** + * Mark an unreachable closed-union branch. A newly unhandled variant fails + * compilation at the call site; a value that escaped its type throws at runtime. + * @param value - the impossible value; typed `never` so a new variant fails to compile at every call site. + * @param context - optional label prefixed into the throw message. + * @returns never — it always throws, rendering the offending value. + */ +export function assertNever(value: never, context?: string): never { + const rendered = (JSON.stringify(value) as string | undefined) ?? String(value) + throw new Error(`unreachable variant${context ? ` in ${context}` : ''}: ${rendered}`) +} diff --git a/packages/sdk/plugin-fetch/src/pacote-fetcher.ts b/packages/sdk/plugin-fetch/src/pacote-fetcher.ts new file mode 100644 index 0000000000..dfb1d790b8 --- /dev/null +++ b/packages/sdk/plugin-fetch/src/pacote-fetcher.ts @@ -0,0 +1,129 @@ +/** + * The npm {@link PluginFetcher}, backed by `pacote`. + * + * Supply-chain safety comes from three layers: (1) {@link resolvePluginSource} + * only ever produces a registry `name@version` spec, so pacote classifies it as + * a registry source and cannot be steered to a git/file/dir spec whose + * lifecycle scripts would run; (2) a registry tarball extract is a plain untar — + * pacote runs no `prepare`/`postinstall` during {@link PacoteFetcher.fetch}; and + * (3) the later wiring step installs with `--ignore-scripts`. The manifest is + * resolved first so extract verifies the tarball against the registry-published + * integrity (a mismatch raises `EINTEGRITY`). + * + * @module @deepseek-ai/dsh-plugin-fetch/pacote-fetcher + */ + +import { extract as pacoteExtract, manifest as pacoteManifest } from 'pacote' +import { createTempDir, type FetchedPlugin, type PluginFetcher } from './fetcher.ts' +import { integrity } from './ids.ts' +import type { NpmSource } from './source.ts' + +/** Temp-dir name prefix for npm fetches. */ +export const NPM_TEMP_PREFIX = 'dsh-plugin-npm-' + +/** The subset of pacote options this fetcher passes through. */ +export interface PacoteFetchOptions { + /** Registry to resolve against; absent uses pacote's default. */ + registry?: string + /** Known resolved tarball URL, forwarded to extract. */ + resolved?: string + /** Expected integrity, forwarded to extract for `EINTEGRITY` verification. */ + integrity?: string +} + +/** The resolved registry manifest fields this fetcher pins from. */ +export interface PacoteResolution { + /** Resolved tarball URL. */ + _resolved: string + /** Registry-published integrity. */ + _integrity: string + /** Concrete resolved version. */ + version: string +} + +/** The extract result fields this fetcher pins from. */ +export interface PacoteExtractResult { + /** Resolved tarball URL of the extracted artifact. */ + resolved: string + /** Integrity of the extracted artifact. */ + integrity: string +} + +/** The pacote surface a {@link PacoteFetcher} depends on; the fetch seam. */ +export interface PacoteApi { + /** Resolve a registry spec to its pinned manifest fields. */ + manifest: (spec: string, options?: PacoteFetchOptions) => Promise + /** Untar a registry spec into `dest`, verifying integrity when supplied. */ + extract: (spec: string, dest: string, options?: PacoteFetchOptions) => Promise +} + +/** The injected collaborators a {@link PacoteFetcher} needs. */ +export interface PacoteFetcherDeps { + /** The pacote resolve/extract surface. */ + pacote: PacoteApi + /** Allocates the fresh temp directory to extract into. */ + createTempDir: (prefix: string) => Promise + /** Registry to resolve against; absent uses pacote's default. */ + registry?: string +} + +/** Fetches an npm plugin source by resolving its manifest, then extracting the verified tarball. */ +export class PacoteFetcher implements PluginFetcher { + readonly kind = 'npm' as const + private readonly deps: PacoteFetcherDeps + + /** Construct with injected pacote, temp-dir, and optional registry seams. */ + constructor(deps: PacoteFetcherDeps) { + this.deps = deps + } + + async fetch(source: NpmSource): Promise { + const spec = `${source.name}@${source.version}` + const registryOptions: PacoteFetchOptions = this.deps.registry !== undefined + ? { registry: this.deps.registry } + : {} + const resolution = await this.deps.pacote.manifest(spec, registryOptions) + const dir = await this.deps.createTempDir(NPM_TEMP_PREFIX) + const extracted = await this.deps.pacote.extract(spec, dir, { + ...registryOptions, + resolved: resolution._resolved, + integrity: resolution._integrity, + }) + return { + dir, + source, + provenance: { + kind: 'npm', + version: resolution.version, + resolved: extracted.resolved, + integrity: integrity(extracted.integrity), + }, + } + } +} + +/** Options for the production npm fetcher. */ +export interface NpmFetchOptions { + /** Registry to resolve against; absent uses pacote's default. */ + registry?: string +} + +/** + * Build the production npm fetcher wired to `pacote`. + * @param options - optional registry override. + * @returns a {@link PacoteFetcher} using the real pacote resolve/extract seam. + */ +export function createPacoteFetcher(options: NpmFetchOptions = {}): PacoteFetcher { + const pacote: PacoteApi = { + manifest: async (spec, pacoteOptions) => { + const resolved = await pacoteManifest(spec, pacoteOptions) + return { _resolved: resolved._resolved, _integrity: resolved._integrity, version: resolved.version } + }, + extract: (spec, dest, pacoteOptions) => pacoteExtract(spec, dest, pacoteOptions), + } + return new PacoteFetcher({ + pacote, + createTempDir, + ...options.registry !== undefined ? { registry: options.registry } : {}, + }) +} diff --git a/packages/sdk/plugin-fetch/src/source.ts b/packages/sdk/plugin-fetch/src/source.ts new file mode 100644 index 0000000000..7d7e3ffcc4 --- /dev/null +++ b/packages/sdk/plugin-fetch/src/source.ts @@ -0,0 +1,126 @@ +/** + * The `PluginSource` discriminated union and the resolver that parses one CLI + * spec string into it. Ambiguous or malformed specs fail loud here — the single + * earliest resolvable point — rather than surfacing as a confusing fetch error. + * + * Grammar: + * - github: `owner/repo[/subdir]#ref` — a `#` unambiguously marks a github ref; + * `ref` is optional and, when omitted, the fetcher pins the default branch. + * - npm: `pkg@version` (scoped `@scope/pkg@version`) — the `@version` is the + * only disambiguator from a bare `owner/repo` github locator. + * + * @module @deepseek-ai/dsh-plugin-fetch/source + */ + +/** A plugin pulled from a github (git tarball) repository. */ +export interface GithubSource { + readonly kind: 'github' + /** Repository owner (user or org). */ + readonly owner: string + /** Repository name. */ + readonly repo: string + /** Path within the repository to extract; absent means the repository root. */ + readonly subdir?: string + /** Branch, tag, or commit; absent means the repository's default branch. */ + readonly ref?: string +} + +/** A plugin pulled from an npm registry by exact package and version spec. */ +export interface NpmSource { + readonly kind: 'npm' + /** Package name, including any `@scope/` prefix. */ + readonly name: string + /** Registry version, range, or dist-tag (non-empty). */ + readonly version: string +} + +/** Every plugin origin `dsh-sdk create ` understands. */ +export type PluginSource = GithubSource | NpmSource + +/** One `/`-separated github name segment (owner, repo, or subdir component). */ +function isNameSegment(segment: string): boolean { + return /^[A-Za-z0-9._-]+$/.test(segment) && segment !== '.' && segment !== '..' +} + +/** A git ref: branch, tag, or commit; permits `/`-nested names, rejects traversal. */ +function isGitRef(ref: string): boolean { + return /^[A-Za-z0-9._/-]+$/.test(ref) + && !ref.includes('..') + && !ref.startsWith('/') + && !ref.endsWith('/') +} + +/** An npm package name, scoped (`@scope/name`) or unscoped. */ +function isNpmPackageName(name: string): boolean { + const segment = /^[a-z0-9][a-z0-9._-]*$/ + if (name.startsWith('@')) { + const slash = name.indexOf('/') + if (slash < 2 || slash === name.length - 1) return false + return segment.test(name.slice(1, slash)) && segment.test(name.slice(slash + 1)) + } + return segment.test(name) +} + +/** Parse a `owner/repo[/subdir]` locator with an optional already-split ref. */ +function tryParseGithubLocator(locator: string, ref: string | undefined): GithubSource | undefined { + if (!/^[^\s@#]+$/.test(locator)) return undefined + const [owner, repo, ...subdirSegments] = locator.split('/') + if (owner === undefined || repo === undefined) return undefined + if (!isNameSegment(owner) || !isNameSegment(repo)) return undefined + if (subdirSegments.some(segment => !isNameSegment(segment))) return undefined + if (ref !== undefined && !isGitRef(ref)) return undefined + const subdir = subdirSegments.join('/') + return { + kind: 'github', + owner, + repo, + ...subdir.length > 0 ? { subdir } : {}, + ...ref !== undefined ? { ref } : {}, + } +} + +/** Parse `pkg@version` (scoped or unscoped); undefined when it is not npm-shaped. */ +function tryParseNpmSource(spec: string): NpmSource | undefined { + if (/[\s#]/.test(spec)) return undefined + // A scoped spec's version `@` follows the scope's `/`; an unscoped spec's is + // the first `@`. A leading `@` with no version `@` yields index 0 (rejected). + const versionAt = spec.startsWith('@') ? spec.indexOf('@', spec.indexOf('/') + 1) : spec.indexOf('@') + if (versionAt <= 0) return undefined + const name = spec.slice(0, versionAt) + const version = spec.slice(versionAt + 1) + if (version.length === 0 || version.includes('/')) return undefined + if (!isNpmPackageName(name)) return undefined + return { kind: 'npm', name, version } +} + +/** + * Parse one `dsh-sdk create ` spec into a {@link PluginSource}. + * @param spec - the raw source argument. + * @returns the discriminated source. + * @throws if the spec is empty, malformed, or ambiguous between github and npm. + */ +export function resolvePluginSource(spec: string): PluginSource { + const trimmed = spec.trim() + if (trimmed.length === 0) throw new Error('plugin source must not be empty') + + const hashIndex = trimmed.indexOf('#') + if (hashIndex !== -1) { + const ref = trimmed.slice(hashIndex + 1) + if (ref.length === 0) { + throw new Error(`github plugin source is missing a ref after '#': ${JSON.stringify(spec)}`) + } + const source = tryParseGithubLocator(trimmed.slice(0, hashIndex), ref) + if (!source) { + throw new Error(`invalid github plugin source: ${JSON.stringify(spec)} — expected "owner/repo[/subdir]#ref"`) + } + return source + } + + const npm = tryParseNpmSource(trimmed) + if (npm) return npm + const github = tryParseGithubLocator(trimmed, undefined) + if (github) return github + throw new Error( + `unrecognized plugin source: ${JSON.stringify(spec)} — expected "owner/repo[/subdir]#ref" (github) or "pkg@version" (npm)`, + ) +} diff --git a/packages/sdk/plugin-fetch/tests/fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/fetcher.spec.ts new file mode 100644 index 0000000000..b68a3d3ae1 --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/fetcher.spec.ts @@ -0,0 +1,64 @@ +import { rm, stat } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { describe, expect, it, vi } from 'vitest' +import { + createTempDir, + fetchPlugin, + type FetchedPlugin, + type PluginFetchers, +} from '../src/fetcher.ts' +import { commitSha } from '../src/ids.ts' +import type { GithubSource, NpmSource, PluginSource } from '../src/source.ts' + +function stubFetchers(): { fetchers: PluginFetchers; github: ReturnType; npm: ReturnType } { + const result = (dir: string): FetchedPlugin => ({ + dir, + source: { kind: 'github', owner: 'o', repo: 'r' }, + provenance: { kind: 'github', sha: commitSha('a'.repeat(40)) }, + }) + const github = vi.fn(async (source: GithubSource) => result(`github:${source.repo}`)) + const npm = vi.fn(async (source: NpmSource) => result(`npm:${source.name}`)) + return { + fetchers: { github: { kind: 'github', fetch: github }, npm: { kind: 'npm', fetch: npm } }, + github, + npm, + } +} + +describe('fetchPlugin', () => { + it('routes a github source to the github fetcher', async () => { + const { fetchers, github, npm } = stubFetchers() + const source: GithubSource = { kind: 'github', owner: 'o', repo: 'r' } + const result = await fetchPlugin(source, fetchers) + expect(github).toHaveBeenCalledWith(source) + expect(npm).not.toHaveBeenCalled() + expect(result.dir).toBe('github:r') + }) + + it('routes an npm source to the npm fetcher', async () => { + const { fetchers, github, npm } = stubFetchers() + const source: NpmSource = { kind: 'npm', name: 'plugin', version: '1.0.0' } + const result = await fetchPlugin(source, fetchers) + expect(npm).toHaveBeenCalledWith(source) + expect(github).not.toHaveBeenCalled() + expect(result.dir).toBe('npm:plugin') + }) + + it('throws on an unknown source kind', () => { + const { fetchers } = stubFetchers() + const bogus = { kind: 'svn' } as unknown as PluginSource + expect(() => fetchPlugin(bogus, fetchers)).toThrow(/unreachable variant in fetchPlugin/) + }) +}) + +describe('createTempDir', () => { + it('creates a fresh empty directory under the OS temp root', async () => { + const dir = await createTempDir('dsh-plugin-fetch-test-') + try { + expect(dir.startsWith(tmpdir())).toBe(true) + expect((await stat(dir)).isDirectory()).toBe(true) + } finally { + await rm(dir, { recursive: true, force: true }) + } + }) +}) diff --git a/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts new file mode 100644 index 0000000000..c0cafb31d5 --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts @@ -0,0 +1,145 @@ +import { afterEach, beforeEach, describe, expect, it, type Mock, vi } from 'vitest' +import { downloadTemplate } from '@bluwy/giget-core' +import { + createGigetFetcher, + defaultResolveRef, + GigetFetcher, + GITHUB_TEMP_PREFIX, + type GigetFetcherDeps, +} from '../src/giget-fetcher.ts' +import type { CommitSha } from '../src/ids.ts' +import type { GithubSource } from '../src/source.ts' + +vi.mock('@bluwy/giget-core', () => ({ downloadTemplate: vi.fn(async (_input: string, options: { dir: string }) => ({ dir: options.dir, source: '', info: { name: '', tar: '' } })) })) + +const SHA = 'a'.repeat(40) + +/** A `fetch` mock typed with the call signature the assertions destructure. */ +function fetchReturning(response: Response): Mock<(url: string, init?: RequestInit) => Promise> { + return vi.fn((_url: string, _init?: RequestInit) => Promise.resolve(response)) +} + +function fakeDeps(overrides: Partial = {}): { + deps: GigetFetcherDeps + download: ReturnType + resolveRef: ReturnType + createTempDir: ReturnType +} { + const download = vi.fn(async () => ({ dir: '/tmp/x' })) + const resolveRef = vi.fn(async () => SHA as CommitSha) + const createTempDir = vi.fn(async () => '/tmp/dsh-plugin-github-abc') + return { deps: { download, resolveRef, createTempDir, ...overrides }, download, resolveRef, createTempDir } +} + +describe('GigetFetcher.fetch', () => { + it('pins the ref to a SHA, downloads that SHA, and reports provenance', async () => { + const { deps, download, resolveRef, createTempDir } = fakeDeps() + const source: GithubSource = { kind: 'github', owner: 'unjs', repo: 'template', ref: 'main' } + const result = await new GigetFetcher(deps).fetch(source) + + expect(resolveRef).toHaveBeenCalledWith(source) + expect(createTempDir).toHaveBeenCalledWith(GITHUB_TEMP_PREFIX) + expect(download).toHaveBeenCalledWith(`unjs/template#${SHA}`, { dir: '/tmp/dsh-plugin-github-abc', force: 'clean' }) + expect(result).toEqual({ + dir: '/tmp/dsh-plugin-github-abc', + source, + provenance: { kind: 'github', sha: SHA }, + }) + }) + + it('includes the subdir in the download input', async () => { + const { deps, download } = fakeDeps() + const source: GithubSource = { kind: 'github', owner: 'o', repo: 'r', subdir: 'packages/plugin' } + await new GigetFetcher(deps).fetch(source) + expect(download).toHaveBeenCalledWith(`o/r/packages/plugin#${SHA}`, expect.anything()) + }) + + it('exposes its source kind', () => { + expect(new GigetFetcher(fakeDeps().deps).kind).toBe('github') + }) +}) + +describe('defaultResolveRef', () => { + afterEach(() => vi.unstubAllGlobals()) + + it('resolves the default branch (HEAD) with no auth header', async () => { + const fetchMock = fetchReturning(new Response(`${SHA}\n`, { status: 200 })) + vi.stubGlobal('fetch', fetchMock) + const sha = await defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r' }) + expect(sha).toBe(SHA) + const [url, init] = fetchMock.mock.calls[0]! + expect(url).toBe('https://api.github.com/repos/o/r/commits/HEAD') + expect((init as RequestInit).headers).toEqual({ Accept: 'application/vnd.github.sha' }) + }) + + it('resolves an explicit ref and sends a bearer token', async () => { + const fetchMock = fetchReturning(new Response(SHA, { status: 200 })) + vi.stubGlobal('fetch', fetchMock) + const sha = await defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r', ref: 'v1.2.3' }, 'secret') + expect(sha).toBe(SHA) + const [url, init] = fetchMock.mock.calls[0]! + expect(url).toBe('https://api.github.com/repos/o/r/commits/v1.2.3') + expect((init as RequestInit).headers).toEqual({ + Accept: 'application/vnd.github.sha', + Authorization: 'Bearer secret', + }) + }) + + it('throws with the HEAD label when the API rejects an unref-ed source', async () => { + vi.stubGlobal('fetch', fetchReturning(new Response('', { status: 404 }))) + await expect(defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r' })).rejects.toThrow( + /cannot resolve github ref o\/r#HEAD: HTTP 404/, + ) + }) + + it('throws with the explicit-ref label when the API rejects', async () => { + vi.stubGlobal('fetch', fetchReturning(new Response('', { status: 403 }))) + await expect( + defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r', ref: 'main' }), + ).rejects.toThrow(/cannot resolve github ref o\/r#main: HTTP 403/) + }) +}) + +describe('createGigetFetcher', () => { + const downloadMock = vi.mocked(downloadTemplate) + let savedToken: string | undefined + + beforeEach(() => { + downloadMock.mockClear() + savedToken = process.env.GITHUB_TOKEN + delete process.env.GITHUB_TOKEN + }) + + afterEach(() => { + vi.unstubAllGlobals() + if (savedToken === undefined) delete process.env.GITHUB_TOKEN + else process.env.GITHUB_TOKEN = savedToken + }) + + it('wires the real download without provider auth when no token is present', async () => { + vi.stubGlobal('fetch', fetchReturning(new Response(SHA, { status: 200 }))) + await createGigetFetcher().fetch({ kind: 'github', owner: 'o', repo: 'r', ref: 'main' }) + const [input, options] = downloadMock.mock.calls[0]! + expect(input).toBe(`o/r#${SHA}`) + expect(options?.dir).toContain(GITHUB_TEMP_PREFIX) + expect(options?.force).toBe('clean') + expect(options?.providerOptions).toBeUndefined() + }) + + it('passes an explicit token to both ref resolution and provider auth', async () => { + const fetchMock = fetchReturning(new Response(SHA, { status: 200 })) + vi.stubGlobal('fetch', fetchMock) + await createGigetFetcher({ token: 'tok' }).fetch({ kind: 'github', owner: 'o', repo: 'r' }) + expect((fetchMock.mock.calls[0]![1] as RequestInit).headers).toMatchObject({ Authorization: 'Bearer tok' }) + const [, options] = downloadMock.mock.calls[0]! + expect(options).toMatchObject({ providerOptions: { auth: 'tok' } }) + }) + + it('reads GITHUB_TOKEN from the environment', async () => { + process.env.GITHUB_TOKEN = 'from-env' + vi.stubGlobal('fetch', fetchReturning(new Response(SHA, { status: 200 }))) + await createGigetFetcher().fetch({ kind: 'github', owner: 'o', repo: 'r' }) + const [, options] = downloadMock.mock.calls[0]! + expect(options).toMatchObject({ providerOptions: { auth: 'from-env' } }) + }) +}) diff --git a/packages/sdk/plugin-fetch/tests/ids.spec.ts b/packages/sdk/plugin-fetch/tests/ids.spec.ts new file mode 100644 index 0000000000..2f1ba6f518 --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/ids.spec.ts @@ -0,0 +1,39 @@ +import { describe, expect, it } from 'vitest' +import { commitSha, integrity } from '../src/ids.ts' + +describe('commitSha', () => { + it('accepts abbreviated and full lowercase hex object ids', () => { + expect(commitSha('abc1234')).toBe('abc1234') + expect(commitSha('a'.repeat(40))).toBe('a'.repeat(40)) + expect(commitSha('0'.repeat(64))).toBe('0'.repeat(64)) + }) + + it.each([ + ['too short', 'abc123'], + ['uppercase', 'ABCDEF1'], + ['non-hex', 'ghijklm'], + ['too long', 'a'.repeat(65)], + ['empty', ''], + ])('rejects an invalid sha (%s)', (_label, value) => { + expect(() => commitSha(value)).toThrow(/invalid commit sha/) + }) +}) + +describe('integrity', () => { + it.each([ + 'sha512-abcABC123+/==', + 'sha384-abcABC123+/', + 'sha256-Zm9vYmFy', + ])('accepts a valid SRI entry (%s)', (value) => { + expect(integrity(value)).toBe(value) + }) + + it.each([ + ['missing algorithm', 'abcABC123'], + ['unsupported algorithm', 'sha1-abcABC123'], + ['illegal base64 char', 'sha512-abc*def'], + ['empty', ''], + ])('rejects an invalid integrity (%s)', (_label, value) => { + expect(() => integrity(value)).toThrow(/invalid subresource integrity/) + }) +}) diff --git a/packages/sdk/plugin-fetch/tests/never.spec.ts b/packages/sdk/plugin-fetch/tests/never.spec.ts new file mode 100644 index 0000000000..224ab66e29 --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/never.spec.ts @@ -0,0 +1,19 @@ +import { describe, expect, it } from 'vitest' +import { assertNever } from '../src/never.ts' + +describe('assertNever', () => { + it('throws with the rendered value and a context label', () => { + expect(() => assertNever('surprise' as never, 'demo')).toThrow( + /unreachable variant in demo: "surprise"/, + ) + }) + + it('omits the context clause when none is given', () => { + expect(() => assertNever(7 as never)).toThrow(/unreachable variant: 7$/) + }) + + it('falls back to String() when the value is not JSON-serializable', () => { + // JSON.stringify(undefined) is undefined, exercising the String() fallback. + expect(() => assertNever(undefined as never)).toThrow(/unreachable variant: undefined$/) + }) +}) diff --git a/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts new file mode 100644 index 0000000000..958e56674e --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts @@ -0,0 +1,108 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { extract as pacoteExtract, manifest as pacoteManifest } from 'pacote' +import { + createPacoteFetcher, + NPM_TEMP_PREFIX, + PacoteFetcher, + type PacoteApi, + type PacoteFetcherDeps, +} from '../src/pacote-fetcher.ts' +import type { NpmSource } from '../src/source.ts' + +vi.mock('pacote', () => ({ manifest: vi.fn(), extract: vi.fn() })) + +const INTEGRITY = 'sha512-abcABC123+/==' +const RESOLVED = 'https://registry.npmjs.org/plugin/-/plugin-1.2.3.tgz' + +function fakePacote(): PacoteApi { + return { + manifest: vi.fn(async () => ({ _resolved: RESOLVED, _integrity: INTEGRITY, version: '1.2.3' })), + extract: vi.fn(async () => ({ resolved: RESOLVED, integrity: INTEGRITY })), + } +} + +function deps(overrides: Partial = {}): PacoteFetcherDeps { + return { + pacote: fakePacote(), + createTempDir: vi.fn(async () => '/tmp/dsh-plugin-npm-abc'), + ...overrides, + } +} + +const SOURCE: NpmSource = { kind: 'npm', name: 'plugin', version: '^1.0.0' } + +describe('PacoteFetcher.fetch', () => { + it('resolves the manifest, extracts with integrity, and reports provenance', async () => { + const d = deps() + const result = await new PacoteFetcher(d).fetch(SOURCE) + + expect(d.pacote.manifest).toHaveBeenCalledWith('plugin@^1.0.0', {}) + expect(d.createTempDir).toHaveBeenCalledWith(NPM_TEMP_PREFIX) + expect(d.pacote.extract).toHaveBeenCalledWith('plugin@^1.0.0', '/tmp/dsh-plugin-npm-abc', { + resolved: RESOLVED, + integrity: INTEGRITY, + }) + expect(result).toEqual({ + dir: '/tmp/dsh-plugin-npm-abc', + source: SOURCE, + provenance: { kind: 'npm', version: '1.2.3', resolved: RESOLVED, integrity: INTEGRITY }, + }) + }) + + it('forwards a configured registry to both manifest and extract', async () => { + const d = deps({ registry: 'https://npm.internal/' }) + await new PacoteFetcher(d).fetch(SOURCE) + expect(d.pacote.manifest).toHaveBeenCalledWith('plugin@^1.0.0', { registry: 'https://npm.internal/' }) + expect(d.pacote.extract).toHaveBeenCalledWith('plugin@^1.0.0', '/tmp/dsh-plugin-npm-abc', { + registry: 'https://npm.internal/', + resolved: RESOLVED, + integrity: INTEGRITY, + }) + }) + + it('rejects a registry integrity that is not a valid SRI', async () => { + const pacote = fakePacote() + pacote.extract = vi.fn(async () => ({ resolved: RESOLVED, integrity: 'not-sri' })) + await expect(new PacoteFetcher(deps({ pacote })).fetch(SOURCE)).rejects.toThrow( + /invalid subresource integrity/, + ) + }) + + it('exposes its source kind', () => { + expect(new PacoteFetcher(deps()).kind).toBe('npm') + }) +}) + +describe('createPacoteFetcher', () => { + const manifestMock = vi.mocked(pacoteManifest) + const extractMock = vi.mocked(pacoteExtract) + + beforeEach(() => { + manifestMock.mockReset() + extractMock.mockReset() + // The real overloaded pacote manifest returns a much wider shape; the fetcher reads only these fields. + manifestMock.mockResolvedValue( + { _resolved: RESOLVED, _integrity: INTEGRITY, version: '1.2.3' } as unknown as Awaited< + ReturnType + >, + ) + extractMock.mockResolvedValue({ from: 'plugin@1.2.3', resolved: RESOLVED, integrity: INTEGRITY }) + }) + + afterEach(() => vi.clearAllMocks()) + + it('wires the real pacote resolve/extract surface', async () => { + const result = await createPacoteFetcher().fetch(SOURCE) + expect(manifestMock).toHaveBeenCalledWith('plugin@^1.0.0', {}) + expect(extractMock).toHaveBeenCalledWith('plugin@^1.0.0', expect.stringContaining(NPM_TEMP_PREFIX), { + resolved: RESOLVED, + integrity: INTEGRITY, + }) + expect(result.provenance).toEqual({ kind: 'npm', version: '1.2.3', resolved: RESOLVED, integrity: INTEGRITY }) + }) + + it('forwards a configured registry through the real surface', async () => { + await createPacoteFetcher({ registry: 'https://npm.internal/' }).fetch(SOURCE) + expect(manifestMock).toHaveBeenCalledWith('plugin@^1.0.0', { registry: 'https://npm.internal/' }) + }) +}) diff --git a/packages/sdk/plugin-fetch/tests/source.spec.ts b/packages/sdk/plugin-fetch/tests/source.spec.ts new file mode 100644 index 0000000000..e585c5dfcf --- /dev/null +++ b/packages/sdk/plugin-fetch/tests/source.spec.ts @@ -0,0 +1,107 @@ +import { describe, expect, it } from 'vitest' +import { resolvePluginSource, type GithubSource, type NpmSource } from '../src/source.ts' + +describe('resolvePluginSource — github', () => { + it('parses owner/repo with a ref', () => { + expect(resolvePluginSource('unjs/template#main')).toEqual({ + kind: 'github', owner: 'unjs', repo: 'template', ref: 'main', + }) + }) + + it('parses a bare owner/repo without a ref', () => { + expect(resolvePluginSource('deepseek-ai/plugin')).toEqual({ + kind: 'github', owner: 'deepseek-ai', repo: 'plugin', + }) + }) + + it('parses a nested subdir with a ref', () => { + expect(resolvePluginSource('owner/repo/packages/plugin#v1.2.3')).toEqual({ + kind: 'github', owner: 'owner', repo: 'repo', subdir: 'packages/plugin', ref: 'v1.2.3', + }) + }) + + it('parses a subdir without a ref', () => { + expect(resolvePluginSource('owner/repo/sub')).toEqual({ + kind: 'github', owner: 'owner', repo: 'repo', subdir: 'sub', + }) + }) + + it('accepts a slash-nested ref', () => { + expect(resolvePluginSource('owner/repo#feature/x')).toEqual({ + kind: 'github', owner: 'owner', repo: 'repo', ref: 'feature/x', + }) + }) + + it('trims surrounding whitespace before parsing', () => { + expect(resolvePluginSource(' owner/repo#main ')).toEqual({ + kind: 'github', owner: 'owner', repo: 'repo', ref: 'main', + }) + }) + + it.each([ + ['empty ref after hash', 'owner/repo#'], + ['single locator segment with hash', 'owner#main'], + ['owner with @ and a hash', 'own@er/repo#main'], + ['ref with whitespace', 'owner/repo#bad ref'], + ['ref with traversal', 'owner/repo#a..b'], + ['ref with a leading slash', 'owner/repo#/main'], + ['ref with a trailing slash', 'owner/repo#main/'], + ['ref with an illegal char', 'owner/repo#ma:in'], + ])('rejects a malformed github spec (%s)', (_label, spec) => { + expect(() => resolvePluginSource(spec)).toThrow(/github plugin source|missing a ref/) + }) +}) + +describe('resolvePluginSource — npm', () => { + it('parses an unscoped name@version', () => { + expect(resolvePluginSource('react@18.2.0')).toEqual({ + kind: 'npm', name: 'react', version: '18.2.0', + }) + }) + + it('parses a scoped name@version', () => { + expect(resolvePluginSource('@deepseek-ai/dsh-tool-foo@0.0.1')).toEqual({ + kind: 'npm', name: '@deepseek-ai/dsh-tool-foo', version: '0.0.1', + }) + }) + + it('accepts a dist-tag as the version', () => { + expect(resolvePluginSource('some-plugin@latest')).toEqual({ + kind: 'npm', name: 'some-plugin', version: 'latest', + }) + }) + + it('accepts a range as the version', () => { + expect(resolvePluginSource('some-plugin@^1.0.0')).toEqual({ + kind: 'npm', name: 'some-plugin', version: '^1.0.0', + }) + }) +}) + +describe('resolvePluginSource — failures', () => { + it.each([ + ['empty', ''], + ['whitespace only', ' '], + ])('rejects a blank spec (%s)', (_label, spec) => { + expect(() => resolvePluginSource(spec)).toThrow(/must not be empty/) + }) + + it.each([ + ['bare word', 'plugin'], + ['internal whitespace', 'owner repo'], + ['empty npm version', 'pkg@'], + ['scoped without version', '@scope/pkg'], + ['scoped with empty scope', '@/pkg@1'], + ['unscoped name with slash and version', 'foo/bar@1'], + ['version containing a slash', 'foo@1/2'], + ['uppercase unscoped name', 'FOO@1.0.0'], + ['uppercase scope segment', '@Scope/pkg@1'], + ['uppercase scoped name segment', '@scope/PKG@1'], + ['empty scoped name segment', '@scope/@1'], + ['dot-only owner', './repo'], + ['traversal subdir segment', 'owner/repo/../x'], + ['double slash subdir', 'owner/repo//sub'], + ])('rejects an unrecognized/ambiguous spec (%s)', (_label, spec) => { + expect(() => resolvePluginSource(spec)).toThrow(/unrecognized plugin source|github plugin source/) + }) +}) diff --git a/packages/sdk/plugin-fetch/tsconfig.json b/packages/sdk/plugin-fetch/tsconfig.json new file mode 100644 index 0000000000..07c2567ff8 --- /dev/null +++ b/packages/sdk/plugin-fetch/tsconfig.json @@ -0,0 +1,15 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../util/brand" + } + ] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9810a9a61e..25a8fadc47 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1223,6 +1223,25 @@ importers: specifier: ^4.0.0-rc.7 version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) + packages/sdk/plugin-fetch: + dependencies: + '@bluwy/giget-core': + specifier: ^0.1.7 + version: 0.1.7 + pacote: + specifier: ^22.0.0 + version: 22.0.0 + devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + '@types/pacote': + specifier: ^11.1.8 + version: 11.1.8 + cordis: + specifier: ^4.0.0-rc.7 + version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) + packages/sdk/scripts: dependencies: '@deepseek-ai/dsh-helper': @@ -2939,6 +2958,10 @@ packages: resolution: {integrity: sha512-6zABk/ECA/QYSCQ1NGiVwwbQerUCZ+TQbp64Q3AgmfNvurHH0j8TtXa1qbShXA6qqkpAj4V5W8pP6mLe1mcMqA==} engines: {node: '>=18'} + '@bluwy/giget-core@0.1.7': + resolution: {integrity: sha512-6XG8TZt8DVYLuGDVSpFJaSMlNowOg5RGecvWbKvlgMoqVbztUAQ3AcWq6oZ5DoCnTzNAPcO7rhkwt8ZVgrS7CQ==} + engines: {node: '>=18'} + '@braintree/sanitize-url@7.1.2': resolution: {integrity: sha512-jigsZK+sMF/cuiB7sERuo9V7N9jx+dhmHHnQyDSVdpZwVutaBu7WvNYqMDLSgFgfB30n452TP3vjDAvFC973mA==} @@ -3392,6 +3415,10 @@ packages: '@noble/hashes': optional: true + '@gar/promise-retry@1.0.3': + resolution: {integrity: sha512-GmzA9ckNokPypTg10pgpeHNQe7ph+iIKKmhKu3Ob9ANkswreCx7R3cKmY781K8QK3AqVL3xVh9A42JvIAbkkSA==} + engines: {node: ^20.17.0 || >=22.9.0} + '@google/genai@1.52.0': resolution: {integrity: sha512-gwSvbpiN/17O9TbsqSsE/OzZcpv5Fo4RQjdngGgogtuB9RsyJ8ZHhX5KjHj1bp5N9snN2eK8LDGXSaWW2hof8Q==} engines: {node: '>=20.0.0'} @@ -3440,6 +3467,10 @@ packages: resolution: {integrity: sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==} engines: {node: '>=12'} + '@isaacs/fs-minipass@4.0.1': + resolution: {integrity: sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==} + engines: {node: '>=18.0.0'} + '@jridgewell/gen-mapping@0.3.13': resolution: {integrity: sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==} @@ -3486,6 +3517,43 @@ packages: '@nodable/entities@2.2.0': resolution: {integrity: sha512-9uGyhaQavEUMC8AIddIjau4NsnsXhou+j5sBAGojCM1oxmQpVKTWR/9JxABD6UAv12vpIms55fPZKFQEhG6uBg==} + '@npmcli/agent@5.0.2': + resolution: {integrity: sha512-EkzGmEsgbQ1rqWkRJe2P0oQHx/ylZozDUNPMXCklLuSFL3GY+QyEfBUjhjCsgGXzh4OGpnHvkboSQgczjP/jJg==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/fs@6.0.0': + resolution: {integrity: sha512-AheOs4swKka/XLtht6xxJDPezlQ7K2IYQ9Y8lST4JLDjnralnWuMM9AE2CdVcgQJ5omrXhsRzM7F7aYmeZBvKQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/git@8.0.0': + resolution: {integrity: sha512-5P1oo+TbxZNAiiMBtpzHA8QyEGh5D69LYLexNWJEDXLdxnAZvT/SLitGJBXxjtCE4ftAcFOS/Tu2185MeIjooQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/installed-package-contents@5.0.0': + resolution: {integrity: sha512-6Ay12sf2Lh7U1ifvnS1mq7TZFeh/rXHMXye+kV7jQrANIubaoVcleeh4HdFumxhsRYwm9OaHycB5lYmSwGrcIQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + hasBin: true + + '@npmcli/node-gyp@6.0.0': + resolution: {integrity: sha512-MFakpea4pcZNlHSTbMi15HK8RY8zl2UpgDtxhZCWOer+KRN3x7HFIMk/fKpOMgR55L4LIcA2qn8IHeyABhIFtw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/package-json@8.0.0': + resolution: {integrity: sha512-agNZzYQ18MR0wKp3Emg1q5QbcC8CXigYp3Z3CvB0Sax9Ge9aF4cVyyuSG+5SbACSrZUKTvMjVULWiE1RJA38wg==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/promise-spawn@10.0.0': + resolution: {integrity: sha512-llZkSzeTsimFx64U+ThT2xQM2uEce8GIQUYvxgbB6ZFvBhV2LP9LeJJb3HT+syG0uCFLsTCHjV9SfC0WNU1vtA==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/redact@5.0.0': + resolution: {integrity: sha512-3zcN5Q3yEmeyxXBzqB6fXPQFzYa2ROsGFSr69W0ArXIAGJqxl/aFECOVPD2kbkYPm0U/EHxFKgclK3UA9WQg5A==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@npmcli/run-script@11.0.0': + resolution: {integrity: sha512-leBRl6F5F0TvWut8m1/aZcMTUHi2vXjKeMJ/Ik1lW7Q7Yy16Dhtkklu+cEqQww1p1NeLnUNzV3+uwpzqRcy9vw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + '@oxc-parser/binding-android-arm-eabi@0.133.0': resolution: {integrity: sha512-l/44caGse+VpnY9gx0yvvc5QnnG3yG1FO3KZgYvNL1GZrfK86zIwAOgGEVlxDyRymzrU/KHiblPFpevKOmJmUA==} engines: {node: ^20.19.0 || >=22.12.0} @@ -4115,6 +4183,30 @@ packages: '@shikijs/vscode-textmate@10.0.2': resolution: {integrity: sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg==} + '@sigstore/bundle@5.0.0': + resolution: {integrity: sha512-wefjygudENbzbQMks1t5u34EP0fFoD0XvaEP7DOUP/sXKvogzEJYFw5E6pegGyp3onGWzVEYKVa3bNZWyTYX+A==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@sigstore/core@4.0.1': + resolution: {integrity: sha512-9v5hRjujn5NXq8o7XFEUgLyAtdr5Iisb4pzM05u3K61IS5q3hP3luWAndk0RkPPLTUFoTbg7Vb84UQ1ZQeajWQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@sigstore/protobuf-specs@0.5.1': + resolution: {integrity: sha512-/ScWUhhoFasJsSRGTVBwId1loQjjnjAfE4djL6ZhrXRpNCmPTnUKF5Jokd58ILseOMjzET3UrMOtJPS9sYeI0g==} + engines: {node: ^18.17.0 || >=20.5.0} + + '@sigstore/sign@5.0.0': + resolution: {integrity: sha512-DSFivqz9/i5AkwZ5fq0YdjaJlc4o1WeS2Zffon0kqtChx0vy4W9NOjkEet9bF2vkzOufX72eVH8kZBIGtcBp1w==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@sigstore/tuf@5.0.0': + resolution: {integrity: sha512-Zyqg9tcHps3uRAlKHLNmsW4ohsUZAjb9G+31r7lg0ICh/JOcadzmJsIRdjKljlRHpaR0K4aJ2kXXIdywdcdMlA==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@sigstore/verify@4.1.0': + resolution: {integrity: sha512-p/s720RiWxLG8XtmfdPfEJOlATA6H/2knFqmtQbFkHKN3IrhWGUwPfpQAf1UnQIEES9IaH6zzhfjkrhTfeSdZw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + '@smithy/core@3.24.7': resolution: {integrity: sha512-KoUi4M1f3BG6kzN1FnCwL7oyFptTbyBJKjR6yhSib+JHRdUmM1o+VwsFtJ66NZCkCzVfJMWRHJNo0R0jznp0Pg==} engines: {node: '>=18.0.0'} @@ -4164,6 +4256,14 @@ packages: peerDependencies: eslint: ^9.0.0 || ^10.0.0 + '@tufjs/canonical-json@2.0.0': + resolution: {integrity: sha512-yVtV8zsdo8qFHe+/3kw81dSLyF7D576A5cCFCi4X7B39tWT7SekaEFUnvnWJHz+9qO7qJTah1JbrDjWKqFtdWA==} + engines: {node: ^16.14.0 || >=18.0.0} + + '@tufjs/models@5.0.0': + resolution: {integrity: sha512-U4mVcdFGOi6pt8n38LdWZp67Svn7ppnU1Pj8SGOVaBi1X4gm+G4ztQlLfkoJbKSHfjA6WeaiJp2A4V83AJF6nQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + '@tybys/wasm-util@0.10.2': resolution: {integrity: sha512-RoBvJ2X0wuKlWFIjrwffGw1IqZHKQqzIchKaadZZfnNpsAYp2mM0h36JtPCjNDAHGgYez/15uMBpfGwchhiMgg==} @@ -4311,18 +4411,36 @@ packages: '@types/ms@2.1.0': resolution: {integrity: sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA==} + '@types/node-fetch@2.6.13': + resolution: {integrity: sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==} + '@types/node@22.20.0': resolution: {integrity: sha512-QWlFW2wf3nTjC13/DqRnBpR4ZO36VJH/JVBkA/vcnmbTBNQIlnObqyqZE1tUR7+Ni23Lda8R1BxMfbXRpCUx5g==} '@types/node@25.9.3': resolution: {integrity: sha512-603BddQMv3pUcr4U2dhujk83N2tTDVr/34wII2B6bJy6g+8WD6yUb11jszNs0gdi4PesVWl7ABt8nYMVpnLUcg==} + '@types/npm-package-arg@6.1.4': + resolution: {integrity: sha512-vDgdbMy2QXHnAruzlv68pUtXCjmqUk3WrBAsRboRovsOmxbfn/WiYCjmecyKjGztnMps5dWp4Uq2prp+Ilo17Q==} + + '@types/npm-registry-fetch@8.0.9': + resolution: {integrity: sha512-7NxvodR5Yrop3pb6+n8jhJNyzwOX0+6F+iagNEoi9u1CGxruYAwZD8pvGc9prIkL0+FdX5Xp0p80J9QPrGUp/g==} + + '@types/npmlog@7.0.0': + resolution: {integrity: sha512-hJWbrKFvxKyWwSUXjZMYTINsSOY6IclhvGOZ97M8ac2tmR9hMwmTnYaMdpGhvju9ctWLTPhCS+eLfQNluiEjQQ==} + + '@types/pacote@11.1.8': + resolution: {integrity: sha512-/XLR0VoTh2JEO0jJg1q/e6Rh9bxjBq9vorJuQmtT7rRrXSiWz7e7NsvXVYJQ0i8JxMlBMPPYDTnrRe7MZRFA8Q==} + '@types/picomatch@3.0.2': resolution: {integrity: sha512-n0i8TD3UDB7paoMMxA3Y65vUncFJXjcUf7lQY7YyKGl6031FNjfsLs6pdLFCy2GNFxItPJG8GvvpbZc2skH7WA==} '@types/retry@0.12.0': resolution: {integrity: sha512-wWKOClTTiizcZhXnPY4wikVAwmdYHp8q6DmC+EJUzAMsycb7HB32Kh9RN4+0gExjmPmZSAQjgURXIGATPegAvA==} + '@types/ssri@7.1.5': + resolution: {integrity: sha512-odD/56S3B51liILSk5aXJlnYt99S6Rt9EFDDqGtJM26rKHApHcwyU/UoYHrzKkdkHMAIquGWCuHtQTbes+FRQw==} + '@types/tough-cookie@4.0.5': resolution: {integrity: sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==} @@ -4537,6 +4655,10 @@ packages: resolution: {integrity: sha512-A9gOqLdi6cV4ibazAjcQufGj0B1y/vDqYrcuP6d/6x8P27gRS8643Dj9o1dEKtB6O7fwxb2FgBmJS2mX7gpvdw==} engines: {node: '>=14.6'} + abbrev@5.0.0: + resolution: {integrity: sha512-/XrFJgzQQQHpti1raDJC6m4ws6aNktmjBlhk8Fdlk7LwCEuDoieEJJY9OFHjfiFJFFRM2tK+Ky/IsfbbmlMu1w==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + accepts@2.0.0: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} @@ -4555,6 +4677,10 @@ packages: resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==} engines: {node: '>= 14'} + agent-base@9.0.0: + resolution: {integrity: sha512-TQf59BsZnytt8GdJKLPfUZ54g/iaUL2OWDSFCCvMOhsHduDQxO8xC4PNeyIkVcA5KwL2phPSv0douC0fgWzmnA==} + engines: {node: '>= 20'} + ajv-formats@3.0.1: resolution: {integrity: sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==} peerDependencies: @@ -4614,6 +4740,9 @@ packages: ast-v8-to-istanbul@1.0.4: resolution: {integrity: sha512-0bC0/4bTSrnwdhU3IsZDwEdojvuPrSg59OYZfKsLRtJZ0u8VBx9DebfqqG8bRdCC0I7vjgxmPi41P0lpkhJHtA==} + asynckit@0.4.0: + resolution: {integrity: sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==} + balanced-match@1.0.2: resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==} @@ -4668,6 +4797,10 @@ packages: resolution: {integrity: sha512-tixWYgm5ZoOD+3g6UTea91eow5z6AAHaho3g0V9CNSNb45gM8SmflpAc+GRd1InC4AqN/07Unrgp56Y94N9hJQ==} engines: {node: '>=20.19.0'} + cacache@21.0.1: + resolution: {integrity: sha512-pTwz/uj3Jyp6WXdJ6fWhR+7LVxVs6RyroQSn7KJwHsSxXuyGSp0pcMVcwSwTpCFq1X2YG8QBe0W+vN+cr0SwzA==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + call-bind-apply-helpers@1.0.2: resolution: {integrity: sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==} engines: {node: '>= 0.4'} @@ -4703,6 +4836,10 @@ packages: resolution: {integrity: sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==} engines: {node: '>= 14.16.0'} + chownr@3.0.0: + resolution: {integrity: sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==} + engines: {node: '>=18'} + color-convert@2.0.1: resolution: {integrity: sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==} engines: {node: '>=7.0.0'} @@ -4710,6 +4847,10 @@ packages: color-name@1.1.4: resolution: {integrity: sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==} + combined-stream@1.0.8: + resolution: {integrity: sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==} + engines: {node: '>= 0.8'} + comma-separated-tokens@2.0.3: resolution: {integrity: sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg==} @@ -4997,6 +5138,10 @@ packages: delaunator@5.1.0: resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==} + delayed-stream@1.0.0: + resolution: {integrity: sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==} + engines: {node: '>=0.4.0'} + depd@2.0.0: resolution: {integrity: sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==} engines: {node: '>= 0.8'} @@ -5090,6 +5235,10 @@ packages: resolution: {integrity: sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA==} engines: {node: '>=20.19.0'} + env-paths@2.2.1: + resolution: {integrity: sha512-+h1lkLKhZMTYjog1VEpJNG7NZJWcuc2DDk/qsqSTRRCOXiLjeQ1d1/udrUGhqMxUgAlwKNZ0cf2uqan5GLuS2A==} + engines: {node: '>=6'} + es-define-property@1.0.1: resolution: {integrity: sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==} engines: {node: '>= 0.4'} @@ -5105,6 +5254,10 @@ packages: resolution: {integrity: sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==} engines: {node: '>= 0.4'} + es-set-tostringtag@2.1.0: + resolution: {integrity: sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==} + engines: {node: '>= 0.4'} + es-toolkit@1.49.0: resolution: {integrity: sha512-G5iZ6Pc/FNRY/soKZHC+TxGDD83rHUDXxzaWhGCX44vAv/tMs56WMusnm/KMNK+luUPsgA9U28cGr4RDlSzL2g==} @@ -5214,6 +5367,9 @@ packages: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} + exponential-backoff@3.1.3: + resolution: {integrity: sha512-ZgEeZXj30q+I0EN+CbSSpIyPaJ5HVQD18Z1m+u1FXbAeT94mr1zw50q4q6jiiC447Nl/YTcIYSAftiGqetwXCA==} + express-rate-limit@8.5.2: resolution: {integrity: sha512-5Kb34ipNX694DH48vN9irak1Qx30nb0PLYHXfJgw4YEjiC3ZEmZJhwOp+VfiCYwFzvFTdB9QkArYS5kXa2cx2A==} engines: {node: '>= 16'} @@ -5301,6 +5457,10 @@ packages: resolution: {integrity: sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==} engines: {node: '>=14'} + form-data@4.0.6: + resolution: {integrity: sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==} + engines: {node: '>= 6'} + formatly@0.3.0: resolution: {integrity: sha512-9XNj/o4wrRFyhSMJOvsuyMwy8aUfBaZ1VrqHVfohyXf0Sw0e+yfKG+xZaY3arGCOMdwFsqObtzVOc1gU9KiT9w==} engines: {node: '>=18.3.0'} @@ -5318,6 +5478,10 @@ packages: resolution: {integrity: sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A==} engines: {node: '>= 0.8'} + fs-minipass@3.0.3: + resolution: {integrity: sha512-XUBA9XClHbnJWSfBzjkm6RvPsyg3sryZt06BEQoXcF7EK/xpGaQYJgQKDJSUH5SGZ76Y7pFx1QBnXz09rU5Fbw==} + engines: {node: ^14.17.0 || ^16.13.0 || >=18.0.0} + fsevents@2.3.3: resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==} engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} @@ -5361,6 +5525,10 @@ packages: deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me hasBin: true + glob@13.0.6: + resolution: {integrity: sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==} + engines: {node: 18 || 20 || >=22} + globals@17.7.0: resolution: {integrity: sha512-Czmyns5dUsq4seFBR/Kdydhmo8y9kC79hiSkPn0YcGtNnYWnrgt0vjrSjx9tspoDGWm2CMarffRuLjM4xUz8xg==} engines: {node: '>=18'} @@ -5380,6 +5548,9 @@ packages: resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==} engines: {node: '>= 0.4'} + graceful-fs@4.2.11: + resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} + hachure-fill@0.5.2: resolution: {integrity: sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==} @@ -5396,6 +5567,10 @@ packages: resolution: {integrity: sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==} engines: {node: '>= 0.4'} + has-tostringtag@1.0.2: + resolution: {integrity: sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==} + engines: {node: '>= 0.4'} + hasown@2.0.4: resolution: {integrity: sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==} engines: {node: '>= 0.4'} @@ -5416,6 +5591,10 @@ packages: hookable@6.1.1: resolution: {integrity: sha512-U9LYDy1CwhMCnprUfeAZWZGByVbhd54hwepegYTK7Pi5NvqEj63ifz5z+xukznehT7i6NIZRu89Ay1AZmRsLEQ==} + hosted-git-info@10.1.1: + resolution: {integrity: sha512-DeOnSPAvOndYKfw075gt8yZzQ7S2hNztw34zBTfhIzLhmBTswIBg5/y+pqu/VD5cYWm5goAFTusDmUEmKZ0PEQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + html-encoding-sniffer@6.0.0: resolution: {integrity: sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg==} engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} @@ -5432,6 +5611,9 @@ packages: htmlparser2@6.1.0: resolution: {integrity: sha512-gyyPk6rgonLFEDGoeRgQNaEUvdJ4ktTmmUh/h2t7s+M8oPpIPxgNACWa+6ESR57kXstwqPiCut0V8NRpcwgU7A==} + http-cache-semantics@4.2.0: + resolution: {integrity: sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ==} + http-errors@2.0.1: resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==} engines: {node: '>= 0.8'} @@ -5440,10 +5622,18 @@ packages: resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} engines: {node: '>= 14'} + http-proxy-agent@9.1.0: + resolution: {integrity: sha512-2NxoveTT58mjYT4n3RPTEfCZGLMbidoO8XEieXfpSYxu+PQJ1qpx4ypwH6N+uF9twBPIvRRgvkvW5HUTYWENig==} + engines: {node: '>= 20'} + https-proxy-agent@7.0.6: resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==} engines: {node: '>= 14'} + https-proxy-agent@9.1.0: + resolution: {integrity: sha512-ag87y7cJJ9/3+GxFr8Oy4O5faDsGRGnBGsJj/YjOSsSx/5eadKLYTMPlzuR6obgoCDDm0abAAZitXXQkMOPSpA==} + engines: {node: '>= 20'} + iconv-lite@0.6.3: resolution: {integrity: sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==} engines: {node: '>=0.10.0'} @@ -5452,6 +5642,10 @@ packages: resolution: {integrity: sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==} engines: {node: '>=0.10.0'} + ignore-walk@9.0.0: + resolution: {integrity: sha512-tCBEZV2z2FNpIDl2vrhiWzIHzs4qOAuIDEO85eS02vZ3L1U3P56qpPL8GuGGAijDktAEaq2swMkO/Fmbo7YmfQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + ignore@5.3.2: resolution: {integrity: sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g==} engines: {node: '>= 4'} @@ -5477,6 +5671,10 @@ packages: inherits@2.0.4: resolution: {integrity: sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==} + ini@7.0.0: + resolution: {integrity: sha512-ifK0CgjALofS5bkrcTy4RaQ9Vx2Knf/eLeIO+NaswQEpH1UblrtTSCIvN71qQDMq0PeQ/SSPojvEJp9vvvfr+w==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + internmap@1.0.1: resolution: {integrity: sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw==} @@ -5520,6 +5718,10 @@ packages: isexe@2.0.0: resolution: {integrity: sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==} + isexe@4.0.0: + resolution: {integrity: sha512-FFUtZMpoZ8RqHS3XeXEmHWLA4thH+ZxCv2lOiPIn1Xc7CxrqhWzNSDzD+/chS/zbYezmiwWLdQC09JdQKmthOw==} + engines: {node: '>=20'} + istanbul-lib-coverage@3.2.2: resolution: {integrity: sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==} engines: {node: '>=8'} @@ -5610,6 +5812,10 @@ packages: json-buffer@3.0.1: resolution: {integrity: sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==} + json-parse-even-better-errors@6.0.0: + resolution: {integrity: sha512-2/8adwnK1/+Fdjyts4r6wSpfANWw8zdNhU9U/Llk59c6O+DjSisPWPykwoL8gZmocP9Dy64S7oie2g+Mia123A==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + json-schema-to-ts@3.1.1: resolution: {integrity: sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==} engines: {node: '>=16'} @@ -5629,6 +5835,10 @@ packages: jsonc-parser@3.3.1: resolution: {integrity: sha512-HUgH65KyejrUFPvHFPbqOY0rsFip3Bo5wb4ngvdi1EpCYWUQDC5V+Y7mZws+DLkr4M//zQJoanu1SP+87Dv1oQ==} + jsonparse@1.3.1: + resolution: {integrity: sha512-POQXvpdL69+CluYsillJ7SUhKvytYjW9vG/GKpnf+xP8UWgYEM/RaMzHHofbALDiKbbP1W8UEYmgGl39WkPZsg==} + engines: {'0': node >= 0.2.0} + jsx-ast-utils-x@0.1.0: resolution: {integrity: sha512-eQQBjBnsVtGacsG9uJNB8qOr3yA8rga4wAaGG1qRcBzSIvfhERLrWxMAM1hp5fcS6Abo8M4+bUBTekYR0qTPQw==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} @@ -5836,6 +6046,10 @@ packages: resolution: {integrity: sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==} engines: {node: '>=10'} + make-fetch-happen@16.0.1: + resolution: {integrity: sha512-uUv1yxHzaKVVEPfcFeGSNov/Cehjv08ovlY8ImTljgL7Q+SiA0dAYLQ6SYVa2kkKqNj4Y3aZEI7xv2teadie0A==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + mark.js@8.11.1: resolution: {integrity: sha512-1I+1qpDt4idfgLQG+BNWmrqku+7/2bi5nLf4YwF8y8zXvmfiTBY3PV3ZibfrjBueCByROpuBjLLFCajqkgYoLQ==} @@ -5998,10 +6212,18 @@ packages: micromark@4.0.2: resolution: {integrity: sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA==} + mime-db@1.52.0: + resolution: {integrity: sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==} + engines: {node: '>= 0.6'} + mime-db@1.54.0: resolution: {integrity: sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==} engines: {node: '>= 0.6'} + mime-types@2.1.35: + resolution: {integrity: sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==} + engines: {node: '>= 0.6'} + mime-types@3.0.2: resolution: {integrity: sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==} engines: {node: '>=18'} @@ -6022,6 +6244,30 @@ packages: minimist@1.2.8: resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==} + minipass-collect@2.0.1: + resolution: {integrity: sha512-D7V8PO9oaz7PWGLbCACuI1qEOsq7UKfLotx/C0Aet43fCUB/wfQ7DYeq2oR/svFJGYDHPr38SHATeaj/ZoKHKw==} + engines: {node: '>=16 || 14 >=14.17'} + + minipass-fetch@6.0.0: + resolution: {integrity: sha512-AWI8bKapGmgx/J0E6IGYSKj8TiHebZkmKWSs8raPSw8KXwgEAJ+Bw3+LSdXHR6T/RHKAWCOYk2MiLrYluaUU6w==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + minipass-flush@1.0.7: + resolution: {integrity: sha512-TbqTz9cUwWyHS2Dy89P3ocAGUGxKjjLuR9z8w4WUTGAVgEj17/4nhgo2Du56i0Fm3Pm30g4iA8Lcqctc76jCzA==} + engines: {node: '>= 8'} + + minipass-pipeline@1.2.4: + resolution: {integrity: sha512-xuIq7cIOt09RPRJ19gdi4b+RiNvDFYe5JH+ggNvBqGqpQXcru3PcRmOZuHBKWK1Txf9+cQ+HMVN4d6z46LZP7A==} + engines: {node: '>=8'} + + minipass-sized@2.0.0: + resolution: {integrity: sha512-zSsHhto5BcUVM2m1LurnXY6M//cGhVaegT71OfOXoprxT6o780GZd792ea6FfrQkuU4usHZIUczAQMRUE2plzA==} + engines: {node: '>=8'} + + minipass@3.3.6: + resolution: {integrity: sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==} + engines: {node: '>=8'} + minipass@7.1.3: resolution: {integrity: sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==} engines: {node: '>=16 || 14 >=14.17'} @@ -6029,12 +6275,20 @@ packages: minisearch@7.2.0: resolution: {integrity: sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==} + minizlib@3.1.0: + resolution: {integrity: sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==} + engines: {node: '>= 18'} + mitt@3.0.1: resolution: {integrity: sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==} mj-context-menu@0.6.1: resolution: {integrity: sha512-7NO5s6n10TIV96d4g2uDpG7ZDpIhMh0QNfGdJw/W47JswFcosz457wqz/b5sAKvl12sxINGFCn80NZHKwxQEXA==} + modern-tar@0.7.6: + resolution: {integrity: sha512-sweCIVXzx1aIGTCdzcMlSZt1h8k5Tmk08VNAuRk3IU28XamGiOH5ypi11g6De2CH7PhYqSSnGy2A/EFhbWnVKg==} + engines: {node: '>=18.0.0'} + mri@1.2.0: resolution: {integrity: sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA==} engines: {node: '>=4'} @@ -6143,6 +6397,44 @@ packages: resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + node-gyp@13.0.1: + resolution: {integrity: sha512-piOr0S10qy5THB+q5BdqkoOx65XL/tjTMUAit3vciPNp+snTOBnGunWH1Rz7XZUxf2T9uFrfT/Ty4+aC3yPeyg==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + hasBin: true + + nopt@10.0.1: + resolution: {integrity: sha512-df3sBr/6ax9hSGuC3CspvLlbnX8cP5L5nZwXF8cGN8l0zSWR6BvzmQ6jPUKjvo6+/xdpkNvEcucBNUdBeeV13g==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + hasBin: true + + npm-bundled@6.0.0: + resolution: {integrity: sha512-EqdodKEW6pYM+dPxA66TZQfMEqVDiuzjDM9edSjuPI1mXUbUJwVxkgqMZSJvs8RTXz2CGq8HUol/AffTZX5g8w==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-install-checks@9.0.0: + resolution: {integrity: sha512-t05Izcgi7p15cpldqoiXYpjzlkTTvBw33sgjmL/JjcvtV0ydbm2O4iEXO8A6smqComu5FAQhUas86HTMQ6Z1Uw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-normalize-package-bin@6.0.0: + resolution: {integrity: sha512-tdt4aFn9QamlhdN3HV2D2ccpBwO5/fyjjbXUxYA6uBjyekMZcZvDq0aSj9t5Jo+tih6AYFnt/cuIRn9013e0Uw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-package-arg@14.0.0: + resolution: {integrity: sha512-69XQh3k+dtGa1p+7RaR57IuG3rCko96xr/nUfN4yDYBXbTYICiWcOpsFKLN2GtGE9cyIljE+f1exnaYt9MvM+Q==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-packlist@11.3.0: + resolution: {integrity: sha512-cS1yVkyriZgQAbiK8PtwhZHEtsFOsKHsCg5Ww2ONckAvXIspgqd6o4WirOzvkupU24iMRZ4xtO4kb2iK2rbnag==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-pick-manifest@12.0.0: + resolution: {integrity: sha512-8Fs3YLrnNOhrCdPNZy18MzNgVC58LTDAFzq1FdZO/p3BHeCC/coz+t4F5Pxabys8HJpyTUorMea26GkXsb4J/Q==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + npm-registry-fetch@20.0.1: + resolution: {integrity: sha512-vzc1svxw/kw1IRjFsLi6gaxe1Olqm88V0tIfu2u5raL0b1gChe6ZEXNkyUlKxUC7s/egt5NxZHkbY18tMKKLfQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + nth-check@2.1.1: resolution: {integrity: sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==} @@ -6199,6 +6491,10 @@ packages: resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==} engines: {node: '>=10'} + p-map@7.0.5: + resolution: {integrity: sha512-e8vJF4XdVkzqqSHguEMz41mQO1wKwxKm5ENrUJQUu9kLDCtn83cxbyHZcszr4QC5zEA7WffRRC4gsTecC7J9oA==} + engines: {node: '>=18'} + p-retry@4.6.2: resolution: {integrity: sha512-312Id396EbJdvRONlngUx0NydfrIQ5lsYu0znKVUzVvArzEIt08V1qhtyESbGVd1FGX7UKtiFp5uwKZdM8wIuQ==} engines: {node: '>=8'} @@ -6209,6 +6505,11 @@ packages: package-manager-detector@1.6.0: resolution: {integrity: sha512-61A5ThoTiDG/C8s8UMZwSorAGwMJ0ERVGj2OjoW5pAalsNOg15+iQiPzrLJ4jhZ1HJzmC2PIHT2oEiH3R5fzNA==} + pacote@22.0.0: + resolution: {integrity: sha512-++VqeOZeL03uGM2MFLk96jGCSt1owBGkyFKoPr+trwNlZhCpjN2RrvwYxt8nTbs1wNMqSFYurq0TafVWkAIHig==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + hasBin: true + pako@1.0.11: resolution: {integrity: sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==} @@ -6247,6 +6548,10 @@ packages: resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==} engines: {node: '>=16 || 14 >=14.18'} + path-scurry@2.0.2: + resolution: {integrity: sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==} + engines: {node: 18 || 20 || >=22} + path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -6289,6 +6594,10 @@ packages: resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==} engines: {node: '>= 0.8.0'} + proc-log@7.0.0: + resolution: {integrity: sha512-FYgfaA69XZ93zaXLoMNQ+ViDXGGBgR8aLh03txzcFhV+9xOXx7+8DLCULrKKpR9+GsH9ZfHm82aSUPpozX0Ztg==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + process-nextick-args@2.0.1: resolution: {integrity: sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==} @@ -6303,6 +6612,15 @@ packages: resolution: {integrity: sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==} engines: {node: '>= 0.10'} + proxy-agent-negotiate@1.1.0: + resolution: {integrity: sha512-N8IBcM3UgCVzz2L2Lqv8DVntDnnC8/hiV4nEDUPkqq72TPUgYWjQc+bdZlBPZK9LzPAvOY//gAt0S0DApoOXWQ==} + engines: {node: '>= 20'} + peerDependencies: + kerberos: ^2.0.0 + peerDependenciesMeta: + kerberos: + optional: true + publint@0.3.21: resolution: {integrity: sha512-OqejcnMV6E9zel2oCrUOJEiiFkGiAAni0A6ibfQNh1k9Gu5z4F+Yso8lllam7AzmV6Do0vp7u3UpZNRBwuXaHQ==} engines: {node: '>=18'} @@ -6495,16 +6813,32 @@ packages: resolution: {integrity: sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==} engines: {node: '>=14'} + sigstore@5.0.0: + resolution: {integrity: sha512-hJqJfoG/e4qFQaauQL00c6J6FrHLBGKtkFvW3JbTSIEFOhLrSjdSM/gWd/yUOfYo/gsERehTXGC1VZWX+9X4Dg==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + sisteransi@1.0.5: resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} slick@1.12.2: resolution: {integrity: sha512-4qdtOGcBjral6YIBCWJ0ljFSKNLz9KkhbWtuGvUyRowl1kxfuE1x/Z/aJcaiilpb3do9bl5K7/1h9XC5wWpY/A==} + smart-buffer@4.2.0: + resolution: {integrity: sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==} + engines: {node: '>= 6.0.0', npm: '>= 3.0.0'} + smol-toml@1.6.1: resolution: {integrity: sha512-dWUG8F5sIIARXih1DTaQAX4SsiTXhInKf1buxdY9DIg4ZYPZK5nGM1VRIYmEbDbsHt7USo99xSLFu5Q1IqTmsg==} engines: {node: '>= 18'} + socks-proxy-agent@10.1.0: + resolution: {integrity: sha512-WlMj/67cEJ6MDI1OcsnjuYKDNDoyPCCYZ249kuuXPiMDw9F8PXkVaQ7YWu3siTydfQ/4BEZcvGzu+aYvz7dDCQ==} + engines: {node: '>= 20'} + + socks@2.8.9: + resolution: {integrity: sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw==} + engines: {node: '>= 10.0.0', npm: '>= 3.0.0'} + source-map-js@1.2.1: resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} engines: {node: '>=0.10.0'} @@ -6516,6 +6850,15 @@ packages: space-separated-tokens@2.0.2: resolution: {integrity: sha512-PEGlAwrG8yXGXRjW32fGbg66JAlOAwbObuqVoJpv/mRgoWDQfgH1wDPvtzWyUSNAXBGSk8h755YDbbcEy3SH2Q==} + spdx-exceptions@2.5.0: + resolution: {integrity: sha512-PiU42r+xO4UbUS1buo3LPJkjlO7430Xn5SVAhdpzzsPHsjbYVflnnFdATgabnLude+Cqu25p6N+g2lw/PFsa4w==} + + spdx-expression-parse@4.0.0: + resolution: {integrity: sha512-Clya5JIij/7C6bRR22+tnGXbc4VKlibKSVj2iHvVeX5iMW7s1SIQlqu699JkODJJIhh/pUu8L0/VLh8xflD+LQ==} + + spdx-license-ids@3.0.23: + resolution: {integrity: sha512-CWLcCCH7VLu13TgOH+r8p1O/Znwhqv/dbb6lqWy67G+pT1kHmeD/+V36AVb/vq8QMIQwVShJ6Ssl5FPh0fuSdw==} + speakingurl@14.0.1: resolution: {integrity: sha512-1POYv7uv2gXoyGFpBCmpDVSNV74IfsWlDW216UPjbWufNf+bSU6GdbDsxdcxtfwb4xlI3yxzOTKClUosxARYrQ==} engines: {node: '>=0.10.0'} @@ -6524,6 +6867,10 @@ packages: resolution: {integrity: sha512-i/VCLG1fvRc95pMHRqG4aQNscv+9aIsqA2oI7ZQS51sTdUcDHYX6cpT8/tqZ+enjs1tKVwbRBWgxut9SWn+f9g==} hasBin: true + ssri@14.0.0: + resolution: {integrity: sha512-jQxKI0yx0ZnTKrqjKkLDV2DXkBQn3k49JVmVqDGcDwKDtGDbImD/GXsq04KD0VVzCQQ9wZJYal3RwR1GzWTSow==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + stackback@0.0.2: resolution: {integrity: sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==} @@ -6584,6 +6931,10 @@ packages: tabbable@6.5.0: resolution: {integrity: sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA==} + tar@7.5.20: + resolution: {integrity: sha512-9FcyK4PA6+WbzlTM9WhQm6vB5W7cP7dUiPsv1g7YDwEQnQ1CGpK3MGlKk/ITVWMk05kHZuBhmVhiv8LZoy/PFQ==} + engines: {node: '>=18'} + tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} @@ -6694,6 +7045,10 @@ packages: engines: {node: '>=18.0.0'} hasBin: true + tuf-js@6.0.0: + resolution: {integrity: sha512-zlJVOIO68hmgo1//X4ENEcTGfuOTAtDPi8PsTsG+FyxD85E/ww1ZnwBbWo/yCEExGpI+Kilg7Z3qCdHX2BoJTQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + type-check@0.4.0: resolution: {integrity: sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew==} engines: {node: '>= 0.8.0'} @@ -6739,6 +7094,10 @@ packages: resolution: {integrity: sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==} engines: {node: '>=20.18.1'} + undici@8.7.0: + resolution: {integrity: sha512-N7iQtfyLhIMOFgQubvmLV26svHpO0bqKnAiWotTQCVKCmWrcGbBotPuW1x+xwYZ2VHdSTVUfPQQnlEt1/LouTQ==} + engines: {node: '>=22.19.0'} + unist-util-is@6.0.1: resolution: {integrity: sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g==} @@ -6772,6 +7131,10 @@ packages: resolution: {integrity: sha512-jOWVmzVceKlVVdwjNSenT4PbGghU0SBIizAev8ofZVgivk/TVHXSbNL8LP6M3spZvkR9/QolkyJavGSX5Cs0UA==} engines: {node: '>=10'} + validate-npm-package-name@8.0.0: + resolution: {integrity: sha512-SCv6OOV6Xj2/3cXy3dGmADluJTNcL3o7hZAglNPTe+WYuEuvxgJzxPrSDLZhF+CwyQOubqgecjMmTJGMVLWjYQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + vary@1.1.2: resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} engines: {node: '>= 0.8'} @@ -6961,6 +7324,11 @@ packages: engines: {node: '>= 8'} hasBin: true + which@7.0.0: + resolution: {integrity: sha512-RancgH2dmbLdHl6LRhEqvklWMgl/Hdnun0Y90KhBOLkMefg8Qa7/Zel8Sm+8HEcP6DEjzsWzpkuBQEZok58isA==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + hasBin: true + why-is-node-running@2.3.0: resolution: {integrity: sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==} engines: {node: '>=8'} @@ -7010,6 +7378,13 @@ packages: xmlchars@2.2.0: resolution: {integrity: sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==} + yallist@4.0.0: + resolution: {integrity: sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==} + + yallist@5.0.0: + resolution: {integrity: sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==} + engines: {node: '>=18'} + yaml@2.9.0: resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==} engines: {node: '>= 14.6'} @@ -7447,6 +7822,10 @@ snapshots: '@bcoe/v8-coverage@1.0.2': {} + '@bluwy/giget-core@0.1.7': + dependencies: + modern-tar: 0.7.6 + '@braintree/sanitize-url@7.1.2': {} '@bramus/specificity@2.4.2': @@ -7774,6 +8153,8 @@ snapshots: '@exodus/bytes@1.15.1': {} + '@gar/promise-retry@1.0.3': {} + '@google/genai@1.52.0(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))': dependencies: google-auth-library: 10.7.0 @@ -7828,6 +8209,10 @@ snapshots: wrap-ansi: 8.1.0 wrap-ansi-cjs: wrap-ansi@7.0.0 + '@isaacs/fs-minipass@4.0.1': + dependencies: + minipass: 7.1.3 + '@jridgewell/gen-mapping@0.3.13': dependencies: '@jridgewell/sourcemap-codec': 1.5.5 @@ -7915,6 +8300,63 @@ snapshots: '@nodable/entities@2.2.0': {} + '@npmcli/agent@5.0.2': + dependencies: + agent-base: 9.0.0 + http-proxy-agent: 9.1.0 + https-proxy-agent: 9.1.0 + lru-cache: 11.5.1 + socks-proxy-agent: 10.1.0 + transitivePeerDependencies: + - kerberos + - supports-color + + '@npmcli/fs@6.0.0': + dependencies: + semver: 7.8.4 + + '@npmcli/git@8.0.0': + dependencies: + '@gar/promise-retry': 1.0.3 + '@npmcli/promise-spawn': 10.0.0 + ini: 7.0.0 + lru-cache: 11.5.1 + npm-pick-manifest: 12.0.0 + proc-log: 7.0.0 + semver: 7.8.4 + which: 7.0.0 + + '@npmcli/installed-package-contents@5.0.0': + dependencies: + npm-bundled: 6.0.0 + npm-normalize-package-bin: 6.0.0 + + '@npmcli/node-gyp@6.0.0': {} + + '@npmcli/package-json@8.0.0': + dependencies: + '@npmcli/git': 8.0.0 + glob: 13.0.6 + hosted-git-info: 10.1.1 + json-parse-even-better-errors: 6.0.0 + proc-log: 7.0.0 + semver: 7.8.4 + spdx-expression-parse: 4.0.0 + + '@npmcli/promise-spawn@10.0.0': + dependencies: + which: 7.0.0 + + '@npmcli/redact@5.0.0': {} + + '@npmcli/run-script@11.0.0': + dependencies: + '@npmcli/node-gyp': 6.0.0 + '@npmcli/package-json': 8.0.0 + '@npmcli/promise-spawn': 10.0.0 + node-gyp: 13.0.1 + proc-log: 7.0.0 + '@oxc-parser/binding-android-arm-eabi@0.133.0': optional: true @@ -8288,6 +8730,39 @@ snapshots: '@shikijs/vscode-textmate@10.0.2': {} + '@sigstore/bundle@5.0.0': + dependencies: + '@sigstore/protobuf-specs': 0.5.1 + + '@sigstore/core@4.0.1': {} + + '@sigstore/protobuf-specs@0.5.1': {} + + '@sigstore/sign@5.0.0': + dependencies: + '@gar/promise-retry': 1.0.3 + '@sigstore/bundle': 5.0.0 + '@sigstore/core': 4.0.1 + '@sigstore/protobuf-specs': 0.5.1 + make-fetch-happen: 16.0.1 + proc-log: 7.0.0 + transitivePeerDependencies: + - kerberos + - supports-color + + '@sigstore/tuf@5.0.0': + dependencies: + '@sigstore/protobuf-specs': 0.5.1 + tuf-js: 6.0.0 + transitivePeerDependencies: + - supports-color + + '@sigstore/verify@4.1.0': + dependencies: + '@sigstore/bundle': 5.0.0 + '@sigstore/core': 4.0.1 + '@sigstore/protobuf-specs': 0.5.1 + '@smithy/core@3.24.7': dependencies: '@aws-crypto/crc32': 5.2.0 @@ -8354,6 +8829,13 @@ snapshots: estraverse: 5.3.0 picomatch: 4.0.4 + '@tufjs/canonical-json@2.0.0': {} + + '@tufjs/models@5.0.0': + dependencies: + '@tufjs/canonical-json': 2.0.0 + minimatch: 10.2.5 + '@tybys/wasm-util@0.10.2': dependencies: tslib: 2.8.1 @@ -8527,6 +9009,11 @@ snapshots: '@types/ms@2.1.0': {} + '@types/node-fetch@2.6.13': + dependencies: + '@types/node': 22.20.0 + form-data: 4.0.6 + '@types/node@22.20.0': dependencies: undici-types: 6.21.0 @@ -8535,10 +9022,35 @@ snapshots: dependencies: undici-types: 7.24.6 + '@types/npm-package-arg@6.1.4': {} + + '@types/npm-registry-fetch@8.0.9': + dependencies: + '@types/node': 22.20.0 + '@types/node-fetch': 2.6.13 + '@types/npm-package-arg': 6.1.4 + '@types/npmlog': 7.0.0 + '@types/ssri': 7.1.5 + + '@types/npmlog@7.0.0': + dependencies: + '@types/node': 22.20.0 + + '@types/pacote@11.1.8': + dependencies: + '@types/node': 22.20.0 + '@types/npm-registry-fetch': 8.0.9 + '@types/npmlog': 7.0.0 + '@types/ssri': 7.1.5 + '@types/picomatch@3.0.2': {} '@types/retry@0.12.0': {} + '@types/ssri@7.1.5': + dependencies: + '@types/node': 22.20.0 + '@types/tough-cookie@4.0.5': {} '@types/trusted-types@2.0.7': @@ -8815,6 +9327,8 @@ snapshots: '@xmldom/xmldom@0.9.10': {} + abbrev@5.0.0: {} + accepts@2.0.0: dependencies: mime-types: 3.0.2 @@ -8828,6 +9342,8 @@ snapshots: agent-base@7.1.4: {} + agent-base@9.0.0: {} + ajv-formats@3.0.1(ajv@8.20.0): optionalDependencies: ajv: 8.20.0 @@ -8895,6 +9411,8 @@ snapshots: estree-walker: 3.0.3 js-tokens: 10.0.0 + asynckit@0.4.0: {} + balanced-match@1.0.2: {} balanced-match@4.0.4: {} @@ -8945,6 +9463,19 @@ snapshots: cac@7.0.0: {} + cacache@21.0.1: + dependencies: + '@npmcli/fs': 6.0.0 + fs-minipass: 3.0.3 + glob: 13.0.6 + lru-cache: 11.5.1 + minipass: 7.1.3 + minipass-collect: 2.0.1 + minipass-flush: 1.0.7 + minipass-pipeline: 1.2.4 + p-map: 7.0.5 + ssri: 14.0.0 + call-bind-apply-helpers@1.0.2: dependencies: es-errors: 1.3.0 @@ -8987,12 +9518,18 @@ snapshots: dependencies: readdirp: 4.1.2 + chownr@3.0.0: {} + color-convert@2.0.1: dependencies: color-name: 1.1.4 color-name@1.1.4: {} + combined-stream@1.0.8: + dependencies: + delayed-stream: 1.0.0 + comma-separated-tokens@2.0.3: {} commander@13.1.0: {} @@ -9298,6 +9835,8 @@ snapshots: dependencies: robust-predicates: 3.0.3 + delayed-stream@1.0.0: {} + depd@2.0.0: {} dequal@2.0.3: {} @@ -9372,6 +9911,8 @@ snapshots: entities@8.0.0: {} + env-paths@2.2.1: {} + es-define-property@1.0.1: {} es-errors@1.3.0: {} @@ -9382,6 +9923,13 @@ snapshots: dependencies: es-errors: 1.3.0 + es-set-tostringtag@2.1.0: + dependencies: + es-errors: 1.3.0 + get-intrinsic: 1.3.0 + has-tostringtag: 1.0.2 + hasown: 2.0.4 + es-toolkit@1.49.0: {} esbuild@0.21.5: @@ -9556,6 +10104,8 @@ snapshots: expect-type@1.3.0: {} + exponential-backoff@3.1.3: {} + express-rate-limit@8.5.2(express@5.2.1): dependencies: express: 5.2.1 @@ -9679,6 +10229,14 @@ snapshots: cross-spawn: 7.0.6 signal-exit: 4.1.0 + form-data@4.0.6: + dependencies: + asynckit: 0.4.0 + combined-stream: 1.0.8 + es-set-tostringtag: 2.1.0 + hasown: 2.0.4 + mime-types: 2.1.35 + formatly@0.3.0: dependencies: fd-package-json: 2.0.0 @@ -9691,6 +10249,10 @@ snapshots: fresh@2.0.0: {} + fs-minipass@3.0.3: + dependencies: + minipass: 7.1.3 + fsevents@2.3.3: optional: true @@ -9753,6 +10315,12 @@ snapshots: package-json-from-dist: 1.0.1 path-scurry: 1.11.1 + glob@13.0.6: + dependencies: + minimatch: 10.2.5 + minipass: 7.1.3 + path-scurry: 2.0.2 + globals@17.7.0: {} globrex@0.1.2: {} @@ -9772,6 +10340,8 @@ snapshots: gopd@1.2.0: {} + graceful-fs@4.2.11: {} + hachure-fill@0.5.2: {} handlebars@4.7.9: @@ -9787,6 +10357,10 @@ snapshots: has-symbols@1.1.0: {} + has-tostringtag@1.0.2: + dependencies: + has-symbols: 1.1.0 + hasown@2.0.4: dependencies: function-bind: 1.1.2 @@ -9815,6 +10389,10 @@ snapshots: hookable@6.1.1: {} + hosted-git-info@10.1.1: + dependencies: + lru-cache: 11.5.1 + html-encoding-sniffer@6.0.0: dependencies: '@exodus/bytes': 1.15.1 @@ -9839,6 +10417,8 @@ snapshots: domutils: 2.8.0 entities: 2.2.0 + http-cache-semantics@4.2.0: {} + http-errors@2.0.1: dependencies: depd: 2.0.0 @@ -9854,6 +10434,15 @@ snapshots: transitivePeerDependencies: - supports-color + http-proxy-agent@9.1.0: + dependencies: + agent-base: 9.0.0 + debug: 4.4.3 + proxy-agent-negotiate: 1.1.0 + transitivePeerDependencies: + - kerberos + - supports-color + https-proxy-agent@7.0.6: dependencies: agent-base: 7.1.4 @@ -9861,6 +10450,15 @@ snapshots: transitivePeerDependencies: - supports-color + https-proxy-agent@9.1.0: + dependencies: + agent-base: 9.0.0 + debug: 4.4.3 + proxy-agent-negotiate: 1.1.0 + transitivePeerDependencies: + - kerberos + - supports-color + iconv-lite@0.6.3: dependencies: safer-buffer: 2.1.2 @@ -9869,6 +10467,10 @@ snapshots: dependencies: safer-buffer: 2.1.2 + ignore-walk@9.0.0: + dependencies: + minimatch: 10.2.5 + ignore@5.3.2: {} ignore@7.0.5: {} @@ -9883,6 +10485,8 @@ snapshots: inherits@2.0.4: {} + ini@7.0.0: {} + internmap@1.0.1: {} internmap@2.0.3: {} @@ -9909,6 +10513,8 @@ snapshots: isexe@2.0.0: {} + isexe@4.0.0: {} + istanbul-lib-coverage@3.2.2: {} istanbul-lib-report@3.0.1: @@ -10001,6 +10607,8 @@ snapshots: json-buffer@3.0.1: {} + json-parse-even-better-errors@6.0.0: {} + json-schema-to-ts@3.1.1: dependencies: '@babel/runtime': 7.29.7 @@ -10016,6 +10624,8 @@ snapshots: jsonc-parser@3.3.1: {} + jsonparse@1.3.1: {} + jsx-ast-utils-x@0.1.0: {} jszip@3.10.1: @@ -10207,6 +10817,24 @@ snapshots: dependencies: semver: 7.8.4 + make-fetch-happen@16.0.1: + dependencies: + '@gar/promise-retry': 1.0.3 + '@npmcli/agent': 5.0.2 + '@npmcli/redact': 5.0.0 + cacache: 21.0.1 + http-cache-semantics: 4.2.0 + minipass: 7.1.3 + minipass-fetch: 6.0.0 + minipass-flush: 1.0.7 + minipass-pipeline: 1.2.4 + negotiator: 1.0.0 + proc-log: 7.0.0 + ssri: 14.0.0 + transitivePeerDependencies: + - kerberos + - supports-color + mark.js@8.11.1: {} markdown-it-mathjax3@4.3.2: @@ -10568,8 +11196,14 @@ snapshots: transitivePeerDependencies: - supports-color + mime-db@1.52.0: {} + mime-db@1.54.0: {} + mime-types@2.1.35: + dependencies: + mime-db: 1.52.0 + mime-types@3.0.2: dependencies: mime-db: 1.54.0 @@ -10586,14 +11220,48 @@ snapshots: minimist@1.2.8: {} + minipass-collect@2.0.1: + dependencies: + minipass: 7.1.3 + + minipass-fetch@6.0.0: + dependencies: + minipass: 7.1.3 + minipass-sized: 2.0.0 + minizlib: 3.1.0 + optionalDependencies: + iconv-lite: 0.7.3 + + minipass-flush@1.0.7: + dependencies: + minipass: 3.3.6 + + minipass-pipeline@1.2.4: + dependencies: + minipass: 3.3.6 + + minipass-sized@2.0.0: + dependencies: + minipass: 7.1.3 + + minipass@3.3.6: + dependencies: + yallist: 4.0.0 + minipass@7.1.3: {} minisearch@7.2.0: {} + minizlib@3.1.0: + dependencies: + minipass: 7.1.3 + mitt@3.0.1: {} mj-context-menu@0.6.1: {} + modern-tar@0.7.6: {} + mri@1.2.0: {} ms@2.1.3: {} @@ -10678,6 +11346,67 @@ snapshots: fetch-blob: 3.2.0 formdata-polyfill: 4.0.10 + node-gyp@13.0.1: + dependencies: + env-paths: 2.2.1 + exponential-backoff: 3.1.3 + graceful-fs: 4.2.11 + nopt: 10.0.1 + proc-log: 7.0.0 + semver: 7.8.4 + tar: 7.5.20 + tinyglobby: 0.2.17 + undici: 8.7.0 + which: 7.0.0 + + nopt@10.0.1: + dependencies: + abbrev: 5.0.0 + + npm-bundled@6.0.0: + dependencies: + npm-normalize-package-bin: 6.0.0 + + npm-install-checks@9.0.0: + dependencies: + semver: 7.8.4 + + npm-normalize-package-bin@6.0.0: {} + + npm-package-arg@14.0.0: + dependencies: + hosted-git-info: 10.1.1 + proc-log: 7.0.0 + semver: 7.8.4 + validate-npm-package-name: 8.0.0 + + npm-packlist@11.3.0: + dependencies: + glob: 13.0.6 + ignore-walk: 9.0.0 + proc-log: 7.0.0 + + npm-pick-manifest@12.0.0: + dependencies: + npm-install-checks: 9.0.0 + npm-normalize-package-bin: 6.0.0 + npm-package-arg: 14.0.0 + semver: 7.8.4 + + npm-registry-fetch@20.0.1: + dependencies: + '@npmcli/redact': 5.0.0 + jsonparse: 1.3.1 + make-fetch-happen: 16.0.1 + minipass: 7.1.3 + minipass-fetch: 6.0.0 + minizlib: 3.1.0 + npm-package-arg: 14.0.0 + proc-log: 7.0.0 + transitivePeerDependencies: + - kerberos + - supports-color + nth-check@2.1.1: dependencies: boolbase: 1.0.0 @@ -10771,6 +11500,8 @@ snapshots: dependencies: p-limit: 3.1.0 + p-map@7.0.5: {} + p-retry@4.6.2: dependencies: '@types/retry': 0.12.0 @@ -10780,6 +11511,29 @@ snapshots: package-manager-detector@1.6.0: {} + pacote@22.0.0: + dependencies: + '@gar/promise-retry': 1.0.3 + '@npmcli/git': 8.0.0 + '@npmcli/installed-package-contents': 5.0.0 + '@npmcli/package-json': 8.0.0 + '@npmcli/promise-spawn': 10.0.0 + '@npmcli/run-script': 11.0.0 + cacache: 21.0.1 + fs-minipass: 3.0.3 + minipass: 7.1.3 + npm-package-arg: 14.0.0 + npm-packlist: 11.3.0 + npm-pick-manifest: 12.0.0 + npm-registry-fetch: 20.0.1 + proc-log: 7.0.0 + sigstore: 5.0.0 + ssri: 14.0.0 + tar: 7.5.20 + transitivePeerDependencies: + - kerberos + - supports-color + pako@1.0.11: {} parse5-htmlparser2-tree-adapter@6.0.1: @@ -10809,6 +11563,11 @@ snapshots: lru-cache: 10.4.3 minipass: 7.1.3 + path-scurry@2.0.2: + dependencies: + lru-cache: 11.5.1 + minipass: 7.1.3 + path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -10838,6 +11597,8 @@ snapshots: prelude-ls@1.2.1: {} + proc-log@7.0.0: {} + process-nextick-args@2.0.1: {} property-information@7.2.0: {} @@ -10853,7 +11614,7 @@ snapshots: '@protobufjs/path': 1.1.2 '@protobufjs/pool': 1.1.0 '@protobufjs/utf8': 1.1.1 - '@types/node': 25.9.3 + '@types/node': 22.20.0 long: 5.3.2 proxy-addr@2.0.7: @@ -10861,6 +11622,8 @@ snapshots: forwarded: 0.2.0 ipaddr.js: 1.9.1 + proxy-agent-negotiate@1.1.0: {} + publint@0.3.21: dependencies: '@publint/pack': 0.1.4 @@ -11144,18 +11907,54 @@ snapshots: signal-exit@4.1.0: {} + sigstore@5.0.0: + dependencies: + '@sigstore/bundle': 5.0.0 + '@sigstore/core': 4.0.1 + '@sigstore/protobuf-specs': 0.5.1 + '@sigstore/sign': 5.0.0 + '@sigstore/tuf': 5.0.0 + '@sigstore/verify': 4.1.0 + transitivePeerDependencies: + - kerberos + - supports-color + sisteransi@1.0.5: {} slick@1.12.2: {} + smart-buffer@4.2.0: {} + smol-toml@1.6.1: {} + socks-proxy-agent@10.1.0: + dependencies: + agent-base: 9.0.0 + debug: 4.4.3 + socks: 2.8.9 + transitivePeerDependencies: + - supports-color + + socks@2.8.9: + dependencies: + ip-address: 10.2.0 + smart-buffer: 4.2.0 + source-map-js@1.2.1: {} source-map@0.6.1: {} space-separated-tokens@2.0.2: {} + spdx-exceptions@2.5.0: {} + + spdx-expression-parse@4.0.0: + dependencies: + spdx-exceptions: 2.5.0 + spdx-license-ids: 3.0.23 + + spdx-license-ids@3.0.23: {} + speakingurl@14.0.1: {} speech-rule-engine@4.1.4: @@ -11164,6 +11963,10 @@ snapshots: commander: 13.1.0 wicked-good-xpath: 1.3.0 + ssri@14.0.0: + dependencies: + minipass: 7.1.3 + stackback@0.0.2: {} statuses@2.0.2: {} @@ -11221,6 +12024,14 @@ snapshots: tabbable@6.5.0: {} + tar@7.5.20: + dependencies: + '@isaacs/fs-minipass': 4.0.1 + chownr: 3.0.0 + minipass: 7.1.3 + minizlib: 3.1.0 + yallist: 5.0.0 + tinybench@2.9.0: {} tinyexec@1.2.4: {} @@ -11301,6 +12112,14 @@ snapshots: optionalDependencies: fsevents: 2.3.3 + tuf-js@6.0.0: + dependencies: + '@gar/promise-retry': 1.0.3 + '@tufjs/models': 5.0.0 + debug: 4.4.3 + transitivePeerDependencies: + - supports-color + type-check@0.4.0: dependencies: prelude-ls: 1.2.1 @@ -11342,6 +12161,8 @@ snapshots: undici@7.28.0: {} + undici@8.7.0: {} + unist-util-is@6.0.1: dependencies: '@types/unist': 3.0.3 @@ -11377,6 +12198,8 @@ snapshots: valid-data-url@3.0.1: {} + validate-npm-package-name@8.0.0: {} + vary@1.1.2: {} vfile-message@4.0.3: @@ -11600,6 +12423,10 @@ snapshots: dependencies: isexe: 2.0.0 + which@7.0.0: + dependencies: + isexe: 4.0.0 + why-is-node-running@2.3.0: dependencies: siginfo: 2.0.0 @@ -11633,6 +12460,10 @@ snapshots: xmlchars@2.2.0: {} + yallist@4.0.0: {} + + yallist@5.0.0: {} + yaml@2.9.0: {} yocto-queue@0.1.0: {} diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 75fb2b75ad..4a845b4554 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -52,6 +52,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, 'packages/sdk/helper': { kind: 'none', reason: 'The project domain edits files and registers no live agent or model surface.' }, + 'packages/sdk/plugin-fetch': { kind: 'none', reason: 'The fetcher acquires plugin sources into a temp dir and registers no live agent or model surface.' }, 'packages/sdk/scripts': { kind: 'indirect', reason: 'The launcher delegates model context to the loaded project plugin tree.' }, 'packages/session-query/session-query': { kind: 'none', reason: 'The trusted query service exposes cloned records only to callers and registers no model surface.' }, 'packages/skill/skill': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-skill.' }, diff --git a/tsconfig.build.json b/tsconfig.build.json index dd13e018c3..9a6133db36 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -98,6 +98,7 @@ { "path": "./packages/mcp/mcp-client" }, { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, - { "path": "./packages/sdk/create-sdk" } + { "path": "./packages/sdk/create-sdk" }, + { "path": "./packages/sdk/plugin-fetch" } ] } diff --git a/tsconfig.json b/tsconfig.json index 22451dc10f..4588c81c72 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -109,6 +109,7 @@ { "path": "./packages/mcp/mcp-client" }, { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, - { "path": "./packages/sdk/create-sdk" } + { "path": "./packages/sdk/create-sdk" }, + { "path": "./packages/sdk/plugin-fetch" } ] } From fe668a6dfe5c9c64453eb78b8f167eb1407c0eb9 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 13:16:51 +0800 Subject: [PATCH 17/73] feat(telemetry): greenfield dsh-sdk telemetry modules Add @deepseek-ai/dsh-telemetry, a launcher-side (non-plugin) library for the ConsentResolver (parses cordis.yml consent + DO_NOT_TRACK/CI), buildTelemetryPayload (redacted cordis.yml + package.json full content, never .env), getOrCreateAnonymousId (random UUID in a per-machine global config file), and TelemetryReporter (fire-and-forget, never blocks or crashes the command). Endpoint is a fixed .invalid placeholder pending the real endpoint. Launcher dispatch wiring and the helper feature-catalog entry are intentionally out of scope. Registers the package in tsconfig references, the module graph, and the README model-experience audit map. Per-file 100% coverage. --- docs/module-graph.md | 3 + packages/sdk/telemetry/README.md | 24 +++ packages/sdk/telemetry/package.json | 35 +++ packages/sdk/telemetry/src/anonymous-id.ts | 106 +++++++++ .../sdk/telemetry/src/consent-resolver.ts | 125 +++++++++++ packages/sdk/telemetry/src/index.ts | 45 ++++ packages/sdk/telemetry/src/payload.ts | 75 +++++++ packages/sdk/telemetry/src/reporter.ts | 149 +++++++++++++ packages/sdk/telemetry/src/secret-redactor.ts | 203 ++++++++++++++++++ .../sdk/telemetry/tests/anonymous-id.spec.ts | 100 +++++++++ .../telemetry/tests/consent-resolver.spec.ts | 131 +++++++++++ packages/sdk/telemetry/tests/payload.spec.ts | 59 +++++ packages/sdk/telemetry/tests/reporter.spec.ts | 134 ++++++++++++ .../telemetry/tests/secret-redactor.spec.ts | 169 +++++++++++++++ packages/sdk/telemetry/tsconfig.json | 13 ++ pnpm-lock.yaml | 13 ++ .../verify-package-readme-model-experience.ts | 1 + tsconfig.build.json | 3 +- tsconfig.json | 3 +- 19 files changed, 1389 insertions(+), 2 deletions(-) create mode 100644 packages/sdk/telemetry/README.md create mode 100644 packages/sdk/telemetry/package.json create mode 100644 packages/sdk/telemetry/src/anonymous-id.ts create mode 100644 packages/sdk/telemetry/src/consent-resolver.ts create mode 100644 packages/sdk/telemetry/src/index.ts create mode 100644 packages/sdk/telemetry/src/payload.ts create mode 100644 packages/sdk/telemetry/src/reporter.ts create mode 100644 packages/sdk/telemetry/src/secret-redactor.ts create mode 100644 packages/sdk/telemetry/tests/anonymous-id.spec.ts create mode 100644 packages/sdk/telemetry/tests/consent-resolver.spec.ts create mode 100644 packages/sdk/telemetry/tests/payload.spec.ts create mode 100644 packages/sdk/telemetry/tests/reporter.spec.ts create mode 100644 packages/sdk/telemetry/tests/secret-redactor.spec.ts create mode 100644 packages/sdk/telemetry/tsconfig.json diff --git a/docs/module-graph.md b/docs/module-graph.md index c9439d5501..4cc2c36eeb 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -140,6 +140,7 @@ flowchart TD pkg_helper["helper"] pkg_plugin_fetch["plugin-fetch"] pkg_scripts["scripts"] + pkg_telemetry["telemetry"] end subgraph group_tasks["packages/tasks"] pkg_tasks["tasks"] @@ -155,6 +156,7 @@ flowchart TD pkg_helper --> pkg_brand pkg_plugin_fetch --> pkg_brand pkg_scripts --> pkg_app_boot + pkg_telemetry --> pkg_brand pkg_llm_deepseek --> pkg_llm pkg_llm_pi_ai --> pkg_llm pkg_session --> pkg_brand @@ -446,6 +448,7 @@ flowchart TD | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) | | [`plugin-fetch`](../packages/sdk/plugin-fetch) | `sdk` | [`brand`](../packages/util/brand) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) | +| [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand) | | [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | | [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) | | [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) | diff --git a/packages/sdk/telemetry/README.md b/packages/sdk/telemetry/README.md new file mode 100644 index 0000000000..3a8b6ac4c2 --- /dev/null +++ b/packages/sdk/telemetry/README.md @@ -0,0 +1,24 @@ +# `@deepseek-ai/dsh-telemetry` + +Launcher-side telemetry primitives for the dsh-sdk toolchain. This is a plain library the launcher imports around each command; it is **not** a Cordis plugin, because `build` and first-init `create` never boot Cordis. Wiring the reporter into the launcher command dispatch and adding the telemetry consent feature to the `dsh-helper` catalog live in their owning packages, not here. + +| Export | Role | +|---|---| +| `SecretRedactor` | Conservative safety backstop: replaces secret-shaped values (secret-like keys, known token shapes, PEM blocks, URL credentials, high-entropy opaque tokens) with a placeholder in both parsed values (`redactValue`) and raw text (`redactText`). Never drops a field or line. | +| `ConsentResolver` | Parses (never boots) a project `cordis.yml` and reads the telemetry entry's enabled/disabled state as consent; `DO_NOT_TRACK`/CI env force a hard opt-out. | +| `buildTelemetryPayload` | Assembles `{command, durationMs, success, cordisYmlContent, packageJsonContent}`, running the redactor over the full `cordis.yml` and `package.json` text. Never reads `.env`. | +| `getOrCreateAnonymousId` | Random UUID persisted in a per-user GLOBAL config file (never in the project, never derived from git). | +| `TelemetryReporter` | Fire-and-forget send: `report()` never blocks or throws; delivery resolves on every path; `flush()` optionally drains in-flight sends within a cap. | + +Consent is carried by the telemetry entry in `cordis.yml`, so disabling telemetry is disabling that entry. When `cordis.yml` does not yet exist (first `create`) consent defaults to allowed; when it exists without a telemetry entry consent defaults to denied — both are configurable on `ConsentResolver`. + +The collection endpoint is a fixed constant (`DSH_TELEMETRY_ENDPOINT`); its `.invalid` placeholder must be replaced with the real endpoint before release. + +## Model Experience + +None, as the reporter sends developer-cycle telemetry from the launcher and never reaches a model request. + +## Known Limitations and Deferred Work + +- **Placeholder endpoint** — `DSH_TELEMETRY_ENDPOINT` points at `.invalid` until the real endpoint is set. +- **Redaction is heuristic** — a conservative backstop, not a guarantee; secrets belong in `.env`, which is never read or reported. diff --git a/packages/sdk/telemetry/package.json b/packages/sdk/telemetry/package.json new file mode 100644 index 0000000000..fcb6efb797 --- /dev/null +++ b/packages/sdk/telemetry/package.json @@ -0,0 +1,35 @@ +{ + "name": "@deepseek-ai/dsh-telemetry", + "description": "Launcher-side dsh-sdk telemetry: secret redaction, consent resolution, anonymous id, payload builder, and fire-and-forget reporter", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "dependencies": { + "yaml": "^2.9.0" + }, + "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/sdk/telemetry/src/anonymous-id.ts b/packages/sdk/telemetry/src/anonymous-id.ts new file mode 100644 index 0000000000..030fefa19f --- /dev/null +++ b/packages/sdk/telemetry/src/anonymous-id.ts @@ -0,0 +1,106 @@ +/** + * Per-machine anonymous telemetry id. + * + * The id is a random UUID persisted in a per-user GLOBAL config file — never in + * the project, and never derived from the git remote, repository URL, or any + * other identifying source (a derived id would make "anonymous" a fiction). The + * same id is reused across projects on one machine so telemetry counts machines, + * not repositories. + * + * @module @deepseek-ai/dsh-telemetry/anonymous-id + */ + +import { randomUUID } from 'node:crypto' +import { mkdir, readFile, writeFile } from 'node:fs/promises' +import { homedir } from 'node:os' +import { dirname, join } from 'node:path' +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** A machine-scoped anonymous telemetry id (random UUID v4). */ +export type AnonymousId = Branded<'AnonymousId'> + +/** Config directory name owned by the DeepSeek Harness across tools. */ +const CONFIG_NAMESPACE = 'deepseek-harness' + +/** Default file, inside the global config dir, storing the anonymous id. */ +export const ANONYMOUS_ID_FILE_NAME = 'telemetry.json' + +const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + +/** Ambient seams for locating and generating the id; every field has a default. */ +export interface AnonymousIdOptions { + /** Environment consulted for `DSH_CONFIG_HOME`/`XDG_CONFIG_HOME`/`APPDATA`; defaults to `process.env`. */ + env?: NodeJS.ProcessEnv + /** Platform string used to pick the Windows path; defaults to `process.platform`. */ + platform?: NodeJS.Platform + /** Home directory resolver; defaults to `os.homedir`. */ + homeDir?: () => string + /** UUID generator; defaults to `crypto.randomUUID` (test seam). */ + randomUUID?: () => string +} + +/** + * Resolve the per-user global config directory for harness tooling. + * Precedence: `DSH_CONFIG_HOME` (explicit override) > `XDG_CONFIG_HOME` > + * platform default (`%APPDATA%` on Windows, else `~/.config`). + * @param options - environment, platform, and home-directory seams. + * @returns absolute config directory path for the harness namespace. + */ +export function globalConfigDir(options: AnonymousIdOptions = {}): string { + const env = options.env ?? process.env + const platform = options.platform ?? process.platform + const home = options.homeDir ?? homedir + if (env.DSH_CONFIG_HOME !== undefined && env.DSH_CONFIG_HOME.length > 0) return env.DSH_CONFIG_HOME + if (env.XDG_CONFIG_HOME !== undefined && env.XDG_CONFIG_HOME.length > 0) { + return join(env.XDG_CONFIG_HOME, CONFIG_NAMESPACE) + } + if (platform === 'win32' && env.APPDATA !== undefined && env.APPDATA.length > 0) { + return join(env.APPDATA, CONFIG_NAMESPACE) + } + return join(home(), '.config', CONFIG_NAMESPACE) +} + +/** Read a valid persisted id from the store, or `undefined` when absent/corrupt. */ +async function readPersistedId(file: string): Promise { + let text: string + try { + text = await readFile(file, 'utf8') + } catch { + // Absent or unreadable: the caller mints and persists a fresh id. + return undefined + } + let parsed: unknown + try { + parsed = JSON.parse(text) + } catch { + // Corrupt JSON: the caller overwrites the store with a fresh id. + return undefined + } + if (parsed !== null && typeof parsed === 'object') { + const value = (parsed as Record).anonymousId + if (typeof value === 'string' && UUID_PATTERN.test(value)) return value as AnonymousId + } + return undefined +} + +/** + * Return the machine's anonymous id, creating and persisting one on first use. + * Persistence is best-effort: a write failure still returns a usable id for the + * current run so telemetry is never blocked by config-dir permissions. + * @param options - config-location and UUID-generation seams. + * @returns the stable per-machine anonymous id. + */ +export async function getOrCreateAnonymousId(options: AnonymousIdOptions = {}): Promise { + const file = join(globalConfigDir(options), ANONYMOUS_ID_FILE_NAME) + const existing = await readPersistedId(file) + if (existing !== undefined) return existing + const generate = options.randomUUID ?? randomUUID + const created = generate() as AnonymousId + try { + await mkdir(dirname(file), { recursive: true }) + await writeFile(file, `${JSON.stringify({ anonymousId: created }, null, 2)}\n`, 'utf8') + } catch { + // Best-effort persistence: return the fresh id even when the store is unwritable. + } + return created +} diff --git a/packages/sdk/telemetry/src/consent-resolver.ts b/packages/sdk/telemetry/src/consent-resolver.ts new file mode 100644 index 0000000000..1d50904f39 --- /dev/null +++ b/packages/sdk/telemetry/src/consent-resolver.ts @@ -0,0 +1,125 @@ +/** + * Consent resolution for dsh-sdk telemetry. + * + * Consent is carried by the telemetry plugin's enabled/disabled state in the + * project `cordis.yml`: an enabled entry means opt-in, a `disabled: true` entry + * means opt-out. The resolver PARSES `cordis.yml` — it never boots a Cordis + * application — because several launcher commands (`build`, `create`) never + * boot Cordis at all. `DO_NOT_TRACK` and CI environment signals force a denial + * regardless of file state. + * + * @module @deepseek-ai/dsh-telemetry/consent-resolver + */ + +import { readFile } from 'node:fs/promises' +import { join } from 'node:path' +import { parseDocument, type ScalarTag } from 'yaml' + +/** Default `cordis.yml` entry name that carries telemetry consent. */ +export const DEFAULT_TELEMETRY_PLUGIN_NAME = '@deepseek-ai/dsh-telemetry' + +/** + * Passthrough for Cordis' `!!js` expression tag so parsing consent never fails + * on projects that inline JavaScript expressions; the resolver only reads plain + * `name`/`disabled` scalars and does not evaluate expressions. + */ +const JS_EXPRESSION_TAG: ScalarTag = { + tag: 'tag:yaml.org,2002:js', + resolve: value => value, +} + +/** Why telemetry is or is not permitted for one command. */ +export type ConsentReason = + | 'enabled' + | 'disabled' + | 'absent' + | 'no-config' + | 'do-not-track' + | 'ci' + | 'unreadable' + +/** Resolved telemetry consent for one command invocation. */ +export interface ConsentDecision { + /** Whether telemetry may be sent. */ + allowed: boolean + /** The signal that determined {@link allowed}. */ + reason: ConsentReason +} + +/** Tuning for {@link ConsentResolver}; every field defaults to a documented value. */ +export interface ConsentResolverOptions { + /** `cordis.yml` entry name whose enabled state carries consent. */ + telemetryPluginName?: string + /** Environment used for `DO_NOT_TRACK`/CI checks; defaults to `process.env`. */ + env?: NodeJS.ProcessEnv + /** Honor `DO_NOT_TRACK`/CI env signals as a hard opt-out. Defaults to `true`. */ + honorEnvOptOut?: boolean + /** Consent when `cordis.yml` does not exist yet (first `create`). Defaults to `true` (telemetry is default-on). */ + allowWhenNoConfig?: boolean + /** Consent when `cordis.yml` exists but has no telemetry entry. Defaults to `false`. */ + allowWhenEntryAbsent?: boolean +} + +/** Whether an environment variable is set to a non-empty, non-"0"/"false" value. */ +function envEnabled(value: string | undefined): boolean { + if (value === undefined) return false + const normalized = value.trim().toLowerCase() + return normalized.length > 0 && normalized !== '0' && normalized !== 'false' +} + +/** Read a `cordis.yml` entry's `name`/`disabled` scalars, tolerating `!!js` tags. */ +function readTelemetryEntry(text: string, pluginName: string): { present: boolean; disabled: boolean } { + const document = parseDocument(text, { customTags: [JS_EXPRESSION_TAG] }) + const contents: unknown = document.toJS({ maxAliasCount: -1 }) + if (!Array.isArray(contents)) return { present: false, disabled: false } + for (const entry of contents) { + if (entry === null || typeof entry !== 'object') continue + const record = entry as Record + if (record.name === pluginName) return { present: true, disabled: record.disabled === true } + } + return { present: false, disabled: false } +} + +/** Resolve telemetry consent by parsing a project's `cordis.yml` and the environment. */ +export class ConsentResolver { + readonly #pluginName: string + readonly #env: NodeJS.ProcessEnv + readonly #honorEnvOptOut: boolean + readonly #allowWhenNoConfig: boolean + readonly #allowWhenEntryAbsent: boolean + + /** @param options - plugin name, environment, and default-decision knobs. */ + constructor(options: ConsentResolverOptions = {}) { + this.#pluginName = options.telemetryPluginName ?? DEFAULT_TELEMETRY_PLUGIN_NAME + this.#env = options.env ?? process.env + this.#honorEnvOptOut = options.honorEnvOptOut ?? true + this.#allowWhenNoConfig = options.allowWhenNoConfig ?? true + this.#allowWhenEntryAbsent = options.allowWhenEntryAbsent ?? false + } + + /** + * Resolve consent for a command run in the given project directory. + * @param projectDir - absolute or relative project root containing `cordis.yml`. + * @returns the consent decision and the signal that produced it. + */ + async resolve(projectDir: string): Promise { + if (this.#honorEnvOptOut) { + if (envEnabled(this.#env.DO_NOT_TRACK)) return { allowed: false, reason: 'do-not-track' } + if (envEnabled(this.#env.CI)) return { allowed: false, reason: 'ci' } + } + let text: string + try { + text = await readFile(join(projectDir, 'cordis.yml'), 'utf8') + } catch (error) { + // Missing cordis.yml is the first-init (`create`) path; any other read + // fault is treated conservatively as its own reason. + if ((error as NodeJS.ErrnoException).code === 'ENOENT') { + return { allowed: this.#allowWhenNoConfig, reason: 'no-config' } + } + return { allowed: false, reason: 'unreadable' } + } + const entry = readTelemetryEntry(text, this.#pluginName) + if (!entry.present) return { allowed: this.#allowWhenEntryAbsent, reason: 'absent' } + return entry.disabled ? { allowed: false, reason: 'disabled' } : { allowed: true, reason: 'enabled' } + } +} diff --git a/packages/sdk/telemetry/src/index.ts b/packages/sdk/telemetry/src/index.ts new file mode 100644 index 0000000000..107956fa39 --- /dev/null +++ b/packages/sdk/telemetry/src/index.ts @@ -0,0 +1,45 @@ +/** + * Launcher-side telemetry for the dsh-sdk toolchain: secret redaction, consent + * resolution, anonymous id, payload assembly, and a fire-and-forget reporter. + * + * This package is a plain library the launcher imports around each command — it + * is NOT a Cordis plugin (several commands never boot Cordis). Wiring it into + * the launcher command dispatch and the helper feature catalog lives outside + * this package. + * + * @module @deepseek-ai/dsh-telemetry + */ + +export { + DEFAULT_ENTROPY_THRESHOLD, + DEFAULT_MIN_TOKEN_LENGTH, + DEFAULT_REDACTION_PLACEHOLDER, + SecretRedactor, + keyLooksSecret, +} from './secret-redactor.ts' +export type { SecretRedactorOptions } from './secret-redactor.ts' +export { + ConsentResolver, + DEFAULT_TELEMETRY_PLUGIN_NAME, +} from './consent-resolver.ts' +export type { + ConsentDecision, + ConsentReason, + ConsentResolverOptions, +} from './consent-resolver.ts' +export { + ANONYMOUS_ID_FILE_NAME, + getOrCreateAnonymousId, + globalConfigDir, +} from './anonymous-id.ts' +export type { AnonymousId, AnonymousIdOptions } from './anonymous-id.ts' +export { buildTelemetryPayload } from './payload.ts' +export type { BuildTelemetryPayloadInput, TelemetryPayload } from './payload.ts' +export { + DEFAULT_FLUSH_TIMEOUT_MS, + DEFAULT_SEND_TIMEOUT_MS, + DSH_TELEMETRY_ENDPOINT, + TELEMETRY_SCHEMA_VERSION, + TelemetryReporter, +} from './reporter.ts' +export type { DeliveryOutcome, TelemetryReporterOptions } from './reporter.ts' diff --git a/packages/sdk/telemetry/src/payload.ts b/packages/sdk/telemetry/src/payload.ts new file mode 100644 index 0000000000..96a7b19d76 --- /dev/null +++ b/packages/sdk/telemetry/src/payload.ts @@ -0,0 +1,75 @@ +/** + * Telemetry payload assembly. + * + * The payload carries the command lifecycle plus the FULL redacted content of + * the project `cordis.yml` and `package.json`. It NEVER reads or includes `.env` + * — secrets live only in `.env`, and the redactor is the backstop for any that + * leak into the two reported files. A file that does not exist (the first + * `create` run) simply omits its field. + * + * @module @deepseek-ai/dsh-telemetry/payload + */ + +import { readFile } from 'node:fs/promises' +import { join } from 'node:path' +import { SecretRedactor } from './secret-redactor.ts' + +/** Project files whose full (redacted) content ships with the payload. */ +const REPORTED_FILES = ['cordis.yml', 'package.json'] as const + +/** One command's telemetry payload. */ +export interface TelemetryPayload { + /** The dsh-sdk command that ran (`start`/`dev`/`build`/`config`/`create`). */ + command: string + /** Wall-clock duration of the command in milliseconds. */ + durationMs: number + /** Whether the command completed without error. */ + success: boolean + /** Redacted full text of the project `cordis.yml`, absent when the file does not exist. */ + cordisYmlContent?: string + /** Redacted full text of the project `package.json`, absent when the file does not exist. */ + packageJsonContent?: string +} + +/** Inputs for {@link buildTelemetryPayload}. */ +export interface BuildTelemetryPayloadInput { + /** The dsh-sdk command that ran. */ + command: string + /** Wall-clock duration of the command in milliseconds. */ + durationMs: number + /** Whether the command completed without error. */ + success: boolean + /** Project root whose `cordis.yml` and `package.json` are read. */ + projectDir: string + /** Redactor applied to reported file content; defaults to a fresh {@link SecretRedactor}. */ + redactor?: SecretRedactor +} + +/** Read a project file's text, returning `undefined` when it cannot be read. */ +async function readReportedFile(projectDir: string, name: string): Promise { + try { + return await readFile(join(projectDir, name), 'utf8') + } catch { + // Missing/unreadable reported file: telemetry omits the field rather than fail. + return undefined + } +} + +/** + * Assemble a redacted telemetry payload for one command invocation. + * @param input - command lifecycle facts, project directory, and optional redactor. + * @returns the payload with redacted `cordis.yml`/`package.json` content. + */ +export async function buildTelemetryPayload(input: BuildTelemetryPayloadInput): Promise { + const redactor = input.redactor ?? new SecretRedactor() + const [cordisYml, packageJson] = await Promise.all( + REPORTED_FILES.map(name => readReportedFile(input.projectDir, name)), + ) + return { + command: input.command, + durationMs: input.durationMs, + success: input.success, + ...cordisYml !== undefined ? { cordisYmlContent: redactor.redactText(cordisYml) } : {}, + ...packageJson !== undefined ? { packageJsonContent: redactor.redactText(packageJson) } : {}, + } +} diff --git a/packages/sdk/telemetry/src/reporter.ts b/packages/sdk/telemetry/src/reporter.ts new file mode 100644 index 0000000000..d41c1db9b7 --- /dev/null +++ b/packages/sdk/telemetry/src/reporter.ts @@ -0,0 +1,149 @@ +/** + * Fire-and-forget telemetry reporter for the dsh-sdk launcher. + * + * The reporter must NEVER block or crash a command: {@link TelemetryReporter.report} + * schedules a detached send and returns immediately, and the underlying delivery + * resolves on every path (consent skip, network failure, non-OK status) instead + * of rejecting. {@link TelemetryReporter.flush} lets the launcher optionally + * drain in-flight sends within a cap before exit. + * + * @module @deepseek-ai/dsh-telemetry/reporter + */ + +import type { ConsentDecision } from './consent-resolver.ts' +import type { TelemetryPayload } from './payload.ts' +import { getOrCreateAnonymousId, type AnonymousId } from './anonymous-id.ts' +import { SecretRedactor } from './secret-redactor.ts' + +/** + * Placeholder collection endpoint. This is a fixed protocol constant, not a + * deployment tunable. + * + * FIXME(ccyu): replace with the real telemetry endpoint before release. The + * `.invalid` TLD guarantees delivery fails harmlessly until then. + */ +export const DSH_TELEMETRY_ENDPOINT = 'https://telemetry.example.invalid/v1/dsh-sdk' + +/** Wire-envelope schema version; bump on any incompatible body change. */ +export const TELEMETRY_SCHEMA_VERSION = 1 + +/** Default per-request send timeout in milliseconds. */ +export const DEFAULT_SEND_TIMEOUT_MS = 3000 + +/** Default cap for {@link TelemetryReporter.flush} in milliseconds. */ +export const DEFAULT_FLUSH_TIMEOUT_MS = 2000 + +/** Outcome of one delivery attempt; delivery never rejects. */ +export type DeliveryOutcome = + | { status: 'skipped'; reason: string } + | { status: 'sent' } + | { status: 'failed'; error: string } + +/** The JSON body posted to the telemetry endpoint. */ +interface TelemetryEnvelope extends TelemetryPayload { + schemaVersion: number + anonymousId: AnonymousId + sentAt: string +} + +/** Injectable seams for {@link TelemetryReporter}; every field has a default. */ +export interface TelemetryReporterOptions { + /** Collection endpoint; defaults to {@link DSH_TELEMETRY_ENDPOINT}. */ + endpoint?: string + /** `fetch` implementation; defaults to the global `fetch`. */ + fetch?: typeof globalThis.fetch + /** Anonymous-id provider; defaults to {@link getOrCreateAnonymousId}. */ + anonymousId?: () => Promise + /** Redactor applied to the assembled envelope as a final backstop; defaults to a fresh {@link SecretRedactor}. */ + redactor?: SecretRedactor + /** Per-request send timeout in milliseconds. */ + timeoutMs?: number + /** Clock for the envelope timestamp; defaults to `Date.now`. */ + now?: () => number +} + +/** Sends telemetry payloads fire-and-forget, swallowing every failure. */ +export class TelemetryReporter { + readonly #endpoint: string + readonly #fetch: typeof globalThis.fetch + readonly #anonymousId: () => Promise + readonly #redactor: SecretRedactor + readonly #timeoutMs: number + readonly #now: () => number + readonly #inflight = new Set>() + + /** @param options - endpoint, transport, id provider, and timing seams. */ + constructor(options: TelemetryReporterOptions = {}) { + this.#endpoint = options.endpoint ?? DSH_TELEMETRY_ENDPOINT + this.#fetch = options.fetch ?? globalThis.fetch + this.#anonymousId = options.anonymousId ?? getOrCreateAnonymousId + this.#redactor = options.redactor ?? new SecretRedactor() + this.#timeoutMs = options.timeoutMs ?? DEFAULT_SEND_TIMEOUT_MS + this.#now = options.now ?? Date.now + } + + /** + * Schedule a detached, non-blocking send. Returns immediately and never + * throws; the send's outcome is observable only through {@link flush}. + * @param payload - the command payload to report. + * @param consent - resolved consent; a denial short-circuits to a skip. + */ + report(payload: TelemetryPayload, consent: ConsentDecision): void { + const pending = this.#deliver(payload, consent) + this.#inflight.add(pending) + void pending.finally(() => this.#inflight.delete(pending)) + } + + /** + * Await in-flight sends up to a timeout so a caller can drain before exit. + * Resolves on the cap regardless of send progress; never rejects. + * @param timeoutMs - maximum time to wait; defaults to {@link DEFAULT_FLUSH_TIMEOUT_MS}. + */ + async flush(timeoutMs: number = DEFAULT_FLUSH_TIMEOUT_MS): Promise { + if (this.#inflight.size === 0) return + const drained = Promise.allSettled([...this.#inflight]).then(() => undefined) + let timer!: ReturnType + const capped = new Promise((resolve) => { + timer = setTimeout(resolve, timeoutMs) + }) + try { + await Promise.race([drained, capped]) + } finally { + clearTimeout(timer) + } + } + + /** Deliver one payload, resolving to an outcome on every path (never rejects). */ + async #deliver(payload: TelemetryPayload, consent: ConsentDecision): Promise { + if (!consent.allowed) return { status: 'skipped', reason: consent.reason } + try { + const envelope: TelemetryEnvelope = { + schemaVersion: TELEMETRY_SCHEMA_VERSION, + anonymousId: await this.#anonymousId(), + sentAt: new Date(this.#now()).toISOString(), + ...payload, + // Idempotent backstop over the only free-form fields, in case a caller + // built the payload without buildTelemetryPayload. Applied to content + // text only so the anonymous id and metadata are never disturbed. + ...payload.cordisYmlContent !== undefined + ? { cordisYmlContent: this.#redactor.redactText(payload.cordisYmlContent) } + : {}, + ...payload.packageJsonContent !== undefined + ? { packageJsonContent: this.#redactor.redactText(payload.packageJsonContent) } + : {}, + } + const response = await this.#fetch(this.#endpoint, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(envelope), + signal: AbortSignal.timeout(this.#timeoutMs), + }) + if (!response.ok) return { status: 'failed', error: `HTTP ${response.status}` } + return { status: 'sent' } + } catch (error) { + // Telemetry is best-effort: network faults, aborts, and id/redaction + // errors are swallowed so the command is never affected. + return { status: 'failed', error: error instanceof Error ? error.message : String(error) } + } + } +} diff --git a/packages/sdk/telemetry/src/secret-redactor.ts b/packages/sdk/telemetry/src/secret-redactor.ts new file mode 100644 index 0000000000..5b9af5f169 --- /dev/null +++ b/packages/sdk/telemetry/src/secret-redactor.ts @@ -0,0 +1,203 @@ +/** + * Conservative secret redactor: the safety backstop that scrubs credential-like + * values from telemetry content before it leaves the machine. + * + * The redactor never drops a field or line — it only replaces the secret-shaped + * VALUE with a fixed placeholder, so the surrounding structure (keys, package + * names, base URLs, dependency pins) stays intact for the maintainer. It leans + * toward redaction on strong signals (secret-like key names, known token + * shapes, PEM blocks, URL credentials, high-entropy opaque tokens) while + * deliberately leaving low-signal values (package names, versions, git SHAs, + * plain URLs, kebab identifiers) untouched, because those are exactly the + * signal telemetry exists to capture. + * + * @module @deepseek-ai/dsh-telemetry/secret-redactor + */ + +/** Default text substituted for a detected secret. */ +export const DEFAULT_REDACTION_PLACEHOLDER = '[REDACTED]' + +/** Default minimum length for the high-entropy opaque-token heuristic. */ +export const DEFAULT_MIN_TOKEN_LENGTH = 24 + +/** Default Shannon-entropy threshold (bits/char) that marks an opaque token secret. */ +export const DEFAULT_ENTROPY_THRESHOLD = 4 + +/** Tuning for {@link SecretRedactor}; every field defaults to a documented constant. */ +export interface SecretRedactorOptions { + /** Replacement text for a detected secret. */ + placeholder?: string + /** Minimum length before the high-entropy heuristic considers an opaque token. */ + minTokenLength?: number + /** Shannon entropy (bits/char) at or above which an opaque token is treated as secret. */ + entropyThreshold?: number +} + +/** + * Regexes for well-known credential shapes. A match anywhere in a candidate + * token marks it secret regardless of length, so short-but-recognizable tokens + * are caught even when the entropy heuristic would not fire. + */ +const KNOWN_SECRET_PATTERNS: readonly RegExp[] = [ + /sk-(?:ant-)?[A-Za-z0-9_-]{10,}/, // OpenAI / DeepSeek / Anthropic style + /gh[pousr]_[A-Za-z0-9]{16,}/, // GitHub personal/oauth/server/refresh tokens + /github_pat_[A-Za-z0-9_]{20,}/, // GitHub fine-grained PAT + /xox[baprs]-[A-Za-z0-9-]{10,}/, // Slack tokens + /AKIA[0-9A-Z]{16}/, // AWS access key id + /AIza[0-9A-Za-z_-]{35}/, // Google API key + /eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/, // JWT +] + +/** + * Key names (normalized to lowercase, separators stripped) whose value is a + * secret. Split by match strategy so short/ambiguous words do not over-match: + * `author` must not trip the `auth` rule. + */ +const KEY_SUBSTRING_INDICATORS: readonly string[] = [ + 'password', 'passwd', 'passphrase', 'secret', 'apikey', 'apisecret', + 'clientsecret', 'privatekey', 'secretkey', 'accesskey', 'credential', + 'connectionstring', 'sastoken', 'xapikey', 'authtoken', 'accesstoken', + 'refreshtoken', 'idtoken', 'sessiontoken', 'bearertoken', +] +const KEY_SUFFIX_INDICATORS: readonly string[] = ['token'] +const KEY_EXACT_INDICATORS: readonly string[] = [ + 'auth', 'authorization', 'cookie', 'bearer', 'dsn', 'signature', +] + +/** + * Whether a key name marks its value as a secret. + * @param key - raw object key or assignment name. + * @returns whether the value under this key must be redacted. + */ +export function keyLooksSecret(key: string): boolean { + const normalized = key.toLowerCase().replace(/[^a-z0-9]/g, '') + if (normalized.length === 0) return false + if (KEY_SUBSTRING_INDICATORS.some(indicator => normalized.includes(indicator))) return true + if (KEY_SUFFIX_INDICATORS.some(indicator => normalized.endsWith(indicator))) return true + return KEY_EXACT_INDICATORS.includes(normalized) +} + +/** Shannon entropy in bits per character. */ +function shannonEntropy(value: string): number { + const counts = new Map() + for (const char of value) counts.set(char, (counts.get(char) ?? 0) + 1) + let entropy = 0 + for (const count of counts.values()) { + const probability = count / value.length + entropy -= probability * Math.log2(probability) + } + return entropy +} + +/** Opaque-token character set (base64/base64url plus common token punctuation). */ +const OPAQUE_TOKEN = /^[A-Za-z0-9+/=_.-]+$/ +/** Version-like leader kept visible (dependency pins, semver). */ +const VERSION_LIKE = /^v?\d+(?:\.\d+)+/ + +/** + * Conservative secret detector and redactor for telemetry content. + * Detection is a pure function of the input; construction only fixes tunables. + */ +export class SecretRedactor { + readonly #placeholder: string + readonly #minTokenLength: number + readonly #entropyThreshold: number + + /** @param options - placeholder text and heuristic thresholds. */ + constructor(options: SecretRedactorOptions = {}) { + this.#placeholder = options.placeholder ?? DEFAULT_REDACTION_PLACEHOLDER + this.#minTokenLength = options.minTokenLength ?? DEFAULT_MIN_TOKEN_LENGTH + this.#entropyThreshold = options.entropyThreshold ?? DEFAULT_ENTROPY_THRESHOLD + } + + /** + * Whether a standalone token value looks like a secret. + * @param value - candidate token, already trimmed of surrounding quotes. + * @returns whether the value should be redacted on its own merits. + */ + isSecretValue(value: string): boolean { + if (KNOWN_SECRET_PATTERNS.some(pattern => pattern.test(value))) return true + if (value.length < this.#minTokenLength) return false + if (!OPAQUE_TOKEN.test(value)) return false + // Git SHAs and integrity digests are hex and public — never a secret we hide. + if (/^[0-9a-fA-F]+$/.test(value)) return false + if (VERSION_LIKE.test(value)) return false + const classes = (/[a-z]/.test(value) ? 1 : 0) + (/[A-Z]/.test(value) ? 1 : 0) + (/[0-9]/.test(value) ? 1 : 0) + return classes >= 3 || shannonEntropy(value) >= this.#entropyThreshold + } + + /** + * Deep-redact a parsed value in place-safe fashion, returning a new structure. + * A secret-named key redacts its string value outright; every other string is + * judged on its own shape. Non-string leaves pass through untouched. + * @param value - parsed JSON-like value (object, array, or primitive). + * @returns a structurally identical value with secret strings replaced. + */ + redactValue(value: T): T { + return this.#redactNode(value, false) as T + } + + #redactNode(value: unknown, keyIsSecret: boolean): unknown { + if (typeof value === 'string') { + return keyIsSecret || this.isSecretValue(value) ? this.#placeholder : value + } + if (Array.isArray(value)) return value.map(item => this.#redactNode(item, false)) + if (value !== null && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, child]) => [key, this.#redactNode(child, keyLooksSecret(key))]), + ) + } + return value + } + + /** + * Redact secrets embedded in raw text (YAML, JSON, or `.env`-style content), + * preserving every line and key while replacing only secret-shaped values. + * @param text - raw file or message text. + * @returns text with detected secrets replaced by the placeholder. + */ + redactText(text: string): string { + let output = this.#redactPemBlocks(text) + output = this.#redactAssignments(output) + output = this.#redactUrlCredentials(output) + output = this.#redactBearerTokens(output) + return this.#redactStandaloneTokens(output) + } + + #redactPemBlocks(text: string): string { + return text.replace( + /-----BEGIN (?:[A-Z ]+ )?PRIVATE KEY-----[\s\S]*?-----END (?:[A-Z ]+ )?PRIVATE KEY-----/g, + this.#placeholder, + ) + } + + #redactAssignments(text: string): string { + // `key: value`, `key = value`, or `"key": "value"` across YAML/JSON/.env. + return text.replace( + /("?)([A-Za-z0-9_.-]+)\1(\s*[:=]\s*)(["']?)([^\n\r"']+)\4/g, + (match, keyQuote: string, key: string, separator: string, valueQuote: string, value: string) => + keyLooksSecret(key) && value.trim().length > 0 + ? `${keyQuote}${key}${keyQuote}${separator}${valueQuote}${this.#placeholder}${valueQuote}` + : match, + ) + } + + #redactUrlCredentials(text: string): string { + // Redact only the password in `scheme://user:password@host`, keeping host visible. + return text.replace( + /([a-z][a-z0-9+.-]*:\/\/[^\s:/@]+:)([^\s/@]+)(@)/gi, + (_match, prefix: string, _password: string, at: string) => `${prefix}${this.#placeholder}${at}`, + ) + } + + #redactBearerTokens(text: string): string { + return text.replace(/(bearer\s+)([a-z0-9._-]{8,})/gi, (_match, prefix: string) => `${prefix}${this.#placeholder}`) + } + + #redactStandaloneTokens(text: string): string { + // `/` is excluded so package names, file paths, and URLs are never split or + // redacted; a secret containing `/` is still scrubbed piecewise. + return text.replace(/[A-Za-z0-9][A-Za-z0-9+=_.-]{7,}/g, token => + this.isSecretValue(token) ? this.#placeholder : token) + } +} diff --git a/packages/sdk/telemetry/tests/anonymous-id.spec.ts b/packages/sdk/telemetry/tests/anonymous-id.spec.ts new file mode 100644 index 0000000000..df8bcffea2 --- /dev/null +++ b/packages/sdk/telemetry/tests/anonymous-id.spec.ts @@ -0,0 +1,100 @@ +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + ANONYMOUS_ID_FILE_NAME, + getOrCreateAnonymousId, + globalConfigDir, +} from '@deepseek-ai/dsh-telemetry' + +const dirs: string[] = [] + +async function tempDir(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-anon-')) + dirs.push(dir) + return dir +} + +afterEach(async () => { + await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) +}) + +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + +describe('globalConfigDir', () => { + it('prefers an explicit DSH_CONFIG_HOME override', () => { + expect(globalConfigDir({ env: { DSH_CONFIG_HOME: '/custom/dsh' } })).toBe('/custom/dsh') + }) + + it('falls back to XDG_CONFIG_HOME under the harness namespace', () => { + expect(globalConfigDir({ env: { XDG_CONFIG_HOME: '/xdg' } })).toBe(join('/xdg', 'deepseek-harness')) + }) + + it('uses %APPDATA% on Windows', () => { + expect(globalConfigDir({ env: { APPDATA: 'C:/Users/x/AppData/Roaming' }, platform: 'win32' })) + .toBe(join('C:/Users/x/AppData/Roaming', 'deepseek-harness')) + }) + + it('falls back to ~/.config on Windows without APPDATA and on posix', () => { + const home = () => '/home/dev' + expect(globalConfigDir({ env: {}, platform: 'win32', homeDir: home })) + .toBe(join('/home/dev', '.config', 'deepseek-harness')) + expect(globalConfigDir({ env: {}, platform: 'linux', homeDir: home })) + .toBe(join('/home/dev', '.config', 'deepseek-harness')) + }) + + it('reads process.env by default', () => { + // No override supplied: the call must not throw and must return an absolute path. + expect(globalConfigDir()).toContain('deepseek-harness') + }) +}) + +describe('getOrCreateAnonymousId', () => { + it('creates, persists, and returns a UUID on first use', async () => { + const dir = await tempDir() + const id = await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } }) + expect(id).toMatch(UUID) + const stored: unknown = JSON.parse(await readFile(join(dir, ANONYMOUS_ID_FILE_NAME), 'utf8')) + expect(stored).toEqual({ anonymousId: id }) + }) + + it('returns the same persisted id on subsequent calls', async () => { + const dir = await tempDir() + const first = await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } }) + const second = await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } }) + expect(second).toBe(first) + }) + + it('uses the injected UUID generator', async () => { + const dir = await tempDir() + const id = await getOrCreateAnonymousId({ + env: { DSH_CONFIG_HOME: dir }, + randomUUID: () => '00000000-0000-4000-8000-000000000000', + }) + expect(id).toBe('00000000-0000-4000-8000-000000000000') + }) + + it('regenerates when the stored file is corrupt JSON', async () => { + const dir = await tempDir() + await writeFile(join(dir, ANONYMOUS_ID_FILE_NAME), 'not json', 'utf8') + const id = await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } }) + expect(id).toMatch(UUID) + }) + + it('regenerates when the stored value is not a valid UUID or object', async () => { + const dir = await tempDir() + await writeFile(join(dir, ANONYMOUS_ID_FILE_NAME), JSON.stringify({ anonymousId: 'nope' }), 'utf8') + expect(await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } })).toMatch(UUID) + await writeFile(join(dir, ANONYMOUS_ID_FILE_NAME), '123', 'utf8') + expect(await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: dir } })).toMatch(UUID) + }) + + it('returns a usable id even when persistence fails', async () => { + const dir = await tempDir() + // A regular file where a directory is expected makes mkdir/writeFile fail. + await writeFile(join(dir, 'blocker'), 'x', 'utf8') + const id = await getOrCreateAnonymousId({ env: { DSH_CONFIG_HOME: join(dir, 'blocker') } }) + expect(id).toMatch(UUID) + }) +}) diff --git a/packages/sdk/telemetry/tests/consent-resolver.spec.ts b/packages/sdk/telemetry/tests/consent-resolver.spec.ts new file mode 100644 index 0000000000..e1f5f90f81 --- /dev/null +++ b/packages/sdk/telemetry/tests/consent-resolver.spec.ts @@ -0,0 +1,131 @@ +import { mkdtemp, mkdir, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { ConsentResolver, DEFAULT_TELEMETRY_PLUGIN_NAME, type ConsentDecision } from '@deepseek-ai/dsh-telemetry' + +const dirs: string[] = [] + +async function projectDir(cordisYml?: string): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-consent-')) + dirs.push(dir) + if (cordisYml !== undefined) await writeFile(join(dir, 'cordis.yml'), cordisYml, 'utf8') + return dir +} + +afterEach(async () => { + await Promise.all(dirs.splice(0).map(dir => import('node:fs/promises').then(fs => fs.rm(dir, { recursive: true, force: true })))) +}) + +const enabledYml = `- id: telemetry\n name: '${DEFAULT_TELEMETRY_PLUGIN_NAME}'\n` + +describe('ConsentResolver environment opt-out', () => { + it('denies when DO_NOT_TRACK is set', async () => { + const decision = await new ConsentResolver({ env: { DO_NOT_TRACK: '1' } }).resolve(await projectDir(enabledYml)) + expect(decision).toEqual({ allowed: false, reason: 'do-not-track' }) + }) + + it('denies when CI is set', async () => { + const decision = await new ConsentResolver({ env: { CI: 'true' } }).resolve(await projectDir(enabledYml)) + expect(decision).toEqual({ allowed: false, reason: 'ci' }) + }) + + it('ignores falsy env values and continues to the file', async () => { + const decision = await new ConsentResolver({ env: { DO_NOT_TRACK: '0', CI: 'false' } }) + .resolve(await projectDir(enabledYml)) + expect(decision).toEqual({ allowed: true, reason: 'enabled' }) + }) + + it('can be told to ignore env opt-out signals', async () => { + const decision = await new ConsentResolver({ env: { DO_NOT_TRACK: '1' }, honorEnvOptOut: false }) + .resolve(await projectDir(enabledYml)) + expect(decision).toEqual({ allowed: true, reason: 'enabled' }) + }) + + it('reads process.env by default', async () => { + const saved = { CI: process.env.CI, DO_NOT_TRACK: process.env.DO_NOT_TRACK } + delete process.env.CI + delete process.env.DO_NOT_TRACK + try { + const decision = await new ConsentResolver().resolve(await projectDir(enabledYml)) + expect(decision).toEqual({ allowed: true, reason: 'enabled' }) + } finally { + if (saved.CI !== undefined) process.env.CI = saved.CI + if (saved.DO_NOT_TRACK !== undefined) process.env.DO_NOT_TRACK = saved.DO_NOT_TRACK + } + }) +}) + +describe('ConsentResolver cordis.yml state', () => { + const resolver = new ConsentResolver({ env: {} }) + + it('allows when the telemetry entry is enabled', async () => { + expect(await resolver.resolve(await projectDir(enabledYml))) + .toEqual({ allowed: true, reason: 'enabled' }) + }) + + it('denies when the telemetry entry is disabled', async () => { + const yml = `- id: telemetry\n name: '${DEFAULT_TELEMETRY_PLUGIN_NAME}'\n disabled: true\n` + expect(await resolver.resolve(await projectDir(yml))) + .toEqual({ allowed: false, reason: 'disabled' }) + }) + + it('tolerates !!js expression tags while reading plain scalars', async () => { + const yml = [ + '- id: telemetry', + ` name: '${DEFAULT_TELEMETRY_PLUGIN_NAME}'`, + '- id: llm', + ' name: \'@deepseek-ai/dsh-llm-deepseek\'', + ' config:', + ' apiKey: !!js process.env.DEEPSEEK_API_KEY', + '', + ].join('\n') + expect(await resolver.resolve(await projectDir(yml))) + .toEqual({ allowed: true, reason: 'enabled' }) + }) + + it('reports absent when cordis.yml has no telemetry entry, defaulting to deny', async () => { + const yml = '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n' + expect(await resolver.resolve(await projectDir(yml))) + .toEqual({ allowed: false, reason: 'absent' }) + }) + + it('can allow when the entry is absent', async () => { + const yml = '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n' + const decision = await new ConsentResolver({ env: {}, allowWhenEntryAbsent: true }).resolve(await projectDir(yml)) + expect(decision).toEqual({ allowed: true, reason: 'absent' }) + }) + + it('skips non-object sequence items and a non-sequence root', async () => { + expect(await resolver.resolve(await projectDir('- just-a-string\n- id: x\n name: y\n'))) + .toEqual({ allowed: false, reason: 'absent' }) + expect(await resolver.resolve(await projectDir('root: not-a-sequence\n'))) + .toEqual({ allowed: false, reason: 'absent' }) + }) + + it('honors a custom telemetry plugin name', async () => { + const yml = '- id: t\n name: \'my-consent-marker\'\n' + const decision = await new ConsentResolver({ env: {}, telemetryPluginName: 'my-consent-marker' }) + .resolve(await projectDir(yml)) + expect(decision).toEqual({ allowed: true, reason: 'enabled' }) + }) +}) + +describe('ConsentResolver missing or unreadable cordis.yml', () => { + it('reports no-config and allows by default on first init', async () => { + expect(await new ConsentResolver({ env: {} }).resolve(await projectDir())) + .toEqual({ allowed: true, reason: 'no-config' }) + }) + + it('can deny on first init', async () => { + const decision = await new ConsentResolver({ env: {}, allowWhenNoConfig: false }).resolve(await projectDir()) + expect(decision).toEqual({ allowed: false, reason: 'no-config' }) + }) + + it('denies with an unreadable reason when cordis.yml is not a regular file', async () => { + const dir = await projectDir() + await mkdir(join(dir, 'cordis.yml')) // a directory where the resolver expects a file + expect(await new ConsentResolver({ env: {} }).resolve(dir)) + .toEqual({ allowed: false, reason: 'unreadable' }) + }) +}) diff --git a/packages/sdk/telemetry/tests/payload.spec.ts b/packages/sdk/telemetry/tests/payload.spec.ts new file mode 100644 index 0000000000..1af5139ef5 --- /dev/null +++ b/packages/sdk/telemetry/tests/payload.spec.ts @@ -0,0 +1,59 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { SecretRedactor, buildTelemetryPayload } from '@deepseek-ai/dsh-telemetry' + +const dirs: string[] = [] + +async function projectDir(files: Record): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-payload-')) + dirs.push(dir) + await Promise.all(Object.entries(files).map(([name, content]) => writeFile(join(dir, name), content, 'utf8'))) + return dir +} + +afterEach(async () => { + await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) +}) + +describe('buildTelemetryPayload', () => { + it('carries lifecycle facts and redacted file content', async () => { + const dir = await projectDir({ + 'cordis.yml': '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n config:\n apiKey: sk-abcdefghij1234567890\n', + 'package.json': '{ "name": "my-app", "config": { "token": "sk-abcdefghij1234567890" } }', + }) + const payload = await buildTelemetryPayload({ command: 'build', durationMs: 42, success: true, projectDir: dir }) + expect(payload.command).toBe('build') + expect(payload.durationMs).toBe(42) + expect(payload.success).toBe(true) + expect(payload.cordisYmlContent).toContain('@deepseek-ai/dsh-llm-deepseek') // package name preserved + expect(payload.cordisYmlContent).not.toContain('sk-abcdefghij1234567890') // secret scrubbed + expect(payload.packageJsonContent).toContain('my-app') + expect(payload.packageJsonContent).not.toContain('sk-abcdefghij1234567890') + }) + + it('omits fields whose files do not exist', async () => { + const dir = await projectDir({ 'cordis.yml': '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n' }) + const payload = await buildTelemetryPayload({ command: 'create', durationMs: 1, success: false, projectDir: dir }) + expect(payload.cordisYmlContent).toBeDefined() + expect('packageJsonContent' in payload).toBe(false) + }) + + it('omits both fields when neither file exists', async () => { + const dir = await projectDir({}) + const payload = await buildTelemetryPayload({ command: 'create', durationMs: 0, success: true, projectDir: dir }) + expect('cordisYmlContent' in payload).toBe(false) + expect('packageJsonContent' in payload).toBe(false) + }) + + it('uses a supplied redactor', async () => { + const dir = await projectDir({ 'package.json': '{ "password": "hunter2" }' }) + const redactor = new SecretRedactor({ placeholder: '<>' }) + const payload = await buildTelemetryPayload({ + command: 'config', durationMs: 5, success: true, projectDir: dir, redactor, + }) + expect(payload.packageJsonContent).toContain('<>') + expect(payload.packageJsonContent).not.toContain('hunter2') + }) +}) diff --git a/packages/sdk/telemetry/tests/reporter.spec.ts b/packages/sdk/telemetry/tests/reporter.spec.ts new file mode 100644 index 0000000000..5d8a490b9e --- /dev/null +++ b/packages/sdk/telemetry/tests/reporter.spec.ts @@ -0,0 +1,134 @@ +import { describe, expect, it, vi } from 'vitest' +import { + DSH_TELEMETRY_ENDPOINT, + SecretRedactor, + TELEMETRY_SCHEMA_VERSION, + TelemetryReporter, + type AnonymousId, + type ConsentDecision, + type TelemetryPayload, +} from '@deepseek-ai/dsh-telemetry' + +const ALLOW: ConsentDecision = { allowed: true, reason: 'enabled' } +const DENY: ConsentDecision = { allowed: false, reason: 'disabled' } +const anon = (value = 'anon-123'): (() => Promise) => async () => value as AnonymousId + +function okResponse(): Response { + return { ok: true } as Response +} + +describe('TelemetryReporter.report', () => { + it('skips delivery when consent is denied', async () => { + const fetchMock = vi.fn(async () => okResponse()) + const reporter = new TelemetryReporter({ fetch: fetchMock, anonymousId: anon() }) + reporter.report({ command: 'build', durationMs: 1, success: true }, DENY) + await reporter.flush(50) + expect(fetchMock).not.toHaveBeenCalled() + }) + + it('posts a redacted envelope when consent is granted', async () => { + const fetchMock = vi.fn(() => Promise.resolve(okResponse())) + const reporter = new TelemetryReporter({ + endpoint: 'https://collector.test/telemetry', + fetch: fetchMock, + anonymousId: anon('anon-xyz'), + redactor: new SecretRedactor(), + now: () => 0, + timeoutMs: 100, + }) + const payload: TelemetryPayload = { + command: 'config', + durationMs: 7, + success: true, + cordisYmlContent: 'apiKey: sk-abcdefghij1234567890\nname: \'@deepseek-ai/dsh-llm-deepseek\'\n', + packageJsonContent: '{ "name": "app" }', + } + reporter.report(payload, ALLOW) + await reporter.flush(50) + + expect(fetchMock).toHaveBeenCalledTimes(1) + const call = fetchMock.mock.calls[0]! + expect(call[0]).toBe('https://collector.test/telemetry') + const init = call[1]! + expect(init.method).toBe('POST') + const body = JSON.parse(init.body as string) as Record + expect(body.schemaVersion).toBe(TELEMETRY_SCHEMA_VERSION) + expect(body.anonymousId).toBe('anon-xyz') + expect(body.sentAt).toBe('1970-01-01T00:00:00.000Z') + expect(body.command).toBe('config') + expect(body.cordisYmlContent).not.toContain('sk-abcdefghij1234567890') + expect(body.cordisYmlContent).toContain('@deepseek-ai/dsh-llm-deepseek') + expect(body.packageJsonContent).toContain('app') + }) + + it('posts an envelope without content fields when they are absent', async () => { + const fetchMock = vi.fn(() => Promise.resolve(okResponse())) + const reporter = new TelemetryReporter({ fetch: fetchMock, anonymousId: anon(), now: () => 0, timeoutMs: 100 }) + reporter.report({ command: 'start', durationMs: 2, success: true }, ALLOW) + await reporter.flush(50) + const body = JSON.parse(fetchMock.mock.calls[0]![1]!.body as string) as Record + expect('cordisYmlContent' in body).toBe(false) + expect('packageJsonContent' in body).toBe(false) + }) + + it('swallows a non-OK HTTP status', async () => { + const fetchMock = vi.fn(async () => ({ ok: false, status: 503 } as Response)) + const reporter = new TelemetryReporter({ fetch: fetchMock, anonymousId: anon(), timeoutMs: 100 }) + reporter.report({ command: 'dev', durationMs: 3, success: true }, ALLOW) + await expect(reporter.flush(50)).resolves.toBeUndefined() + expect(fetchMock).toHaveBeenCalledTimes(1) + }) + + it('swallows a transport failure', async () => { + const fetchMock = vi.fn(async () => { throw new Error('network down') }) + const reporter = new TelemetryReporter({ fetch: fetchMock, anonymousId: anon(), timeoutMs: 100 }) + reporter.report({ command: 'dev', durationMs: 3, success: false }, ALLOW) + await expect(reporter.flush(50)).resolves.toBeUndefined() + }) + + it('swallows a non-Error transport rejection', async () => { + const fetchMock = vi.fn(async () => { throw 'boom' }) + const reporter = new TelemetryReporter({ fetch: fetchMock, anonymousId: anon(), timeoutMs: 100 }) + reporter.report({ command: 'dev', durationMs: 3, success: false }, ALLOW) + await expect(reporter.flush(50)).resolves.toBeUndefined() + }) + + it('swallows a failure while resolving the anonymous id, never sending', async () => { + const fetchMock = vi.fn(async () => okResponse()) + const reporter = new TelemetryReporter({ + fetch: fetchMock, + anonymousId: async () => { throw new Error('config unwritable') }, + timeoutMs: 100, + }) + reporter.report({ command: 'build', durationMs: 1, success: true }, ALLOW) + await reporter.flush(50) + expect(fetchMock).not.toHaveBeenCalled() + }) +}) + +describe('TelemetryReporter.flush', () => { + it('returns immediately when nothing is in flight', async () => { + const reporter = new TelemetryReporter({ fetch: vi.fn(async () => okResponse()), anonymousId: anon() }) + await expect(reporter.flush()).resolves.toBeUndefined() + }) + + it('resolves on the timeout cap when a send never settles', async () => { + const reporter = new TelemetryReporter({ + fetch: () => new Promise(() => {}), + anonymousId: anon(), + timeoutMs: 10, + }) + reporter.report({ command: 'start', durationMs: 1, success: true }, ALLOW) + const started = Date.now() + await reporter.flush(15) + expect(Date.now() - started).toBeLessThan(1000) + }) +}) + +describe('TelemetryReporter defaults', () => { + it('defaults the endpoint and transport seams without options', () => { + const reporter = new TelemetryReporter() + expect(reporter).toBeInstanceOf(TelemetryReporter) + expect(DSH_TELEMETRY_ENDPOINT).toContain('.invalid') + }) +}) diff --git a/packages/sdk/telemetry/tests/secret-redactor.spec.ts b/packages/sdk/telemetry/tests/secret-redactor.spec.ts new file mode 100644 index 0000000000..c3250d9ace --- /dev/null +++ b/packages/sdk/telemetry/tests/secret-redactor.spec.ts @@ -0,0 +1,169 @@ +import { describe, expect, it } from 'vitest' +import { + DEFAULT_ENTROPY_THRESHOLD, + DEFAULT_MIN_TOKEN_LENGTH, + DEFAULT_REDACTION_PLACEHOLDER, + SecretRedactor, + keyLooksSecret, +} from '@deepseek-ai/dsh-telemetry' + +const REDACTED = DEFAULT_REDACTION_PLACEHOLDER + +describe('exported defaults', () => { + it('expose the documented tunable defaults', () => { + expect(DEFAULT_REDACTION_PLACEHOLDER).toBe('[REDACTED]') + expect(DEFAULT_MIN_TOKEN_LENGTH).toBe(24) + expect(DEFAULT_ENTROPY_THRESHOLD).toBe(4) + }) +}) + +describe('keyLooksSecret', () => { + it('matches secret substrings across casings and separators', () => { + for (const key of ['password', 'API_KEY', 'apiKey', 'clientSecret', 'x-api-key', 'privateKey', 'CREDENTIALS']) { + expect(keyLooksSecret(key)).toBe(true) + } + }) + + it('matches *token as a suffix but not tokenizer', () => { + expect(keyLooksSecret('accessToken')).toBe(true) + expect(keyLooksSecret('token')).toBe(true) + expect(keyLooksSecret('tokenizer')).toBe(false) + }) + + it('matches short ambiguous words only as whole keys', () => { + expect(keyLooksSecret('auth')).toBe(true) + expect(keyLooksSecret('authorization')).toBe(true) + expect(keyLooksSecret('cookie')).toBe(true) + expect(keyLooksSecret('author')).toBe(false) + }) + + it('does not match ordinary config keys', () => { + for (const key of ['name', 'version', 'model', 'baseURL', 'timeout', 'path', 'pass']) { + expect(keyLooksSecret(key)).toBe(false) + } + }) + + it('returns false for a key with no alphanumerics', () => { + expect(keyLooksSecret('---')).toBe(false) + }) +}) + +describe('SecretRedactor.isSecretValue', () => { + const redactor = new SecretRedactor() + + it('detects known token shapes regardless of length', () => { + expect(redactor.isSecretValue('sk-abcdefghij1234567890')).toBe(true) + expect(redactor.isSecretValue('sk-ant-abcdefghij1234567890')).toBe(true) + expect(redactor.isSecretValue('ghp_abcdefghijklmnop1234')).toBe(true) + expect(redactor.isSecretValue('github_pat_abcdefghijklmnopqrst')).toBe(true) + expect(redactor.isSecretValue('xoxb-abcdefghij-klmno')).toBe(true) + expect(redactor.isSecretValue('AKIA1234567890ABCDEF')).toBe(true) + expect(redactor.isSecretValue(`AIza${'a'.repeat(35)}`)).toBe(true) + expect(redactor.isSecretValue('eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.abcdefghijklmnop')).toBe(true) + }) + + it('detects high-entropy opaque tokens with three character classes', () => { + // Non-hex letters keep it off the hex-digest exemption; three classes trip the rule. + expect(redactor.isSecretValue('zX9zX9zX9zX9zX9zX9zX9zX9')).toBe(true) + }) + + it('detects high-entropy opaque tokens by entropy even within two classes', () => { + // 30 distinct lowercase+digit chars: entropy ~4.9, only two classes. + const token = 'abcdefghijklmnopqrstuvwxyz0123' + expect(token.length).toBeGreaterThanOrEqual(DEFAULT_MIN_TOKEN_LENGTH) + expect(redactor.isSecretValue(token)).toBe(true) + }) + + it('leaves short values, non-opaque text, hex digests, and versions untouched', () => { + expect(redactor.isSecretValue('deepseek-chat')).toBe(false) // short + expect(redactor.isSecretValue('a token with spaces here!!')).toBe(false) // not opaque + expect(redactor.isSecretValue('a'.repeat(40))).toBe(false) // low entropy, one class + expect(redactor.isSecretValue('abcdef0123456789abcdef0123456789abcdef01')).toBe(false) // 40-hex git SHA + expect(redactor.isSecretValue('1.2.3.4.5.6.7.8.9.10.11.12')).toBe(false) // version-like + expect(redactor.isSecretValue('ZXQPZXQPZXQPZXQPZXQPZXQP')).toBe(false) // uppercase only, low entropy + }) + + it('honors a custom entropy threshold', () => { + const strict = new SecretRedactor({ entropyThreshold: 100 }) + // Two-class token can no longer trip the entropy branch under an impossible threshold. + expect(strict.isSecretValue('abcdefghijklmnopqrstuvwxyz0123')).toBe(false) + }) +}) + +describe('SecretRedactor.redactValue', () => { + const redactor = new SecretRedactor() + + it('redacts secret-keyed strings and secret-shaped strings, keeping structure', () => { + const result = redactor.redactValue({ + apiKey: 'short-not-shaped', + name: 'my-package', + token: 'sk-abcdefghij1234567890', + count: 3, + enabled: true, + missing: null, + nested: { password: 'p', note: 'plain text value' }, + list: ['harmless', 'sk-abcdefghij1234567890'], + }) + expect(result).toEqual({ + apiKey: REDACTED, // redacted by key even though the value is not secret-shaped + name: 'my-package', + token: REDACTED, + count: 3, + enabled: true, + missing: null, + nested: { password: REDACTED, note: 'plain text value' }, + list: ['harmless', REDACTED], + }) + }) + + it('redacts a top-level secret string and passes through primitives', () => { + expect(redactor.redactValue('sk-abcdefghij1234567890')).toBe(REDACTED) + expect(redactor.redactValue('plain')).toBe('plain') + expect(redactor.redactValue(42)).toBe(42) + expect(redactor.redactValue(null)).toBeNull() + }) +}) + +describe('SecretRedactor.redactText', () => { + const redactor = new SecretRedactor() + + it('redacts PEM private key blocks', () => { + const text = '-----BEGIN RSA PRIVATE KEY-----\nMIIabc\ndef==\n-----END RSA PRIVATE KEY-----' + expect(redactor.redactText(text)).toBe(REDACTED) + }) + + it('redacts secret-keyed assignments across YAML, JSON, and .env', () => { + expect(redactor.redactText('password: hunter2')).toBe(`password: ${REDACTED}`) + expect(redactor.redactText('apiKey: "sk-abcdefghij1234567890"')).toBe(`apiKey: "${REDACTED}"`) + expect(redactor.redactText('"token": "abcdefgh"')).toBe(`"token": "${REDACTED}"`) + expect(redactor.redactText('API_KEY=sk-abcdefghij1234567890')).toBe(`API_KEY=${REDACTED}`) + }) + + it('keeps non-secret assignments and whitespace-only secret values intact', () => { + expect(redactor.redactText('model: deepseek-chat')).toBe('model: deepseek-chat') + expect(redactor.redactText('password: \n')).toBe('password: \n') + }) + + it('redacts only the password in URL credentials, keeping the host', () => { + expect(redactor.redactText('url: https://user:s3cretPass@api.deepseek.com/v1')) + .toBe(`url: https://user:${REDACTED}@api.deepseek.com/v1`) + }) + + it('redacts bearer tokens embedded in free text', () => { + expect(redactor.redactText('sending Bearer abcdefgh12345678 now')) + .toBe(`sending Bearer ${REDACTED} now`) + }) + + it('redacts standalone secret-shaped tokens while keeping package names and paths', () => { + expect(redactor.redactText('key sk-abcdefghij1234567890 end')) + .toBe(`key ${REDACTED} end`) + expect(redactor.redactText('name: @deepseek-ai/dsh-telemetry')).toBe('name: @deepseek-ai/dsh-telemetry') + expect(redactor.redactText('path: ./plugins/local-plugin/src/index.ts')) + .toBe('path: ./plugins/local-plugin/src/index.ts') + }) + + it('is idempotent on already-redacted text', () => { + const once = redactor.redactText('password: hunter2') + expect(redactor.redactText(once)).toBe(once) + }) +}) diff --git a/packages/sdk/telemetry/tsconfig.json b/packages/sdk/telemetry/tsconfig.json new file mode 100644 index 0000000000..8acc8f11c5 --- /dev/null +++ b/packages/sdk/telemetry/tsconfig.json @@ -0,0 +1,13 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { "path": "../../util/brand" } + ] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 25a8fadc47..0acc85fcab 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1267,6 +1267,19 @@ importers: specifier: ^4.22.4 version: 4.22.4 + packages/sdk/telemetry: + dependencies: + yaml: + specifier: ^2.9.0 + version: 2.9.0 + devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + cordis: + specifier: ^4.0.0-rc.7 + version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) + packages/session-persistence/session-persistence: devDependencies: '@deepseek-ai/dsh-session': diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 4a845b4554..004338fa2d 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -54,6 +54,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/sdk/helper': { kind: 'none', reason: 'The project domain edits files and registers no live agent or model surface.' }, 'packages/sdk/plugin-fetch': { kind: 'none', reason: 'The fetcher acquires plugin sources into a temp dir and registers no live agent or model surface.' }, 'packages/sdk/scripts': { kind: 'indirect', reason: 'The launcher delegates model context to the loaded project plugin tree.' }, + 'packages/sdk/telemetry': { kind: 'none', reason: 'The launcher-side reporter sends developer-cycle telemetry and registers no live agent or model surface.' }, 'packages/session-query/session-query': { kind: 'none', reason: 'The trusted query service exposes cloned records only to callers and registers no model surface.' }, 'packages/skill/skill': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-skill.' }, 'packages/skill/skill-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-skill.' }, diff --git a/tsconfig.build.json b/tsconfig.build.json index 9a6133db36..9359d7a531 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -99,6 +99,7 @@ { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, { "path": "./packages/sdk/create-sdk" }, - { "path": "./packages/sdk/plugin-fetch" } + { "path": "./packages/sdk/plugin-fetch" }, + { "path": "./packages/sdk/telemetry" } ] } diff --git a/tsconfig.json b/tsconfig.json index 4588c81c72..48c5187fdc 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -110,6 +110,7 @@ { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, { "path": "./packages/sdk/create-sdk" }, - { "path": "./packages/sdk/plugin-fetch" } + { "path": "./packages/sdk/plugin-fetch" }, + { "path": "./packages/sdk/telemetry" } ] } From a66b98d2705934cbf51c46beaa9c737b9c04176c Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 14:03:39 +0800 Subject: [PATCH 18/73] fix(telemetry): report unless telemetry entry is explicitly disabled Per ccyu's decision (option A), remove the consent asymmetry in ConsentResolver: telemetry is OFF only when cordis.yml has a telemetry entry with disabled: true. A cordis.yml with no telemetry entry now reports (allowWhenEntryAbsent defaults to true) rather than denying. No cordis.yml, an enabled entry, and DO_NOT_TRACK/CI are unchanged. Both no-config and absent-entry defaults stay configurable. Updates the module/README docs and the unit tests (file-present-but-no-entry and non-object/non-sequence roots now report). Per-file 100% coverage holds. --- packages/sdk/telemetry/README.md | 2 +- packages/sdk/telemetry/src/consent-resolver.ts | 16 ++++++++-------- .../sdk/telemetry/tests/consent-resolver.spec.ts | 16 ++++++++-------- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/packages/sdk/telemetry/README.md b/packages/sdk/telemetry/README.md index 3a8b6ac4c2..43d13d3dcc 100644 --- a/packages/sdk/telemetry/README.md +++ b/packages/sdk/telemetry/README.md @@ -10,7 +10,7 @@ Launcher-side telemetry primitives for the dsh-sdk toolchain. This is a plain li | `getOrCreateAnonymousId` | Random UUID persisted in a per-user GLOBAL config file (never in the project, never derived from git). | | `TelemetryReporter` | Fire-and-forget send: `report()` never blocks or throws; delivery resolves on every path; `flush()` optionally drains in-flight sends within a cap. | -Consent is carried by the telemetry entry in `cordis.yml`, so disabling telemetry is disabling that entry. When `cordis.yml` does not yet exist (first `create`) consent defaults to allowed; when it exists without a telemetry entry consent defaults to denied — both are configurable on `ConsentResolver`. +Consent is carried by the telemetry entry in `cordis.yml`, so disabling telemetry is disabling that entry. Telemetry reports by default and is off only when a present telemetry entry is explicitly `disabled`: a missing `cordis.yml` (first `create`), an enabled entry, or a `cordis.yml` with no telemetry entry all report. `DO_NOT_TRACK`/CI always deny. The no-config and absent-entry defaults are configurable on `ConsentResolver`. The collection endpoint is a fixed constant (`DSH_TELEMETRY_ENDPOINT`); its `.invalid` placeholder must be replaced with the real endpoint before release. diff --git a/packages/sdk/telemetry/src/consent-resolver.ts b/packages/sdk/telemetry/src/consent-resolver.ts index 1d50904f39..a4327dc9f7 100644 --- a/packages/sdk/telemetry/src/consent-resolver.ts +++ b/packages/sdk/telemetry/src/consent-resolver.ts @@ -1,12 +1,12 @@ /** * Consent resolution for dsh-sdk telemetry. * - * Consent is carried by the telemetry plugin's enabled/disabled state in the - * project `cordis.yml`: an enabled entry means opt-in, a `disabled: true` entry - * means opt-out. The resolver PARSES `cordis.yml` — it never boots a Cordis - * application — because several launcher commands (`build`, `create`) never - * boot Cordis at all. `DO_NOT_TRACK` and CI environment signals force a denial - * regardless of file state. + * Telemetry is OFF only when `cordis.yml` contains a telemetry entry that is + * explicitly `disabled`; every other file state reports (no `cordis.yml`, an + * enabled entry, or no telemetry entry at all). The resolver PARSES `cordis.yml` + * — it never boots a Cordis application — because several launcher commands + * (`build`, `create`) never boot Cordis at all. `DO_NOT_TRACK` and CI + * environment signals force a denial regardless of file state. * * @module @deepseek-ai/dsh-telemetry/consent-resolver */ @@ -56,7 +56,7 @@ export interface ConsentResolverOptions { honorEnvOptOut?: boolean /** Consent when `cordis.yml` does not exist yet (first `create`). Defaults to `true` (telemetry is default-on). */ allowWhenNoConfig?: boolean - /** Consent when `cordis.yml` exists but has no telemetry entry. Defaults to `false`. */ + /** Consent when `cordis.yml` exists but has no telemetry entry. Defaults to `true` (report unless a present entry is disabled). */ allowWhenEntryAbsent?: boolean } @@ -94,7 +94,7 @@ export class ConsentResolver { this.#env = options.env ?? process.env this.#honorEnvOptOut = options.honorEnvOptOut ?? true this.#allowWhenNoConfig = options.allowWhenNoConfig ?? true - this.#allowWhenEntryAbsent = options.allowWhenEntryAbsent ?? false + this.#allowWhenEntryAbsent = options.allowWhenEntryAbsent ?? true } /** diff --git a/packages/sdk/telemetry/tests/consent-resolver.spec.ts b/packages/sdk/telemetry/tests/consent-resolver.spec.ts index e1f5f90f81..ca0cec3bbd 100644 --- a/packages/sdk/telemetry/tests/consent-resolver.spec.ts +++ b/packages/sdk/telemetry/tests/consent-resolver.spec.ts @@ -84,23 +84,23 @@ describe('ConsentResolver cordis.yml state', () => { .toEqual({ allowed: true, reason: 'enabled' }) }) - it('reports absent when cordis.yml has no telemetry entry, defaulting to deny', async () => { + it('reports (allows) when cordis.yml has no telemetry entry', async () => { const yml = '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n' expect(await resolver.resolve(await projectDir(yml))) - .toEqual({ allowed: false, reason: 'absent' }) + .toEqual({ allowed: true, reason: 'absent' }) }) - it('can allow when the entry is absent', async () => { + it('can be told to deny when the entry is absent', async () => { const yml = '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n' - const decision = await new ConsentResolver({ env: {}, allowWhenEntryAbsent: true }).resolve(await projectDir(yml)) - expect(decision).toEqual({ allowed: true, reason: 'absent' }) + const decision = await new ConsentResolver({ env: {}, allowWhenEntryAbsent: false }).resolve(await projectDir(yml)) + expect(decision).toEqual({ allowed: false, reason: 'absent' }) }) - it('skips non-object sequence items and a non-sequence root', async () => { + it('skips non-object sequence items and a non-sequence root, still reporting absent', async () => { expect(await resolver.resolve(await projectDir('- just-a-string\n- id: x\n name: y\n'))) - .toEqual({ allowed: false, reason: 'absent' }) + .toEqual({ allowed: true, reason: 'absent' }) expect(await resolver.resolve(await projectDir('root: not-a-sequence\n'))) - .toEqual({ allowed: false, reason: 'absent' }) + .toEqual({ allowed: true, reason: 'absent' }) }) it('honors a custom telemetry plugin name', async () => { From 472a683bdc76f33cee9d2dd492623f91bb0f4b48 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 15:21:37 +0800 Subject: [PATCH 19/73] feat(create-sdk): headless creation via --config/--config-json + NDJSON + skill Add a headless create path: --config / --config-json supply a structured project spec (answers + feature plan) that drives CreateWizard through a HeadlessPromptPort, bypassing the TTY. --json emits NDJSON lifecycle events (done / action-required / error) so an agent can fill a missing input and re-run. Ship a thin SKILL.md playbook for agent-driven creation. Per-file 100% coverage. --- packages/sdk/create-sdk/README.md | 6 +- packages/sdk/create-sdk/src/args.ts | 12 +++ packages/sdk/create-sdk/src/command.ts | 42 ++++++-- packages/sdk/create-sdk/src/headless.ts | 98 ++++++++++++++++++ packages/sdk/create-sdk/tests/create.spec.ts | 102 +++++++++++++++++++ skills/create-dsh-sdk-project/SKILL.md | 57 +++++++++++ 6 files changed, 307 insertions(+), 10 deletions(-) create mode 100644 packages/sdk/create-sdk/src/headless.ts create mode 100644 skills/create-dsh-sdk-project/SKILL.md diff --git a/packages/sdk/create-sdk/README.md b/packages/sdk/create-sdk/README.md index 66cddc030a..a5fa51b480 100644 --- a/packages/sdk/create-sdk/README.md +++ b/packages/sdk/create-sdk/README.md @@ -6,14 +6,14 @@ The supported package surface is the `create-sdk` bin. The package root exports The initializer rejects every existing target path, creates one `SdkProject` edit session, validates and commits it, then asks whether to install NPM dependencies and build. Install or build failures keep the generated project and print a retry command. -Public flags are `[directory]`, `--description`, `--provider`, `--base-url`, `--api-key`, `--model`, `--interface`, `--pm`, and `--install`/`--no-install`. Flags prefill matching questions, but creation always requires a TTY. +Public flags are `[directory]`, `--description`, `--provider`, `--base-url`, `--api-key`, `--model`, `--interface`, `--pm`, `--install`/`--no-install`, plus the headless flags `--config ` / `--config-json ` and `--json`. Interactive flags prefill matching questions; a headless spec (`--config`/`--config-json`) supplies every answer and its feature plan up front, so creation runs without a TTY and drives through a `HeadlessPromptPort` that fails loud on any missing required answer. `--json` emits NDJSON lifecycle events (`done` / `action-required` / `error`) so an agent can fill the named missing input and re-run. The provider choice is DeepSeek or a custom endpoint backed by `llm-pi-ai`. DeepSeek asks only for an API key and uses the public endpoint plus `deepseek-v4-flash`; custom also asks for a base URL. An empty key requires confirmation and creates a commented empty `.env` variable so provider startup fails clearly until it is filled. Existing plugin defaults are omitted; required SDK presets remain typed against the owning package's Config. ## Model Experience -Indirectly, through the generated project composition and its selected runtime plugins. +Indirectly, through the generated project composition and its selected runtime plugins; the headless `--config-json` + `--json` surface additionally lets an agent create a project end to end and react to `action-required` events. ## Known Limitations and Deferred Work -- **TTY-only creation** — flags prefill questions, but the wizard still requires an interactive terminal before it writes a project. +- **Headless local plugins** — the headless spec supplies project answers and the feature plan; scaffolding a local plugin (the interactive none/plugin/tool choice) is not yet expressible in the spec and defaults to none. diff --git a/packages/sdk/create-sdk/src/args.ts b/packages/sdk/create-sdk/src/args.ts index 521132caa9..897159bd7c 100644 --- a/packages/sdk/create-sdk/src/args.ts +++ b/packages/sdk/create-sdk/src/args.ts @@ -19,6 +19,9 @@ export interface CreateArgs { packageManager?: PackageManagerName install?: boolean linkWorkspace?: boolean + config?: string + configJson?: string + json?: boolean help: boolean } @@ -32,6 +35,9 @@ interface CommanderCreateOptions { pm?: PackageManagerName install?: boolean linkWorkspace?: boolean + config?: string + configJson?: string + json?: boolean help?: boolean } @@ -60,6 +66,9 @@ function createProgram(): Command { .addOption(new Option('--install').default(undefined)) .addOption(new Option('--no-install').default(undefined)) .option('--link-workspace') + .option('--config ') + .option('--config-json ') + .addOption(new Option('--json').default(undefined)) } /** Parse create-sdk positionals/options through Commander into a domain-neutral value. */ @@ -79,6 +88,9 @@ export function parseCreateArgs(argv: readonly string[]): CreateArgs { ...options.pm === undefined ? {} : { packageManager: options.pm }, ...options.install === undefined ? {} : { install: options.install }, ...options.linkWorkspace ? { linkWorkspace: true } : {}, + ...options.config === undefined ? {} : { config: options.config }, + ...options.configJson === undefined ? {} : { configJson: options.configJson }, + ...options.json === undefined ? {} : { json: options.json }, help: options.help ?? false, } } diff --git a/packages/sdk/create-sdk/src/command.ts b/packages/sdk/create-sdk/src/command.ts index 0c7076c90b..a7855ea027 100644 --- a/packages/sdk/create-sdk/src/command.ts +++ b/packages/sdk/create-sdk/src/command.ts @@ -7,12 +7,15 @@ import { readFile } from 'node:fs/promises' import { ClackPromptPort, + HeadlessPromptError, + HeadlessPromptPort, PromptCancelledError, type PackageManagerVersionProbe, type PromptPort, } from '@deepseek-ai/dsh-helper' -import { parseCreateArgs } from './args.ts' +import { parseCreateArgs, type CreateArgs } from './args.ts' import { CreateWizard, type ResolvedCreateRequest } from './create-wizard.ts' +import { resolveHeadless } from './headless.ts' import { scaffoldProject, type ScaffoldResult } from './project-scaffolder.ts' import { CREATE_TEMPLATES, packageManagerTemplateModel } from './templates/create-templates.ts' @@ -46,16 +49,18 @@ export async function createProject( context.stdout.write(CREATE_TEMPLATES.usage.render({})) return undefined } - if (!context.port && (!context.stdin.isTTY || !context.stdout.isTTY)) { - throw new Error('create-sdk requires an interactive TTY') + const headless = await resolveHeadless(args) + if (!headless && !context.port && (!context.stdin.isTTY || !context.stdout.isTTY)) { + throw new Error('create-sdk requires an interactive TTY, --config , or --config-json ') } const wizard = new CreateWizard({ - args, + args: headless ? headless.args : args, /* v8 ignore next -- production TTY wiring is exercised by the built-bin smoke */ - port: context.port ?? new ClackPromptPort(context.stdin, context.stdout), + port: context.port ?? (headless ? new HeadlessPromptPort() : new ClackPromptPort(context.stdin, context.stdout)), cwd: context.cwd, releaseVersion: context.releaseVersion ?? await readCreateSdkVersion(), ...context.versionProbe ? { versionProbe: context.versionProbe } : {}, + ...headless?.features ? { features: headless.features } : {}, }) const resolved = await wizard.run() const result = await scaffoldProject(resolved.directory, resolved.request) @@ -87,6 +92,17 @@ export async function createProject( return result } +/** Whether NDJSON lifecycle events were requested, tolerating unparseable argv. */ +function wantsJsonEvents(argv: readonly string[]): boolean { + let parsed: CreateArgs + try { + parsed = parseCreateArgs(argv) + } catch { + return false + } + return parsed.json === true +} + /** Run the create command with process defaults and convert cancellation to a clean exit. */ export async function runCreateCommand( argv: readonly string[] = process.argv.slice(2), @@ -97,15 +113,27 @@ export async function runCreateCommand( stderr: process.stderr, }, ): Promise { + const json = wantsJsonEvents(argv) + const emit = (event: Record): void => { + context.stdout.write(`${JSON.stringify(event)}\n`) + } try { await createProject(argv, context) + if (json) emit({ type: 'done' }) return 0 } catch (error) { if (error instanceof PromptCancelledError) { - context.stderr.write('create-sdk: cancelled\n') + if (json) emit({ type: 'error', reason: 'cancelled' }) + else context.stderr.write('create-sdk: cancelled\n') return 1 } - context.stderr.write(`create-sdk: ${error instanceof Error ? error.message : String(error)}\n`) + if (json && error instanceof HeadlessPromptError) { + emit({ type: 'action-required', prompt: error.prompt }) + return 1 + } + const message = error instanceof Error ? error.message : String(error) + if (json) emit({ type: 'error', message }) + else context.stderr.write(`create-sdk: ${message}\n`) return 1 } } diff --git a/packages/sdk/create-sdk/src/headless.ts b/packages/sdk/create-sdk/src/headless.ts new file mode 100644 index 0000000000..e32a053457 --- /dev/null +++ b/packages/sdk/create-sdk/src/headless.ts @@ -0,0 +1,98 @@ +/** + * Headless create input: a structured project spec supplied by an agent or CI + * instead of interactive prompts. + * + * @module @deepseek-ai/create-sdk/headless + */ + +import { readFile } from 'node:fs/promises' +import type { FeatureSelection, PackageManagerName, RunInterface } from '@deepseek-ai/dsh-helper' +import type { CreateArgs } from './args.ts' + +/** + * Structured, non-interactive create input. Scalar fields mirror {@link CreateArgs} + * project answers; `features` is the headless feature plan handed to `CreateWizard` + * (the interactive tree/suggests prompts are skipped). Absent required answers make + * the run fail loud through `HeadlessPromptPort` rather than blocking. + */ +export interface HeadlessCreateSpec { + directory?: string + description?: string + provider?: 'deepseek' | 'custom' + baseURL?: string + apiKey?: string + model?: string + interface?: RunInterface + pm?: PackageManagerName + install?: boolean + linkWorkspace?: boolean + features?: readonly FeatureSelection[] +} + +/** Resolved headless input: the args the wizard reads plus the feature plan. */ +export interface ResolvedHeadless { + args: CreateArgs + features: readonly FeatureSelection[] | undefined +} + +function asRecord(value: unknown, source: string): Record { + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new Error(`${source}: expected a JSON object`) + } + return value as Record +} + +/** Parse and shallow-validate a headless spec from JSON text. */ +export function parseHeadlessSpec(text: string, source: string): HeadlessCreateSpec { + let parsed: unknown + try { + parsed = JSON.parse(text) + } catch (error) { + /* v8 ignore next -- JSON.parse only throws Error instances; the String() branch is defensive */ + throw new Error(`${source}: invalid JSON (${error instanceof Error ? error.message : String(error)})`) + } + const record = asRecord(parsed, source) + if (record.features !== undefined && !Array.isArray(record.features)) { + throw new Error(`${source}: "features" must be an array`) + } + return record +} + +/** + * Load a headless spec from `--config-json` (inline) or `--config` (a JSON file), + * returning `undefined` when neither is supplied. + * @param args - parsed create args. + * @param readFileText - file reader seam for tests. + * @returns the resolved args + feature plan, or `undefined` for interactive runs. + */ +export async function resolveHeadless( + args: CreateArgs, + readFileText: (path: string) => Promise = path => readFile(path, 'utf8'), +): Promise { + let text: string + let source: string + if (args.configJson !== undefined) { + text = args.configJson + source = '--config-json' + } else if (args.config !== undefined) { + source = args.config + text = await readFileText(args.config) + } else { + return undefined + } + const spec = parseHeadlessSpec(text, source) + const resolvedArgs: CreateArgs = { + ...spec.directory === undefined ? {} : { directory: spec.directory }, + ...spec.description === undefined ? {} : { description: spec.description }, + ...spec.provider === undefined ? {} : { provider: spec.provider }, + ...spec.baseURL === undefined ? {} : { baseURL: spec.baseURL }, + ...spec.apiKey === undefined ? {} : { apiKey: spec.apiKey }, + ...spec.model === undefined ? {} : { model: spec.model }, + ...spec.interface === undefined ? {} : { runInterface: spec.interface }, + ...spec.pm === undefined ? {} : { packageManager: spec.pm }, + ...spec.install === undefined ? {} : { install: spec.install }, + ...spec.linkWorkspace ? { linkWorkspace: true } : {}, + help: false, + } + return { args: resolvedArgs, features: spec.features } +} diff --git a/packages/sdk/create-sdk/tests/create.spec.ts b/packages/sdk/create-sdk/tests/create.spec.ts index f4b66dc8ff..1f2cdb75fc 100644 --- a/packages/sdk/create-sdk/tests/create.spec.ts +++ b/packages/sdk/create-sdk/tests/create.spec.ts @@ -30,6 +30,7 @@ import { type CreateCommandContext, } from '../src/command.ts' import { CreateWizard } from '../src/create-wizard.ts' +import { resolveHeadless } from '../src/headless.ts' import { scaffoldProject } from '../src/project-scaffolder.ts' class ScriptedPort implements PromptPort { @@ -482,6 +483,52 @@ describe('create command composition', () => { await expect(createProject(argv('agent', false), context)).rejects.toThrow('interactive TTY') }) + it('creates headlessly from --config-json with no TTY', async () => { + const root = await mkdtemp(join(tmpdir(), 'create-headless-cmd-')) + temporary.push(root) + const spec = JSON.stringify({ + directory: 'agent', description: 'test', provider: 'deepseek', apiKey: 'key', + model: 'deepseek-v4-flash', interface: 'embed', pm: 'npm', install: false, + features: [{ id: 'persistence', options: ['jsonl'] }], + }) + const context = commandContext(root) + context.stdin.isTTY = false + context.stdout.isTTY = false + const result = await createProject(['--config-json', spec], context) + expect(result?.project.root).toBe(join(root, 'agent')) + }) + + it('emits NDJSON lifecycle events under --json', async () => { + const root = await mkdtemp(join(tmpdir(), 'create-headless-json-')) + temporary.push(root) + const base = { + description: 'test', model: 'deepseek-v4-flash', interface: 'embed', pm: 'npm', install: false, + } + const ok = commandContext(root) + ok.stdin.isTTY = false + ok.stdout.isTTY = false + const okSpec = JSON.stringify({ ...base, directory: 'done-agent', provider: 'deepseek', apiKey: 'key', features: [] }) + await expect(runCreateCommand(['--config-json', okSpec, '--json'], ok)).resolves.toBe(0) + expect(ok.readStdout()).toContain('{"type":"done"}') + + const missing = commandContext(root) + missing.stdin.isTTY = false + missing.stdout.isTTY = false + const missingSpec = JSON.stringify({ ...base, directory: 'miss-agent', provider: 'custom', baseURL: 'https://x', features: [] }) + await expect(runCreateCommand(['--config-json', missingSpec, '--json'], missing)).resolves.toBe(1) + expect(missing.readStdout()).toContain('"type":"action-required"') + + const broken = commandContext(root) + broken.stdin.isTTY = false + broken.stdout.isTTY = false + await expect(runCreateCommand(['--config-json', '{bad', '--json'], broken)).resolves.toBe(1) + expect(broken.readStdout()).toContain('"type":"error"') + + const cancelled = commandContext(root, new ScriptedPort([ScriptedPort.cancel])) + await expect(runCreateCommand(['--json', ...argv('cancel-agent', false)], cancelled)).resolves.toBe(1) + expect(cancelled.readStdout()).toContain('"reason":"cancelled"') + }) + it('creates through an injected prompt port and delegates optional setup', async () => { const root = await mkdtemp(join(tmpdir(), 'create-command-success-')) temporary.push(root) @@ -550,3 +597,58 @@ describe('create command composition', () => { await expect(runCreateCommand(['--help'], help)).resolves.toBe(0) }) }) + +describe('resolveHeadless', () => { + it('returns undefined without a config source', async () => { + expect(await resolveHeadless(parseCreateArgs(['agent']))).toBeUndefined() + }) + + it('maps every inline --config-json field into args plus the feature plan', async () => { + const spec = JSON.stringify({ + directory: 'a', description: 'd', provider: 'custom', baseURL: 'https://x', apiKey: 'k', + model: 'm', interface: 'acp', pm: 'pnpm', install: true, linkWorkspace: true, + features: [{ id: 'todo', options: ['default'] }], + }) + const resolved = await resolveHeadless(parseCreateArgs(['--config-json', spec])) + expect(resolved?.args).toMatchObject({ + directory: 'a', description: 'd', provider: 'custom', baseURL: 'https://x', apiKey: 'k', + model: 'm', runInterface: 'acp', packageManager: 'pnpm', install: true, linkWorkspace: true, help: false, + }) + expect(resolved?.features).toEqual([{ id: 'todo', options: ['default'] }]) + }) + + it('reads --config from a file via the injected reader and omits absent fields', async () => { + const resolved = await resolveHeadless( + parseCreateArgs(['--config', '/spec.json']), + async () => JSON.stringify({ description: 'from-file' }), + ) + expect(resolved?.args.description).toBe('from-file') + expect(resolved?.args.directory).toBeUndefined() + expect(resolved?.args.linkWorkspace).toBeUndefined() + expect(resolved?.features).toBeUndefined() + }) + + it('reads --config from disk with the default reader', async () => { + const dir = await mkdtemp(join(tmpdir(), 'create-headless-file-')) + temporary.push(dir) + const file = join(dir, 'spec.json') + await writeFile(file, JSON.stringify({ description: 'on-disk' })) + const resolved = await resolveHeadless(parseCreateArgs(['--config', file])) + expect(resolved?.args.description).toBe('on-disk') + }) + + it('fails loud on invalid JSON, a non-object root, or a non-array features field', async () => { + await expect(resolveHeadless(parseCreateArgs(['--config-json', '{bad']))).rejects.toThrow('invalid JSON') + await expect(resolveHeadless(parseCreateArgs(['--config-json', '[]']))).rejects.toThrow('expected a JSON object') + await expect(resolveHeadless(parseCreateArgs(['--config-json', 'null']))).rejects.toThrow('expected a JSON object') + await expect(resolveHeadless(parseCreateArgs(['--config-json', '5']))).rejects.toThrow('expected a JSON object') + await expect(resolveHeadless(parseCreateArgs(['--config-json', '{"features":1}']))).rejects.toThrow('must be an array') + }) + + it('accepts a minimal spec, leaving unspecified answers undefined', async () => { + const resolved = await resolveHeadless(parseCreateArgs(['--config-json', '{"directory":"x"}'])) + expect(resolved?.args.directory).toBe('x') + expect(resolved?.args.description).toBeUndefined() + expect(resolved?.features).toBeUndefined() + }) +}) diff --git a/skills/create-dsh-sdk-project/SKILL.md b/skills/create-dsh-sdk-project/SKILL.md new file mode 100644 index 0000000000..5b984f3b86 --- /dev/null +++ b/skills/create-dsh-sdk-project/SKILL.md @@ -0,0 +1,57 @@ +--- +name: create-dsh-sdk-project +description: Create a DeepSeek Harness SDK project non-interactively (headless), driven by an agent instead of the interactive wizard. Use when asked to scaffold a new DSH SDK project without a terminal. +--- + +# Create a DeepSeek Harness SDK project headlessly + +The `create-sdk` initializer normally runs an interactive wizard. To create a project +**without a terminal**, pass a structured spec and ask for machine-readable events: + +```sh +npm create @deepseek-ai/sdk -- --config-json '' --json +``` + +- `--config-json ''` supplies the whole spec inline (no prompts). Alternatively + `--config ` reads the same spec from a file. +- `--json` makes the command emit one NDJSON lifecycle event per line to stdout. + +## Spec shape + +All fields are optional except those a chosen feature requires. Unsupplied answers that +have a sensible default are taken from it; a *required* answer with no default (a secret, +a custom provider base URL, a required feature option) makes the run fail loud rather than +block. + +```json +{ + "directory": "my-agent", + "description": "A DeepSeek Harness agent", + "provider": "deepseek", + "apiKey": "", + "model": "deepseek-v4-flash", + "interface": "stdio", + "pm": "npm", + "install": false, + "features": [ + { "id": "persistence", "options": ["sqlite"] }, + { "id": "web", "options": ["exa"], "secrets": { "apiKey": "" } } + ] +} +``` + +`features` is the complete set of optional features to enable, each with its chosen +options and any secrets/values it needs. The interactive feature tree and its +recommended-feature prompts are skipped in headless mode. + +## Reacting to events + +Each line of stdout is one JSON object: + +- `{"type":"done"}` — the project was created (and installed, if `install` was true). +- `{"type":"action-required","prompt":""}` — a required answer was missing. + Add the corresponding field to the spec (e.g. an `apiKey`, a feature secret, a custom + `baseURL`) and re-run. +- `{"type":"error","message":""}` — the run failed for another reason. + +Iterate: read `action-required`, fill the named input into the spec, re-run until `done`. From 8052370155b4d45f1eba8764e89932962339e6a2 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 15:40:34 +0800 Subject: [PATCH 20/73] =?UTF-8?q?chore(sdk):=20drop=20dsh-plugin-fetch=20(?= =?UTF-8?q?giget/pacote)=20=E2=80=94=20#2=20will=20use=20native=20npm/pnpm?= =?UTF-8?q?=20deps?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit External-plugin creation will add a package-manager-native dependency (github:owner/repo#ref or pkg@version) plus a cordis mount instead of fetching tarballs into a temp dir, so the giget/pacote fetch package is no longer needed. --- docs/module-graph.md | 3 - packages/sdk/README.md | 1 - packages/sdk/plugin-fetch/README.md | 32 - packages/sdk/plugin-fetch/package.json | 37 - packages/sdk/plugin-fetch/src/fetcher.ts | 90 -- .../sdk/plugin-fetch/src/giget-fetcher.ts | 116 --- packages/sdk/plugin-fetch/src/ids.ts | 39 - packages/sdk/plugin-fetch/src/index.ts | 48 - packages/sdk/plugin-fetch/src/never.ts | 19 - .../sdk/plugin-fetch/src/pacote-fetcher.ts | 129 --- packages/sdk/plugin-fetch/src/source.ts | 126 --- .../sdk/plugin-fetch/tests/fetcher.spec.ts | 64 -- .../plugin-fetch/tests/giget-fetcher.spec.ts | 145 --- packages/sdk/plugin-fetch/tests/ids.spec.ts | 39 - packages/sdk/plugin-fetch/tests/never.spec.ts | 19 - .../plugin-fetch/tests/pacote-fetcher.spec.ts | 108 --- .../sdk/plugin-fetch/tests/source.spec.ts | 107 --- packages/sdk/plugin-fetch/tsconfig.json | 15 - pnpm-lock.yaml | 831 ------------------ .../verify-package-readme-model-experience.ts | 1 - tsconfig.build.json | 1 - tsconfig.json | 1 - 22 files changed, 1971 deletions(-) delete mode 100644 packages/sdk/plugin-fetch/README.md delete mode 100644 packages/sdk/plugin-fetch/package.json delete mode 100644 packages/sdk/plugin-fetch/src/fetcher.ts delete mode 100644 packages/sdk/plugin-fetch/src/giget-fetcher.ts delete mode 100644 packages/sdk/plugin-fetch/src/ids.ts delete mode 100644 packages/sdk/plugin-fetch/src/index.ts delete mode 100644 packages/sdk/plugin-fetch/src/never.ts delete mode 100644 packages/sdk/plugin-fetch/src/pacote-fetcher.ts delete mode 100644 packages/sdk/plugin-fetch/src/source.ts delete mode 100644 packages/sdk/plugin-fetch/tests/fetcher.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tests/ids.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tests/never.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tests/source.spec.ts delete mode 100644 packages/sdk/plugin-fetch/tsconfig.json diff --git a/docs/module-graph.md b/docs/module-graph.md index 4cc2c36eeb..240772529b 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -138,7 +138,6 @@ flowchart TD end subgraph group_sdk["packages/sdk"] pkg_helper["helper"] - pkg_plugin_fetch["plugin-fetch"] pkg_scripts["scripts"] pkg_telemetry["telemetry"] end @@ -154,7 +153,6 @@ flowchart TD pkg_llm --> pkg_brand pkg_code_runtime_worker --> pkg_code_runtime pkg_helper --> pkg_brand - pkg_plugin_fetch --> pkg_brand pkg_scripts --> pkg_app_boot pkg_telemetry --> pkg_brand pkg_llm_deepseek --> pkg_llm @@ -446,7 +444,6 @@ flowchart TD | [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand) | | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime) | | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) | -| [`plugin-fetch`](../packages/sdk/plugin-fetch) | `sdk` | [`brand`](../packages/util/brand) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) | | [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand) | | [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | diff --git a/packages/sdk/README.md b/packages/sdk/README.md index 92c8794e64..953d52ea76 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -9,7 +9,6 @@ The [feature RFC](../../docs/rfc/proposed/feature/2026-07-14-sdk-developer-proje | [`helper`](helper/README.md) | Project aggregate, edit session, builtin features, project documents, templates, package managers, and prompt abstraction | | [`scripts`](scripts/README.md) | The `dsh-sdk` launcher: `start`, `dev`, `build`, and interactive `config` | | [`create-sdk`](create-sdk/README.md) | The `npm create @deepseek-ai/sdk` initializer | -| [`plugin-fetch`](plugin-fetch/README.md) | Fetch an external plugin (github/npm) into a temp dir — pinned and un-executed — for `dsh-sdk create` | `@deepseek-ai/create-sdk` is the one package-name exception to the repository's `@deepseek-ai/dsh-*` rule: npm's scoped initializer convention requires that name for `npm create @deepseek-ai/sdk`. diff --git a/packages/sdk/plugin-fetch/README.md b/packages/sdk/plugin-fetch/README.md deleted file mode 100644 index 801dc2a2a6..0000000000 --- a/packages/sdk/plugin-fetch/README.md +++ /dev/null @@ -1,32 +0,0 @@ -# `@deepseek-ai/dsh-plugin-fetch` - -Fetch an external Cordis plugin into a temp directory — pinned to an immutable commit or integrity and never executed — for the forthcoming `dsh-sdk create ` command. - -The package parses a source spec into a `PluginSource`, dispatches to the matching `PluginFetcher`, and returns a common `FetchedPlugin` (temp dir + immutable provenance) that the wiring step pins into `package.json`, mounts in `cordis.yml`, and installs with `--ignore-scripts`. - -| Export | Role | -|---|---| -| `resolvePluginSource(spec)` → `PluginSource` | Parse `owner/repo[/subdir]#ref` (github) or `pkg@version` (npm); fail loud on an ambiguous or malformed spec | -| `PluginFetcher` | The fetch seam: resolve the pin BEFORE download, extract without executing pulled code | -| `GigetFetcher` / `createGigetFetcher()` | Github fetcher over `@bluwy/giget-core`: resolve `#ref` to a commit SHA, download that SHA | -| `PacoteFetcher` / `createPacoteFetcher()` | Npm fetcher over `pacote`: resolve the manifest, then extract the tarball verified against its integrity | -| `fetchPlugin(source, fetchers)` → `FetchedPlugin` | Dispatch one source to its fetcher by discriminant tag | - -## Safety model — confirm-before-run, not run-on-fetch - -A fetch only downloads and unpacks; it runs no install, no `postinstall`/`prepare`, and no degit-style template actions. - -- **github** uses `@bluwy/giget-core` (one runtime dependency, `modern-tar`; no CLI, install, or JSON-registry surface) so a fetch can only download and untar a tarball. The commit is pinned first: `GigetFetcher` resolves `#ref` — or the default branch when absent — to an immutable SHA via the GitHub commits API, then downloads that SHA. Provenance carries the SHA so wiring pins `github:owner/repo#`. -- **npm** uses `pacote`. Registry-only is enforced upstream: `resolvePluginSource` produces only a `name@version` registry spec, so pacote never sees a git/file/dir spec whose lifecycle scripts would run, and a registry tarball extract is a plain untar. The manifest is resolved first so extract verifies the artifact against the registry-published integrity (a mismatch raises `EINTEGRITY`). Provenance carries the exact version, resolved URL, and integrity. - -Both network boundaries (giget download, GitHub ref resolution, pacote, temp-dir allocation) are constructor-injected, so the fetch logic is unit-tested without network; the `create*Fetcher()` factories wire the real libraries. - -## Model Experience - -None, as this developer tooling acquires plugin sources for the SDK launcher and registers no live agent or model surface. - -## Known Limitations and Deferred Work - -- **Wiring is not here yet** — pinning `package.json`, mounting `cordis.yml` through `ProjectEditSession` with a confirmed diff, and `install --ignore-scripts` land with the `dsh-sdk create` command. This package stops at a fetched, pinned temp directory. -- **npm registry authentication** — `PacoteFetcher` targets a public or default-configured registry; private-registry auth beyond pacote's ambient npm config is deferred. -- **Template-repo initialization** — the whole-project init mode (`dsh-sdk create` from a template repository) is out of scope; this package fetches a single plugin into an existing project. diff --git a/packages/sdk/plugin-fetch/package.json b/packages/sdk/plugin-fetch/package.json deleted file mode 100644 index e8e38090e6..0000000000 --- a/packages/sdk/plugin-fetch/package.json +++ /dev/null @@ -1,37 +0,0 @@ -{ - "name": "@deepseek-ai/dsh-plugin-fetch", - "description": "Fetch an external Cordis plugin (github or npm) into a temp dir, pinned and un-executed, for dsh-sdk create", - "version": "0.0.1", - "private": true, - "type": "module", - "main": "lib/index.js", - "types": "lib/types/index.d.ts", - "exports": { - ".": { - "types": "./lib/types/index.d.ts", - "default": "./lib/index.js" - }, - "./src/*": "./src/*", - "./package.json": "./package.json" - }, - "files": [ - "lib/index.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" - ], - "license": "BSD-3-Clause", - "dependencies": { - "@bluwy/giget-core": "^0.1.7", - "pacote": "^22.0.0" - }, - "peerDependencies": { - "@deepseek-ai/dsh-brand": "^0.0.1", - "cordis": "^4.0.0-rc.7" - }, - "devDependencies": { - "@deepseek-ai/dsh-brand": "workspace:^", - "@types/pacote": "^11.1.8", - "cordis": "^4.0.0-rc.7" - } -} diff --git a/packages/sdk/plugin-fetch/src/fetcher.ts b/packages/sdk/plugin-fetch/src/fetcher.ts deleted file mode 100644 index 7c3d3a763c..0000000000 --- a/packages/sdk/plugin-fetch/src/fetcher.ts +++ /dev/null @@ -1,90 +0,0 @@ -/** - * The `PluginFetcher` seam, its common `FetchedPlugin` result, and the - * tag-dispatched entry point. A fetcher acquires one plugin source into a fresh - * temp directory WITHOUT executing any pulled code, and reports immutable - * provenance the wiring step pins the dependency to. - * - * @module @deepseek-ai/dsh-plugin-fetch/fetcher - */ - -import { mkdtemp } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import { assertNever } from './never.ts' -import type { CommitSha, Integrity } from './ids.ts' -import type { GithubSource, NpmSource, PluginSource } from './source.ts' - -/** Immutable pin for a github fetch: the resolved commit the tarball came from. */ -export interface GithubProvenance { - readonly kind: 'github' - readonly sha: CommitSha -} - -/** Immutable pin for an npm fetch: exact version, tarball URL, and integrity. */ -export interface NpmProvenance { - readonly kind: 'npm' - /** Concrete resolved version (e.g. `1.2.3`), never the requested range/tag. */ - readonly version: string - /** Tarball URL the artifact resolved to. */ - readonly resolved: string - /** Subresource integrity the artifact was verified against. */ - readonly integrity: Integrity -} - -/** Provenance a fetch records so wiring can pin an immutable dependency. */ -export type PluginProvenance = GithubProvenance | NpmProvenance - -/** The common result of fetching any plugin source. */ -export interface FetchedPlugin { - /** Absolute temp directory holding the extracted, UN-executed source. */ - readonly dir: string - /** The source that produced this fetch, echoed for the wiring step. */ - readonly source: PluginSource - /** Immutable provenance to pin the dependency during wiring. */ - readonly provenance: PluginProvenance -} - -/** - * A fetcher for one source kind. Implementations resolve the immutable pin - * BEFORE download and must never run lifecycle scripts or template actions. - */ -export interface PluginFetcher { - /** The single source kind this fetcher handles. */ - readonly kind: S['kind'] - /** - * Fetch one source into a fresh temp directory. - * @param source - the resolved source to fetch. - * @returns the temp dir plus immutable provenance. - */ - fetch(source: S): Promise -} - -/** The per-kind fetchers {@link fetchPlugin} dispatches across. */ -export interface PluginFetchers { - readonly github: PluginFetcher - readonly npm: PluginFetcher -} - -/** - * Dispatch one source to its fetcher by discriminant tag. - * @param source - the resolved plugin source. - * @param fetchers - the per-kind fetchers to route across. - * @returns the fetch result from the matching fetcher. - */ -export function fetchPlugin(source: PluginSource, fetchers: PluginFetchers): Promise { - switch (source.kind) { - case 'github': return fetchers.github.fetch(source) - case 'npm': return fetchers.npm.fetch(source) - default: return assertNever(source, 'fetchPlugin') - } -} - -/** - * Create a fresh, empty temp directory for one fetch — the default temp-dir - * seam shared by the concrete fetchers. - * @param prefix - a `mkdtemp` name prefix identifying the fetch kind. - * @returns the absolute path of the created directory. - */ -export function createTempDir(prefix: string): Promise { - return mkdtemp(join(tmpdir(), prefix)) -} diff --git a/packages/sdk/plugin-fetch/src/giget-fetcher.ts b/packages/sdk/plugin-fetch/src/giget-fetcher.ts deleted file mode 100644 index 55e001063c..0000000000 --- a/packages/sdk/plugin-fetch/src/giget-fetcher.ts +++ /dev/null @@ -1,116 +0,0 @@ -/** - * The github {@link PluginFetcher}, backed by `@bluwy/giget-core`. - * - * `@bluwy/giget-core` is chosen over unjs `giget`: it carries a single runtime - * dependency (`modern-tar`) versus giget's CLI/registry stack, and it dropped - * the `install` and JSON-registry options entirely, so a fetch can only ever - * download and untar a tarball — never run install or degit-style actions. That - * is exactly the "extract, never execute" guarantee this feature needs. - * - * The commit is pinned BEFORE download: {@link GigetFetcher} resolves `#ref` to - * an immutable SHA (default via the GitHub commits API), then downloads that - * SHA. Provenance carries the SHA so wiring pins `github:owner/repo#`. - * - * @module @deepseek-ai/dsh-plugin-fetch/giget-fetcher - */ - -import { downloadTemplate } from '@bluwy/giget-core' -import { createTempDir, type FetchedPlugin, type PluginFetcher } from './fetcher.ts' -import { commitSha, type CommitSha } from './ids.ts' -import type { GithubSource } from './source.ts' - -/** Temp-dir name prefix for github fetches. */ -export const GITHUB_TEMP_PREFIX = 'dsh-plugin-github-' - -/** Downloads a giget input string into `dir`; the tarball-extraction seam. */ -export type DownloadTemplate = ( - input: string, - options: { dir: string; force: 'clean' }, -) => Promise<{ dir: string }> - -/** Resolves a github source's ref to an immutable commit SHA before download. */ -export type ResolveRef = (source: GithubSource) => Promise - -/** The injected collaborators a {@link GigetFetcher} needs. */ -export interface GigetFetcherDeps { - /** Downloads a pinned giget input into a directory. */ - download: DownloadTemplate - /** Resolves `source.ref` (or the default branch) to a commit SHA. */ - resolveRef: ResolveRef - /** Allocates the fresh temp directory to download into. */ - createTempDir: (prefix: string) => Promise -} - -/** Build the giget input string that pins a github source to a commit SHA. */ -function gigetInput(source: GithubSource, sha: CommitSha): string { - const path = source.subdir ? `${source.owner}/${source.repo}/${source.subdir}` : `${source.owner}/${source.repo}` - return `${path}#${sha}` -} - -/** A human-readable label for one github source, for error messages. */ -function githubLabel(source: GithubSource): string { - return `${source.owner}/${source.repo}#${source.ref ?? 'HEAD'}` -} - -/** - * Resolve a github source's ref to an immutable SHA via the GitHub commits API. - * Uses the `application/vnd.github.sha` media type, which returns the resolved - * commit id as plain text. - * @param source - the github source; an absent `ref` resolves the default branch (`HEAD`). - * @param token - optional bearer token for private repositories. - * @returns the resolved immutable commit SHA. - * @throws if the GitHub API rejects the request. - */ -export async function defaultResolveRef(source: GithubSource, token?: string): Promise { - const ref = source.ref ?? 'HEAD' - const url = `https://api.github.com/repos/${source.owner}/${source.repo}/commits/${ref}` - const headers: Record = { Accept: 'application/vnd.github.sha' } - if (token !== undefined) headers.Authorization = `Bearer ${token}` - const response = await fetch(url, { headers }) - if (!response.ok) { - throw new Error(`cannot resolve github ref ${githubLabel(source)}: HTTP ${response.status}`) - } - return commitSha((await response.text()).trim()) -} - -/** Fetches a github plugin source by pinning `#ref` to a commit SHA, then downloading it. */ -export class GigetFetcher implements PluginFetcher { - readonly kind = 'github' as const - private readonly deps: GigetFetcherDeps - - /** Construct with injected download, ref-resolution, and temp-dir seams. */ - constructor(deps: GigetFetcherDeps) { - this.deps = deps - } - - async fetch(source: GithubSource): Promise { - const sha = await this.deps.resolveRef(source) - const dir = await this.deps.createTempDir(GITHUB_TEMP_PREFIX) - await this.deps.download(gigetInput(source, sha), { dir, force: 'clean' }) - return { dir, source, provenance: { kind: 'github', sha } } - } -} - -/** Options for the production github fetcher. */ -export interface GithubFetchOptions { - /** Bearer token for private repositories; defaults to `GITHUB_TOKEN`. */ - token?: string -} - -/** - * Build the production github fetcher wired to `@bluwy/giget-core` and the - * GitHub commits API. - * @param options - optional token override (else `process.env.GITHUB_TOKEN`). - * @returns a {@link GigetFetcher} using the real download and ref-resolution seams. - */ -export function createGigetFetcher(options: GithubFetchOptions = {}): GigetFetcher { - const token = options.token ?? process.env.GITHUB_TOKEN - return new GigetFetcher({ - download: (input, downloadOptions) => downloadTemplate(input, { - ...downloadOptions, - ...token !== undefined ? { providerOptions: { auth: token } } : {}, - }), - resolveRef: source => defaultResolveRef(source, token), - createTempDir, - }) -} diff --git a/packages/sdk/plugin-fetch/src/ids.ts b/packages/sdk/plugin-fetch/src/ids.ts deleted file mode 100644 index a3a2ffa575..0000000000 --- a/packages/sdk/plugin-fetch/src/ids.ts +++ /dev/null @@ -1,39 +0,0 @@ -/** - * Branded provenance identities owned by the plugin-fetch layer. Both cross the - * fetch → wiring boundary and are opaque tokens that must not be confused with - * ordinary strings (a package name, a URL) at that seam. - * - * @module @deepseek-ai/dsh-plugin-fetch/ids - */ - -import type { Branded } from '@deepseek-ai/dsh-brand' - -/** An immutable git commit object id a github fetch pins to. */ -export type CommitSha = Branded<'CommitSha'> - -/** - * Construct a {@link CommitSha}, validating the hexadecimal object-id shape. - * @param value - lowercase hex of an abbreviated or full commit id (7–64 chars, covering SHA-1 and SHA-256). - * @returns the branded commit id. - */ -export function commitSha(value: string): CommitSha { - if (!/^[0-9a-f]{7,64}$/.test(value)) { - throw new Error(`invalid commit sha: ${JSON.stringify(value)}`) - } - return value as CommitSha -} - -/** A Subresource Integrity string an npm fetch pins to. */ -export type Integrity = Branded<'Integrity'> - -/** - * Construct an {@link Integrity}, validating the SRI `-` shape. - * @param value - a single SRI entry using sha256, sha384, or sha512. - * @returns the branded integrity string. - */ -export function integrity(value: string): Integrity { - if (!/^sha(256|384|512)-[A-Za-z0-9+/]+={0,2}$/.test(value)) { - throw new Error(`invalid subresource integrity: ${JSON.stringify(value)}`) - } - return value as Integrity -} diff --git a/packages/sdk/plugin-fetch/src/index.ts b/packages/sdk/plugin-fetch/src/index.ts deleted file mode 100644 index d4b9682ff5..0000000000 --- a/packages/sdk/plugin-fetch/src/index.ts +++ /dev/null @@ -1,48 +0,0 @@ -/** - * Fetch an external Cordis plugin (github or npm) into a temp directory — - * pinned to an immutable commit/integrity and never executed — for the - * `dsh-sdk create ` command. Parses a source spec, dispatches to the - * matching fetcher, and returns a common {@link FetchedPlugin} the wiring step - * pins and mounts. - * - * @module @deepseek-ai/dsh-plugin-fetch - */ - -export { resolvePluginSource } from './source.ts' -export type { GithubSource, NpmSource, PluginSource } from './source.ts' -export { commitSha, integrity } from './ids.ts' -export type { CommitSha, Integrity } from './ids.ts' -export { createTempDir, fetchPlugin } from './fetcher.ts' -export type { - FetchedPlugin, - GithubProvenance, - NpmProvenance, - PluginFetcher, - PluginFetchers, - PluginProvenance, -} from './fetcher.ts' -export { - createGigetFetcher, - defaultResolveRef, - GigetFetcher, - GITHUB_TEMP_PREFIX, -} from './giget-fetcher.ts' -export type { - DownloadTemplate, - GigetFetcherDeps, - GithubFetchOptions, - ResolveRef, -} from './giget-fetcher.ts' -export { - createPacoteFetcher, - NPM_TEMP_PREFIX, - PacoteFetcher, -} from './pacote-fetcher.ts' -export type { - NpmFetchOptions, - PacoteApi, - PacoteExtractResult, - PacoteFetcherDeps, - PacoteFetchOptions, - PacoteResolution, -} from './pacote-fetcher.ts' diff --git a/packages/sdk/plugin-fetch/src/never.ts b/packages/sdk/plugin-fetch/src/never.ts deleted file mode 100644 index 664ab3293d..0000000000 --- a/packages/sdk/plugin-fetch/src/never.ts +++ /dev/null @@ -1,19 +0,0 @@ -/** - * Exhaustiveness helper for this package's closed unions. Kept local so the - * SDK plugin-fetch tooling stays free of the model-runtime `dsh-llm` dependency - * that owns the shared `assertNever`. - * - * @module @deepseek-ai/dsh-plugin-fetch/never - */ - -/** - * Mark an unreachable closed-union branch. A newly unhandled variant fails - * compilation at the call site; a value that escaped its type throws at runtime. - * @param value - the impossible value; typed `never` so a new variant fails to compile at every call site. - * @param context - optional label prefixed into the throw message. - * @returns never — it always throws, rendering the offending value. - */ -export function assertNever(value: never, context?: string): never { - const rendered = (JSON.stringify(value) as string | undefined) ?? String(value) - throw new Error(`unreachable variant${context ? ` in ${context}` : ''}: ${rendered}`) -} diff --git a/packages/sdk/plugin-fetch/src/pacote-fetcher.ts b/packages/sdk/plugin-fetch/src/pacote-fetcher.ts deleted file mode 100644 index dfb1d790b8..0000000000 --- a/packages/sdk/plugin-fetch/src/pacote-fetcher.ts +++ /dev/null @@ -1,129 +0,0 @@ -/** - * The npm {@link PluginFetcher}, backed by `pacote`. - * - * Supply-chain safety comes from three layers: (1) {@link resolvePluginSource} - * only ever produces a registry `name@version` spec, so pacote classifies it as - * a registry source and cannot be steered to a git/file/dir spec whose - * lifecycle scripts would run; (2) a registry tarball extract is a plain untar — - * pacote runs no `prepare`/`postinstall` during {@link PacoteFetcher.fetch}; and - * (3) the later wiring step installs with `--ignore-scripts`. The manifest is - * resolved first so extract verifies the tarball against the registry-published - * integrity (a mismatch raises `EINTEGRITY`). - * - * @module @deepseek-ai/dsh-plugin-fetch/pacote-fetcher - */ - -import { extract as pacoteExtract, manifest as pacoteManifest } from 'pacote' -import { createTempDir, type FetchedPlugin, type PluginFetcher } from './fetcher.ts' -import { integrity } from './ids.ts' -import type { NpmSource } from './source.ts' - -/** Temp-dir name prefix for npm fetches. */ -export const NPM_TEMP_PREFIX = 'dsh-plugin-npm-' - -/** The subset of pacote options this fetcher passes through. */ -export interface PacoteFetchOptions { - /** Registry to resolve against; absent uses pacote's default. */ - registry?: string - /** Known resolved tarball URL, forwarded to extract. */ - resolved?: string - /** Expected integrity, forwarded to extract for `EINTEGRITY` verification. */ - integrity?: string -} - -/** The resolved registry manifest fields this fetcher pins from. */ -export interface PacoteResolution { - /** Resolved tarball URL. */ - _resolved: string - /** Registry-published integrity. */ - _integrity: string - /** Concrete resolved version. */ - version: string -} - -/** The extract result fields this fetcher pins from. */ -export interface PacoteExtractResult { - /** Resolved tarball URL of the extracted artifact. */ - resolved: string - /** Integrity of the extracted artifact. */ - integrity: string -} - -/** The pacote surface a {@link PacoteFetcher} depends on; the fetch seam. */ -export interface PacoteApi { - /** Resolve a registry spec to its pinned manifest fields. */ - manifest: (spec: string, options?: PacoteFetchOptions) => Promise - /** Untar a registry spec into `dest`, verifying integrity when supplied. */ - extract: (spec: string, dest: string, options?: PacoteFetchOptions) => Promise -} - -/** The injected collaborators a {@link PacoteFetcher} needs. */ -export interface PacoteFetcherDeps { - /** The pacote resolve/extract surface. */ - pacote: PacoteApi - /** Allocates the fresh temp directory to extract into. */ - createTempDir: (prefix: string) => Promise - /** Registry to resolve against; absent uses pacote's default. */ - registry?: string -} - -/** Fetches an npm plugin source by resolving its manifest, then extracting the verified tarball. */ -export class PacoteFetcher implements PluginFetcher { - readonly kind = 'npm' as const - private readonly deps: PacoteFetcherDeps - - /** Construct with injected pacote, temp-dir, and optional registry seams. */ - constructor(deps: PacoteFetcherDeps) { - this.deps = deps - } - - async fetch(source: NpmSource): Promise { - const spec = `${source.name}@${source.version}` - const registryOptions: PacoteFetchOptions = this.deps.registry !== undefined - ? { registry: this.deps.registry } - : {} - const resolution = await this.deps.pacote.manifest(spec, registryOptions) - const dir = await this.deps.createTempDir(NPM_TEMP_PREFIX) - const extracted = await this.deps.pacote.extract(spec, dir, { - ...registryOptions, - resolved: resolution._resolved, - integrity: resolution._integrity, - }) - return { - dir, - source, - provenance: { - kind: 'npm', - version: resolution.version, - resolved: extracted.resolved, - integrity: integrity(extracted.integrity), - }, - } - } -} - -/** Options for the production npm fetcher. */ -export interface NpmFetchOptions { - /** Registry to resolve against; absent uses pacote's default. */ - registry?: string -} - -/** - * Build the production npm fetcher wired to `pacote`. - * @param options - optional registry override. - * @returns a {@link PacoteFetcher} using the real pacote resolve/extract seam. - */ -export function createPacoteFetcher(options: NpmFetchOptions = {}): PacoteFetcher { - const pacote: PacoteApi = { - manifest: async (spec, pacoteOptions) => { - const resolved = await pacoteManifest(spec, pacoteOptions) - return { _resolved: resolved._resolved, _integrity: resolved._integrity, version: resolved.version } - }, - extract: (spec, dest, pacoteOptions) => pacoteExtract(spec, dest, pacoteOptions), - } - return new PacoteFetcher({ - pacote, - createTempDir, - ...options.registry !== undefined ? { registry: options.registry } : {}, - }) -} diff --git a/packages/sdk/plugin-fetch/src/source.ts b/packages/sdk/plugin-fetch/src/source.ts deleted file mode 100644 index 7d7e3ffcc4..0000000000 --- a/packages/sdk/plugin-fetch/src/source.ts +++ /dev/null @@ -1,126 +0,0 @@ -/** - * The `PluginSource` discriminated union and the resolver that parses one CLI - * spec string into it. Ambiguous or malformed specs fail loud here — the single - * earliest resolvable point — rather than surfacing as a confusing fetch error. - * - * Grammar: - * - github: `owner/repo[/subdir]#ref` — a `#` unambiguously marks a github ref; - * `ref` is optional and, when omitted, the fetcher pins the default branch. - * - npm: `pkg@version` (scoped `@scope/pkg@version`) — the `@version` is the - * only disambiguator from a bare `owner/repo` github locator. - * - * @module @deepseek-ai/dsh-plugin-fetch/source - */ - -/** A plugin pulled from a github (git tarball) repository. */ -export interface GithubSource { - readonly kind: 'github' - /** Repository owner (user or org). */ - readonly owner: string - /** Repository name. */ - readonly repo: string - /** Path within the repository to extract; absent means the repository root. */ - readonly subdir?: string - /** Branch, tag, or commit; absent means the repository's default branch. */ - readonly ref?: string -} - -/** A plugin pulled from an npm registry by exact package and version spec. */ -export interface NpmSource { - readonly kind: 'npm' - /** Package name, including any `@scope/` prefix. */ - readonly name: string - /** Registry version, range, or dist-tag (non-empty). */ - readonly version: string -} - -/** Every plugin origin `dsh-sdk create ` understands. */ -export type PluginSource = GithubSource | NpmSource - -/** One `/`-separated github name segment (owner, repo, or subdir component). */ -function isNameSegment(segment: string): boolean { - return /^[A-Za-z0-9._-]+$/.test(segment) && segment !== '.' && segment !== '..' -} - -/** A git ref: branch, tag, or commit; permits `/`-nested names, rejects traversal. */ -function isGitRef(ref: string): boolean { - return /^[A-Za-z0-9._/-]+$/.test(ref) - && !ref.includes('..') - && !ref.startsWith('/') - && !ref.endsWith('/') -} - -/** An npm package name, scoped (`@scope/name`) or unscoped. */ -function isNpmPackageName(name: string): boolean { - const segment = /^[a-z0-9][a-z0-9._-]*$/ - if (name.startsWith('@')) { - const slash = name.indexOf('/') - if (slash < 2 || slash === name.length - 1) return false - return segment.test(name.slice(1, slash)) && segment.test(name.slice(slash + 1)) - } - return segment.test(name) -} - -/** Parse a `owner/repo[/subdir]` locator with an optional already-split ref. */ -function tryParseGithubLocator(locator: string, ref: string | undefined): GithubSource | undefined { - if (!/^[^\s@#]+$/.test(locator)) return undefined - const [owner, repo, ...subdirSegments] = locator.split('/') - if (owner === undefined || repo === undefined) return undefined - if (!isNameSegment(owner) || !isNameSegment(repo)) return undefined - if (subdirSegments.some(segment => !isNameSegment(segment))) return undefined - if (ref !== undefined && !isGitRef(ref)) return undefined - const subdir = subdirSegments.join('/') - return { - kind: 'github', - owner, - repo, - ...subdir.length > 0 ? { subdir } : {}, - ...ref !== undefined ? { ref } : {}, - } -} - -/** Parse `pkg@version` (scoped or unscoped); undefined when it is not npm-shaped. */ -function tryParseNpmSource(spec: string): NpmSource | undefined { - if (/[\s#]/.test(spec)) return undefined - // A scoped spec's version `@` follows the scope's `/`; an unscoped spec's is - // the first `@`. A leading `@` with no version `@` yields index 0 (rejected). - const versionAt = spec.startsWith('@') ? spec.indexOf('@', spec.indexOf('/') + 1) : spec.indexOf('@') - if (versionAt <= 0) return undefined - const name = spec.slice(0, versionAt) - const version = spec.slice(versionAt + 1) - if (version.length === 0 || version.includes('/')) return undefined - if (!isNpmPackageName(name)) return undefined - return { kind: 'npm', name, version } -} - -/** - * Parse one `dsh-sdk create ` spec into a {@link PluginSource}. - * @param spec - the raw source argument. - * @returns the discriminated source. - * @throws if the spec is empty, malformed, or ambiguous between github and npm. - */ -export function resolvePluginSource(spec: string): PluginSource { - const trimmed = spec.trim() - if (trimmed.length === 0) throw new Error('plugin source must not be empty') - - const hashIndex = trimmed.indexOf('#') - if (hashIndex !== -1) { - const ref = trimmed.slice(hashIndex + 1) - if (ref.length === 0) { - throw new Error(`github plugin source is missing a ref after '#': ${JSON.stringify(spec)}`) - } - const source = tryParseGithubLocator(trimmed.slice(0, hashIndex), ref) - if (!source) { - throw new Error(`invalid github plugin source: ${JSON.stringify(spec)} — expected "owner/repo[/subdir]#ref"`) - } - return source - } - - const npm = tryParseNpmSource(trimmed) - if (npm) return npm - const github = tryParseGithubLocator(trimmed, undefined) - if (github) return github - throw new Error( - `unrecognized plugin source: ${JSON.stringify(spec)} — expected "owner/repo[/subdir]#ref" (github) or "pkg@version" (npm)`, - ) -} diff --git a/packages/sdk/plugin-fetch/tests/fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/fetcher.spec.ts deleted file mode 100644 index b68a3d3ae1..0000000000 --- a/packages/sdk/plugin-fetch/tests/fetcher.spec.ts +++ /dev/null @@ -1,64 +0,0 @@ -import { rm, stat } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { describe, expect, it, vi } from 'vitest' -import { - createTempDir, - fetchPlugin, - type FetchedPlugin, - type PluginFetchers, -} from '../src/fetcher.ts' -import { commitSha } from '../src/ids.ts' -import type { GithubSource, NpmSource, PluginSource } from '../src/source.ts' - -function stubFetchers(): { fetchers: PluginFetchers; github: ReturnType; npm: ReturnType } { - const result = (dir: string): FetchedPlugin => ({ - dir, - source: { kind: 'github', owner: 'o', repo: 'r' }, - provenance: { kind: 'github', sha: commitSha('a'.repeat(40)) }, - }) - const github = vi.fn(async (source: GithubSource) => result(`github:${source.repo}`)) - const npm = vi.fn(async (source: NpmSource) => result(`npm:${source.name}`)) - return { - fetchers: { github: { kind: 'github', fetch: github }, npm: { kind: 'npm', fetch: npm } }, - github, - npm, - } -} - -describe('fetchPlugin', () => { - it('routes a github source to the github fetcher', async () => { - const { fetchers, github, npm } = stubFetchers() - const source: GithubSource = { kind: 'github', owner: 'o', repo: 'r' } - const result = await fetchPlugin(source, fetchers) - expect(github).toHaveBeenCalledWith(source) - expect(npm).not.toHaveBeenCalled() - expect(result.dir).toBe('github:r') - }) - - it('routes an npm source to the npm fetcher', async () => { - const { fetchers, github, npm } = stubFetchers() - const source: NpmSource = { kind: 'npm', name: 'plugin', version: '1.0.0' } - const result = await fetchPlugin(source, fetchers) - expect(npm).toHaveBeenCalledWith(source) - expect(github).not.toHaveBeenCalled() - expect(result.dir).toBe('npm:plugin') - }) - - it('throws on an unknown source kind', () => { - const { fetchers } = stubFetchers() - const bogus = { kind: 'svn' } as unknown as PluginSource - expect(() => fetchPlugin(bogus, fetchers)).toThrow(/unreachable variant in fetchPlugin/) - }) -}) - -describe('createTempDir', () => { - it('creates a fresh empty directory under the OS temp root', async () => { - const dir = await createTempDir('dsh-plugin-fetch-test-') - try { - expect(dir.startsWith(tmpdir())).toBe(true) - expect((await stat(dir)).isDirectory()).toBe(true) - } finally { - await rm(dir, { recursive: true, force: true }) - } - }) -}) diff --git a/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts deleted file mode 100644 index c0cafb31d5..0000000000 --- a/packages/sdk/plugin-fetch/tests/giget-fetcher.spec.ts +++ /dev/null @@ -1,145 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it, type Mock, vi } from 'vitest' -import { downloadTemplate } from '@bluwy/giget-core' -import { - createGigetFetcher, - defaultResolveRef, - GigetFetcher, - GITHUB_TEMP_PREFIX, - type GigetFetcherDeps, -} from '../src/giget-fetcher.ts' -import type { CommitSha } from '../src/ids.ts' -import type { GithubSource } from '../src/source.ts' - -vi.mock('@bluwy/giget-core', () => ({ downloadTemplate: vi.fn(async (_input: string, options: { dir: string }) => ({ dir: options.dir, source: '', info: { name: '', tar: '' } })) })) - -const SHA = 'a'.repeat(40) - -/** A `fetch` mock typed with the call signature the assertions destructure. */ -function fetchReturning(response: Response): Mock<(url: string, init?: RequestInit) => Promise> { - return vi.fn((_url: string, _init?: RequestInit) => Promise.resolve(response)) -} - -function fakeDeps(overrides: Partial = {}): { - deps: GigetFetcherDeps - download: ReturnType - resolveRef: ReturnType - createTempDir: ReturnType -} { - const download = vi.fn(async () => ({ dir: '/tmp/x' })) - const resolveRef = vi.fn(async () => SHA as CommitSha) - const createTempDir = vi.fn(async () => '/tmp/dsh-plugin-github-abc') - return { deps: { download, resolveRef, createTempDir, ...overrides }, download, resolveRef, createTempDir } -} - -describe('GigetFetcher.fetch', () => { - it('pins the ref to a SHA, downloads that SHA, and reports provenance', async () => { - const { deps, download, resolveRef, createTempDir } = fakeDeps() - const source: GithubSource = { kind: 'github', owner: 'unjs', repo: 'template', ref: 'main' } - const result = await new GigetFetcher(deps).fetch(source) - - expect(resolveRef).toHaveBeenCalledWith(source) - expect(createTempDir).toHaveBeenCalledWith(GITHUB_TEMP_PREFIX) - expect(download).toHaveBeenCalledWith(`unjs/template#${SHA}`, { dir: '/tmp/dsh-plugin-github-abc', force: 'clean' }) - expect(result).toEqual({ - dir: '/tmp/dsh-plugin-github-abc', - source, - provenance: { kind: 'github', sha: SHA }, - }) - }) - - it('includes the subdir in the download input', async () => { - const { deps, download } = fakeDeps() - const source: GithubSource = { kind: 'github', owner: 'o', repo: 'r', subdir: 'packages/plugin' } - await new GigetFetcher(deps).fetch(source) - expect(download).toHaveBeenCalledWith(`o/r/packages/plugin#${SHA}`, expect.anything()) - }) - - it('exposes its source kind', () => { - expect(new GigetFetcher(fakeDeps().deps).kind).toBe('github') - }) -}) - -describe('defaultResolveRef', () => { - afterEach(() => vi.unstubAllGlobals()) - - it('resolves the default branch (HEAD) with no auth header', async () => { - const fetchMock = fetchReturning(new Response(`${SHA}\n`, { status: 200 })) - vi.stubGlobal('fetch', fetchMock) - const sha = await defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r' }) - expect(sha).toBe(SHA) - const [url, init] = fetchMock.mock.calls[0]! - expect(url).toBe('https://api.github.com/repos/o/r/commits/HEAD') - expect((init as RequestInit).headers).toEqual({ Accept: 'application/vnd.github.sha' }) - }) - - it('resolves an explicit ref and sends a bearer token', async () => { - const fetchMock = fetchReturning(new Response(SHA, { status: 200 })) - vi.stubGlobal('fetch', fetchMock) - const sha = await defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r', ref: 'v1.2.3' }, 'secret') - expect(sha).toBe(SHA) - const [url, init] = fetchMock.mock.calls[0]! - expect(url).toBe('https://api.github.com/repos/o/r/commits/v1.2.3') - expect((init as RequestInit).headers).toEqual({ - Accept: 'application/vnd.github.sha', - Authorization: 'Bearer secret', - }) - }) - - it('throws with the HEAD label when the API rejects an unref-ed source', async () => { - vi.stubGlobal('fetch', fetchReturning(new Response('', { status: 404 }))) - await expect(defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r' })).rejects.toThrow( - /cannot resolve github ref o\/r#HEAD: HTTP 404/, - ) - }) - - it('throws with the explicit-ref label when the API rejects', async () => { - vi.stubGlobal('fetch', fetchReturning(new Response('', { status: 403 }))) - await expect( - defaultResolveRef({ kind: 'github', owner: 'o', repo: 'r', ref: 'main' }), - ).rejects.toThrow(/cannot resolve github ref o\/r#main: HTTP 403/) - }) -}) - -describe('createGigetFetcher', () => { - const downloadMock = vi.mocked(downloadTemplate) - let savedToken: string | undefined - - beforeEach(() => { - downloadMock.mockClear() - savedToken = process.env.GITHUB_TOKEN - delete process.env.GITHUB_TOKEN - }) - - afterEach(() => { - vi.unstubAllGlobals() - if (savedToken === undefined) delete process.env.GITHUB_TOKEN - else process.env.GITHUB_TOKEN = savedToken - }) - - it('wires the real download without provider auth when no token is present', async () => { - vi.stubGlobal('fetch', fetchReturning(new Response(SHA, { status: 200 }))) - await createGigetFetcher().fetch({ kind: 'github', owner: 'o', repo: 'r', ref: 'main' }) - const [input, options] = downloadMock.mock.calls[0]! - expect(input).toBe(`o/r#${SHA}`) - expect(options?.dir).toContain(GITHUB_TEMP_PREFIX) - expect(options?.force).toBe('clean') - expect(options?.providerOptions).toBeUndefined() - }) - - it('passes an explicit token to both ref resolution and provider auth', async () => { - const fetchMock = fetchReturning(new Response(SHA, { status: 200 })) - vi.stubGlobal('fetch', fetchMock) - await createGigetFetcher({ token: 'tok' }).fetch({ kind: 'github', owner: 'o', repo: 'r' }) - expect((fetchMock.mock.calls[0]![1] as RequestInit).headers).toMatchObject({ Authorization: 'Bearer tok' }) - const [, options] = downloadMock.mock.calls[0]! - expect(options).toMatchObject({ providerOptions: { auth: 'tok' } }) - }) - - it('reads GITHUB_TOKEN from the environment', async () => { - process.env.GITHUB_TOKEN = 'from-env' - vi.stubGlobal('fetch', fetchReturning(new Response(SHA, { status: 200 }))) - await createGigetFetcher().fetch({ kind: 'github', owner: 'o', repo: 'r' }) - const [, options] = downloadMock.mock.calls[0]! - expect(options).toMatchObject({ providerOptions: { auth: 'from-env' } }) - }) -}) diff --git a/packages/sdk/plugin-fetch/tests/ids.spec.ts b/packages/sdk/plugin-fetch/tests/ids.spec.ts deleted file mode 100644 index 2f1ba6f518..0000000000 --- a/packages/sdk/plugin-fetch/tests/ids.spec.ts +++ /dev/null @@ -1,39 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { commitSha, integrity } from '../src/ids.ts' - -describe('commitSha', () => { - it('accepts abbreviated and full lowercase hex object ids', () => { - expect(commitSha('abc1234')).toBe('abc1234') - expect(commitSha('a'.repeat(40))).toBe('a'.repeat(40)) - expect(commitSha('0'.repeat(64))).toBe('0'.repeat(64)) - }) - - it.each([ - ['too short', 'abc123'], - ['uppercase', 'ABCDEF1'], - ['non-hex', 'ghijklm'], - ['too long', 'a'.repeat(65)], - ['empty', ''], - ])('rejects an invalid sha (%s)', (_label, value) => { - expect(() => commitSha(value)).toThrow(/invalid commit sha/) - }) -}) - -describe('integrity', () => { - it.each([ - 'sha512-abcABC123+/==', - 'sha384-abcABC123+/', - 'sha256-Zm9vYmFy', - ])('accepts a valid SRI entry (%s)', (value) => { - expect(integrity(value)).toBe(value) - }) - - it.each([ - ['missing algorithm', 'abcABC123'], - ['unsupported algorithm', 'sha1-abcABC123'], - ['illegal base64 char', 'sha512-abc*def'], - ['empty', ''], - ])('rejects an invalid integrity (%s)', (_label, value) => { - expect(() => integrity(value)).toThrow(/invalid subresource integrity/) - }) -}) diff --git a/packages/sdk/plugin-fetch/tests/never.spec.ts b/packages/sdk/plugin-fetch/tests/never.spec.ts deleted file mode 100644 index 224ab66e29..0000000000 --- a/packages/sdk/plugin-fetch/tests/never.spec.ts +++ /dev/null @@ -1,19 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { assertNever } from '../src/never.ts' - -describe('assertNever', () => { - it('throws with the rendered value and a context label', () => { - expect(() => assertNever('surprise' as never, 'demo')).toThrow( - /unreachable variant in demo: "surprise"/, - ) - }) - - it('omits the context clause when none is given', () => { - expect(() => assertNever(7 as never)).toThrow(/unreachable variant: 7$/) - }) - - it('falls back to String() when the value is not JSON-serializable', () => { - // JSON.stringify(undefined) is undefined, exercising the String() fallback. - expect(() => assertNever(undefined as never)).toThrow(/unreachable variant: undefined$/) - }) -}) diff --git a/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts b/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts deleted file mode 100644 index 958e56674e..0000000000 --- a/packages/sdk/plugin-fetch/tests/pacote-fetcher.spec.ts +++ /dev/null @@ -1,108 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { extract as pacoteExtract, manifest as pacoteManifest } from 'pacote' -import { - createPacoteFetcher, - NPM_TEMP_PREFIX, - PacoteFetcher, - type PacoteApi, - type PacoteFetcherDeps, -} from '../src/pacote-fetcher.ts' -import type { NpmSource } from '../src/source.ts' - -vi.mock('pacote', () => ({ manifest: vi.fn(), extract: vi.fn() })) - -const INTEGRITY = 'sha512-abcABC123+/==' -const RESOLVED = 'https://registry.npmjs.org/plugin/-/plugin-1.2.3.tgz' - -function fakePacote(): PacoteApi { - return { - manifest: vi.fn(async () => ({ _resolved: RESOLVED, _integrity: INTEGRITY, version: '1.2.3' })), - extract: vi.fn(async () => ({ resolved: RESOLVED, integrity: INTEGRITY })), - } -} - -function deps(overrides: Partial = {}): PacoteFetcherDeps { - return { - pacote: fakePacote(), - createTempDir: vi.fn(async () => '/tmp/dsh-plugin-npm-abc'), - ...overrides, - } -} - -const SOURCE: NpmSource = { kind: 'npm', name: 'plugin', version: '^1.0.0' } - -describe('PacoteFetcher.fetch', () => { - it('resolves the manifest, extracts with integrity, and reports provenance', async () => { - const d = deps() - const result = await new PacoteFetcher(d).fetch(SOURCE) - - expect(d.pacote.manifest).toHaveBeenCalledWith('plugin@^1.0.0', {}) - expect(d.createTempDir).toHaveBeenCalledWith(NPM_TEMP_PREFIX) - expect(d.pacote.extract).toHaveBeenCalledWith('plugin@^1.0.0', '/tmp/dsh-plugin-npm-abc', { - resolved: RESOLVED, - integrity: INTEGRITY, - }) - expect(result).toEqual({ - dir: '/tmp/dsh-plugin-npm-abc', - source: SOURCE, - provenance: { kind: 'npm', version: '1.2.3', resolved: RESOLVED, integrity: INTEGRITY }, - }) - }) - - it('forwards a configured registry to both manifest and extract', async () => { - const d = deps({ registry: 'https://npm.internal/' }) - await new PacoteFetcher(d).fetch(SOURCE) - expect(d.pacote.manifest).toHaveBeenCalledWith('plugin@^1.0.0', { registry: 'https://npm.internal/' }) - expect(d.pacote.extract).toHaveBeenCalledWith('plugin@^1.0.0', '/tmp/dsh-plugin-npm-abc', { - registry: 'https://npm.internal/', - resolved: RESOLVED, - integrity: INTEGRITY, - }) - }) - - it('rejects a registry integrity that is not a valid SRI', async () => { - const pacote = fakePacote() - pacote.extract = vi.fn(async () => ({ resolved: RESOLVED, integrity: 'not-sri' })) - await expect(new PacoteFetcher(deps({ pacote })).fetch(SOURCE)).rejects.toThrow( - /invalid subresource integrity/, - ) - }) - - it('exposes its source kind', () => { - expect(new PacoteFetcher(deps()).kind).toBe('npm') - }) -}) - -describe('createPacoteFetcher', () => { - const manifestMock = vi.mocked(pacoteManifest) - const extractMock = vi.mocked(pacoteExtract) - - beforeEach(() => { - manifestMock.mockReset() - extractMock.mockReset() - // The real overloaded pacote manifest returns a much wider shape; the fetcher reads only these fields. - manifestMock.mockResolvedValue( - { _resolved: RESOLVED, _integrity: INTEGRITY, version: '1.2.3' } as unknown as Awaited< - ReturnType - >, - ) - extractMock.mockResolvedValue({ from: 'plugin@1.2.3', resolved: RESOLVED, integrity: INTEGRITY }) - }) - - afterEach(() => vi.clearAllMocks()) - - it('wires the real pacote resolve/extract surface', async () => { - const result = await createPacoteFetcher().fetch(SOURCE) - expect(manifestMock).toHaveBeenCalledWith('plugin@^1.0.0', {}) - expect(extractMock).toHaveBeenCalledWith('plugin@^1.0.0', expect.stringContaining(NPM_TEMP_PREFIX), { - resolved: RESOLVED, - integrity: INTEGRITY, - }) - expect(result.provenance).toEqual({ kind: 'npm', version: '1.2.3', resolved: RESOLVED, integrity: INTEGRITY }) - }) - - it('forwards a configured registry through the real surface', async () => { - await createPacoteFetcher({ registry: 'https://npm.internal/' }).fetch(SOURCE) - expect(manifestMock).toHaveBeenCalledWith('plugin@^1.0.0', { registry: 'https://npm.internal/' }) - }) -}) diff --git a/packages/sdk/plugin-fetch/tests/source.spec.ts b/packages/sdk/plugin-fetch/tests/source.spec.ts deleted file mode 100644 index e585c5dfcf..0000000000 --- a/packages/sdk/plugin-fetch/tests/source.spec.ts +++ /dev/null @@ -1,107 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { resolvePluginSource, type GithubSource, type NpmSource } from '../src/source.ts' - -describe('resolvePluginSource — github', () => { - it('parses owner/repo with a ref', () => { - expect(resolvePluginSource('unjs/template#main')).toEqual({ - kind: 'github', owner: 'unjs', repo: 'template', ref: 'main', - }) - }) - - it('parses a bare owner/repo without a ref', () => { - expect(resolvePluginSource('deepseek-ai/plugin')).toEqual({ - kind: 'github', owner: 'deepseek-ai', repo: 'plugin', - }) - }) - - it('parses a nested subdir with a ref', () => { - expect(resolvePluginSource('owner/repo/packages/plugin#v1.2.3')).toEqual({ - kind: 'github', owner: 'owner', repo: 'repo', subdir: 'packages/plugin', ref: 'v1.2.3', - }) - }) - - it('parses a subdir without a ref', () => { - expect(resolvePluginSource('owner/repo/sub')).toEqual({ - kind: 'github', owner: 'owner', repo: 'repo', subdir: 'sub', - }) - }) - - it('accepts a slash-nested ref', () => { - expect(resolvePluginSource('owner/repo#feature/x')).toEqual({ - kind: 'github', owner: 'owner', repo: 'repo', ref: 'feature/x', - }) - }) - - it('trims surrounding whitespace before parsing', () => { - expect(resolvePluginSource(' owner/repo#main ')).toEqual({ - kind: 'github', owner: 'owner', repo: 'repo', ref: 'main', - }) - }) - - it.each([ - ['empty ref after hash', 'owner/repo#'], - ['single locator segment with hash', 'owner#main'], - ['owner with @ and a hash', 'own@er/repo#main'], - ['ref with whitespace', 'owner/repo#bad ref'], - ['ref with traversal', 'owner/repo#a..b'], - ['ref with a leading slash', 'owner/repo#/main'], - ['ref with a trailing slash', 'owner/repo#main/'], - ['ref with an illegal char', 'owner/repo#ma:in'], - ])('rejects a malformed github spec (%s)', (_label, spec) => { - expect(() => resolvePluginSource(spec)).toThrow(/github plugin source|missing a ref/) - }) -}) - -describe('resolvePluginSource — npm', () => { - it('parses an unscoped name@version', () => { - expect(resolvePluginSource('react@18.2.0')).toEqual({ - kind: 'npm', name: 'react', version: '18.2.0', - }) - }) - - it('parses a scoped name@version', () => { - expect(resolvePluginSource('@deepseek-ai/dsh-tool-foo@0.0.1')).toEqual({ - kind: 'npm', name: '@deepseek-ai/dsh-tool-foo', version: '0.0.1', - }) - }) - - it('accepts a dist-tag as the version', () => { - expect(resolvePluginSource('some-plugin@latest')).toEqual({ - kind: 'npm', name: 'some-plugin', version: 'latest', - }) - }) - - it('accepts a range as the version', () => { - expect(resolvePluginSource('some-plugin@^1.0.0')).toEqual({ - kind: 'npm', name: 'some-plugin', version: '^1.0.0', - }) - }) -}) - -describe('resolvePluginSource — failures', () => { - it.each([ - ['empty', ''], - ['whitespace only', ' '], - ])('rejects a blank spec (%s)', (_label, spec) => { - expect(() => resolvePluginSource(spec)).toThrow(/must not be empty/) - }) - - it.each([ - ['bare word', 'plugin'], - ['internal whitespace', 'owner repo'], - ['empty npm version', 'pkg@'], - ['scoped without version', '@scope/pkg'], - ['scoped with empty scope', '@/pkg@1'], - ['unscoped name with slash and version', 'foo/bar@1'], - ['version containing a slash', 'foo@1/2'], - ['uppercase unscoped name', 'FOO@1.0.0'], - ['uppercase scope segment', '@Scope/pkg@1'], - ['uppercase scoped name segment', '@scope/PKG@1'], - ['empty scoped name segment', '@scope/@1'], - ['dot-only owner', './repo'], - ['traversal subdir segment', 'owner/repo/../x'], - ['double slash subdir', 'owner/repo//sub'], - ])('rejects an unrecognized/ambiguous spec (%s)', (_label, spec) => { - expect(() => resolvePluginSource(spec)).toThrow(/unrecognized plugin source|github plugin source/) - }) -}) diff --git a/packages/sdk/plugin-fetch/tsconfig.json b/packages/sdk/plugin-fetch/tsconfig.json deleted file mode 100644 index 07c2567ff8..0000000000 --- a/packages/sdk/plugin-fetch/tsconfig.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "extends": "../../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib/types" - }, - "include": [ - "src" - ], - "references": [ - { - "path": "../../util/brand" - } - ] -} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 0acc85fcab..cafebfaae6 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1223,25 +1223,6 @@ importers: specifier: ^4.0.0-rc.7 version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) - packages/sdk/plugin-fetch: - dependencies: - '@bluwy/giget-core': - specifier: ^0.1.7 - version: 0.1.7 - pacote: - specifier: ^22.0.0 - version: 22.0.0 - devDependencies: - '@deepseek-ai/dsh-brand': - specifier: workspace:^ - version: link:../../util/brand - '@types/pacote': - specifier: ^11.1.8 - version: 11.1.8 - cordis: - specifier: ^4.0.0-rc.7 - version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) - packages/sdk/scripts: dependencies: '@deepseek-ai/dsh-helper': @@ -2971,10 +2952,6 @@ packages: resolution: {integrity: sha512-6zABk/ECA/QYSCQ1NGiVwwbQerUCZ+TQbp64Q3AgmfNvurHH0j8TtXa1qbShXA6qqkpAj4V5W8pP6mLe1mcMqA==} engines: {node: '>=18'} - '@bluwy/giget-core@0.1.7': - resolution: {integrity: sha512-6XG8TZt8DVYLuGDVSpFJaSMlNowOg5RGecvWbKvlgMoqVbztUAQ3AcWq6oZ5DoCnTzNAPcO7rhkwt8ZVgrS7CQ==} - engines: {node: '>=18'} - '@braintree/sanitize-url@7.1.2': resolution: {integrity: sha512-jigsZK+sMF/cuiB7sERuo9V7N9jx+dhmHHnQyDSVdpZwVutaBu7WvNYqMDLSgFgfB30n452TP3vjDAvFC973mA==} @@ -3428,10 +3405,6 @@ packages: '@noble/hashes': optional: true - '@gar/promise-retry@1.0.3': - resolution: {integrity: sha512-GmzA9ckNokPypTg10pgpeHNQe7ph+iIKKmhKu3Ob9ANkswreCx7R3cKmY781K8QK3AqVL3xVh9A42JvIAbkkSA==} - engines: {node: ^20.17.0 || >=22.9.0} - '@google/genai@1.52.0': resolution: {integrity: sha512-gwSvbpiN/17O9TbsqSsE/OzZcpv5Fo4RQjdngGgogtuB9RsyJ8ZHhX5KjHj1bp5N9snN2eK8LDGXSaWW2hof8Q==} engines: {node: '>=20.0.0'} @@ -3480,10 +3453,6 @@ packages: resolution: {integrity: sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==} engines: {node: '>=12'} - '@isaacs/fs-minipass@4.0.1': - resolution: {integrity: sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==} - engines: {node: '>=18.0.0'} - '@jridgewell/gen-mapping@0.3.13': resolution: {integrity: sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==} @@ -3530,43 +3499,6 @@ packages: '@nodable/entities@2.2.0': resolution: {integrity: sha512-9uGyhaQavEUMC8AIddIjau4NsnsXhou+j5sBAGojCM1oxmQpVKTWR/9JxABD6UAv12vpIms55fPZKFQEhG6uBg==} - '@npmcli/agent@5.0.2': - resolution: {integrity: sha512-EkzGmEsgbQ1rqWkRJe2P0oQHx/ylZozDUNPMXCklLuSFL3GY+QyEfBUjhjCsgGXzh4OGpnHvkboSQgczjP/jJg==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/fs@6.0.0': - resolution: {integrity: sha512-AheOs4swKka/XLtht6xxJDPezlQ7K2IYQ9Y8lST4JLDjnralnWuMM9AE2CdVcgQJ5omrXhsRzM7F7aYmeZBvKQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/git@8.0.0': - resolution: {integrity: sha512-5P1oo+TbxZNAiiMBtpzHA8QyEGh5D69LYLexNWJEDXLdxnAZvT/SLitGJBXxjtCE4ftAcFOS/Tu2185MeIjooQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/installed-package-contents@5.0.0': - resolution: {integrity: sha512-6Ay12sf2Lh7U1ifvnS1mq7TZFeh/rXHMXye+kV7jQrANIubaoVcleeh4HdFumxhsRYwm9OaHycB5lYmSwGrcIQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - hasBin: true - - '@npmcli/node-gyp@6.0.0': - resolution: {integrity: sha512-MFakpea4pcZNlHSTbMi15HK8RY8zl2UpgDtxhZCWOer+KRN3x7HFIMk/fKpOMgR55L4LIcA2qn8IHeyABhIFtw==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/package-json@8.0.0': - resolution: {integrity: sha512-agNZzYQ18MR0wKp3Emg1q5QbcC8CXigYp3Z3CvB0Sax9Ge9aF4cVyyuSG+5SbACSrZUKTvMjVULWiE1RJA38wg==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/promise-spawn@10.0.0': - resolution: {integrity: sha512-llZkSzeTsimFx64U+ThT2xQM2uEce8GIQUYvxgbB6ZFvBhV2LP9LeJJb3HT+syG0uCFLsTCHjV9SfC0WNU1vtA==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/redact@5.0.0': - resolution: {integrity: sha512-3zcN5Q3yEmeyxXBzqB6fXPQFzYa2ROsGFSr69W0ArXIAGJqxl/aFECOVPD2kbkYPm0U/EHxFKgclK3UA9WQg5A==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@npmcli/run-script@11.0.0': - resolution: {integrity: sha512-leBRl6F5F0TvWut8m1/aZcMTUHi2vXjKeMJ/Ik1lW7Q7Yy16Dhtkklu+cEqQww1p1NeLnUNzV3+uwpzqRcy9vw==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - '@oxc-parser/binding-android-arm-eabi@0.133.0': resolution: {integrity: sha512-l/44caGse+VpnY9gx0yvvc5QnnG3yG1FO3KZgYvNL1GZrfK86zIwAOgGEVlxDyRymzrU/KHiblPFpevKOmJmUA==} engines: {node: ^20.19.0 || >=22.12.0} @@ -4196,30 +4128,6 @@ packages: '@shikijs/vscode-textmate@10.0.2': resolution: {integrity: sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg==} - '@sigstore/bundle@5.0.0': - resolution: {integrity: sha512-wefjygudENbzbQMks1t5u34EP0fFoD0XvaEP7DOUP/sXKvogzEJYFw5E6pegGyp3onGWzVEYKVa3bNZWyTYX+A==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@sigstore/core@4.0.1': - resolution: {integrity: sha512-9v5hRjujn5NXq8o7XFEUgLyAtdr5Iisb4pzM05u3K61IS5q3hP3luWAndk0RkPPLTUFoTbg7Vb84UQ1ZQeajWQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@sigstore/protobuf-specs@0.5.1': - resolution: {integrity: sha512-/ScWUhhoFasJsSRGTVBwId1loQjjnjAfE4djL6ZhrXRpNCmPTnUKF5Jokd58ILseOMjzET3UrMOtJPS9sYeI0g==} - engines: {node: ^18.17.0 || >=20.5.0} - - '@sigstore/sign@5.0.0': - resolution: {integrity: sha512-DSFivqz9/i5AkwZ5fq0YdjaJlc4o1WeS2Zffon0kqtChx0vy4W9NOjkEet9bF2vkzOufX72eVH8kZBIGtcBp1w==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@sigstore/tuf@5.0.0': - resolution: {integrity: sha512-Zyqg9tcHps3uRAlKHLNmsW4ohsUZAjb9G+31r7lg0ICh/JOcadzmJsIRdjKljlRHpaR0K4aJ2kXXIdywdcdMlA==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - '@sigstore/verify@4.1.0': - resolution: {integrity: sha512-p/s720RiWxLG8XtmfdPfEJOlATA6H/2knFqmtQbFkHKN3IrhWGUwPfpQAf1UnQIEES9IaH6zzhfjkrhTfeSdZw==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - '@smithy/core@3.24.7': resolution: {integrity: sha512-KoUi4M1f3BG6kzN1FnCwL7oyFptTbyBJKjR6yhSib+JHRdUmM1o+VwsFtJ66NZCkCzVfJMWRHJNo0R0jznp0Pg==} engines: {node: '>=18.0.0'} @@ -4269,14 +4177,6 @@ packages: peerDependencies: eslint: ^9.0.0 || ^10.0.0 - '@tufjs/canonical-json@2.0.0': - resolution: {integrity: sha512-yVtV8zsdo8qFHe+/3kw81dSLyF7D576A5cCFCi4X7B39tWT7SekaEFUnvnWJHz+9qO7qJTah1JbrDjWKqFtdWA==} - engines: {node: ^16.14.0 || >=18.0.0} - - '@tufjs/models@5.0.0': - resolution: {integrity: sha512-U4mVcdFGOi6pt8n38LdWZp67Svn7ppnU1Pj8SGOVaBi1X4gm+G4ztQlLfkoJbKSHfjA6WeaiJp2A4V83AJF6nQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - '@tybys/wasm-util@0.10.2': resolution: {integrity: sha512-RoBvJ2X0wuKlWFIjrwffGw1IqZHKQqzIchKaadZZfnNpsAYp2mM0h36JtPCjNDAHGgYez/15uMBpfGwchhiMgg==} @@ -4424,36 +4324,18 @@ packages: '@types/ms@2.1.0': resolution: {integrity: sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA==} - '@types/node-fetch@2.6.13': - resolution: {integrity: sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==} - '@types/node@22.20.0': resolution: {integrity: sha512-QWlFW2wf3nTjC13/DqRnBpR4ZO36VJH/JVBkA/vcnmbTBNQIlnObqyqZE1tUR7+Ni23Lda8R1BxMfbXRpCUx5g==} '@types/node@25.9.3': resolution: {integrity: sha512-603BddQMv3pUcr4U2dhujk83N2tTDVr/34wII2B6bJy6g+8WD6yUb11jszNs0gdi4PesVWl7ABt8nYMVpnLUcg==} - '@types/npm-package-arg@6.1.4': - resolution: {integrity: sha512-vDgdbMy2QXHnAruzlv68pUtXCjmqUk3WrBAsRboRovsOmxbfn/WiYCjmecyKjGztnMps5dWp4Uq2prp+Ilo17Q==} - - '@types/npm-registry-fetch@8.0.9': - resolution: {integrity: sha512-7NxvodR5Yrop3pb6+n8jhJNyzwOX0+6F+iagNEoi9u1CGxruYAwZD8pvGc9prIkL0+FdX5Xp0p80J9QPrGUp/g==} - - '@types/npmlog@7.0.0': - resolution: {integrity: sha512-hJWbrKFvxKyWwSUXjZMYTINsSOY6IclhvGOZ97M8ac2tmR9hMwmTnYaMdpGhvju9ctWLTPhCS+eLfQNluiEjQQ==} - - '@types/pacote@11.1.8': - resolution: {integrity: sha512-/XLR0VoTh2JEO0jJg1q/e6Rh9bxjBq9vorJuQmtT7rRrXSiWz7e7NsvXVYJQ0i8JxMlBMPPYDTnrRe7MZRFA8Q==} - '@types/picomatch@3.0.2': resolution: {integrity: sha512-n0i8TD3UDB7paoMMxA3Y65vUncFJXjcUf7lQY7YyKGl6031FNjfsLs6pdLFCy2GNFxItPJG8GvvpbZc2skH7WA==} '@types/retry@0.12.0': resolution: {integrity: sha512-wWKOClTTiizcZhXnPY4wikVAwmdYHp8q6DmC+EJUzAMsycb7HB32Kh9RN4+0gExjmPmZSAQjgURXIGATPegAvA==} - '@types/ssri@7.1.5': - resolution: {integrity: sha512-odD/56S3B51liILSk5aXJlnYt99S6Rt9EFDDqGtJM26rKHApHcwyU/UoYHrzKkdkHMAIquGWCuHtQTbes+FRQw==} - '@types/tough-cookie@4.0.5': resolution: {integrity: sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==} @@ -4668,10 +4550,6 @@ packages: resolution: {integrity: sha512-A9gOqLdi6cV4ibazAjcQufGj0B1y/vDqYrcuP6d/6x8P27gRS8643Dj9o1dEKtB6O7fwxb2FgBmJS2mX7gpvdw==} engines: {node: '>=14.6'} - abbrev@5.0.0: - resolution: {integrity: sha512-/XrFJgzQQQHpti1raDJC6m4ws6aNktmjBlhk8Fdlk7LwCEuDoieEJJY9OFHjfiFJFFRM2tK+Ky/IsfbbmlMu1w==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - accepts@2.0.0: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} @@ -4690,10 +4568,6 @@ packages: resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==} engines: {node: '>= 14'} - agent-base@9.0.0: - resolution: {integrity: sha512-TQf59BsZnytt8GdJKLPfUZ54g/iaUL2OWDSFCCvMOhsHduDQxO8xC4PNeyIkVcA5KwL2phPSv0douC0fgWzmnA==} - engines: {node: '>= 20'} - ajv-formats@3.0.1: resolution: {integrity: sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==} peerDependencies: @@ -4753,9 +4627,6 @@ packages: ast-v8-to-istanbul@1.0.4: resolution: {integrity: sha512-0bC0/4bTSrnwdhU3IsZDwEdojvuPrSg59OYZfKsLRtJZ0u8VBx9DebfqqG8bRdCC0I7vjgxmPi41P0lpkhJHtA==} - asynckit@0.4.0: - resolution: {integrity: sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==} - balanced-match@1.0.2: resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==} @@ -4810,10 +4681,6 @@ packages: resolution: {integrity: sha512-tixWYgm5ZoOD+3g6UTea91eow5z6AAHaho3g0V9CNSNb45gM8SmflpAc+GRd1InC4AqN/07Unrgp56Y94N9hJQ==} engines: {node: '>=20.19.0'} - cacache@21.0.1: - resolution: {integrity: sha512-pTwz/uj3Jyp6WXdJ6fWhR+7LVxVs6RyroQSn7KJwHsSxXuyGSp0pcMVcwSwTpCFq1X2YG8QBe0W+vN+cr0SwzA==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - call-bind-apply-helpers@1.0.2: resolution: {integrity: sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==} engines: {node: '>= 0.4'} @@ -4849,10 +4716,6 @@ packages: resolution: {integrity: sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==} engines: {node: '>= 14.16.0'} - chownr@3.0.0: - resolution: {integrity: sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==} - engines: {node: '>=18'} - color-convert@2.0.1: resolution: {integrity: sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==} engines: {node: '>=7.0.0'} @@ -4860,10 +4723,6 @@ packages: color-name@1.1.4: resolution: {integrity: sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==} - combined-stream@1.0.8: - resolution: {integrity: sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==} - engines: {node: '>= 0.8'} - comma-separated-tokens@2.0.3: resolution: {integrity: sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg==} @@ -5151,10 +5010,6 @@ packages: delaunator@5.1.0: resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==} - delayed-stream@1.0.0: - resolution: {integrity: sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==} - engines: {node: '>=0.4.0'} - depd@2.0.0: resolution: {integrity: sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==} engines: {node: '>= 0.8'} @@ -5248,10 +5103,6 @@ packages: resolution: {integrity: sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA==} engines: {node: '>=20.19.0'} - env-paths@2.2.1: - resolution: {integrity: sha512-+h1lkLKhZMTYjog1VEpJNG7NZJWcuc2DDk/qsqSTRRCOXiLjeQ1d1/udrUGhqMxUgAlwKNZ0cf2uqan5GLuS2A==} - engines: {node: '>=6'} - es-define-property@1.0.1: resolution: {integrity: sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==} engines: {node: '>= 0.4'} @@ -5267,10 +5118,6 @@ packages: resolution: {integrity: sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==} engines: {node: '>= 0.4'} - es-set-tostringtag@2.1.0: - resolution: {integrity: sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==} - engines: {node: '>= 0.4'} - es-toolkit@1.49.0: resolution: {integrity: sha512-G5iZ6Pc/FNRY/soKZHC+TxGDD83rHUDXxzaWhGCX44vAv/tMs56WMusnm/KMNK+luUPsgA9U28cGr4RDlSzL2g==} @@ -5380,9 +5227,6 @@ packages: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} - exponential-backoff@3.1.3: - resolution: {integrity: sha512-ZgEeZXj30q+I0EN+CbSSpIyPaJ5HVQD18Z1m+u1FXbAeT94mr1zw50q4q6jiiC447Nl/YTcIYSAftiGqetwXCA==} - express-rate-limit@8.5.2: resolution: {integrity: sha512-5Kb34ipNX694DH48vN9irak1Qx30nb0PLYHXfJgw4YEjiC3ZEmZJhwOp+VfiCYwFzvFTdB9QkArYS5kXa2cx2A==} engines: {node: '>= 16'} @@ -5470,10 +5314,6 @@ packages: resolution: {integrity: sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==} engines: {node: '>=14'} - form-data@4.0.6: - resolution: {integrity: sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==} - engines: {node: '>= 6'} - formatly@0.3.0: resolution: {integrity: sha512-9XNj/o4wrRFyhSMJOvsuyMwy8aUfBaZ1VrqHVfohyXf0Sw0e+yfKG+xZaY3arGCOMdwFsqObtzVOc1gU9KiT9w==} engines: {node: '>=18.3.0'} @@ -5491,10 +5331,6 @@ packages: resolution: {integrity: sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A==} engines: {node: '>= 0.8'} - fs-minipass@3.0.3: - resolution: {integrity: sha512-XUBA9XClHbnJWSfBzjkm6RvPsyg3sryZt06BEQoXcF7EK/xpGaQYJgQKDJSUH5SGZ76Y7pFx1QBnXz09rU5Fbw==} - engines: {node: ^14.17.0 || ^16.13.0 || >=18.0.0} - fsevents@2.3.3: resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==} engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} @@ -5538,10 +5374,6 @@ packages: deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me hasBin: true - glob@13.0.6: - resolution: {integrity: sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==} - engines: {node: 18 || 20 || >=22} - globals@17.7.0: resolution: {integrity: sha512-Czmyns5dUsq4seFBR/Kdydhmo8y9kC79hiSkPn0YcGtNnYWnrgt0vjrSjx9tspoDGWm2CMarffRuLjM4xUz8xg==} engines: {node: '>=18'} @@ -5561,9 +5393,6 @@ packages: resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==} engines: {node: '>= 0.4'} - graceful-fs@4.2.11: - resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} - hachure-fill@0.5.2: resolution: {integrity: sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==} @@ -5580,10 +5409,6 @@ packages: resolution: {integrity: sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==} engines: {node: '>= 0.4'} - has-tostringtag@1.0.2: - resolution: {integrity: sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==} - engines: {node: '>= 0.4'} - hasown@2.0.4: resolution: {integrity: sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==} engines: {node: '>= 0.4'} @@ -5604,10 +5429,6 @@ packages: hookable@6.1.1: resolution: {integrity: sha512-U9LYDy1CwhMCnprUfeAZWZGByVbhd54hwepegYTK7Pi5NvqEj63ifz5z+xukznehT7i6NIZRu89Ay1AZmRsLEQ==} - hosted-git-info@10.1.1: - resolution: {integrity: sha512-DeOnSPAvOndYKfw075gt8yZzQ7S2hNztw34zBTfhIzLhmBTswIBg5/y+pqu/VD5cYWm5goAFTusDmUEmKZ0PEQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - html-encoding-sniffer@6.0.0: resolution: {integrity: sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg==} engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} @@ -5624,9 +5445,6 @@ packages: htmlparser2@6.1.0: resolution: {integrity: sha512-gyyPk6rgonLFEDGoeRgQNaEUvdJ4ktTmmUh/h2t7s+M8oPpIPxgNACWa+6ESR57kXstwqPiCut0V8NRpcwgU7A==} - http-cache-semantics@4.2.0: - resolution: {integrity: sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ==} - http-errors@2.0.1: resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==} engines: {node: '>= 0.8'} @@ -5635,18 +5453,10 @@ packages: resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} engines: {node: '>= 14'} - http-proxy-agent@9.1.0: - resolution: {integrity: sha512-2NxoveTT58mjYT4n3RPTEfCZGLMbidoO8XEieXfpSYxu+PQJ1qpx4ypwH6N+uF9twBPIvRRgvkvW5HUTYWENig==} - engines: {node: '>= 20'} - https-proxy-agent@7.0.6: resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==} engines: {node: '>= 14'} - https-proxy-agent@9.1.0: - resolution: {integrity: sha512-ag87y7cJJ9/3+GxFr8Oy4O5faDsGRGnBGsJj/YjOSsSx/5eadKLYTMPlzuR6obgoCDDm0abAAZitXXQkMOPSpA==} - engines: {node: '>= 20'} - iconv-lite@0.6.3: resolution: {integrity: sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==} engines: {node: '>=0.10.0'} @@ -5655,10 +5465,6 @@ packages: resolution: {integrity: sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==} engines: {node: '>=0.10.0'} - ignore-walk@9.0.0: - resolution: {integrity: sha512-tCBEZV2z2FNpIDl2vrhiWzIHzs4qOAuIDEO85eS02vZ3L1U3P56qpPL8GuGGAijDktAEaq2swMkO/Fmbo7YmfQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - ignore@5.3.2: resolution: {integrity: sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g==} engines: {node: '>= 4'} @@ -5684,10 +5490,6 @@ packages: inherits@2.0.4: resolution: {integrity: sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==} - ini@7.0.0: - resolution: {integrity: sha512-ifK0CgjALofS5bkrcTy4RaQ9Vx2Knf/eLeIO+NaswQEpH1UblrtTSCIvN71qQDMq0PeQ/SSPojvEJp9vvvfr+w==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - internmap@1.0.1: resolution: {integrity: sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw==} @@ -5731,10 +5533,6 @@ packages: isexe@2.0.0: resolution: {integrity: sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==} - isexe@4.0.0: - resolution: {integrity: sha512-FFUtZMpoZ8RqHS3XeXEmHWLA4thH+ZxCv2lOiPIn1Xc7CxrqhWzNSDzD+/chS/zbYezmiwWLdQC09JdQKmthOw==} - engines: {node: '>=20'} - istanbul-lib-coverage@3.2.2: resolution: {integrity: sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==} engines: {node: '>=8'} @@ -5825,10 +5623,6 @@ packages: json-buffer@3.0.1: resolution: {integrity: sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==} - json-parse-even-better-errors@6.0.0: - resolution: {integrity: sha512-2/8adwnK1/+Fdjyts4r6wSpfANWw8zdNhU9U/Llk59c6O+DjSisPWPykwoL8gZmocP9Dy64S7oie2g+Mia123A==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - json-schema-to-ts@3.1.1: resolution: {integrity: sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==} engines: {node: '>=16'} @@ -5848,10 +5642,6 @@ packages: jsonc-parser@3.3.1: resolution: {integrity: sha512-HUgH65KyejrUFPvHFPbqOY0rsFip3Bo5wb4ngvdi1EpCYWUQDC5V+Y7mZws+DLkr4M//zQJoanu1SP+87Dv1oQ==} - jsonparse@1.3.1: - resolution: {integrity: sha512-POQXvpdL69+CluYsillJ7SUhKvytYjW9vG/GKpnf+xP8UWgYEM/RaMzHHofbALDiKbbP1W8UEYmgGl39WkPZsg==} - engines: {'0': node >= 0.2.0} - jsx-ast-utils-x@0.1.0: resolution: {integrity: sha512-eQQBjBnsVtGacsG9uJNB8qOr3yA8rga4wAaGG1qRcBzSIvfhERLrWxMAM1hp5fcS6Abo8M4+bUBTekYR0qTPQw==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} @@ -6059,10 +5849,6 @@ packages: resolution: {integrity: sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==} engines: {node: '>=10'} - make-fetch-happen@16.0.1: - resolution: {integrity: sha512-uUv1yxHzaKVVEPfcFeGSNov/Cehjv08ovlY8ImTljgL7Q+SiA0dAYLQ6SYVa2kkKqNj4Y3aZEI7xv2teadie0A==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - mark.js@8.11.1: resolution: {integrity: sha512-1I+1qpDt4idfgLQG+BNWmrqku+7/2bi5nLf4YwF8y8zXvmfiTBY3PV3ZibfrjBueCByROpuBjLLFCajqkgYoLQ==} @@ -6225,18 +6011,10 @@ packages: micromark@4.0.2: resolution: {integrity: sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA==} - mime-db@1.52.0: - resolution: {integrity: sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==} - engines: {node: '>= 0.6'} - mime-db@1.54.0: resolution: {integrity: sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==} engines: {node: '>= 0.6'} - mime-types@2.1.35: - resolution: {integrity: sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==} - engines: {node: '>= 0.6'} - mime-types@3.0.2: resolution: {integrity: sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==} engines: {node: '>=18'} @@ -6257,30 +6035,6 @@ packages: minimist@1.2.8: resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==} - minipass-collect@2.0.1: - resolution: {integrity: sha512-D7V8PO9oaz7PWGLbCACuI1qEOsq7UKfLotx/C0Aet43fCUB/wfQ7DYeq2oR/svFJGYDHPr38SHATeaj/ZoKHKw==} - engines: {node: '>=16 || 14 >=14.17'} - - minipass-fetch@6.0.0: - resolution: {integrity: sha512-AWI8bKapGmgx/J0E6IGYSKj8TiHebZkmKWSs8raPSw8KXwgEAJ+Bw3+LSdXHR6T/RHKAWCOYk2MiLrYluaUU6w==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - minipass-flush@1.0.7: - resolution: {integrity: sha512-TbqTz9cUwWyHS2Dy89P3ocAGUGxKjjLuR9z8w4WUTGAVgEj17/4nhgo2Du56i0Fm3Pm30g4iA8Lcqctc76jCzA==} - engines: {node: '>= 8'} - - minipass-pipeline@1.2.4: - resolution: {integrity: sha512-xuIq7cIOt09RPRJ19gdi4b+RiNvDFYe5JH+ggNvBqGqpQXcru3PcRmOZuHBKWK1Txf9+cQ+HMVN4d6z46LZP7A==} - engines: {node: '>=8'} - - minipass-sized@2.0.0: - resolution: {integrity: sha512-zSsHhto5BcUVM2m1LurnXY6M//cGhVaegT71OfOXoprxT6o780GZd792ea6FfrQkuU4usHZIUczAQMRUE2plzA==} - engines: {node: '>=8'} - - minipass@3.3.6: - resolution: {integrity: sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==} - engines: {node: '>=8'} - minipass@7.1.3: resolution: {integrity: sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==} engines: {node: '>=16 || 14 >=14.17'} @@ -6288,20 +6042,12 @@ packages: minisearch@7.2.0: resolution: {integrity: sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==} - minizlib@3.1.0: - resolution: {integrity: sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==} - engines: {node: '>= 18'} - mitt@3.0.1: resolution: {integrity: sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==} mj-context-menu@0.6.1: resolution: {integrity: sha512-7NO5s6n10TIV96d4g2uDpG7ZDpIhMh0QNfGdJw/W47JswFcosz457wqz/b5sAKvl12sxINGFCn80NZHKwxQEXA==} - modern-tar@0.7.6: - resolution: {integrity: sha512-sweCIVXzx1aIGTCdzcMlSZt1h8k5Tmk08VNAuRk3IU28XamGiOH5ypi11g6De2CH7PhYqSSnGy2A/EFhbWnVKg==} - engines: {node: '>=18.0.0'} - mri@1.2.0: resolution: {integrity: sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA==} engines: {node: '>=4'} @@ -6410,44 +6156,6 @@ packages: resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} - node-gyp@13.0.1: - resolution: {integrity: sha512-piOr0S10qy5THB+q5BdqkoOx65XL/tjTMUAit3vciPNp+snTOBnGunWH1Rz7XZUxf2T9uFrfT/Ty4+aC3yPeyg==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - hasBin: true - - nopt@10.0.1: - resolution: {integrity: sha512-df3sBr/6ax9hSGuC3CspvLlbnX8cP5L5nZwXF8cGN8l0zSWR6BvzmQ6jPUKjvo6+/xdpkNvEcucBNUdBeeV13g==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - hasBin: true - - npm-bundled@6.0.0: - resolution: {integrity: sha512-EqdodKEW6pYM+dPxA66TZQfMEqVDiuzjDM9edSjuPI1mXUbUJwVxkgqMZSJvs8RTXz2CGq8HUol/AffTZX5g8w==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-install-checks@9.0.0: - resolution: {integrity: sha512-t05Izcgi7p15cpldqoiXYpjzlkTTvBw33sgjmL/JjcvtV0ydbm2O4iEXO8A6smqComu5FAQhUas86HTMQ6Z1Uw==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-normalize-package-bin@6.0.0: - resolution: {integrity: sha512-tdt4aFn9QamlhdN3HV2D2ccpBwO5/fyjjbXUxYA6uBjyekMZcZvDq0aSj9t5Jo+tih6AYFnt/cuIRn9013e0Uw==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-package-arg@14.0.0: - resolution: {integrity: sha512-69XQh3k+dtGa1p+7RaR57IuG3rCko96xr/nUfN4yDYBXbTYICiWcOpsFKLN2GtGE9cyIljE+f1exnaYt9MvM+Q==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-packlist@11.3.0: - resolution: {integrity: sha512-cS1yVkyriZgQAbiK8PtwhZHEtsFOsKHsCg5Ww2ONckAvXIspgqd6o4WirOzvkupU24iMRZ4xtO4kb2iK2rbnag==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-pick-manifest@12.0.0: - resolution: {integrity: sha512-8Fs3YLrnNOhrCdPNZy18MzNgVC58LTDAFzq1FdZO/p3BHeCC/coz+t4F5Pxabys8HJpyTUorMea26GkXsb4J/Q==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - - npm-registry-fetch@20.0.1: - resolution: {integrity: sha512-vzc1svxw/kw1IRjFsLi6gaxe1Olqm88V0tIfu2u5raL0b1gChe6ZEXNkyUlKxUC7s/egt5NxZHkbY18tMKKLfQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - nth-check@2.1.1: resolution: {integrity: sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==} @@ -6504,10 +6212,6 @@ packages: resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==} engines: {node: '>=10'} - p-map@7.0.5: - resolution: {integrity: sha512-e8vJF4XdVkzqqSHguEMz41mQO1wKwxKm5ENrUJQUu9kLDCtn83cxbyHZcszr4QC5zEA7WffRRC4gsTecC7J9oA==} - engines: {node: '>=18'} - p-retry@4.6.2: resolution: {integrity: sha512-312Id396EbJdvRONlngUx0NydfrIQ5lsYu0znKVUzVvArzEIt08V1qhtyESbGVd1FGX7UKtiFp5uwKZdM8wIuQ==} engines: {node: '>=8'} @@ -6518,11 +6222,6 @@ packages: package-manager-detector@1.6.0: resolution: {integrity: sha512-61A5ThoTiDG/C8s8UMZwSorAGwMJ0ERVGj2OjoW5pAalsNOg15+iQiPzrLJ4jhZ1HJzmC2PIHT2oEiH3R5fzNA==} - pacote@22.0.0: - resolution: {integrity: sha512-++VqeOZeL03uGM2MFLk96jGCSt1owBGkyFKoPr+trwNlZhCpjN2RrvwYxt8nTbs1wNMqSFYurq0TafVWkAIHig==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - hasBin: true - pako@1.0.11: resolution: {integrity: sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==} @@ -6561,10 +6260,6 @@ packages: resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==} engines: {node: '>=16 || 14 >=14.18'} - path-scurry@2.0.2: - resolution: {integrity: sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==} - engines: {node: 18 || 20 || >=22} - path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -6607,10 +6302,6 @@ packages: resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==} engines: {node: '>= 0.8.0'} - proc-log@7.0.0: - resolution: {integrity: sha512-FYgfaA69XZ93zaXLoMNQ+ViDXGGBgR8aLh03txzcFhV+9xOXx7+8DLCULrKKpR9+GsH9ZfHm82aSUPpozX0Ztg==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - process-nextick-args@2.0.1: resolution: {integrity: sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==} @@ -6625,15 +6316,6 @@ packages: resolution: {integrity: sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==} engines: {node: '>= 0.10'} - proxy-agent-negotiate@1.1.0: - resolution: {integrity: sha512-N8IBcM3UgCVzz2L2Lqv8DVntDnnC8/hiV4nEDUPkqq72TPUgYWjQc+bdZlBPZK9LzPAvOY//gAt0S0DApoOXWQ==} - engines: {node: '>= 20'} - peerDependencies: - kerberos: ^2.0.0 - peerDependenciesMeta: - kerberos: - optional: true - publint@0.3.21: resolution: {integrity: sha512-OqejcnMV6E9zel2oCrUOJEiiFkGiAAni0A6ibfQNh1k9Gu5z4F+Yso8lllam7AzmV6Do0vp7u3UpZNRBwuXaHQ==} engines: {node: '>=18'} @@ -6826,32 +6508,16 @@ packages: resolution: {integrity: sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==} engines: {node: '>=14'} - sigstore@5.0.0: - resolution: {integrity: sha512-hJqJfoG/e4qFQaauQL00c6J6FrHLBGKtkFvW3JbTSIEFOhLrSjdSM/gWd/yUOfYo/gsERehTXGC1VZWX+9X4Dg==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - sisteransi@1.0.5: resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} slick@1.12.2: resolution: {integrity: sha512-4qdtOGcBjral6YIBCWJ0ljFSKNLz9KkhbWtuGvUyRowl1kxfuE1x/Z/aJcaiilpb3do9bl5K7/1h9XC5wWpY/A==} - smart-buffer@4.2.0: - resolution: {integrity: sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==} - engines: {node: '>= 6.0.0', npm: '>= 3.0.0'} - smol-toml@1.6.1: resolution: {integrity: sha512-dWUG8F5sIIARXih1DTaQAX4SsiTXhInKf1buxdY9DIg4ZYPZK5nGM1VRIYmEbDbsHt7USo99xSLFu5Q1IqTmsg==} engines: {node: '>= 18'} - socks-proxy-agent@10.1.0: - resolution: {integrity: sha512-WlMj/67cEJ6MDI1OcsnjuYKDNDoyPCCYZ249kuuXPiMDw9F8PXkVaQ7YWu3siTydfQ/4BEZcvGzu+aYvz7dDCQ==} - engines: {node: '>= 20'} - - socks@2.8.9: - resolution: {integrity: sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw==} - engines: {node: '>= 10.0.0', npm: '>= 3.0.0'} - source-map-js@1.2.1: resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} engines: {node: '>=0.10.0'} @@ -6863,15 +6529,6 @@ packages: space-separated-tokens@2.0.2: resolution: {integrity: sha512-PEGlAwrG8yXGXRjW32fGbg66JAlOAwbObuqVoJpv/mRgoWDQfgH1wDPvtzWyUSNAXBGSk8h755YDbbcEy3SH2Q==} - spdx-exceptions@2.5.0: - resolution: {integrity: sha512-PiU42r+xO4UbUS1buo3LPJkjlO7430Xn5SVAhdpzzsPHsjbYVflnnFdATgabnLude+Cqu25p6N+g2lw/PFsa4w==} - - spdx-expression-parse@4.0.0: - resolution: {integrity: sha512-Clya5JIij/7C6bRR22+tnGXbc4VKlibKSVj2iHvVeX5iMW7s1SIQlqu699JkODJJIhh/pUu8L0/VLh8xflD+LQ==} - - spdx-license-ids@3.0.23: - resolution: {integrity: sha512-CWLcCCH7VLu13TgOH+r8p1O/Znwhqv/dbb6lqWy67G+pT1kHmeD/+V36AVb/vq8QMIQwVShJ6Ssl5FPh0fuSdw==} - speakingurl@14.0.1: resolution: {integrity: sha512-1POYv7uv2gXoyGFpBCmpDVSNV74IfsWlDW216UPjbWufNf+bSU6GdbDsxdcxtfwb4xlI3yxzOTKClUosxARYrQ==} engines: {node: '>=0.10.0'} @@ -6880,10 +6537,6 @@ packages: resolution: {integrity: sha512-i/VCLG1fvRc95pMHRqG4aQNscv+9aIsqA2oI7ZQS51sTdUcDHYX6cpT8/tqZ+enjs1tKVwbRBWgxut9SWn+f9g==} hasBin: true - ssri@14.0.0: - resolution: {integrity: sha512-jQxKI0yx0ZnTKrqjKkLDV2DXkBQn3k49JVmVqDGcDwKDtGDbImD/GXsq04KD0VVzCQQ9wZJYal3RwR1GzWTSow==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - stackback@0.0.2: resolution: {integrity: sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==} @@ -6944,10 +6597,6 @@ packages: tabbable@6.5.0: resolution: {integrity: sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA==} - tar@7.5.20: - resolution: {integrity: sha512-9FcyK4PA6+WbzlTM9WhQm6vB5W7cP7dUiPsv1g7YDwEQnQ1CGpK3MGlKk/ITVWMk05kHZuBhmVhiv8LZoy/PFQ==} - engines: {node: '>=18'} - tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} @@ -7058,10 +6707,6 @@ packages: engines: {node: '>=18.0.0'} hasBin: true - tuf-js@6.0.0: - resolution: {integrity: sha512-zlJVOIO68hmgo1//X4ENEcTGfuOTAtDPi8PsTsG+FyxD85E/ww1ZnwBbWo/yCEExGpI+Kilg7Z3qCdHX2BoJTQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - type-check@0.4.0: resolution: {integrity: sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew==} engines: {node: '>= 0.8.0'} @@ -7107,10 +6752,6 @@ packages: resolution: {integrity: sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==} engines: {node: '>=20.18.1'} - undici@8.7.0: - resolution: {integrity: sha512-N7iQtfyLhIMOFgQubvmLV26svHpO0bqKnAiWotTQCVKCmWrcGbBotPuW1x+xwYZ2VHdSTVUfPQQnlEt1/LouTQ==} - engines: {node: '>=22.19.0'} - unist-util-is@6.0.1: resolution: {integrity: sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g==} @@ -7144,10 +6785,6 @@ packages: resolution: {integrity: sha512-jOWVmzVceKlVVdwjNSenT4PbGghU0SBIizAev8ofZVgivk/TVHXSbNL8LP6M3spZvkR9/QolkyJavGSX5Cs0UA==} engines: {node: '>=10'} - validate-npm-package-name@8.0.0: - resolution: {integrity: sha512-SCv6OOV6Xj2/3cXy3dGmADluJTNcL3o7hZAglNPTe+WYuEuvxgJzxPrSDLZhF+CwyQOubqgecjMmTJGMVLWjYQ==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - vary@1.1.2: resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} engines: {node: '>= 0.8'} @@ -7337,11 +6974,6 @@ packages: engines: {node: '>= 8'} hasBin: true - which@7.0.0: - resolution: {integrity: sha512-RancgH2dmbLdHl6LRhEqvklWMgl/Hdnun0Y90KhBOLkMefg8Qa7/Zel8Sm+8HEcP6DEjzsWzpkuBQEZok58isA==} - engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} - hasBin: true - why-is-node-running@2.3.0: resolution: {integrity: sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==} engines: {node: '>=8'} @@ -7391,13 +7023,6 @@ packages: xmlchars@2.2.0: resolution: {integrity: sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==} - yallist@4.0.0: - resolution: {integrity: sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==} - - yallist@5.0.0: - resolution: {integrity: sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==} - engines: {node: '>=18'} - yaml@2.9.0: resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==} engines: {node: '>= 14.6'} @@ -7835,10 +7460,6 @@ snapshots: '@bcoe/v8-coverage@1.0.2': {} - '@bluwy/giget-core@0.1.7': - dependencies: - modern-tar: 0.7.6 - '@braintree/sanitize-url@7.1.2': {} '@bramus/specificity@2.4.2': @@ -8166,8 +7787,6 @@ snapshots: '@exodus/bytes@1.15.1': {} - '@gar/promise-retry@1.0.3': {} - '@google/genai@1.52.0(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))': dependencies: google-auth-library: 10.7.0 @@ -8222,10 +7841,6 @@ snapshots: wrap-ansi: 8.1.0 wrap-ansi-cjs: wrap-ansi@7.0.0 - '@isaacs/fs-minipass@4.0.1': - dependencies: - minipass: 7.1.3 - '@jridgewell/gen-mapping@0.3.13': dependencies: '@jridgewell/sourcemap-codec': 1.5.5 @@ -8313,63 +7928,6 @@ snapshots: '@nodable/entities@2.2.0': {} - '@npmcli/agent@5.0.2': - dependencies: - agent-base: 9.0.0 - http-proxy-agent: 9.1.0 - https-proxy-agent: 9.1.0 - lru-cache: 11.5.1 - socks-proxy-agent: 10.1.0 - transitivePeerDependencies: - - kerberos - - supports-color - - '@npmcli/fs@6.0.0': - dependencies: - semver: 7.8.4 - - '@npmcli/git@8.0.0': - dependencies: - '@gar/promise-retry': 1.0.3 - '@npmcli/promise-spawn': 10.0.0 - ini: 7.0.0 - lru-cache: 11.5.1 - npm-pick-manifest: 12.0.0 - proc-log: 7.0.0 - semver: 7.8.4 - which: 7.0.0 - - '@npmcli/installed-package-contents@5.0.0': - dependencies: - npm-bundled: 6.0.0 - npm-normalize-package-bin: 6.0.0 - - '@npmcli/node-gyp@6.0.0': {} - - '@npmcli/package-json@8.0.0': - dependencies: - '@npmcli/git': 8.0.0 - glob: 13.0.6 - hosted-git-info: 10.1.1 - json-parse-even-better-errors: 6.0.0 - proc-log: 7.0.0 - semver: 7.8.4 - spdx-expression-parse: 4.0.0 - - '@npmcli/promise-spawn@10.0.0': - dependencies: - which: 7.0.0 - - '@npmcli/redact@5.0.0': {} - - '@npmcli/run-script@11.0.0': - dependencies: - '@npmcli/node-gyp': 6.0.0 - '@npmcli/package-json': 8.0.0 - '@npmcli/promise-spawn': 10.0.0 - node-gyp: 13.0.1 - proc-log: 7.0.0 - '@oxc-parser/binding-android-arm-eabi@0.133.0': optional: true @@ -8743,39 +8301,6 @@ snapshots: '@shikijs/vscode-textmate@10.0.2': {} - '@sigstore/bundle@5.0.0': - dependencies: - '@sigstore/protobuf-specs': 0.5.1 - - '@sigstore/core@4.0.1': {} - - '@sigstore/protobuf-specs@0.5.1': {} - - '@sigstore/sign@5.0.0': - dependencies: - '@gar/promise-retry': 1.0.3 - '@sigstore/bundle': 5.0.0 - '@sigstore/core': 4.0.1 - '@sigstore/protobuf-specs': 0.5.1 - make-fetch-happen: 16.0.1 - proc-log: 7.0.0 - transitivePeerDependencies: - - kerberos - - supports-color - - '@sigstore/tuf@5.0.0': - dependencies: - '@sigstore/protobuf-specs': 0.5.1 - tuf-js: 6.0.0 - transitivePeerDependencies: - - supports-color - - '@sigstore/verify@4.1.0': - dependencies: - '@sigstore/bundle': 5.0.0 - '@sigstore/core': 4.0.1 - '@sigstore/protobuf-specs': 0.5.1 - '@smithy/core@3.24.7': dependencies: '@aws-crypto/crc32': 5.2.0 @@ -8842,13 +8367,6 @@ snapshots: estraverse: 5.3.0 picomatch: 4.0.4 - '@tufjs/canonical-json@2.0.0': {} - - '@tufjs/models@5.0.0': - dependencies: - '@tufjs/canonical-json': 2.0.0 - minimatch: 10.2.5 - '@tybys/wasm-util@0.10.2': dependencies: tslib: 2.8.1 @@ -9022,11 +8540,6 @@ snapshots: '@types/ms@2.1.0': {} - '@types/node-fetch@2.6.13': - dependencies: - '@types/node': 22.20.0 - form-data: 4.0.6 - '@types/node@22.20.0': dependencies: undici-types: 6.21.0 @@ -9035,35 +8548,10 @@ snapshots: dependencies: undici-types: 7.24.6 - '@types/npm-package-arg@6.1.4': {} - - '@types/npm-registry-fetch@8.0.9': - dependencies: - '@types/node': 22.20.0 - '@types/node-fetch': 2.6.13 - '@types/npm-package-arg': 6.1.4 - '@types/npmlog': 7.0.0 - '@types/ssri': 7.1.5 - - '@types/npmlog@7.0.0': - dependencies: - '@types/node': 22.20.0 - - '@types/pacote@11.1.8': - dependencies: - '@types/node': 22.20.0 - '@types/npm-registry-fetch': 8.0.9 - '@types/npmlog': 7.0.0 - '@types/ssri': 7.1.5 - '@types/picomatch@3.0.2': {} '@types/retry@0.12.0': {} - '@types/ssri@7.1.5': - dependencies: - '@types/node': 22.20.0 - '@types/tough-cookie@4.0.5': {} '@types/trusted-types@2.0.7': @@ -9340,8 +8828,6 @@ snapshots: '@xmldom/xmldom@0.9.10': {} - abbrev@5.0.0: {} - accepts@2.0.0: dependencies: mime-types: 3.0.2 @@ -9355,8 +8841,6 @@ snapshots: agent-base@7.1.4: {} - agent-base@9.0.0: {} - ajv-formats@3.0.1(ajv@8.20.0): optionalDependencies: ajv: 8.20.0 @@ -9424,8 +8908,6 @@ snapshots: estree-walker: 3.0.3 js-tokens: 10.0.0 - asynckit@0.4.0: {} - balanced-match@1.0.2: {} balanced-match@4.0.4: {} @@ -9476,19 +8958,6 @@ snapshots: cac@7.0.0: {} - cacache@21.0.1: - dependencies: - '@npmcli/fs': 6.0.0 - fs-minipass: 3.0.3 - glob: 13.0.6 - lru-cache: 11.5.1 - minipass: 7.1.3 - minipass-collect: 2.0.1 - minipass-flush: 1.0.7 - minipass-pipeline: 1.2.4 - p-map: 7.0.5 - ssri: 14.0.0 - call-bind-apply-helpers@1.0.2: dependencies: es-errors: 1.3.0 @@ -9531,18 +9000,12 @@ snapshots: dependencies: readdirp: 4.1.2 - chownr@3.0.0: {} - color-convert@2.0.1: dependencies: color-name: 1.1.4 color-name@1.1.4: {} - combined-stream@1.0.8: - dependencies: - delayed-stream: 1.0.0 - comma-separated-tokens@2.0.3: {} commander@13.1.0: {} @@ -9848,8 +9311,6 @@ snapshots: dependencies: robust-predicates: 3.0.3 - delayed-stream@1.0.0: {} - depd@2.0.0: {} dequal@2.0.3: {} @@ -9924,8 +9385,6 @@ snapshots: entities@8.0.0: {} - env-paths@2.2.1: {} - es-define-property@1.0.1: {} es-errors@1.3.0: {} @@ -9936,13 +9395,6 @@ snapshots: dependencies: es-errors: 1.3.0 - es-set-tostringtag@2.1.0: - dependencies: - es-errors: 1.3.0 - get-intrinsic: 1.3.0 - has-tostringtag: 1.0.2 - hasown: 2.0.4 - es-toolkit@1.49.0: {} esbuild@0.21.5: @@ -10117,8 +9569,6 @@ snapshots: expect-type@1.3.0: {} - exponential-backoff@3.1.3: {} - express-rate-limit@8.5.2(express@5.2.1): dependencies: express: 5.2.1 @@ -10242,14 +9692,6 @@ snapshots: cross-spawn: 7.0.6 signal-exit: 4.1.0 - form-data@4.0.6: - dependencies: - asynckit: 0.4.0 - combined-stream: 1.0.8 - es-set-tostringtag: 2.1.0 - hasown: 2.0.4 - mime-types: 2.1.35 - formatly@0.3.0: dependencies: fd-package-json: 2.0.0 @@ -10262,10 +9704,6 @@ snapshots: fresh@2.0.0: {} - fs-minipass@3.0.3: - dependencies: - minipass: 7.1.3 - fsevents@2.3.3: optional: true @@ -10328,12 +9766,6 @@ snapshots: package-json-from-dist: 1.0.1 path-scurry: 1.11.1 - glob@13.0.6: - dependencies: - minimatch: 10.2.5 - minipass: 7.1.3 - path-scurry: 2.0.2 - globals@17.7.0: {} globrex@0.1.2: {} @@ -10353,8 +9785,6 @@ snapshots: gopd@1.2.0: {} - graceful-fs@4.2.11: {} - hachure-fill@0.5.2: {} handlebars@4.7.9: @@ -10370,10 +9800,6 @@ snapshots: has-symbols@1.1.0: {} - has-tostringtag@1.0.2: - dependencies: - has-symbols: 1.1.0 - hasown@2.0.4: dependencies: function-bind: 1.1.2 @@ -10402,10 +9828,6 @@ snapshots: hookable@6.1.1: {} - hosted-git-info@10.1.1: - dependencies: - lru-cache: 11.5.1 - html-encoding-sniffer@6.0.0: dependencies: '@exodus/bytes': 1.15.1 @@ -10430,8 +9852,6 @@ snapshots: domutils: 2.8.0 entities: 2.2.0 - http-cache-semantics@4.2.0: {} - http-errors@2.0.1: dependencies: depd: 2.0.0 @@ -10447,15 +9867,6 @@ snapshots: transitivePeerDependencies: - supports-color - http-proxy-agent@9.1.0: - dependencies: - agent-base: 9.0.0 - debug: 4.4.3 - proxy-agent-negotiate: 1.1.0 - transitivePeerDependencies: - - kerberos - - supports-color - https-proxy-agent@7.0.6: dependencies: agent-base: 7.1.4 @@ -10463,15 +9874,6 @@ snapshots: transitivePeerDependencies: - supports-color - https-proxy-agent@9.1.0: - dependencies: - agent-base: 9.0.0 - debug: 4.4.3 - proxy-agent-negotiate: 1.1.0 - transitivePeerDependencies: - - kerberos - - supports-color - iconv-lite@0.6.3: dependencies: safer-buffer: 2.1.2 @@ -10480,10 +9882,6 @@ snapshots: dependencies: safer-buffer: 2.1.2 - ignore-walk@9.0.0: - dependencies: - minimatch: 10.2.5 - ignore@5.3.2: {} ignore@7.0.5: {} @@ -10498,8 +9896,6 @@ snapshots: inherits@2.0.4: {} - ini@7.0.0: {} - internmap@1.0.1: {} internmap@2.0.3: {} @@ -10526,8 +9922,6 @@ snapshots: isexe@2.0.0: {} - isexe@4.0.0: {} - istanbul-lib-coverage@3.2.2: {} istanbul-lib-report@3.0.1: @@ -10620,8 +10014,6 @@ snapshots: json-buffer@3.0.1: {} - json-parse-even-better-errors@6.0.0: {} - json-schema-to-ts@3.1.1: dependencies: '@babel/runtime': 7.29.7 @@ -10637,8 +10029,6 @@ snapshots: jsonc-parser@3.3.1: {} - jsonparse@1.3.1: {} - jsx-ast-utils-x@0.1.0: {} jszip@3.10.1: @@ -10830,24 +10220,6 @@ snapshots: dependencies: semver: 7.8.4 - make-fetch-happen@16.0.1: - dependencies: - '@gar/promise-retry': 1.0.3 - '@npmcli/agent': 5.0.2 - '@npmcli/redact': 5.0.0 - cacache: 21.0.1 - http-cache-semantics: 4.2.0 - minipass: 7.1.3 - minipass-fetch: 6.0.0 - minipass-flush: 1.0.7 - minipass-pipeline: 1.2.4 - negotiator: 1.0.0 - proc-log: 7.0.0 - ssri: 14.0.0 - transitivePeerDependencies: - - kerberos - - supports-color - mark.js@8.11.1: {} markdown-it-mathjax3@4.3.2: @@ -11209,14 +10581,8 @@ snapshots: transitivePeerDependencies: - supports-color - mime-db@1.52.0: {} - mime-db@1.54.0: {} - mime-types@2.1.35: - dependencies: - mime-db: 1.52.0 - mime-types@3.0.2: dependencies: mime-db: 1.54.0 @@ -11233,48 +10599,14 @@ snapshots: minimist@1.2.8: {} - minipass-collect@2.0.1: - dependencies: - minipass: 7.1.3 - - minipass-fetch@6.0.0: - dependencies: - minipass: 7.1.3 - minipass-sized: 2.0.0 - minizlib: 3.1.0 - optionalDependencies: - iconv-lite: 0.7.3 - - minipass-flush@1.0.7: - dependencies: - minipass: 3.3.6 - - minipass-pipeline@1.2.4: - dependencies: - minipass: 3.3.6 - - minipass-sized@2.0.0: - dependencies: - minipass: 7.1.3 - - minipass@3.3.6: - dependencies: - yallist: 4.0.0 - minipass@7.1.3: {} minisearch@7.2.0: {} - minizlib@3.1.0: - dependencies: - minipass: 7.1.3 - mitt@3.0.1: {} mj-context-menu@0.6.1: {} - modern-tar@0.7.6: {} - mri@1.2.0: {} ms@2.1.3: {} @@ -11359,67 +10691,6 @@ snapshots: fetch-blob: 3.2.0 formdata-polyfill: 4.0.10 - node-gyp@13.0.1: - dependencies: - env-paths: 2.2.1 - exponential-backoff: 3.1.3 - graceful-fs: 4.2.11 - nopt: 10.0.1 - proc-log: 7.0.0 - semver: 7.8.4 - tar: 7.5.20 - tinyglobby: 0.2.17 - undici: 8.7.0 - which: 7.0.0 - - nopt@10.0.1: - dependencies: - abbrev: 5.0.0 - - npm-bundled@6.0.0: - dependencies: - npm-normalize-package-bin: 6.0.0 - - npm-install-checks@9.0.0: - dependencies: - semver: 7.8.4 - - npm-normalize-package-bin@6.0.0: {} - - npm-package-arg@14.0.0: - dependencies: - hosted-git-info: 10.1.1 - proc-log: 7.0.0 - semver: 7.8.4 - validate-npm-package-name: 8.0.0 - - npm-packlist@11.3.0: - dependencies: - glob: 13.0.6 - ignore-walk: 9.0.0 - proc-log: 7.0.0 - - npm-pick-manifest@12.0.0: - dependencies: - npm-install-checks: 9.0.0 - npm-normalize-package-bin: 6.0.0 - npm-package-arg: 14.0.0 - semver: 7.8.4 - - npm-registry-fetch@20.0.1: - dependencies: - '@npmcli/redact': 5.0.0 - jsonparse: 1.3.1 - make-fetch-happen: 16.0.1 - minipass: 7.1.3 - minipass-fetch: 6.0.0 - minizlib: 3.1.0 - npm-package-arg: 14.0.0 - proc-log: 7.0.0 - transitivePeerDependencies: - - kerberos - - supports-color - nth-check@2.1.1: dependencies: boolbase: 1.0.0 @@ -11513,8 +10784,6 @@ snapshots: dependencies: p-limit: 3.1.0 - p-map@7.0.5: {} - p-retry@4.6.2: dependencies: '@types/retry': 0.12.0 @@ -11524,29 +10793,6 @@ snapshots: package-manager-detector@1.6.0: {} - pacote@22.0.0: - dependencies: - '@gar/promise-retry': 1.0.3 - '@npmcli/git': 8.0.0 - '@npmcli/installed-package-contents': 5.0.0 - '@npmcli/package-json': 8.0.0 - '@npmcli/promise-spawn': 10.0.0 - '@npmcli/run-script': 11.0.0 - cacache: 21.0.1 - fs-minipass: 3.0.3 - minipass: 7.1.3 - npm-package-arg: 14.0.0 - npm-packlist: 11.3.0 - npm-pick-manifest: 12.0.0 - npm-registry-fetch: 20.0.1 - proc-log: 7.0.0 - sigstore: 5.0.0 - ssri: 14.0.0 - tar: 7.5.20 - transitivePeerDependencies: - - kerberos - - supports-color - pako@1.0.11: {} parse5-htmlparser2-tree-adapter@6.0.1: @@ -11576,11 +10822,6 @@ snapshots: lru-cache: 10.4.3 minipass: 7.1.3 - path-scurry@2.0.2: - dependencies: - lru-cache: 11.5.1 - minipass: 7.1.3 - path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -11610,8 +10851,6 @@ snapshots: prelude-ls@1.2.1: {} - proc-log@7.0.0: {} - process-nextick-args@2.0.1: {} property-information@7.2.0: {} @@ -11635,8 +10874,6 @@ snapshots: forwarded: 0.2.0 ipaddr.js: 1.9.1 - proxy-agent-negotiate@1.1.0: {} - publint@0.3.21: dependencies: '@publint/pack': 0.1.4 @@ -11920,54 +11157,18 @@ snapshots: signal-exit@4.1.0: {} - sigstore@5.0.0: - dependencies: - '@sigstore/bundle': 5.0.0 - '@sigstore/core': 4.0.1 - '@sigstore/protobuf-specs': 0.5.1 - '@sigstore/sign': 5.0.0 - '@sigstore/tuf': 5.0.0 - '@sigstore/verify': 4.1.0 - transitivePeerDependencies: - - kerberos - - supports-color - sisteransi@1.0.5: {} slick@1.12.2: {} - smart-buffer@4.2.0: {} - smol-toml@1.6.1: {} - socks-proxy-agent@10.1.0: - dependencies: - agent-base: 9.0.0 - debug: 4.4.3 - socks: 2.8.9 - transitivePeerDependencies: - - supports-color - - socks@2.8.9: - dependencies: - ip-address: 10.2.0 - smart-buffer: 4.2.0 - source-map-js@1.2.1: {} source-map@0.6.1: {} space-separated-tokens@2.0.2: {} - spdx-exceptions@2.5.0: {} - - spdx-expression-parse@4.0.0: - dependencies: - spdx-exceptions: 2.5.0 - spdx-license-ids: 3.0.23 - - spdx-license-ids@3.0.23: {} - speakingurl@14.0.1: {} speech-rule-engine@4.1.4: @@ -11976,10 +11177,6 @@ snapshots: commander: 13.1.0 wicked-good-xpath: 1.3.0 - ssri@14.0.0: - dependencies: - minipass: 7.1.3 - stackback@0.0.2: {} statuses@2.0.2: {} @@ -12037,14 +11234,6 @@ snapshots: tabbable@6.5.0: {} - tar@7.5.20: - dependencies: - '@isaacs/fs-minipass': 4.0.1 - chownr: 3.0.0 - minipass: 7.1.3 - minizlib: 3.1.0 - yallist: 5.0.0 - tinybench@2.9.0: {} tinyexec@1.2.4: {} @@ -12125,14 +11314,6 @@ snapshots: optionalDependencies: fsevents: 2.3.3 - tuf-js@6.0.0: - dependencies: - '@gar/promise-retry': 1.0.3 - '@tufjs/models': 5.0.0 - debug: 4.4.3 - transitivePeerDependencies: - - supports-color - type-check@0.4.0: dependencies: prelude-ls: 1.2.1 @@ -12174,8 +11355,6 @@ snapshots: undici@7.28.0: {} - undici@8.7.0: {} - unist-util-is@6.0.1: dependencies: '@types/unist': 3.0.3 @@ -12211,8 +11390,6 @@ snapshots: valid-data-url@3.0.1: {} - validate-npm-package-name@8.0.0: {} - vary@1.1.2: {} vfile-message@4.0.3: @@ -12436,10 +11613,6 @@ snapshots: dependencies: isexe: 2.0.0 - which@7.0.0: - dependencies: - isexe: 4.0.0 - why-is-node-running@2.3.0: dependencies: siginfo: 2.0.0 @@ -12473,10 +11646,6 @@ snapshots: xmlchars@2.2.0: {} - yallist@4.0.0: {} - - yallist@5.0.0: {} - yaml@2.9.0: {} yocto-queue@0.1.0: {} diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 004338fa2d..125b06c28c 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -52,7 +52,6 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, 'packages/sdk/helper': { kind: 'none', reason: 'The project domain edits files and registers no live agent or model surface.' }, - 'packages/sdk/plugin-fetch': { kind: 'none', reason: 'The fetcher acquires plugin sources into a temp dir and registers no live agent or model surface.' }, 'packages/sdk/scripts': { kind: 'indirect', reason: 'The launcher delegates model context to the loaded project plugin tree.' }, 'packages/sdk/telemetry': { kind: 'none', reason: 'The launcher-side reporter sends developer-cycle telemetry and registers no live agent or model surface.' }, 'packages/session-query/session-query': { kind: 'none', reason: 'The trusted query service exposes cloned records only to callers and registers no model surface.' }, diff --git a/tsconfig.build.json b/tsconfig.build.json index 9359d7a531..386728bf3b 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -99,7 +99,6 @@ { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, { "path": "./packages/sdk/create-sdk" }, - { "path": "./packages/sdk/plugin-fetch" }, { "path": "./packages/sdk/telemetry" } ] } diff --git a/tsconfig.json b/tsconfig.json index 48c5187fdc..47d1067e88 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -110,7 +110,6 @@ { "path": "./packages/sdk/helper" }, { "path": "./packages/sdk/scripts" }, { "path": "./packages/sdk/create-sdk" }, - { "path": "./packages/sdk/plugin-fetch" }, { "path": "./packages/sdk/telemetry" } ] } From 16485392e48dc7fed028d948be6339e89dcc1d53 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 15:43:56 +0800 Subject: [PATCH 21/73] docs(sdk): retarget #2 to native npm/pnpm dependency + cordis mount --- docs/sdk-后续工作-设计.md | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/docs/sdk-后续工作-设计.md b/docs/sdk-后续工作-设计.md index a2096cf42f..d281052cee 100644 --- a/docs/sdk-后续工作-设计.md +++ b/docs/sdk-后续工作-设计.md @@ -10,7 +10,7 @@ | 块 | 做什么 | 核心对象 | 结论要点 | |---|---|---|---| | **#1 headless + skill** | create/config 非交互化，agent 端到端建项目 | `HeadlessPrompter` + `CreationDriver`(NDJSON) | 无 spec 文件、传结构化对象；薄 SKILL.md 入口；beyond-eve | -| **#2 建插件** | `dsh-sdk create ` 拉插件并接线 | `PluginSource` + `PluginFetcher`(giget/pacote) | 只解压不执行、锁版本、经 `ProjectEditSession` 显式接线 | +| **#2 建插件** | `dsh-sdk create ` 加依赖并挂载 | PM 原生 `add` + `ProjectEditSession` cordis 挂载 | npm/pnpm 原生依赖（`github:#sha` / `pkg@version`），不用 giget/pacote | | **#3 遥测** | 每个 `dsh-sdk` 命令上报 | `TelemetryReporter` / `ConsentResolver` / `SecretRedactor` | 发 cordis.yml+package.json 全文；不发 `.env`、疑似密钥脱敏；关闭 = cordis.yml 有明确 disabled 的遥测条目（甲）| | **#4 交互测试** | 覆盖 wizard 各分支、快照 cordis.yml | `WizardHarness` + clack mock 注入 | 注入流为主、真 PTY 仅 1–2 个可选 smoke | @@ -106,23 +106,17 @@ SDK 初版（`packages/sdk/*`）已经落地三个包： - **skill**：核心是 headless 内核；agent 传参直接建完，缺必答项就响亮失败让 agent 补答。附一层**薄 SKILL.md**（指向内核、教 agent 驱动），让"通过 skill 创建"字面落地。 - **比 eve 更进一步**：eve 把 headless 原语（`runHeadless` + 非阻塞 Prompter + NDJSON）造好了，却没接到它的 skill——它的 SKILL.md 只指向半交互 CLI，且 agent 跑 `eve init` 时只打印指引、打回给人。我们把 **skill → headless 内核接通**，才真正做到"headless 为 skill 服务"。 -### 4.2 `dsh-sdk create ` 建插件（#2） +### 4.2 `dsh-sdk create ` 建插件（#2，简单版） -**目标**：从 github repo 或 npm 包拉一个插件进现有项目并接线；安全第一。 +**目标**：把一个 github repo 或 npm 包当**依赖**加进现有项目并挂载；用包管理器原生能力，**不引 giget/pacote**。 -**设计（只解压不执行 + 锁版本 + 显式接线）**： +**设计（PM 原生依赖 + cordis 挂载）**： -- **`PluginSource`（判别联合）**：`GithubSource`（`owner/repo[/subdir]#ref`）| `NpmSource`（`pkg@version`）。由 spec 字符串解析而来。 -- **`PluginFetcher`（seam）**：把源抓进 temp 目录，**绝不执行被拉代码的生命周期脚本**。 - - `GigetFetcher`（github/git）：giget；`#ref` 先解析成 commit SHA 再下、记进 lock。 - - `PacoteFetcher`（npm）：pacote `extract`（只解包不跑 postinstall），带 `integrity` 校验。来源类型限定放在**上游 `resolvePluginSource`**（只产出 `name@version`）作为主保证，不依赖 pacote 的 `allowRegistry`（`@types/pacote` 无此选项，且 registry tarball extract 本就不跑脚本）。 -- **接线（显式可审）**： - 1. `package.json` 精确锁版本（npm：exact + integrity；github：`github:owner/repo#`）。 - 2. 经 `ProjectEditSession` 改 `cordis.yml` 挂插件——**给 diff、要确认**再写。 - 3. `install --ignore-scripts`（pnpm v10 默认亦拦依赖 build 脚本）。 - 4. 打印清单（dep spec + 锁的 ref/integrity + cordis.yml diff）。 -- **信任模型**：学 `npm create` 的手感，但把信任反过来——**confirm-before-run，而非 run-on-fetch**。 -- **repo 初始化模式**（从模板仓库整体建项目）同走 giget（优于 degit——degit 的 `degit.json` 会自动跑动作）；建远程新仓可用 `gh repo create --template`。注：eve 不支持 template-repo init，这是我们的自有取舍。 +- **来源**：npm（`pkg@version`）或 github（`github:owner/repo#ref`，推荐锁 commit SHA）。npm/pnpm/yarn 原生支持这两种依赖来源，自己解析包名、把 commit/integrity 钉进 lockfile。 +- **流程**：`dsh-sdk create ` → 确认（TTY guard 同 config）→ 用项目的包管理器 `add `（PM 解析名字 / 装依赖 / 写 lockfile）→ 读回新增依赖名 → 经 `ProjectEditSession` 挂一条 cordis 条目引用它 → commit。 +- **不落 `plugins/`**：外部插件是 node_modules 依赖，不是本地生成插件（后者才走 `LocalPluginBlueprint` + 文件生成）。 +- **构建张力（暂缓）**：源码型 github 插件装时要 `prepare` build（pnpm 10.26 默认禁、需 `allowBuilds` 放行）——"预编译-only vs 允许构建"的取舍留到以后；先做能跑的简单版，交给 PM 默认行为。 +- **弃用**：早期设计的 `PluginSource`/`GigetFetcher`/`PacoteFetcher`（抓 tarball 到 temp 再接线）已随 `dsh-plugin-fetch` 包一起撤掉——PM 原生依赖覆盖了它。 ### 4.3 遥测（#3） @@ -185,7 +179,7 @@ SDK 初版（`packages/sdk/*`）已经落地三个包： |---|---|---| | 地基 | dsh-helper, create-sdk, dsh-scripts | `HeadlessPromptPort`（实现已有 `PromptPort`）+ prefill 补全 + NDJSON + 命令注册/催表扩展点整理 | | #1 headless+skill | create-sdk, dsh-scripts, （新）skill 包 | 结构化 spec 入口 `--config-json`/`--config`、薄 SKILL.md | -| #2 建插件 | （新）fetcher 包, dsh-scripts, dsh-helper | `PluginSource`、`GigetFetcher`/`PacoteFetcher`、`dsh-sdk create ` 注册、经 `ProjectEditSession` 接线+diff、新依赖 giget/pacote | +| #2 建插件 | dsh-scripts, dsh-helper | `dsh-sdk create ` 命令、PM `add`、经 `ProjectEditSession` 挂 cordis 条目、无新依赖 | | #3 遥测 | （新）telemetry 包, dsh-scripts, dsh-helper | `TelemetryReporter`/`ConsentResolver`/`SecretRedactor`、launcher 接线、催表遥测 feature、内置 endpoint、全局 UUID | | #4 测试 | packages/support, （已可注入）create-sdk/dsh-scripts | `WizardHarness`、create/config 的 `test.each` cordis.yml 快照、可选 PTY smoke | From ca7533880e8c90ef7e812d426e82cc2838c5aeb1 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 16:01:48 +0800 Subject: [PATCH 22/73] =?UTF-8?q?feat(dsh-sdk):=20create=20=20?= =?UTF-8?q?=E2=80=94=20add=20external=20plugin=20as=20native=20PM=20depend?= =?UTF-8?q?ency=20+=20cordis=20mount?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit dsh-sdk create adds a github (github:owner/repo#ref) or npm (pkg@version) plugin as a package-manager-native dependency, then mounts the resolved dependency in cordis.yml through ProjectEditSession. Adds PackageManager.add(spec) and ProjectEditSession.addExternalPlugin(id, packageName). No giget/pacote. Per-file 100% coverage on the new/changed files. --- .../src/package-managers/package-manager.ts | 24 +++++ .../src/project/project-edit-session.ts | 17 ++++ packages/sdk/helper/tests/documents.spec.ts | 5 + packages/sdk/helper/tests/project.spec.ts | 22 +++++ packages/sdk/scripts/README.md | 1 + packages/sdk/scripts/src/args.ts | 6 +- packages/sdk/scripts/src/command.ts | 5 + packages/sdk/scripts/src/create-plugin.ts | 91 +++++++++++++++++++ packages/sdk/scripts/tests/scripts.spec.ts | 60 ++++++++++++ 9 files changed, 230 insertions(+), 1 deletion(-) create mode 100644 packages/sdk/scripts/src/create-plugin.ts diff --git a/packages/sdk/helper/src/package-managers/package-manager.ts b/packages/sdk/helper/src/package-managers/package-manager.ts index a6f2fdd103..6e6eda478a 100644 --- a/packages/sdk/helper/src/package-managers/package-manager.ts +++ b/packages/sdk/helper/src/package-managers/package-manager.ts @@ -148,6 +148,25 @@ export abstract class PackageManager { await this.runChecked(runner, this.buildCommand(), cwd, 'build') } + /** + * Build add-dependency command arguments for one already-normalized source spec. + * @param spec - a package-manager-native dependency source (`pkg@version` or `github:owner/repo#ref`). + * @returns arguments following the manager executable. + */ + addCommand(spec: string): readonly string[] { + return ['add', spec] + } + + /** + * Add one dependency from a native source spec and fail on non-zero or signalled exit. + * @param spec - a package-manager-native dependency source. + * @param cwd - project directory. + * @param runner - optional subprocess boundary. + */ + async add(spec: string, cwd: string, runner: CommandRunner = new NodeCommandRunner()): Promise { + await this.runChecked(runner, this.addCommand(spec), cwd, 'add') + } + private async runChecked(runner: CommandRunner, args: readonly string[], cwd: string, operation: string): Promise { const result = await runner.run(this.name, args, cwd) if (result.signal !== null) { @@ -184,6 +203,11 @@ export class NpmPackageManager extends PackageManager { override linkSpec(relativePath: string): string { return `file:${relativePath}` } + + /** npm adds a dependency through `install ` rather than an `add` verb. */ + override addCommand(spec: string): readonly string[] { + return ['install', spec] + } } /** pnpm workspace behavior. */ diff --git a/packages/sdk/helper/src/project/project-edit-session.ts b/packages/sdk/helper/src/project/project-edit-session.ts index 9b2c885ac5..0d0a1cb6dd 100644 --- a/packages/sdk/helper/src/project/project-edit-session.ts +++ b/packages/sdk/helper/src/project/project-edit-session.ts @@ -220,6 +220,23 @@ export class ProjectEditSession implements FeatureProjectView { this.addedPlugins.add(entry.id) } + /** + * Mount a Cordis entry for an external dependency the package manager has already + * added (github or npm), without generating files or re-adding the dependency. + * @param id - stable Cordis config entry id. + * @param packageName - the installed dependency's package name. + */ + addExternalPlugin(id: string, packageName: string): void { + this.assertOpen() + if (!this.manifest().npmDependency(packageName)) { + throw new Error(`external plugin dependency is not installed: ${packageName}`) + } + const cordis = this.cordis() + if (cordis.entry(id)) throw new Error(`Cordis config entry already exists: ${id}`) + cordis.addEntry({ id, name: packageName }) + this.addedPlugins.add(id) + } + /** Enable or disable one custom/manual Cordis config entry by stable id. */ setCustomPluginDisabled(id: string, disabled: boolean): void { this.assertOpen() diff --git a/packages/sdk/helper/tests/documents.spec.ts b/packages/sdk/helper/tests/documents.spec.ts index 1ba8b9bd68..3ce5eeccea 100644 --- a/packages/sdk/helper/tests/documents.spec.ts +++ b/packages/sdk/helper/tests/documents.spec.ts @@ -298,6 +298,11 @@ describe('package manager strategies', () => { await npm.install('/tmp', runner) await npm.build('/tmp', runner) expect(calls).toEqual([['npm', 'install'], ['npm', 'run', 'build']]) + await npm.add('some-pkg@1.0.0', '/tmp', runner) + const pnpm = createPackageManager('pnpm', '10.0.0') + await pnpm.add('github:o/r#sha', '/tmp', runner) + expect(calls).toContainEqual(['npm', 'install', 'some-pkg@1.0.0']) + expect(calls).toContainEqual(['pnpm', 'add', 'github:o/r#sha']) const failed: CommandRunner = { run: async () => ({ exitCode: 2, signal: null }) } await expect(npm.install('/tmp', failed)).rejects.toThrow('exited with code 2') const killed: CommandRunner = { run: async () => ({ exitCode: null, signal: 'SIGTERM' }) } diff --git a/packages/sdk/helper/tests/project.spec.ts b/packages/sdk/helper/tests/project.spec.ts index d55bb16be9..6a6268b554 100644 --- a/packages/sdk/helper/tests/project.spec.ts +++ b/packages/sdk/helper/tests/project.spec.ts @@ -690,6 +690,28 @@ describe('SdkProject and ProjectEditSession', () => { expect(committed.packageManifest().dependencies?.['@deepseek-ai/dsh-subagent']).toMatch(/^file:/) expect(createBuiltinRegistry(committed.profile).get(featureId('subagent')).inspect(committed).state).toBe('absent') }) + + it('mounts an external plugin dependency and rejects missing deps or duplicate entries', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-external-plugin-')) + temporary.push(root) + const creation = request() + const project = SdkProject.create(root, creation) + const registry = createBuiltinRegistry(project.profile) + const edit = project.edit(registry) + for (const item of creation.features) edit.installFeature(registry.get(item.id), item) + await edit.commit() + const manifestPath = join(root, 'package.json') + const manifest = JSON.parse(await readFile(manifestPath, 'utf8')) as { dependencies?: Record } + manifest.dependencies = { ...manifest.dependencies, 'ext-plugin': 'github:o/r#sha' } + await writeFile(manifestPath, JSON.stringify(manifest, null, 2)) + const reopened = await SdkProject.open(root) + const edit2 = reopened.edit(createBuiltinRegistry(reopened.profile)) + edit2.addExternalPlugin('ext-plugin', 'ext-plugin') + expect(() => { edit2.addExternalPlugin('ext-plugin', 'ext-plugin') }).toThrow('already exists') + expect(() => { edit2.addExternalPlugin('missing', 'not-a-dep') }).toThrow('not installed') + const commit = await edit2.commit() + expect(commit.project.cordis.entry('ext-plugin')?.name).toBe('ext-plugin') + }) }) describe('extension points', () => { diff --git a/packages/sdk/scripts/README.md b/packages/sdk/scripts/README.md index e87c375a4f..39d706177e 100644 --- a/packages/sdk/scripts/README.md +++ b/packages/sdk/scripts/README.md @@ -8,6 +8,7 @@ The `dsh-sdk` launcher owns SDK project startup and configuration. | `dsh-sdk dev [target] [-- args…]` | Register TypeScript and local-workspace source resolution, then use the start path | | `dsh-sdk build [args…]` | Invoke the project's installed tsdown with the project arguments | | `dsh-sdk config` | Open one interactive edit session, review accumulated changes, commit once, and install once when NPM dependencies changed | +| `dsh-sdk create ` | Add an external Cordis plugin from a native package-manager source (`pkg@version` or `github:owner/repo#ref`): confirm, ` add `, then mount the resolved dependency in `cordis.yml`. No giget/pacote; the package manager resolves and pins the source (github deps build via their own `prepare` under the manager's policy) | `ProjectBuild(tsdownConfig)` and `PluginBuild(tsdownConfig)` are exported only from `@deepseek-ai/dsh-scripts/dev/tsdown-config`. Development and production read the same `cordis.yml`. diff --git a/packages/sdk/scripts/src/args.ts b/packages/sdk/scripts/src/args.ts index 1b91269592..4d1ce867de 100644 --- a/packages/sdk/scripts/src/args.ts +++ b/packages/sdk/scripts/src/args.ts @@ -8,12 +8,13 @@ import { parseArgs as parseNodeArgs } from 'node:util' import { Command } from 'commander' /** Commands implemented by the dsh-sdk launcher. */ -type DshSdkCommand = 'start' | 'dev' | 'build' | 'config' +type DshSdkCommand = 'start' | 'dev' | 'build' | 'config' | 'create' /** Parsed dsh-sdk invocation. */ export interface DshSdkArgs { command?: DshSdkCommand target?: string + source?: string forwarded: readonly string[] help: boolean } @@ -60,6 +61,9 @@ export function parseDshSdkArgs(argv: readonly string[]): DshSdkArgs { program.command('config').helpOption(false).action(() => { parsed = { command: 'config', forwarded: [], help: false } }) + program.command('create ').helpOption(false).action((source: string) => { + parsed = { command: 'create', source, forwarded: [], help: false } + }) program.parse([...launcherArgv], { from: 'user' }) /* v8 ignore next -- every registered Commander action above assigns parsed or Commander throws */ if (!parsed) throw new Error('dsh-sdk command did not resolve') diff --git a/packages/sdk/scripts/src/command.ts b/packages/sdk/scripts/src/command.ts index 9351cfa39c..5174ec5ded 100644 --- a/packages/sdk/scripts/src/command.ts +++ b/packages/sdk/scripts/src/command.ts @@ -7,6 +7,7 @@ import { parseDshSdkArgs } from './args.ts' import { runProjectBuild } from './build.ts' import { runConfigCommand, type ConfigCommandContext } from './config.ts' +import { runCreatePluginCommand } from './create-plugin.ts' import { runSDK } from './runtime.ts' import { DSH_SDK_TEMPLATES } from './templates/dsh-sdk-templates.ts' @@ -19,6 +20,7 @@ export interface DshSdkCommandContext extends ConfigCommandContext { run?: typeof runSDK build?: typeof runProjectBuild config?: typeof runConfigCommand + createPlugin?: typeof runCreatePluginCommand } /** Run one parsed dsh-sdk command and return its process exit code. */ @@ -40,6 +42,7 @@ export async function runDshSdkCommand( const run = context.run ?? runSDK const build = context.build ?? runProjectBuild const config = context.config ?? runConfigCommand + const createPlugin = context.createPlugin ?? runCreatePluginCommand switch (args.command) { case 'start': await run(args.target, { cwd: context.cwd, argv: args.forwarded }); break case 'dev': await run(args.target, { cwd: context.cwd, dev: true, argv: args.forwarded }); break @@ -49,6 +52,8 @@ export async function runDshSdkCommand( if (result.installError) return 1 break } + /* v8 ignore next -- Commander requires , so create never dispatches without it */ + case 'create': await createPlugin(args.source ?? '', context); break } return 0 } catch (error) { diff --git a/packages/sdk/scripts/src/create-plugin.ts b/packages/sdk/scripts/src/create-plugin.ts new file mode 100644 index 0000000000..f658451fde --- /dev/null +++ b/packages/sdk/scripts/src/create-plugin.ts @@ -0,0 +1,91 @@ +/** + * dsh-sdk create command: add an external Cordis plugin (github or npm) as a + * native package-manager dependency and mount it in cordis.yml. + * + * @module @deepseek-ai/dsh-scripts/create-plugin + */ + +import { readFile } from 'node:fs/promises' +import { join } from 'node:path' +import { + ClackPromptPort, + ConfirmQuestion, + SdkProject, + createBuiltinRegistry, + requireAnswer, + type PackageManager, + type ProjectCommitResult, + type PromptPort, +} from '@deepseek-ai/dsh-helper' + +/** Process and interaction slice required by dsh-sdk create. */ +export interface CreatePluginContext { + cwd: string + stdin: NodeJS.ReadStream + stdout: NodeJS.WriteStream + port?: PromptPort + add?: (manager: PackageManager, spec: string, cwd: string) => Promise +} + +/** Result of a create run; `undefined` when the confirmation was declined. */ +export type CreatePluginResult = ProjectCommitResult | undefined + +/** Derive a stable cordis entry id from a package name's last path segment. */ +function pluginId(packageName: string): string { + const base = packageName.startsWith('@') ? packageName.slice(packageName.indexOf('/') + 1) : packageName + const id = base.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '') + /* v8 ignore next -- a valid npm package name always yields a non-empty id */ + if (!id) throw new Error(`cannot derive a plugin id from package name: ${packageName}`) + return id +} + +/** Read the direct dependency names declared in a project's package.json. */ +async function dependencyNames(cwd: string): Promise> { + const manifest = JSON.parse(await readFile(join(cwd, 'package.json'), 'utf8')) as { + dependencies?: Record + } + /* v8 ignore next -- generated projects always declare a dependencies map */ + return new Set(Object.keys(manifest.dependencies ?? {})) +} + +/** + * Add one external plugin dependency to the current project and mount it. + * @param source - a package-manager-native source (`pkg@version` or `github:owner/repo#ref`). + * @param context - process, interaction, and dependency-add boundaries. + * @returns the commit result, or `undefined` when the confirmation was declined. + */ +export async function runCreatePluginCommand( + source: string, + context: CreatePluginContext, +): Promise { + const spec = source.trim() + if (!spec) throw new Error('dsh-sdk create requires a plugin source (pkg@version or github:owner/repo#ref)') + if (!context.port && (!context.stdin.isTTY || !context.stdout.isTTY)) { + throw new Error('dsh-sdk create requires an interactive TTY') + } + const project = await SdkProject.open(context.cwd) + /* v8 ignore next -- production TTY wiring is exercised by the built-bin smoke */ + const port = context.port ?? new ClackPromptPort(context.stdin, context.stdout) + const confirmed = requireAnswer(await new ConfirmQuestion({ + id: 'create.confirm', + message: `Add plugin '${spec}' as a dependency and mount it in cordis.yml?`, + initialValue: true, + }).resolve(port)) + if (!confirmed) return undefined + + const before = await dependencyNames(context.cwd) + /* v8 ignore next -- production package-manager wiring is exercised by the built-bin smoke */ + const add = context.add ?? ((manager, source, cwd) => manager.add(source, cwd)) + await add(project.profile.packageManager, spec, context.cwd) + const after = await dependencyNames(context.cwd) + const added = [...after].filter(name => !before.has(name)) + if (added.length === 0) throw new Error(`dsh-sdk create: '${spec}' added no new dependency`) + + const reopened = await SdkProject.open(context.cwd) + const registry = createBuiltinRegistry(reopened.profile) + const edit = reopened.edit(registry) + for (const packageName of added) edit.addExternalPlugin(pluginId(packageName), packageName) + const commit = await edit.commit() + context.stdout.write(`Mounted ${added.join(', ')} in cordis.yml.\n`) + return commit +} diff --git a/packages/sdk/scripts/tests/scripts.spec.ts b/packages/sdk/scripts/tests/scripts.spec.ts index 42c74f5631..faa025926e 100644 --- a/packages/sdk/scripts/tests/scripts.spec.ts +++ b/packages/sdk/scripts/tests/scripts.spec.ts @@ -31,6 +31,7 @@ import { PluginBuild, ProjectBuild, runProjectBuild } from '../src/build.ts' import { runDshSdkCommand, type DshSdkCommandContext } from '../src/command.ts' import { runConfigCommand } from '../src/config.ts' import { ConfigWorkflow, type ConfigPlan } from '../src/config/config-workflow.ts' +import { runCreatePluginCommand } from '../src/create-plugin.ts' import { initialize, resolve as resolveLocalPlugin } from '../src/local-plugin-loader-hooks.ts' const temporary: string[] = [] @@ -559,3 +560,62 @@ describe('ConfigWorkflow', () => { expect(output.read()).toContain('Disable feature: ask-user') }) }) + +describe('dsh-sdk create', () => { + const writeDependency = (name: string) => async (_m: unknown, spec: string, cwd: string): Promise => { + const path = join(cwd, 'package.json') + const manifest = JSON.parse(await readFile(path, 'utf8')) as { dependencies?: Record } + manifest.dependencies = { ...manifest.dependencies, [name]: spec } + await writeFile(path, JSON.stringify(manifest, null, 2)) + } + + it('adds a dependency and mounts it after confirmation', async () => { + const project = await committedProject() + const context = { ...commandContext(project.root), port: new QueuePort([true]), add: writeDependency('my-ext-plugin') } + const result = await runCreatePluginCommand('github:o/r#sha', context) + expect(result?.project.cordis.entry('my-ext-plugin')?.name).toBe('my-ext-plugin') + expect(context.readStdout()).toContain('Mounted my-ext-plugin') + }) + + it('derives the cordis id from a scoped package name', async () => { + const project = await committedProject() + const context = { ...commandContext(project.root), port: new QueuePort([true]), add: writeDependency('@acme/cool-plugin') } + const result = await runCreatePluginCommand('@acme/cool-plugin@1.0.0', context) + expect(result?.project.cordis.entry('cool-plugin')?.name).toBe('@acme/cool-plugin') + }) + + it('returns undefined and adds nothing when declined', async () => { + const project = await committedProject() + let added = false + const context = { + ...commandContext(project.root), + port: new QueuePort([false]), + add: async () => { added = true }, + } + await expect(runCreatePluginCommand('pkg@1.0.0', context)).resolves.toBeUndefined() + expect(added).toBe(false) + }) + + it('rejects an empty source, a non-TTY session, and a no-op add', async () => { + const project = await committedProject() + await expect(runCreatePluginCommand(' ', { ...commandContext(project.root), port: new QueuePort([]) })) + .rejects.toThrow('requires a plugin source') + const noTty = commandContext(project.root) + noTty.stdin.isTTY = false + noTty.stdout.isTTY = false + await expect(runCreatePluginCommand('pkg@1.0.0', noTty)).rejects.toThrow('interactive TTY') + const noOutTty = commandContext(project.root) + noOutTty.stdout.isTTY = false + await expect(runCreatePluginCommand('pkg@1.0.0', noOutTty)).rejects.toThrow('interactive TTY') + await expect(runCreatePluginCommand('pkg@1.0.0', { + ...commandContext(project.root), port: new QueuePort([true]), add: async () => {}, + })).rejects.toThrow('added no new dependency') + }) + + it('dispatches create through the launcher', async () => { + const project = await committedProject() + const context = commandContext(project.root) + context.createPlugin = async () => undefined + await expect(runDshSdkCommand(['create', 'pkg@1.0.0'], context)).resolves.toBe(0) + }) +}) From 560ce3b539bc24ea994cfbe9476c21ecd834162e Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Fri, 17 Jul 2026 20:28:31 +0800 Subject: [PATCH 23/73] feat(dsh-sdk): launcher telemetry reporting around every command Wrap runDshSdkCommand so each command times itself and, in a finally block, resolves consent (option A) and sends one best-effort, fire-and-forget telemetry event (redacted cordis.yml + package.json content; never reads .env). Never affects the command's exit code. Adds dsh-scripts -> dsh-telemetry dependency. Default-on via absent consent entry; opt-out by a disabled telemetry entry. The config/create wizard opt-out toggle is deferred (see design doc). --- docs/sdk-后续工作-设计.md | 3 ++ packages/sdk/scripts/package.json | 1 + packages/sdk/scripts/src/command.ts | 15 +++++- packages/sdk/scripts/src/telemetry.ts | 63 ++++++++++++++++++++++ packages/sdk/scripts/tests/scripts.spec.ts | 47 ++++++++++++++++ packages/sdk/scripts/tsconfig.json | 1 + pnpm-lock.yaml | 3 ++ 7 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 packages/sdk/scripts/src/telemetry.ts diff --git a/docs/sdk-后续工作-设计.md b/docs/sdk-后续工作-设计.md index d281052cee..1395ece5e0 100644 --- a/docs/sdk-后续工作-设计.md +++ b/docs/sdk-后续工作-设计.md @@ -133,6 +133,9 @@ SDK 初版（`packages/sdk/*`）已经落地三个包： - **endpoint**：内置在代码里。 - **consent 承载**：遥测作为 `create` 时默认打开的 feature 写进 cordis.yml（对用户可见、随项目）。 +> **接线现状（读码修正）**：launcher 上报已接通——`runDshSdkCommand` 计时包住每条命令，`finally` 里 resolve consent（甲）→ 建 redacted payload（cordis.yml+package.json 全文、不读 .env）→ fire-and-forget 上报 + flush，best-effort 永不影响命令结果。**默认开**：无遥测条目 → 甲 → 上报。**opt-out 现状**：在 cordis.yml 手动加一条 `disabled` 的 `@deepseek-ai/dsh-telemetry` 条目即关（`ConsentResolver` 读到 disabled → 不报；disabled 条目 cordis 不加载，故不会因"它不是运行时插件"而 boot 失败）。 +> **暂缓（催表 opt-out 开关）**：把"关遥测"做成 config/create 向导里的勾选项还没做。关键约束：`@deepseek-ai/dsh-telemetry` 是 **launcher 库、不是 cordis 运行时插件**，所以 consent 条目只能以 **disabled 形态**存在（enabled=无条目=甲默认报；要关才写 disabled 条目），不能像普通 feature 那样挂一个 enabled 的可 boot 条目。向导化这个"只在关闭时才出现条目"的特殊语义留作后续。 + **在案取舍**：发全文会把第三方（含私有 scoped）包名、cordis 配置值（base-url/路径）暴露给 endpoint 持有方；主流工具都不发这些（Turbo 排除包名、Angular 禁模块名）。ccyu 作为本 SDK 维护者接受此暴露——目的即掌握开发者用了哪些 plugin/依赖/配置。 ### 4.4 交互测试（#4） diff --git a/packages/sdk/scripts/package.json b/packages/sdk/scripts/package.json index ba441a528c..6fdbcbc3da 100644 --- a/packages/sdk/scripts/package.json +++ b/packages/sdk/scripts/package.json @@ -32,6 +32,7 @@ "license": "BSD-3-Clause", "dependencies": { "@deepseek-ai/dsh-helper": "workspace:^", + "@deepseek-ai/dsh-telemetry": "workspace:^", "commander": "^15.0.0", "node-addon-require-builtin": "^0.1.0" }, diff --git a/packages/sdk/scripts/src/command.ts b/packages/sdk/scripts/src/command.ts index 5174ec5ded..ebf9b06846 100644 --- a/packages/sdk/scripts/src/command.ts +++ b/packages/sdk/scripts/src/command.ts @@ -9,6 +9,7 @@ import { runProjectBuild } from './build.ts' import { runConfigCommand, type ConfigCommandContext } from './config.ts' import { runCreatePluginCommand } from './create-plugin.ts' import { runSDK } from './runtime.ts' +import { reportCommandTelemetry, type CommandTelemetryEvent } from './telemetry.ts' import { DSH_SDK_TEMPLATES } from './templates/dsh-sdk-templates.ts' /** Injectable process and command boundaries used by the dsh-sdk bin. */ @@ -21,6 +22,7 @@ export interface DshSdkCommandContext extends ConfigCommandContext { build?: typeof runProjectBuild config?: typeof runConfigCommand createPlugin?: typeof runCreatePluginCommand + telemetry?: (event: CommandTelemetryEvent) => Promise } /** Run one parsed dsh-sdk command and return its process exit code. */ @@ -33,12 +35,16 @@ export async function runDshSdkCommand( stderr: process.stderr, }, ): Promise { + const startedAt = Date.now() + let command: string | undefined + let success = true try { const args = parseDshSdkArgs(argv) if (args.help || !args.command) { context.stdout.write(DSH_SDK_TEMPLATES.usage.render({})) return 0 } + command = args.command const run = context.run ?? runSDK const build = context.build ?? runProjectBuild const config = context.config ?? runConfigCommand @@ -49,7 +55,7 @@ export async function runDshSdkCommand( case 'build': await build(args.forwarded, context.cwd); break case 'config': { const result = await config(context) - if (result.installError) return 1 + if (result.installError) { success = false; return 1 } break } /* v8 ignore next -- Commander requires , so create never dispatches without it */ @@ -57,7 +63,14 @@ export async function runDshSdkCommand( } return 0 } catch (error) { + success = false context.stderr.write(`dsh-sdk: ${error instanceof Error ? error.message : String(error)}\n`) return 1 + } finally { + if (command !== undefined) { + /* v8 ignore next -- production telemetry wiring is exercised by the built-bin smoke */ + const telemetry = context.telemetry ?? reportCommandTelemetry + await telemetry({ command, cwd: context.cwd, durationMs: Date.now() - startedAt, success }) + } } } diff --git a/packages/sdk/scripts/src/telemetry.ts b/packages/sdk/scripts/src/telemetry.ts new file mode 100644 index 0000000000..1ef74fdbb7 --- /dev/null +++ b/packages/sdk/scripts/src/telemetry.ts @@ -0,0 +1,63 @@ +/** + * Launcher-side telemetry wiring: resolve consent and send one fire-and-forget + * event around each dsh-sdk command. Best-effort — never affects the command's + * outcome or exit code. + * + * @module @deepseek-ai/dsh-scripts/telemetry + */ + +import { + ConsentResolver, + TelemetryReporter, + buildTelemetryPayload, + type ConsentDecision, +} from '@deepseek-ai/dsh-telemetry' + +/** One command's telemetry lifecycle facts. */ +export interface CommandTelemetryEvent { + /** The dsh-sdk command that ran. */ + command: string + /** Project directory whose consent, `cordis.yml`, and `package.json` are read. */ + cwd: string + /** Wall-clock duration in milliseconds. */ + durationMs: number + /** Whether the command completed without error. */ + success: boolean +} + +/** Injectable consent and delivery seams for tests. */ +export interface CommandTelemetryDeps { + resolve?: (cwd: string) => Promise + reporter?: Pick +} + +/** + * Resolve consent for the project and, when allowed, assemble and send one + * telemetry event, draining in-flight sends before returning. Swallows every + * error so telemetry can never change a command's result. + * @param event - the command lifecycle facts. + * @param deps - consent and delivery seams; defaults hit the real endpoint. + */ +export async function reportCommandTelemetry( + event: CommandTelemetryEvent, + deps: CommandTelemetryDeps = {}, +): Promise { + try { + /* v8 ignore next -- the production ConsentResolver is exercised by the built-bin smoke */ + const resolve = deps.resolve ?? (cwd => new ConsentResolver().resolve(cwd)) + const consent = await resolve(event.cwd) + if (!consent.allowed) return + const payload = await buildTelemetryPayload({ + command: event.command, + durationMs: event.durationMs, + success: event.success, + projectDir: event.cwd, + }) + /* v8 ignore next -- the production TelemetryReporter is exercised by the built-bin smoke */ + const reporter = deps.reporter ?? new TelemetryReporter() + reporter.report(payload, consent) + await reporter.flush() + } catch { + // Telemetry is best-effort; a consent, payload, or delivery fault never reaches the command. + } +} diff --git a/packages/sdk/scripts/tests/scripts.spec.ts b/packages/sdk/scripts/tests/scripts.spec.ts index faa025926e..3ab59f13d8 100644 --- a/packages/sdk/scripts/tests/scripts.spec.ts +++ b/packages/sdk/scripts/tests/scripts.spec.ts @@ -32,6 +32,7 @@ import { runDshSdkCommand, type DshSdkCommandContext } from '../src/command.ts' import { runConfigCommand } from '../src/config.ts' import { ConfigWorkflow, type ConfigPlan } from '../src/config/config-workflow.ts' import { runCreatePluginCommand } from '../src/create-plugin.ts' +import { reportCommandTelemetry, type CommandTelemetryEvent } from '../src/telemetry.ts' import { initialize, resolve as resolveLocalPlugin } from '../src/local-plugin-loader-hooks.ts' const temporary: string[] = [] @@ -619,3 +620,49 @@ describe('dsh-sdk create', () => { await expect(runDshSdkCommand(['create', 'pkg@1.0.0'], context)).resolves.toBe(0) }) }) + +describe('command telemetry', () => { + it('reports when consent allows and skips when denied or faulting', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-telemetry-')) + temporary.push(dir) + const sent: unknown[] = [] + const reporter = { report: () => { sent.push(1) }, flush: async () => {} } + await reportCommandTelemetry( + { command: 'build', cwd: dir, durationMs: 5, success: true }, + { resolve: async () => ({ allowed: true, reason: 'absent' }), reporter }, + ) + expect(sent).toHaveLength(1) + await reportCommandTelemetry( + { command: 'build', cwd: dir, durationMs: 5, success: true }, + { resolve: async () => ({ allowed: false, reason: 'disabled' }), reporter }, + ) + expect(sent).toHaveLength(1) + await expect(reportCommandTelemetry( + { command: 'build', cwd: dir, durationMs: 5, success: true }, + { resolve: async () => { throw new Error('boom') }, reporter }, + )).resolves.toBeUndefined() + expect(sent).toHaveLength(1) + }) + + it('emits a telemetry event carrying each command outcome', async () => { + const project = await committedProject() + const events: CommandTelemetryEvent[] = [] + const context = commandContext(project.root) + context.telemetry = async (event) => { events.push(event) } + context.build = async () => {} + await expect(runDshSdkCommand(['build'], context)).resolves.toBe(0) + expect(events).toHaveLength(1) + expect(events[0]).toMatchObject({ command: 'build', cwd: project.root, success: true }) + + await runDshSdkCommand([], context) + expect(events).toHaveLength(1) + + context.build = async () => { throw new Error('boom') } + await expect(runDshSdkCommand(['build'], context)).resolves.toBe(1) + expect(events[1]).toMatchObject({ command: 'build', success: false }) + + context.config = async () => ({ installError: new Error('offline') }) + await expect(runDshSdkCommand(['config'], context)).resolves.toBe(1) + expect(events.at(-1)).toMatchObject({ command: 'config', success: false }) + }) +}) diff --git a/packages/sdk/scripts/tsconfig.json b/packages/sdk/scripts/tsconfig.json index 848de9a314..461c86c06d 100644 --- a/packages/sdk/scripts/tsconfig.json +++ b/packages/sdk/scripts/tsconfig.json @@ -7,6 +7,7 @@ "include": ["src"], "references": [ { "path": "../helper" }, + { "path": "../telemetry" }, { "path": "../../ui/app-boot" }, { "path": "../../../vendor/cordis" } ] diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index cafebfaae6..68ee8e2640 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1228,6 +1228,9 @@ importers: '@deepseek-ai/dsh-helper': specifier: workspace:^ version: link:../helper + '@deepseek-ai/dsh-telemetry': + specifier: workspace:^ + version: link:../telemetry commander: specifier: ^15.0.0 version: 15.0.0 From ed418827c28845bc5e5d44ea9a5bcb6c275c24d9 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 18 Jul 2026 16:32:08 +0800 Subject: [PATCH 24/73] =?UTF-8?q?chore(sdk):=20satisfy=20CI=20gates=20?= =?UTF-8?q?=E2=80=94=20config=20catalog,=20md-wrap,=20knip?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Regenerate docs/config-catalog.md for the telemetry package, unwrap the design doc's multi-line blockquote paragraphs, and stop exporting the create-sdk headless spec internals that have no external consumer. --- docs/config-catalog.md | 1 + docs/sdk-后续工作-设计.md | 2 ++ packages/sdk/create-sdk/src/headless.ts | 4 ++-- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 3e3d8141fa..7cccb69e0a 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1432,4 +1432,5 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them. - `@deepseek-ai/dsh-scripts` ([`packages/sdk/scripts/src/index.ts`](../packages/sdk/scripts/src/index.ts)) - `@deepseek-ai/dsh-subagent-inprocess` ([`packages/subagent/subagent-inprocess/src/index.ts`](../packages/subagent/subagent-inprocess/src/index.ts)) - `@deepseek-ai/dsh-subagent-subprocess` ([`packages/subagent/subagent-subprocess/src/index.ts`](../packages/subagent/subagent-subprocess/src/index.ts)) +- `@deepseek-ai/dsh-telemetry` ([`packages/sdk/telemetry/src/index.ts`](../packages/sdk/telemetry/src/index.ts)) - `@deepseek-ai/dsh-timeout` ([`packages/util/timeout/src/index.ts`](../packages/util/timeout/src/index.ts)) diff --git a/docs/sdk-后续工作-设计.md b/docs/sdk-后续工作-设计.md index 1395ece5e0..5f968c27ee 100644 --- a/docs/sdk-后续工作-设计.md +++ b/docs/sdk-后续工作-设计.md @@ -1,6 +1,7 @@ # DeepSeek Harness SDK 后续工作设计 > 状态：设计成稿，供通读与评审。定案后由 ccyu 转正式 RFC 并双语化。本文件为临时设计文档，不走 doc-sync / 文档预算门禁。 +> > 一句话：**SDK 初版已合并；本轮把"创建项目""创建插件""遥测""交互测试"四块补齐，核心是抽出一个既撑交互又撑 headless 的创建内核，其余三块围绕它扩展。** ## 0. 总览（一屏读完） @@ -134,6 +135,7 @@ SDK 初版（`packages/sdk/*`）已经落地三个包： - **consent 承载**：遥测作为 `create` 时默认打开的 feature 写进 cordis.yml（对用户可见、随项目）。 > **接线现状（读码修正）**：launcher 上报已接通——`runDshSdkCommand` 计时包住每条命令，`finally` 里 resolve consent（甲）→ 建 redacted payload（cordis.yml+package.json 全文、不读 .env）→ fire-and-forget 上报 + flush，best-effort 永不影响命令结果。**默认开**：无遥测条目 → 甲 → 上报。**opt-out 现状**：在 cordis.yml 手动加一条 `disabled` 的 `@deepseek-ai/dsh-telemetry` 条目即关（`ConsentResolver` 读到 disabled → 不报；disabled 条目 cordis 不加载，故不会因"它不是运行时插件"而 boot 失败）。 +> > **暂缓（催表 opt-out 开关）**：把"关遥测"做成 config/create 向导里的勾选项还没做。关键约束：`@deepseek-ai/dsh-telemetry` 是 **launcher 库、不是 cordis 运行时插件**，所以 consent 条目只能以 **disabled 形态**存在（enabled=无条目=甲默认报；要关才写 disabled 条目），不能像普通 feature 那样挂一个 enabled 的可 boot 条目。向导化这个"只在关闭时才出现条目"的特殊语义留作后续。 **在案取舍**：发全文会把第三方（含私有 scoped）包名、cordis 配置值（base-url/路径）暴露给 endpoint 持有方；主流工具都不发这些（Turbo 排除包名、Angular 禁模块名）。ccyu 作为本 SDK 维护者接受此暴露——目的即掌握开发者用了哪些 plugin/依赖/配置。 diff --git a/packages/sdk/create-sdk/src/headless.ts b/packages/sdk/create-sdk/src/headless.ts index e32a053457..164405e14f 100644 --- a/packages/sdk/create-sdk/src/headless.ts +++ b/packages/sdk/create-sdk/src/headless.ts @@ -15,7 +15,7 @@ import type { CreateArgs } from './args.ts' * (the interactive tree/suggests prompts are skipped). Absent required answers make * the run fail loud through `HeadlessPromptPort` rather than blocking. */ -export interface HeadlessCreateSpec { +interface HeadlessCreateSpec { directory?: string description?: string provider?: 'deepseek' | 'custom' @@ -43,7 +43,7 @@ function asRecord(value: unknown, source: string): Record { } /** Parse and shallow-validate a headless spec from JSON text. */ -export function parseHeadlessSpec(text: string, source: string): HeadlessCreateSpec { +function parseHeadlessSpec(text: string, source: string): HeadlessCreateSpec { let parsed: unknown try { parsed = JSON.parse(text) From 78f21bf703df0b32c1a4196b489565e30c6319e2 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 18 Jul 2026 21:25:56 +0800 Subject: [PATCH 25/73] fix(sdk): list create and headless flags in usage help dsh-sdk's usage template predates the create command and create-sdk's predates --config/--config-json/--json, so --help hid both surfaces the README already documents. Pin each with a help-output assertion. --- packages/sdk/create-sdk/src/templates/assets/usage.txt.tpl | 3 +++ packages/sdk/create-sdk/tests/create.spec.ts | 1 + packages/sdk/scripts/src/templates/assets/usage.txt.tpl | 1 + packages/sdk/scripts/tests/scripts.spec.ts | 1 + 4 files changed, 6 insertions(+) diff --git a/packages/sdk/create-sdk/src/templates/assets/usage.txt.tpl b/packages/sdk/create-sdk/src/templates/assets/usage.txt.tpl index 2b734eb753..32f4d5c6d2 100644 --- a/packages/sdk/create-sdk/src/templates/assets/usage.txt.tpl +++ b/packages/sdk/create-sdk/src/templates/assets/usage.txt.tpl @@ -9,3 +9,6 @@ Options: --interface --pm --install / --no-install + --config + --config-json + --json diff --git a/packages/sdk/create-sdk/tests/create.spec.ts b/packages/sdk/create-sdk/tests/create.spec.ts index 1f2cdb75fc..0e4abd37c4 100644 --- a/packages/sdk/create-sdk/tests/create.spec.ts +++ b/packages/sdk/create-sdk/tests/create.spec.ts @@ -477,6 +477,7 @@ describe('create command composition', () => { context.stdout.isTTY = false await expect(createProject(['--help'], context)).resolves.toBeUndefined() expect(context.readStdout()).toContain('Usage: create-sdk') + expect(context.readStdout()).toContain('--config-json ') expect(context.readStdout()).not.toContain('--link-workspace') await expect(createProject(argv('agent', false), context)).rejects.toThrow('interactive TTY') context.stdin.isTTY = true diff --git a/packages/sdk/scripts/src/templates/assets/usage.txt.tpl b/packages/sdk/scripts/src/templates/assets/usage.txt.tpl index d4122198f5..b372c65d17 100644 --- a/packages/sdk/scripts/src/templates/assets/usage.txt.tpl +++ b/packages/sdk/scripts/src/templates/assets/usage.txt.tpl @@ -5,3 +5,4 @@ Commands: dev [target] [-- args...] Start with TypeScript and local-plugin source resolution build [args...] Run the project's installed tsdown config Interactively edit project features + create Add an external plugin dependency (pkg@version or github:owner/repo#ref) and mount it in cordis.yml diff --git a/packages/sdk/scripts/tests/scripts.spec.ts b/packages/sdk/scripts/tests/scripts.spec.ts index 3ab59f13d8..8b887d74db 100644 --- a/packages/sdk/scripts/tests/scripts.spec.ts +++ b/packages/sdk/scripts/tests/scripts.spec.ts @@ -168,6 +168,7 @@ describe('Commander launcher arguments', () => { await expect(runDshSdkCommand(['unknown'], context)).resolves.toBe(1) await expect(runDshSdkCommand([], context)).resolves.toBe(0) expect(context.readStdout()).toContain('Usage: dsh-sdk') + expect(context.readStdout()).toContain('create ') const defaults = commandContext(root) await writeFile(join(root, 'main.mjs'), 'export function main() { return "ok" }\n') From 013db6e8f02eb8f4c6c0bdd0f21f62d69f2483da Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 18 Jul 2026 21:50:46 +0800 Subject: [PATCH 26/73] fix(create-sdk): keep --json stdout pure NDJSON The SKILL.md contract says every stdout line is one JSON event, but createProject wrote the Created/Next-steps templates to stdout and the default install/build path inherited the launcher's stdio, so package- manager child output interleaved with the event stream. Under --json, route human progress to stderr and run install/build through a NodeCommandRunner that redirects child stdout+stderr to stderr. --- packages/sdk/create-sdk/src/command.ts | 13 ++++++--- packages/sdk/create-sdk/tests/create.spec.ts | 18 +++++++++++++ .../src/package-managers/package-manager.ts | 27 ++++++++++++++++--- packages/sdk/helper/tests/documents.spec.ts | 14 ++++++++++ 4 files changed, 65 insertions(+), 7 deletions(-) diff --git a/packages/sdk/create-sdk/src/command.ts b/packages/sdk/create-sdk/src/command.ts index a7855ea027..9897db2a21 100644 --- a/packages/sdk/create-sdk/src/command.ts +++ b/packages/sdk/create-sdk/src/command.ts @@ -9,6 +9,7 @@ import { ClackPromptPort, HeadlessPromptError, HeadlessPromptPort, + NodeCommandRunner, PromptCancelledError, type PackageManagerVersionProbe, type PromptPort, @@ -45,6 +46,9 @@ export async function createProject( context: CreateCommandContext, ): Promise { const args = parseCreateArgs(argv) + // Under --json, stdout carries only NDJSON events: human-readable progress + // and package-manager child output move to stderr. + const progress = args.json === true ? context.stderr : context.stdout if (args.help) { context.stdout.write(CREATE_TEMPLATES.usage.render({})) return undefined @@ -64,7 +68,7 @@ export async function createProject( }) const resolved = await wizard.run() const result = await scaffoldProject(resolved.directory, resolved.request) - context.stdout.write(CREATE_TEMPLATES.created.render({ + progress.write(CREATE_TEMPLATES.created.render({ name: resolved.request.name, directory: resolved.directory, })) @@ -72,8 +76,9 @@ export async function createProject( try { if (context.setup) await context.setup(resolved) else { - await resolved.request.packageManager.install(resolved.directory) - await resolved.request.packageManager.build(resolved.directory) + const runner = args.json === true ? new NodeCommandRunner(context.stderr) : new NodeCommandRunner() + await resolved.request.packageManager.install(resolved.directory, runner) + await resolved.request.packageManager.build(resolved.directory, runner) } } catch (error) { context.stderr.write(CREATE_TEMPLATES.setupFailure.render({ @@ -84,7 +89,7 @@ export async function createProject( throw error } } - context.stdout.write(CREATE_TEMPLATES.nextSteps.render({ + progress.write(CREATE_TEMPLATES.nextSteps.render({ directory: resolved.directory, setupRequired: !resolved.install, ...packageManagerTemplateModel(resolved.request.packageManager), diff --git a/packages/sdk/create-sdk/tests/create.spec.ts b/packages/sdk/create-sdk/tests/create.spec.ts index 0e4abd37c4..362e5046f9 100644 --- a/packages/sdk/create-sdk/tests/create.spec.ts +++ b/packages/sdk/create-sdk/tests/create.spec.ts @@ -8,6 +8,7 @@ import { HeadlessPromptPort, LocalPluginBlueprint, featureId, + NodeCommandRunner, NpmPackageManager, type FeatureSelection, type NestedMultiSelectValue, @@ -511,6 +512,12 @@ describe('create command composition', () => { const okSpec = JSON.stringify({ ...base, directory: 'done-agent', provider: 'deepseek', apiKey: 'key', features: [] }) await expect(runCreateCommand(['--config-json', okSpec, '--json'], ok)).resolves.toBe(0) expect(ok.readStdout()).toContain('{"type":"done"}') + // stdout stays pure NDJSON: every line parses, human progress goes to stderr + for (const line of ok.readStdout().split('\n').filter(line => line.length > 0)) { + expect(() => { JSON.parse(line) }).not.toThrow() + } + expect(ok.readStderr()).toContain('Created done-agent') + expect(ok.readStderr()).toContain('Next: cd') const missing = commandContext(root) missing.stdin.isTTY = false @@ -564,6 +571,17 @@ describe('create command composition', () => { await createProject(argv('agent', true), context) expect(install).toHaveBeenCalledOnce() expect(build).toHaveBeenCalledOnce() + const spec = JSON.stringify({ + directory: 'json-agent', description: 'test', provider: 'deepseek', apiKey: 'key', + model: 'deepseek-v4-flash', interface: 'embed', pm: 'npm', install: true, features: [], + }) + const json = commandContext(root) + json.stdin.isTTY = false + json.stdout.isTTY = false + await createProject(['--config-json', spec, '--json'], json) + // json mode hands install/build a runner that redirects child output to stderr + expect(install).toHaveBeenCalledTimes(2) + expect(install.mock.calls[1]?.[1]).toBeInstanceOf(NodeCommandRunner) install.mockRestore() build.mockRestore() }) diff --git a/packages/sdk/helper/src/package-managers/package-manager.ts b/packages/sdk/helper/src/package-managers/package-manager.ts index 6e6eda478a..8d6b617977 100644 --- a/packages/sdk/helper/src/package-managers/package-manager.ts +++ b/packages/sdk/helper/src/package-managers/package-manager.ts @@ -58,17 +58,38 @@ export function scrubEnvironment(environment: NodeJS.ProcessEnv = process.env): /** Node child-process command runner with inherited stdio and quiescent completion. */ export class NodeCommandRunner implements CommandRunner { - /** Spawn one child and settle only after its exit. */ + private readonly output: NodeJS.WritableStream | undefined + + /** + * @param output - redirect target for child stdout+stderr; the child inherits + * this process's stdio when absent. Callers whose own stdout carries a machine + * protocol (create-sdk --json NDJSON) redirect child output to keep the + * protocol stream pure. + */ + constructor(output?: NodeJS.WritableStream) { + this.output = output + } + + /** Spawn one child and settle only after exit, with redirected stdio drained. */ run(command: string, args: readonly string[], cwd: string): Promise { return new Promise((resolve, reject) => { + const output = this.output + if (output === undefined) { + const child = spawn(command, [...args], { cwd, env: scrubEnvironment(), stdio: 'inherit', shell: false }) + child.once('error', reject) + child.once('exit', (exitCode, signal) => { resolve({ exitCode, signal }) }) + return + } const child = spawn(command, [...args], { cwd, env: scrubEnvironment(), - stdio: 'inherit', + stdio: ['inherit', 'pipe', 'pipe'], shell: false, }) + child.stdout.pipe(output, { end: false }) + child.stderr.pipe(output, { end: false }) child.once('error', reject) - child.once('exit', (exitCode, signal) => { resolve({ exitCode, signal }) }) + child.once('close', (exitCode, signal) => { resolve({ exitCode, signal }) }) }) } } diff --git a/packages/sdk/helper/tests/documents.spec.ts b/packages/sdk/helper/tests/documents.spec.ts index 3ce5eeccea..469887036a 100644 --- a/packages/sdk/helper/tests/documents.spec.ts +++ b/packages/sdk/helper/tests/documents.spec.ts @@ -1,6 +1,7 @@ import { chmod, mkdtemp, mkdir, readFile, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { Writable } from 'node:stream' import { afterEach, describe, expect, it } from 'vitest' import { CordisYamlFile, JsExpression } from '../src/documents/cordis-yaml-file.ts' import { EnvFile } from '../src/documents/env-file.ts' @@ -325,6 +326,19 @@ describe('package manager strategies', () => { const runner = new NodeCommandRunner() await expect(runner.run(process.execPath, ['-e', ''], root)).resolves.toEqual({ exitCode: 0, signal: null }) await expect(runner.run('missing-dsh-command', [], root)).rejects.toThrow() + let redirected = '' + const output = new Writable({ + write(chunk, _encoding, callback) { redirected += String(chunk); callback() }, + }) + const redirecting = new NodeCommandRunner(output) + await expect(redirecting.run( + process.execPath, + ['-e', 'process.stdout.write("child-out"); process.stderr.write("child-err")'], + root, + )).resolves.toEqual({ exitCode: 0, signal: null }) + expect(redirected).toContain('child-out') + expect(redirected).toContain('child-err') + await expect(redirecting.run('missing-dsh-command', [], root)).rejects.toThrow() }) it('discovers and rewrites a repository-local NPM dependency closure', async () => { From e674f4c20e21710b8b9ea14706c0955a2b681706 Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 18 Jul 2026 21:53:47 +0800 Subject: [PATCH 27/73] fix(telemetry): withhold package.json when cordis.yml is absent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit buildTelemetryPayload read the two reported files independently, so a dsh-sdk command mistakenly run in an arbitrary non-SDK directory (no cordis.yml, e.g. any unrelated repo) still uploaded that directory's package.json — dependency names and metadata of a project that never opted into the SDK toolchain. Gate the manifest on cordis.yml presence: without the config the directory is not an SDK project and its manifest is not ours to report. Consent semantics are unchanged. --- packages/sdk/telemetry/README.md | 2 +- packages/sdk/telemetry/src/payload.ts | 13 ++++++++++--- packages/sdk/telemetry/tests/payload.spec.ts | 12 +++++++++++- 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/packages/sdk/telemetry/README.md b/packages/sdk/telemetry/README.md index 43d13d3dcc..8ec3da2c78 100644 --- a/packages/sdk/telemetry/README.md +++ b/packages/sdk/telemetry/README.md @@ -6,7 +6,7 @@ Launcher-side telemetry primitives for the dsh-sdk toolchain. This is a plain li |---|---| | `SecretRedactor` | Conservative safety backstop: replaces secret-shaped values (secret-like keys, known token shapes, PEM blocks, URL credentials, high-entropy opaque tokens) with a placeholder in both parsed values (`redactValue`) and raw text (`redactText`). Never drops a field or line. | | `ConsentResolver` | Parses (never boots) a project `cordis.yml` and reads the telemetry entry's enabled/disabled state as consent; `DO_NOT_TRACK`/CI env force a hard opt-out. | -| `buildTelemetryPayload` | Assembles `{command, durationMs, success, cordisYmlContent, packageJsonContent}`, running the redactor over the full `cordis.yml` and `package.json` text. Never reads `.env`. | +| `buildTelemetryPayload` | Assembles `{command, durationMs, success, cordisYmlContent, packageJsonContent}`, running the redactor over the full `cordis.yml` and `package.json` text. Never reads `.env`; `package.json` ships only alongside a `cordis.yml`, so a command run in a non-SDK directory never uploads that directory's unrelated manifest. | | `getOrCreateAnonymousId` | Random UUID persisted in a per-user GLOBAL config file (never in the project, never derived from git). | | `TelemetryReporter` | Fire-and-forget send: `report()` never blocks or throws; delivery resolves on every path; `flush()` optionally drains in-flight sends within a cap. | diff --git a/packages/sdk/telemetry/src/payload.ts b/packages/sdk/telemetry/src/payload.ts index 96a7b19d76..505cedb872 100644 --- a/packages/sdk/telemetry/src/payload.ts +++ b/packages/sdk/telemetry/src/payload.ts @@ -5,7 +5,9 @@ * the project `cordis.yml` and `package.json`. It NEVER reads or includes `.env` * — secrets live only in `.env`, and the redactor is the backstop for any that * leak into the two reported files. A file that does not exist (the first - * `create` run) simply omits its field. + * `create` run) simply omits its field, and `package.json` ships only when + * `cordis.yml` is present: without it the directory is not an SDK project, and + * its manifest belongs to whatever unrelated project the command ran in. * * @module @deepseek-ai/dsh-telemetry/payload */ @@ -27,7 +29,7 @@ export interface TelemetryPayload { success: boolean /** Redacted full text of the project `cordis.yml`, absent when the file does not exist. */ cordisYmlContent?: string - /** Redacted full text of the project `package.json`, absent when the file does not exist. */ + /** Redacted full text of the project `package.json`, absent when it or `cordis.yml` does not exist. */ packageJsonContent?: string } @@ -70,6 +72,11 @@ export async function buildTelemetryPayload(input: BuildTelemetryPayloadInput): durationMs: input.durationMs, success: input.success, ...cordisYml !== undefined ? { cordisYmlContent: redactor.redactText(cordisYml) } : {}, - ...packageJson !== undefined ? { packageJsonContent: redactor.redactText(packageJson) } : {}, + // package.json is an SDK-project manifest only alongside cordis.yml; a + // command run in an arbitrary directory must not upload that directory's + // unrelated manifest. + ...cordisYml !== undefined && packageJson !== undefined + ? { packageJsonContent: redactor.redactText(packageJson) } + : {}, } } diff --git a/packages/sdk/telemetry/tests/payload.spec.ts b/packages/sdk/telemetry/tests/payload.spec.ts index 1af5139ef5..ed3ab2f82f 100644 --- a/packages/sdk/telemetry/tests/payload.spec.ts +++ b/packages/sdk/telemetry/tests/payload.spec.ts @@ -47,8 +47,18 @@ describe('buildTelemetryPayload', () => { expect('packageJsonContent' in payload).toBe(false) }) + it('withholds package.json when cordis.yml is absent (not an SDK project)', async () => { + const dir = await projectDir({ 'package.json': '{ "name": "unrelated-repo" }' }) + const payload = await buildTelemetryPayload({ command: 'build', durationMs: 3, success: false, projectDir: dir }) + expect('cordisYmlContent' in payload).toBe(false) + expect('packageJsonContent' in payload).toBe(false) + }) + it('uses a supplied redactor', async () => { - const dir = await projectDir({ 'package.json': '{ "password": "hunter2" }' }) + const dir = await projectDir({ + 'cordis.yml': '- id: llm\n name: \'@deepseek-ai/dsh-llm-deepseek\'\n', + 'package.json': '{ "password": "hunter2" }', + }) const redactor = new SecretRedactor({ placeholder: '<>' }) const payload = await buildTelemetryPayload({ command: 'config', durationMs: 5, success: true, projectDir: dir, redactor, From 7a5d3ae5ce9bf7b1354decba22d7e0dd9ca7a22d Mon Sep 17 00:00:00 2001 From: imccyu <276526105+imccyu@users.noreply.github.com> Date: Sat, 18 Jul 2026 21:56:52 +0800 Subject: [PATCH 28/73] fix(telemetry): stop bearer redaction from eating plain prose The bearer rule matched any 8+ run of letters after the word, so package.json prose like "uses bearer authentication" lost its following word to the placeholder. Real bearer credentials always carry a digit; require one in the candidate token. --- packages/sdk/telemetry/src/secret-redactor.ts | 7 ++++++- packages/sdk/telemetry/tests/secret-redactor.spec.ts | 7 +++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/packages/sdk/telemetry/src/secret-redactor.ts b/packages/sdk/telemetry/src/secret-redactor.ts index 5b9af5f169..087ba2284a 100644 --- a/packages/sdk/telemetry/src/secret-redactor.ts +++ b/packages/sdk/telemetry/src/secret-redactor.ts @@ -191,7 +191,12 @@ export class SecretRedactor { } #redactBearerTokens(text: string): string { - return text.replace(/(bearer\s+)([a-z0-9._-]{8,})/gi, (_match, prefix: string) => `${prefix}${this.#placeholder}`) + // The candidate must contain a digit: real bearer credentials are never + // letters-only, while prose like "bearer authentication" is. + return text.replace( + /(bearer\s+)((?=[a-z._-]*[0-9])[a-z0-9._-]{8,})/gi, + (_match, prefix: string) => `${prefix}${this.#placeholder}`, + ) } #redactStandaloneTokens(text: string): string { diff --git a/packages/sdk/telemetry/tests/secret-redactor.spec.ts b/packages/sdk/telemetry/tests/secret-redactor.spec.ts index c3250d9ace..77d89d968e 100644 --- a/packages/sdk/telemetry/tests/secret-redactor.spec.ts +++ b/packages/sdk/telemetry/tests/secret-redactor.spec.ts @@ -154,6 +154,13 @@ describe('SecretRedactor.redactText', () => { .toBe(`sending Bearer ${REDACTED} now`) }) + it('keeps letters-only prose after the word bearer intact', () => { + expect(redactor.redactText('uses bearer authentication for requests')) + .toBe('uses bearer authentication for requests') + expect(redactor.redactText('"description": "bearer token-helper middleware"')) + .toBe('"description": "bearer token-helper middleware"') + }) + it('redacts standalone secret-shaped tokens while keeping package names and paths', () => { expect(redactor.redactText('key sk-abcdefghij1234567890 end')) .toBe(`key ${REDACTED} end`) From 86d97845dc7a3be558a5b494be667b039b72d9b7 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 12:14:37 +0800 Subject: [PATCH 29/73] docs(agent): remove stale pre-step contracts --- packages/core/agent-loop/src/tool-calls.ts | 3 ++- packages/core/agent/README.md | 1 - 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/core/agent-loop/src/tool-calls.ts b/packages/core/agent-loop/src/tool-calls.ts index 57bfb081b4..13105d3082 100644 --- a/packages/core/agent-loop/src/tool-calls.ts +++ b/packages/core/agent-loop/src/tool-calls.ts @@ -106,7 +106,8 @@ function parseArguments(raw: string): unknown { * before start; an exclusive reclassification waits for the current pool to * drain and remains for the caller's next barrier. Results and contexts commit * in model order. Abort stops starts, drains and commits started calls, accepts - * their contexts into the owning batch, and throws. + * their contexts into the owning batch, records results for skipped calls, and + * returns an aborted outcome. */ async function runGroup( ctx: Context, diff --git a/packages/core/agent/README.md b/packages/core/agent/README.md index 8d1870795d..5979b2bb08 100644 --- a/packages/core/agent/README.md +++ b/packages/core/agent/README.md @@ -77,4 +77,3 @@ The handle every plugin programs against: - **No public step-only abort** — `cancel()` clears ALL pending work (queued + steering + in-flight); an abort that preserves queued prompts returns only with a named consumer ([stop-surface RFC](../../../docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md)). - **`HookContext` carries exactly one `MessageSource`** — contributions from several plugins merged onto one tool call collapse under one source; mixed provenance is unrepresentable. - **`SessionStartSource` reserves `'clear'`/`'compact'` with no emitter yet** — only `'startup'`/`'resume'` occur until the driving subsystems land (`TODO(compaction)`). -- **`agent/pre-step`'s `fullSystemPrompt`/`sessionPrefix` parameters are a flagged smell** — compaction is their only consumer; a lazy prompt provider or a compaction-specific pressure seam is the marked revisit. From 7538cef967b745d9817b9502363699d494d6d069 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 12:35:43 +0800 Subject: [PATCH 30/73] docs(agent): describe cancelled post-step checkpoint --- docs/cordis-catalog/events.md | 12 ++++++------ docs/core-data-structures/core.md | 2 +- docs/event-producer-consumer.md | 10 +++++----- packages/cordis/tool-cordis/src/api-catalog.ts | 2 +- packages/core/agent/src/types.ts | 9 +++++---- website/zh-CN/api/harness/events.md | 16 ++++++++-------- 6 files changed, 26 insertions(+), 25 deletions(-) diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index f7fe327f80..1dd62f0b76 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -47,11 +47,11 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:310`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:311`](../../packages/core/agent/src/types.ts) ### `agent/post-step` — serial -Awaited serial checkpoint after the response, tool results, injected context, and steering are durable but before `step/end`. +Awaited serial checkpoint after the response, real or synthetic tool results, injected context, and steering are durable but before `step/end`. A cancelled tool batch reaches this checkpoint with an aborted signal. ```ts cordis-catalog 'agent/post-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void @@ -59,7 +59,7 @@ Awaited serial checkpoint after the response, tool results, injected context, an Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:263`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:264`](../../packages/core/agent/src/types.ts) ### `agent/pre-step` — serial @@ -119,7 +119,7 @@ Recover a model-request failure after its failed step has closed. `retry` opens Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:277`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:278`](../../packages/core/agent/src/types.ts) ### `agent/session-prefix` — waterfall @@ -179,7 +179,7 @@ Override whether the turn continues. The default continues after tool calls or s Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:287`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:288`](../../packages/core/agent/src/types.ts) ### `agent/turn-stop` — serial @@ -191,7 +191,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:297`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:298`](../../packages/core/agent/src/types.ts) ## `agent-loop/*` diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index c50134550d..15fd4f2942 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -456,7 +456,7 @@ It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the type RequestErrorDecision = { action: 'fail' } | { action: 'retry' } ``` -`agent/post-step` is the awaited successful-step checkpoint after assistant output, tool results, buffered context, and steering are durable. Its signature is `(agent, turn, step, signal)`; replayable facts remain in the session log rather than a transient payload. +`agent/post-step` is awaited after assistant output, real or synthetic tool results, buffered context, and steering are durable but before `step/end`. A cancelled tool batch reaches it with an aborted signal after draining; its signature is `(agent, turn, step, signal)`, and replayable facts remain in the session log rather than a transient payload. `agent/turn-stop` returns the stop-only `ContinuationStop` subset or `undefined`. The loop calls this serial checkpoint after folding the ordinary decision, its reason, and pending steering; a stop is terminal and discards pending steering. diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 2e39b77563..abba29627b 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -10,19 +10,19 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:362`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | [`stdio`](../packages/ui/stdio) | | `agent/created` | `emit` | [`packages/core/agent/src/types.ts:147`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio) | | `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:156`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`stdio`](../packages/ui/stdio) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:310`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | -| `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:263`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:311`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | | `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:204`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`time-context`](../packages/context/time-context), [`user-approval`](../packages/ui/user-approval) | | `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:214`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | | `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:175`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | | `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:226`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp) | -| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:277`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:278`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic) | | `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:241`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill), [`workspace-context`](../packages/context/workspace-context) | | `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:188`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`stdio`](../packages/ui/stdio) | | `agent/status` | `emit` | [`packages/core/agent/src/types.ts:165`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`stdio`](../packages/ui/stdio) | | `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:252`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:287`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:297`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | +| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:288`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:298`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:31`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/ui/acp) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:61`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:70`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 116366b910..6555aed2a4 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -323,7 +323,7 @@ export const EVENT_API: readonly EventApiEntry[] = [ name: 'agent/post-step', mode: 'serial', signature: '\'agent/post-step\'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void', - summary: 'Awaited serial checkpoint after the response, tool results, injected context, and steering are durable but before `step/end`.', + summary: 'Awaited serial checkpoint after the response, real or synthetic tool results, injected context, and steering are durable but before `step/end`.', }, { name: 'agent/pre-step', diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index 0c79c822f2..702861f407 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -251,11 +251,12 @@ declare module 'cordis' { */ 'agent/step-result'(this: Scoped, agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise /** - * Awaited serial checkpoint after the response, tool results, injected - * context, and steering are durable but before `step/end`. - * @param agent - the agent that completed the step. + * Awaited serial checkpoint after the response, real or synthetic tool + * results, injected context, and steering are durable but before `step/end`. + * A cancelled tool batch reaches this checkpoint with an aborted signal. + * @param agent - the agent whose step is settling. * @param turn - the open turn number. - * @param step - the completed step number. + * @param step - the open step number. * @param signal - the turn abort signal. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @mode serial diff --git a/website/zh-CN/api/harness/events.md b/website/zh-CN/api/harness/events.md index 0a7bb0a5f6..bc2a9e7e4b 100644 --- a/website/zh-CN/api/harness/events.md +++ b/website/zh-CN/api/harness/events.md @@ -49,7 +49,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w - `step` — the step at which the failure surfaced. - `error` — the failure, verbatim. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L310) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L311) ### agent/post-step @@ -59,14 +59,14 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w 'agent/post-step'(this: Scoped, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise | void ``` -Awaited serial checkpoint after the response, tool results, injected context, and steering are durable but before `step/end`. +Awaited serial checkpoint after the response, real or synthetic tool results, injected context, and steering are durable but before `step/end`. A cancelled tool batch reaches this checkpoint with an aborted signal. -- `agent` — the agent that completed the step. +- `agent` — the agent whose step is settling. - `turn` — the open turn number. -- `step` — the completed step number. +- `step` — the open step number. - `signal` — the turn abort signal. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L263) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L264) ### agent/pre-step @@ -151,7 +151,7 @@ Recover a model-request failure after its failed step has closed. `retry` opens - `retryAttempt` — zero-based number of prior recovery retries. - `signal` — the turn abort signal. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L277) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L278) ### agent/session-prefix @@ -230,7 +230,7 @@ Override whether the turn continues. The default continues after tool calls or s - `turn` — the turn being continued or stopped. - `defaultDecision` — what the loop would do absent an override. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L287) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L288) ### agent/turn-stop @@ -245,7 +245,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a - `agent` — the agent whose composed continuation outcome may be stopped. - `turn` — the turn at its terminal-stop checkpoint. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L297) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L298) ## agent-loop/* From 85ae6848875f96c3199367ccf33454a46a3e39a9 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 12:40:56 +0800 Subject: [PATCH 31/73] docs: sync type-equivalent JSDoc --- docs/core-data-structures/compaction.md | 3 ++- docs/core-data-structures/core.md | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/core-data-structures/compaction.md b/docs/core-data-structures/compaction.md index eae8d2e727..f6cdfd5b8b 100644 --- a/docs/core-data-structures/compaction.md +++ b/docs/core-data-structures/compaction.md @@ -54,7 +54,8 @@ interface CompactionResult { Automatic callers state why policy is running; implementations may treat confirmed overflow more aggressively than ordinary pressure. ```ts type-equiv -export type CompactionTrigger = 'pressure' | 'context-overflow' +/** Why automatic policy is asking a backend to consider compaction. */ +type CompactionTrigger = 'pressure' | 'context-overflow' ``` `CompactService` exposes `compactIfNeeded(agent, trigger, signal)` for automatic `pressure` or `context-overflow` policy, returning `null` when no safe work exists, and `compactRegion(...)` for an explicit inclusive surface range. Implementations must forward the supplied signal to summarization. The seam owns no pricing API: the singleton [`ctx.tokenMeter`](token-meter.md) directly owns estimation and replay, while `dsh-compact-basic` owns retention, event sequencing, routed summarization calls, and their configuration. diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 4d32b7796f..0eaede9345 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -446,12 +446,14 @@ type ContinuationDecision = `agent/request-error` receives the original `RequestError`, whose optional provider-neutral `code` supports stable routing without message parsing: ```ts type-equiv +/** Model-request failure with an optional machine-routable provider code. */ type RequestError = Error & { code?: string } ``` It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` preserves that error: ```ts type-equiv +/** Failed-request recovery decision; `retry` opens another numbered step while listeners delegate by calling `next()`. */ type RequestErrorDecision = { action: 'fail' } | { action: 'retry' } ``` From 954c2ad2b5a2c9cedae72b3e0aa6e4efacd2a106 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 13:03:15 +0800 Subject: [PATCH 32/73] docs(llm-pi-ai): track overflow classification gap --- packages/llm/llm-pi-ai/src/stream.ts | 3 +++ 1 file changed, 3 insertions(+) diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index c3aae46c44..f7a0d6ee7f 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -30,6 +30,9 @@ export function mapUsage(usage: PiUsage): TokenUsage { function classifyPiAiError(message: string): string { if (/\b(?:401|403)\b/.test(message)) return 'AUTH' if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT' + // TODO: Classify the full message with pi-ai's isContextOverflow() and the + // resolved model's contextWindow so provider-specific and usage-based overflows + // reach automatic compaction. if (isContextWindowExceededError(message)) return CONTEXT_WINDOW_EXCEEDED_CODE if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' From edaaa50fe375cf1a058367289d0bd553880af293 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 13:03:31 +0800 Subject: [PATCH 33/73] test(acp): snapshot cancelled queued tool calls --- examples/acp-agent/tests/acp.snapshot.ts | 1 + .../snapshots/cancel-tool-calls/input.json | 11 +++++++++ .../cancel-tool-calls/replay.override.json | 15 ++++++++++++ .../snapshots/cancel-tool-calls/session.jsonl | 20 ++++++++++++++++ .../cancel-tool-calls/stdout.golden.jsonl | 7 ++++++ packages/support/acp-snapshot/src/harness.ts | 24 ++++++++----------- .../tests/fixtures/fake-acp-agent.ts | 20 +++++++++++++++- .../acp-snapshot/tests/harness.spec.ts | 10 ++++++++ 8 files changed, 93 insertions(+), 15 deletions(-) create mode 100644 examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json create mode 100644 examples/acp-agent/tests/snapshots/cancel-tool-calls/replay.override.json create mode 100644 examples/acp-agent/tests/snapshots/cancel-tool-calls/session.jsonl create mode 100644 examples/acp-agent/tests/snapshots/cancel-tool-calls/stdout.golden.jsonl diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts index 124117afef..d60d8b9724 100644 --- a/examples/acp-agent/tests/acp.snapshot.ts +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -103,6 +103,7 @@ const SCENARIOS: Scenario[] = [ configPath: WORKSPACE_CONTEXT_CONFIG, }, { name: 'cancel', hasModelTurn: true, recorded: false, overridden: true }, + { name: 'cancel-tool-calls', hasModelTurn: true, recorded: false, overridden: true }, { name: 'subagent-spawn', hasModelTurn: true, recorded: true }, { name: 'subagent-multi', hasModelTurn: true, recorded: true }, { name: 'subagent-fork', hasModelTurn: true, recorded: true }, diff --git a/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json b/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json new file mode 100644 index 0000000000..3610e1f436 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json @@ -0,0 +1,11 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { + "op": "promptAndCancel", + "text": "Run two shell commands: wait for cancellation, then write skipped.txt.", + "afterUpdate": "tool_call" + } + ] +} diff --git a/examples/acp-agent/tests/snapshots/cancel-tool-calls/replay.override.json b/examples/acp-agent/tests/snapshots/cancel-tool-calls/replay.override.json new file mode 100644 index 0000000000..c0aa7730d7 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel-tool-calls/replay.override.json @@ -0,0 +1,15 @@ +[ + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "tool-call" }, + { "type": "tool-call-delta", "index": 0, "id": "call_wait", "name": "bash", "argumentsDelta": "{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}" }, + { "type": "block-end", "index": 0, "block": { "type": "tool-call", "id": "call_wait", "name": "bash", "arguments": "{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}" } }, + { "type": "block-start", "index": 1, "blockType": "tool-call" }, + { "type": "tool-call-delta", "index": 1, "id": "call_skipped", "name": "bash", "argumentsDelta": "{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}" }, + { "type": "block-end", "index": 1, "block": { "type": "tool-call", "id": "call_skipped", "name": "bash", "arguments": "{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}" } }, + { "type": "usage", "usage": { "inputTokens": 10, "outputTokens": 10 } }, + { "type": "finish", "reason": { "kind": "tool-calls" } } + ] + } +] diff --git a/examples/acp-agent/tests/snapshots/cancel-tool-calls/session.jsonl b/examples/acp-agent/tests/snapshots/cancel-tool-calls/session.jsonl new file mode 100644 index 0000000000..e6d18515a6 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel-tool-calls/session.jsonl @@ -0,0 +1,20 @@ +{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} +{"type":"turn/start","seq":0,"time":1784437195072,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1784437195072,"data":{"content":[{"type":"text","text":"Run two shell commands: wait for cancellation, then write skipped.txt."}],"source":{"kind":"user"}},"surfaceOp":"append"} +{"type":"step/start","seq":2,"time":1784437195076,"data":{"turn":1,"step":1}} +{"type":"request/header","seq":3,"time":1784437195076,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"assistant/chunk","seq":4,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":5,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"call_wait","name":"bash","argumentsDelta":"{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}"}}} +{"type":"assistant/chunk","seq":6,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"call_wait","name":"bash","arguments":"{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}"}}}} +{"type":"assistant/chunk","seq":7,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":8,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_skipped","name":"bash","argumentsDelta":"{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}"}}} +{"type":"assistant/chunk","seq":9,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_skipped","name":"bash","arguments":"{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}"}}}} +{"type":"assistant/chunk","seq":10,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":10}}}} +{"type":"assistant/chunk","seq":11,"time":1784437195077,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":12,"time":1784437195078,"data":{"turn":1,"step":1,"content":[{"type":"tool-call","id":"call_wait","name":"bash","arguments":"{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}"},{"type":"tool-call","id":"call_skipped","name":"bash","arguments":"{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}"}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":10,"outputTokens":10}},"sourceEventSeqs":[4,5,6,7,8,9,10,11],"surfaceOp":"append"} +{"type":"tool/call","seq":13,"time":1784437195078,"data":{"turn":1,"step":1,"callId":"call_wait","name":"bash","arguments":"{\"command\":\"node -e \\\"setInterval(() => {}, 1000)\\\"\",\"description\":\"Wait until cancellation\"}"}} +{"type":"tool/result","seq":14,"time":1784437195089,"data":{"turn":1,"step":1,"callId":"call_wait","content":[{"type":"text","text":"Error: command aborted"}],"isError":true},"sourceEventSeqs":[13],"surfaceOp":"append"} +{"type":"tool/call","seq":15,"time":1784437195089,"data":{"turn":1,"step":1,"callId":"call_skipped","name":"bash","arguments":"{\"command\":\"printf skipped > skipped.txt\",\"description\":\"Write skipped marker\"}"}} +{"type":"tool/result","seq":16,"time":1784437195089,"data":{"turn":1,"step":1,"callId":"call_skipped","content":[{"type":"text","text":"Error: tool call skipped because the step was aborted before execution"}],"isError":true,"error":{"name":"AbortError","code":"ABORTED"}},"sourceEventSeqs":[15],"surfaceOp":"append"} +{"type":"step/end","seq":17,"time":1784437195090,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":18,"time":1784437195090,"data":{"turn":1,"reason":{"kind":"aborted","reason":"session/cancel"}}} diff --git a/examples/acp-agent/tests/snapshots/cancel-tool-calls/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/cancel-tool-calls/stdout.golden.jsonl new file mode 100644 index 0000000000..11178b9bc7 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel-tool-calls/stdout.golden.jsonl @@ -0,0 +1,7 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","configOptions":[{"id":"model","name":"Model","description":"Sets this session's provider and model.","category":"model","type":"select","currentValue":"[\"deepseek\",\"deepseek-v4-flash\"]","options":[{"value":"[\"deepseek\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"}]},{"id":"permission","name":"Permissions","description":"The session permission preset: each choice bundles a sandbox mode and an approval policy.","category":"mode","type":"select","currentValue":"danger-full-access","options":[{"value":"workspace-write","name":"workspace-write","description":"Write inside the workspace and permitted temporary directories; wider retries require approval."},{"value":"danger-full-access","name":"danger-full-access","description":"Full file access without approval prompts."}]}]}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_wait","title":"node -e \"setInterval(() => {}, 1000)\"","kind":"execute","status":"in_progress","rawInput":"node -e \"setInterval(() => {}, 1000)\"","content":[{"type":"content","content":{"type":"text","text":"Wait until cancellation"}}]}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"cancelled"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_wait","status":"failed","content":[{"type":"content","content":{"type":"text","text":"```console\nError: command aborted\n```"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_skipped","title":"printf skipped > skipped.txt","kind":"execute","status":"in_progress","rawInput":"printf skipped > skipped.txt","content":[{"type":"content","content":{"type":"text","text":"Write skipped marker"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_skipped","status":"failed","content":[{"type":"content","content":{"type":"text","text":"```console\nError: tool call skipped because the step was aborted before execution\n```"}}]}}} diff --git a/packages/support/acp-snapshot/src/harness.ts b/packages/support/acp-snapshot/src/harness.ts index e3efca8da9..ca00056d93 100644 --- a/packages/support/acp-snapshot/src/harness.ts +++ b/packages/support/acp-snapshot/src/harness.ts @@ -37,11 +37,10 @@ export type { AgentUnderTest } from './launcher.ts' * (random) session id into a `{{sessionId}}` variable that later steps * reference, since a committed file cannot know the id in advance. * - * `promptAndCancel` sends a prompt WITHOUT awaiting its response, waits until - * the client observes the first streamed `agent_message_chunk` (so the emitted - * frames deterministically precede the cancellation), then cancels the turn — - * the only way to exercise a cancel deterministically (a plain `prompt` step - * awaits the response, which a cancel/hang scenario would block on forever). + * `promptAndCancel` starts a prompt without awaiting completion, waits until + * the client observes the selected update (`agent_message_chunk` by default), + * then cancels and awaits completion. This keeps update/cancel order + * deterministic for fixtures that a plain `prompt` cannot drive. */ export type InputStep = | { op: 'initialize'; terminalOutput?: boolean } @@ -49,7 +48,7 @@ export type InputStep = | { op: 'newSessionExpectError'; additionalDirectories?: string[] } | { op: 'prompt'; text: string } | { op: 'promptExpectError'; text: string } - | { op: 'promptAndCancel'; text: string } + | { op: 'promptAndCancel'; text: string; afterUpdate?: 'agent_message_chunk' | 'tool_call' } | { op: 'cancel' } | { op: 'setConfigOption'; configId: string; value: string } | { op: 'setConfigOptionExpectError'; configId: string; value: string } @@ -342,15 +341,12 @@ async function runStep( case 'promptAndCancel': { const sessionId = getSessionId() if (sessionId === undefined) throw new Error('snapshot-harness: promptAndCancel before newSession') - // Dispatch the prompt WITHOUT awaiting (a hang fixture never resolves on - // its own). To pin frame order deterministically, wait until the client - // has OBSERVED the hang's streamed agent_message_chunk before cancelling — - // so those update frames always precede the cancelled prompt response in - // the transcript (without this, the late chunk and the response race). - // Then cancel and await the prompt, which the bridge settles as - // `cancelled` once the abort propagates. + // Dispatch without awaiting because the fixture does not settle on its + // own. Waiting for the selected update pins it before cancellation and + // the cancelled prompt response in the transcript. const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) - await waitForUpdate(u => u.sessionUpdate === 'agent_message_chunk') + const afterUpdate = step.afterUpdate ?? 'agent_message_chunk' + await waitForUpdate(u => u.sessionUpdate === afterUpdate) await client.cancel({ sessionId }) await promptDone return diff --git a/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts index 5dd5524ed0..dd8ca9ce92 100644 --- a/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts +++ b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts @@ -33,6 +33,8 @@ interface Behavior { rejectExtraDirs?: boolean /** How `session/prompt` settles: a clean response, a JSON-RPC error, or a hang until `session/cancel`. */ prompt?: 'respond' | 'error' | 'hang-until-cancel' + /** Emit a tool call instead of a message chunk before parking a cancellable prompt. */ + cancelAtToolCall?: boolean /** Before responding to a prompt, send a `session/request_permission` request and echo its outcome as a chunk. */ permissionProbe?: boolean /** Echo the `DSH_SNAPSHOT_*` env the harness set as a chunk (spec-side env-plumbing assertions). */ @@ -126,7 +128,23 @@ async function handlePrompt(id: number | string): Promise { params: { sessionId, update: { sessionUpdate: 'agent_thought_chunk', content: { type: 'text', text: 'mulling' } } }, }) } - chunk('thinking about it') + if (behavior.cancelAtToolCall === true) { + send({ + method: 'session/update', + params: { + sessionId, + update: { + sessionUpdate: 'tool_call', + toolCallId: 'call_fake_1', + title: 'fake tool', + kind: 'execute', + status: 'in_progress', + }, + }, + }) + } else { + chunk('thinking about it') + } if (behavior.echoEnv === true) { chunk(`env:${JSON.stringify({ mode: process.env.DSH_SNAPSHOT, diff --git a/packages/support/acp-snapshot/tests/harness.spec.ts b/packages/support/acp-snapshot/tests/harness.spec.ts index 2b9d6e032f..be03b27c78 100644 --- a/packages/support/acp-snapshot/tests/harness.spec.ts +++ b/packages/support/acp-snapshot/tests/harness.spec.ts @@ -334,6 +334,16 @@ describe('runScenario', () => { expect(result.rawStdout.indexOf('thinking about it')).toBeLessThan(result.rawStdout.indexOf('cancelled')) }) + it('promptAndCancel can wait for a tool call before cancelling', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ prompt: 'hang-until-cancel', cancelAtToolCall: true }) + const result = await runScenario( + { steps: [...boot, { op: 'promptAndCancel', text: 'hang', afterUpdate: 'tool_call' }] }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('"sessionUpdate":"tool_call"') + expect(result.rawStdout.indexOf('"sessionUpdate":"tool_call"')).toBeLessThan(result.rawStdout.indexOf('cancelled')) + }) + it('promptExpectError swallows a model-error response as the expected outcome', { timeout: 20_000 }, async () => { const { fixtureFile } = await scenario({ prompt: 'error' }) const result = await runScenario( From d709a8a1a4aab517ffdf75059f681b6ad5cc8e91 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 13:09:16 +0800 Subject: [PATCH 34/73] test(acp): await cancelled tool updates --- .../snapshots/cancel-tool-calls/input.json | 3 ++- packages/support/acp-snapshot/src/harness.ts | 16 +++++++++++++--- .../tests/fixtures/fake-acp-agent.ts | 15 +++++++++++++++ .../support/acp-snapshot/tests/harness.spec.ts | 18 +++++++++++++++--- 4 files changed, 45 insertions(+), 7 deletions(-) diff --git a/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json b/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json index 3610e1f436..0f40e9d8b6 100644 --- a/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json +++ b/examples/acp-agent/tests/snapshots/cancel-tool-calls/input.json @@ -5,7 +5,8 @@ { "op": "promptAndCancel", "text": "Run two shell commands: wait for cancellation, then write skipped.txt.", - "afterUpdate": "tool_call" + "afterUpdate": "tool_call", + "waitForToolCallUpdate": "call_skipped" } ] } diff --git a/packages/support/acp-snapshot/src/harness.ts b/packages/support/acp-snapshot/src/harness.ts index ca00056d93..542b6d6df3 100644 --- a/packages/support/acp-snapshot/src/harness.ts +++ b/packages/support/acp-snapshot/src/harness.ts @@ -39,8 +39,8 @@ export type { AgentUnderTest } from './launcher.ts' * * `promptAndCancel` starts a prompt without awaiting completion, waits until * the client observes the selected update (`agent_message_chunk` by default), - * then cancels and awaits completion. This keeps update/cancel order - * deterministic for fixtures that a plain `prompt` cannot drive. + * then cancels and awaits completion. A named `waitForToolCallUpdate` keeps the + * step open for a terminal tool update that may follow the prompt response. */ export type InputStep = | { op: 'initialize'; terminalOutput?: boolean } @@ -48,7 +48,12 @@ export type InputStep = | { op: 'newSessionExpectError'; additionalDirectories?: string[] } | { op: 'prompt'; text: string } | { op: 'promptExpectError'; text: string } - | { op: 'promptAndCancel'; text: string; afterUpdate?: 'agent_message_chunk' | 'tool_call' } + | { + op: 'promptAndCancel' + text: string + afterUpdate?: 'agent_message_chunk' | 'tool_call' + waitForToolCallUpdate?: string + } | { op: 'cancel' } | { op: 'setConfigOption'; configId: string; value: string } | { op: 'setConfigOptionExpectError'; configId: string; value: string } @@ -347,8 +352,13 @@ async function runStep( const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) const afterUpdate = step.afterUpdate ?? 'agent_message_chunk' await waitForUpdate(u => u.sessionUpdate === afterUpdate) + // Arm this before cancellation so a fast tool drain cannot outrun the waiter. + const toolCallUpdateDone = step.waitForToolCallUpdate === undefined + ? undefined + : waitForUpdate(u => u.sessionUpdate === 'tool_call_update' && u.toolCallId === step.waitForToolCallUpdate) await client.cancel({ sessionId }) await promptDone + if (toolCallUpdateDone !== undefined) await toolCallUpdateDone return } case 'cancel': { diff --git a/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts index dd8ca9ce92..43b5ee3fb3 100644 --- a/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts +++ b/packages/support/acp-snapshot/tests/fixtures/fake-acp-agent.ts @@ -35,6 +35,8 @@ interface Behavior { prompt?: 'respond' | 'error' | 'hang-until-cancel' /** Emit a tool call instead of a message chunk before parking a cancellable prompt. */ cancelAtToolCall?: boolean + /** Emit the parked tool call's terminal update after answering cancellation. */ + cancelToolCallUpdate?: boolean /** Before responding to a prompt, send a `session/request_permission` request and echo its outcome as a chunk. */ permissionProbe?: boolean /** Echo the `DSH_SNAPSHOT_*` env the harness set as a chunk (spec-side env-plumbing assertions). */ @@ -247,6 +249,19 @@ function handleFrame(frame: Record): void { const parked = parkedPromptId parkedPromptId = null respond(parked, { stopReason: 'cancelled' }) + if (behavior.cancelToolCallUpdate === true) { + send({ + method: 'session/update', + params: { + sessionId, + update: { + sessionUpdate: 'tool_call_update', + toolCallId: 'call_fake_1', + status: 'failed', + }, + }, + }) + } } return default: diff --git a/packages/support/acp-snapshot/tests/harness.spec.ts b/packages/support/acp-snapshot/tests/harness.spec.ts index be03b27c78..3d174b3c3e 100644 --- a/packages/support/acp-snapshot/tests/harness.spec.ts +++ b/packages/support/acp-snapshot/tests/harness.spec.ts @@ -334,14 +334,26 @@ describe('runScenario', () => { expect(result.rawStdout.indexOf('thinking about it')).toBeLessThan(result.rawStdout.indexOf('cancelled')) }) - it('promptAndCancel can wait for a tool call before cancelling', { timeout: 20_000 }, async () => { - const { fixtureFile } = await scenario({ prompt: 'hang-until-cancel', cancelAtToolCall: true }) + it('promptAndCancel can bracket cancellation with tool-call updates', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ + prompt: 'hang-until-cancel', + cancelAtToolCall: true, + cancelToolCallUpdate: true, + }) const result = await runScenario( - { steps: [...boot, { op: 'promptAndCancel', text: 'hang', afterUpdate: 'tool_call' }] }, + { + steps: [...boot, { + op: 'promptAndCancel', + text: 'hang', + afterUpdate: 'tool_call', + waitForToolCallUpdate: 'call_fake_1', + }], + }, { agent: AGENT, mode: 'replay', fixtureFile }, ) expect(result.rawStdout).toContain('"sessionUpdate":"tool_call"') expect(result.rawStdout.indexOf('"sessionUpdate":"tool_call"')).toBeLessThan(result.rawStdout.indexOf('cancelled')) + expect(result.rawStdout.indexOf('cancelled')).toBeLessThan(result.rawStdout.indexOf('"sessionUpdate":"tool_call_update"')) }) it('promptExpectError swallows a model-error response as the expected outcome', { timeout: 20_000 }, async () => { From 40ad78ee0bdcddea49acd4adabe92da94c0e0280 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 13:13:36 +0800 Subject: [PATCH 35/73] docs: condense merged architecture contract --- docs/architecture.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 32df16a3cd..193dbf56bf 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -83,7 +83,7 @@ forever: snapshot the derived messages (the reconstruction boundary) 'step/start' agent/request (config only) -> log request/header -> llm/stream (frozen) - on final adapter failure or terminal in-band error/aborted finish: + on final adapter-path or terminal in-band failure: 'step/end' agent/request-error(original error, consecutive retry attempt, signal) retry in the next numbered step or preserve the original error @@ -108,15 +108,15 @@ forever: Each step assembles ordered prompt sections, tool schemas, and `{{name}}` variables; unknown or valueless references fail the turn. `dsh-system-prompt` owns the harness identity and default persona, which an agent scope may shadow. The loop supplies `model` and `cwd` ([prompt-ownership RFC](rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md)). -Context accepted during tool execution—including async `agent.inject()` notices and post-tool `additionalContexts`—waits for settlement, then follows every recorded result. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering while the step signal remains open. Ordinary leftover steering becomes queued input. Terminal `agent/turn-stop` runs after continuation and steering folding, stays authoritative through turn close and flush, and discards later steering while preserving queued prompts. +Context accepted during tool execution—including async `agent.inject()` notices and post-tool `additionalContexts`—waits for settlement, then follows every recorded result. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering before signal closure. Leftover steering becomes queued input. Terminal `agent/turn-stop` runs after continuation and steering folding, stays authoritative through turn close and flush, and discards later steering while preserving queued prompts. -`dsh-compact-basic` uses those checkpoints for routed-envelope pressure and canonical-overflow recovery; only a tool-balanced surface replacement authorizes retry ([RFC](rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)). +`dsh-compact-basic` handles pressure and canonical overflow at these checkpoints; retry requires a balanced surface replacement ([RFC](rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)). ### Failure Boundaries -The turn is the containment boundary. Final adapter selection, dispatch, iteration, and terminal in-band failures close the step before `agent/request-error`; retry reconstructs a new numbered step, while the default preserves the provider error. Attempts reset after success. +The turn is the containment boundary. Final adapter-path and terminal in-band failures close the step before `agent/request-error`; retry starts a new numbered step, while the default preserves the provider error. Attempts reset on success. -Other plugin and step failures use `agent/error`. Cancellation and disposal beat recovery; undispatched model tool calls receive synthetic `tool/call` and `ABORTED` result pairs before turn closure. `cancel()` clears queues and aborts active work; disposal awaits quiescence before unregistering. +Other failures use `agent/error`. Cancellation and disposal beat recovery; undispatched model tool calls receive synthetic `tool/call` and `ABORTED` result pairs before turn closure. `cancel()` clears queues and aborts active work; disposal awaits quiescence before unregistering. Every session event is turn-enclosed. Reloading preserves an interrupted tail and closes it with a synthetic `interrupted` turn end. Failures after durable turn close report only through `agent/error` because no safe in-turn position remains. Each turn has one `TurnEndReason`; [TurnEndReasonMap](core-data-structures/session.md#why-a-turn-ended-turnendreasonmap) owns the variants. From 1fdcd5a77314717391462d377e6991622db4eb31 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 13:42:21 +0800 Subject: [PATCH 36/73] ci: run pi-ai OpenAI e2e through Azure --- .github/workflows/pi-ai-provider-e2e.yml | 20 ++++++++------- packages/llm/llm-pi-ai/tests/adapter.spec.ts | 19 ++++++++++++++ .../llm/llm-pi-ai/tests/provider-apis.e2e.ts | 25 +++++++++++-------- 3 files changed, 45 insertions(+), 19 deletions(-) diff --git a/.github/workflows/pi-ai-provider-e2e.yml b/.github/workflows/pi-ai-provider-e2e.yml index 7a273c52f3..4be5f0c13a 100644 --- a/.github/workflows/pi-ai-provider-e2e.yml +++ b/.github/workflows/pi-ai-provider-e2e.yml @@ -1,12 +1,12 @@ -name: E2E (pi-ai OpenAI and Anthropic) +name: E2E (pi-ai Azure OpenAI and Anthropic) # This suite spends tokens against two external providers and is intentionally # opt-in. It has no push, pull_request, schedule, or workflow_call trigger. on: workflow_dispatch: inputs: - openai_model: - description: OpenAI model from pi-ai's installed catalog + azure_openai_model: + description: Azure OpenAI model from pi-ai's installed catalog required: true default: gpt-5.5 type: string @@ -22,7 +22,7 @@ permissions: jobs: e2e: runs-on: ubuntu-latest - name: OpenAI Responses + Anthropic Messages + name: Azure OpenAI Responses + Anthropic Messages timeout-minutes: 20 steps: - uses: actions/checkout@v6 @@ -52,12 +52,12 @@ jobs: # dispatched CI run must fail instead of reporting an all-skipped green. - name: Preflight (require provider API keys) env: - OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY_EXTERNAL }} + AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY_EXTERNAL }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }} run: | set -euo pipefail missing=0 - for name in OPENAI_API_KEY ANTHROPIC_API_KEY; do + for name in AZURE_OPENAI_API_KEY ANTHROPIC_API_KEY; do if [ -z "${!name:-}" ]; then echo "::error::${name} is empty. Configure the corresponding *_EXTERNAL repository secret." missing=1 @@ -65,11 +65,13 @@ jobs: done exit "$missing" - - name: E2E tests (real OpenAI and Anthropic APIs) + - name: E2E tests (real Azure OpenAI and Anthropic APIs) env: - OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY_EXTERNAL }} + AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY_EXTERNAL }} + AZURE_OPENAI_API_VERSION: v1 ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }} - DSH_PI_AI_OPENAI_MODEL: ${{ inputs.openai_model }} + DSH_PI_AI_AZURE_OPENAI_MODEL: ${{ inputs.azure_openai_model }} + DSH_PI_AI_AZURE_OPENAI_BASE_URL: https://openai-routerhub-resource.services.ai.azure.com/api/projects/openai/openai/ DSH_PI_AI_ANTHROPIC_MODEL: ${{ inputs.anthropic_model }} DSH_E2E_MAX_WORKERS: 2 run: >- diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 559c36bf27..3ebb23ee50 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -167,6 +167,25 @@ describe('PiAiAdapter provider routing', () => { expect(server.paths).toEqual(['/v1/responses']) }) + it('uses Azure OpenAI Responses with the configured project base path and API key', async () => { + vi.stubEnv('AZURE_OPENAI_API_VERSION', 'v1') + const server = await mockServer([{ status: 401, body: JSON.stringify({ error: { message: 'expected mock failure' } }) }]) + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(LlmPiAi, { + providers: [{ + provider: 'azure-openai-responses', + apiKey: 'test-key', + baseURL: `${server.url}/api/projects/openai/openai/`, + maxRetries: 0, + }], + }) + const result = await assemble(ctx, { provider: 'azure-openai-responses', model: 'gpt-5.5', messages: [] }) + expect(result.finish.kind).toBe('error') + expect(server.paths).toEqual(['/api/projects/openai/openai/responses?api-version=v1']) + expect(server.headers[0]?.['api-key']).toBe('test-key') + }) + it.each([ [401, 'AUTH'], [400, 'INVALID_REQUEST'], diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index 7fffba4618..1ed0959785 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -3,22 +3,26 @@ import { Context } from 'cordis' import LlmService, { CallId } from '@deepseek-ai/dsh-llm' import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' -import type { PiAiReplayState } from '@deepseek-ai/dsh-llm-pi-ai' +import type { PiAiReplayState } from '../src/replay.ts' import { assemble, type AssembledResult } from './assemble.ts' interface ProviderCase { - provider: 'openai' | 'anthropic' - api: 'openai-responses' | 'anthropic-messages' + provider: 'azure-openai-responses' | 'anthropic' + api: 'azure-openai-responses' | 'anthropic-messages' model: string apiKey?: string + baseURL?: string } +const azureOpenAIBaseURL = process.env.DSH_PI_AI_AZURE_OPENAI_BASE_URL ?? process.env.AZURE_OPENAI_BASE_URL + const providerCases: ProviderCase[] = [ { - provider: 'openai', - api: 'openai-responses', - model: process.env.DSH_PI_AI_OPENAI_MODEL ?? 'gpt-5.5', - ...process.env.OPENAI_API_KEY ? { apiKey: process.env.OPENAI_API_KEY } : {}, + provider: 'azure-openai-responses', + api: 'azure-openai-responses', + model: process.env.DSH_PI_AI_AZURE_OPENAI_MODEL ?? 'gpt-5.5', + ...process.env.AZURE_OPENAI_API_KEY ? { apiKey: process.env.AZURE_OPENAI_API_KEY } : {}, + ...azureOpenAIBaseURL ? { baseURL: azureOpenAIBaseURL } : {}, }, { provider: 'anthropic', @@ -38,6 +42,7 @@ async function harness(): Promise { providers: providerCases.map(profile => ({ provider: profile.provider, ...profile.apiKey === undefined ? {} : { apiKey: profile.apiKey }, + ...profile.baseURL === undefined ? {} : { baseURL: profile.baseURL }, })), }) return ctx @@ -90,7 +95,7 @@ for (const profile of providerCases) { provider: profile.provider, model: profile.model, messages: ask('Reply with exactly the word: pong'), - maxTokens: 64, + maxTokens: 1024, }) expect(result.finish.kind).toBe('stop') @@ -108,7 +113,7 @@ for (const profile of providerCases) { model: profile.model, messages: prompt, tools: [lookupTool], - maxTokens: 256, + maxTokens: 2048, }) expect(first.finish.kind).toBe('tool-calls') @@ -134,7 +139,7 @@ for (const profile of providerCases) { }, ], tools: [lookupTool], - maxTokens: 256, + maxTokens: 2048, }) expect(second.finish.kind).toBe('stop') From 51425dd61a481257a72f0351e7a564439e470be0 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 13:51:25 +0800 Subject: [PATCH 37/73] test(llm-pi-ai): surface provider e2e failures --- packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index 1ed0959785..b830bfe08b 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -63,6 +63,13 @@ function textOf(result: AssembledResult): string { .join('') } +function expectFinish(result: AssembledResult, expected: 'stop' | 'tool-calls'): void { + if (result.finish.kind === 'error') { + throw new Error(`provider request failed (${result.finish.code ?? 'unknown'}): ${result.finish.message}`) + } + expect(result.finish.kind).toBe(expected) +} + function expectNativeReplay(result: AssembledResult, profile: ProviderCase): PiAiReplayState { const replayState = result.message.provenance?.replayState expect(replayState).toMatchObject({ @@ -98,7 +105,7 @@ for (const profile of providerCases) { maxTokens: 1024, }) - expect(result.finish.kind).toBe('stop') + expectFinish(result, 'stop') expect(textOf(result).toLowerCase()).toContain('pong') expect(result.usage?.inputTokens).toBeGreaterThan(0) expect(result.usage?.outputTokens).toBeGreaterThan(0) @@ -116,7 +123,7 @@ for (const profile of providerCases) { maxTokens: 2048, }) - expect(first.finish.kind).toBe('tool-calls') + expectFinish(first, 'tool-calls') const call = first.message.content.find(block => block.type === 'tool-call') expect(call).toBeDefined() expect(call!.name).toBe('lookup_code') @@ -142,7 +149,7 @@ for (const profile of providerCases) { maxTokens: 2048, }) - expect(second.finish.kind).toBe('stop') + expectFinish(second, 'stop') expect(textOf(second).toLowerCase()).toContain('ocean') expect(expectNativeReplay(second, profile).stopReason).toBe('stop') }) From 39dea6107ea7ec00994142b078b04e12506dc7a9 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 14:06:18 +0800 Subject: [PATCH 38/73] fix(ci): target Azure Foundry OpenAI v1 route --- .github/workflows/pi-ai-provider-e2e.yml | 5 ++--- packages/llm/llm-pi-ai/tests/adapter.spec.ts | 12 ++++++------ .../llm/llm-pi-ai/tests/provider-apis.e2e.ts | 19 +++++++++++-------- 3 files changed, 19 insertions(+), 17 deletions(-) diff --git a/.github/workflows/pi-ai-provider-e2e.yml b/.github/workflows/pi-ai-provider-e2e.yml index 4be5f0c13a..d198abf5b5 100644 --- a/.github/workflows/pi-ai-provider-e2e.yml +++ b/.github/workflows/pi-ai-provider-e2e.yml @@ -68,10 +68,9 @@ jobs: - name: E2E tests (real Azure OpenAI and Anthropic APIs) env: AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY_EXTERNAL }} - AZURE_OPENAI_API_VERSION: v1 ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }} - DSH_PI_AI_AZURE_OPENAI_MODEL: ${{ inputs.azure_openai_model }} - DSH_PI_AI_AZURE_OPENAI_BASE_URL: https://openai-routerhub-resource.services.ai.azure.com/api/projects/openai/openai/ + DSH_PI_AI_OPENAI_MODEL: ${{ inputs.azure_openai_model }} + DSH_PI_AI_OPENAI_BASE_URL: https://openai-routerhub-resource.services.ai.azure.com/api/projects/openai/openai/v1 DSH_PI_AI_ANTHROPIC_MODEL: ${{ inputs.anthropic_model }} DSH_E2E_MAX_WORKERS: 2 run: >- diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 3ebb23ee50..26fbb392de 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -167,22 +167,22 @@ describe('PiAiAdapter provider routing', () => { expect(server.paths).toEqual(['/v1/responses']) }) - it('uses Azure OpenAI Responses with the configured project base path and API key', async () => { - vi.stubEnv('AZURE_OPENAI_API_VERSION', 'v1') + it('uses OpenAI Responses against an Azure project v1 path with its API key header', async () => { const server = await mockServer([{ status: 401, body: JSON.stringify({ error: { message: 'expected mock failure' } }) }]) const ctx = new Context() await ctx.plugin(LlmService) await ctx.plugin(LlmPiAi, { providers: [{ - provider: 'azure-openai-responses', + provider: 'openai', apiKey: 'test-key', - baseURL: `${server.url}/api/projects/openai/openai/`, + baseURL: `${server.url}/api/projects/openai/openai/v1`, + headers: { 'api-key': 'test-key' }, maxRetries: 0, }], }) - const result = await assemble(ctx, { provider: 'azure-openai-responses', model: 'gpt-5.5', messages: [] }) + const result = await assemble(ctx, { provider: 'openai', model: 'gpt-5.5', messages: [] }) expect(result.finish.kind).toBe('error') - expect(server.paths).toEqual(['/api/projects/openai/openai/responses?api-version=v1']) + expect(server.paths).toEqual(['/api/projects/openai/openai/v1/responses']) expect(server.headers[0]?.['api-key']).toBe('test-key') }) diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index b830bfe08b..a2fcc2c143 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -7,22 +7,24 @@ import type { PiAiReplayState } from '../src/replay.ts' import { assemble, type AssembledResult } from './assemble.ts' interface ProviderCase { - provider: 'azure-openai-responses' | 'anthropic' - api: 'azure-openai-responses' | 'anthropic-messages' + provider: 'openai' | 'anthropic' + api: 'openai-responses' | 'anthropic-messages' model: string apiKey?: string baseURL?: string + headers?: Record } -const azureOpenAIBaseURL = process.env.DSH_PI_AI_AZURE_OPENAI_BASE_URL ?? process.env.AZURE_OPENAI_BASE_URL +const openAIBaseURL = process.env.DSH_PI_AI_OPENAI_BASE_URL +const azureOpenAIKey = process.env.AZURE_OPENAI_API_KEY const providerCases: ProviderCase[] = [ { - provider: 'azure-openai-responses', - api: 'azure-openai-responses', - model: process.env.DSH_PI_AI_AZURE_OPENAI_MODEL ?? 'gpt-5.5', - ...process.env.AZURE_OPENAI_API_KEY ? { apiKey: process.env.AZURE_OPENAI_API_KEY } : {}, - ...azureOpenAIBaseURL ? { baseURL: azureOpenAIBaseURL } : {}, + provider: 'openai', + api: 'openai-responses', + model: process.env.DSH_PI_AI_OPENAI_MODEL ?? 'gpt-5.5', + ...azureOpenAIKey ? { apiKey: azureOpenAIKey, headers: { 'api-key': azureOpenAIKey } } : {}, + ...openAIBaseURL ? { baseURL: openAIBaseURL } : {}, }, { provider: 'anthropic', @@ -43,6 +45,7 @@ async function harness(): Promise { provider: profile.provider, ...profile.apiKey === undefined ? {} : { apiKey: profile.apiKey }, ...profile.baseURL === undefined ? {} : { baseURL: profile.baseURL }, + ...profile.headers === undefined ? {} : { headers: profile.headers }, })), }) return ctx From f21ed1bfa05449045b6d1b425def6e1c44c90d5e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 14:06:47 +0800 Subject: [PATCH 39/73] refactor(core): use initiator in loop internals --- ...2026-07-15-agent-initiator-scope.i18n.yaml | 4 +-- .../2026-07-15-agent-initiator-scope.md | 6 ++-- .../2026-07-15-agent-initiator-scope.zh.md | 6 ++-- packages/core/agent-loop/README.md | 2 +- packages/core/agent-loop/src/agent.ts | 2 +- packages/core/agent-loop/src/loop.ts | 28 ++++++++++--------- packages/core/agent-loop/src/tool-calls.ts | 9 ++---- 7 files changed, 28 insertions(+), 29 deletions(-) diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml index 9d9e11522f..7c9a43da69 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-15-agent-initiator-scope.md: 480bb56f811567406e964ca94fca04fd3d4ace62 -2026-07-15-agent-initiator-scope.zh.md: 29ea3a47f42d607a4584b888ace232de124e7598 +2026-07-15-agent-initiator-scope.md: 61aad24c13d66f206fed81ae6eb074666b23d0d2 +2026-07-15-agent-initiator-scope.zh.md: e420e3cd5fa4b2b2f868be11080c37f29b2ce313 diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md index 480bb56f81..61aad24c13 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md @@ -16,7 +16,7 @@ The mandatory `ctx.agents` service uses Node `AsyncLocalStorage` to carry the in `currentInitiator()` reads optionally, `requireInitiator()` throws `no initiating agent is active`, and `withInitiator(agent, operation)` preserves the operation's exact synchronous value or Promise. `withoutInitiator(operation)` establishes a clearing boundary for work that must not inherit an Agent. Session remains derived as `agent.session`; turn, step, tool call, `signal`, model, `cwd`, sandbox, and authorization stay with their existing owners. -`AgentLoop` already injects `ctx.agents` and wraps each concrete driver's complete `runLoop` lifetime in `agents.withInitiator(agent, ...)`. Concurrent drivers therefore receive independent stores, a child driver shadows its parent, and the parent store returns when the child boundary settles. Creation, persistence load, and unpublished `setup(agentCtx)` remain outside the child's driver boundary: creation initiated by a parent runs under the parent identity, while `agentCtx.agent` explicitly identifies the child. +`AgentLoop` already injects `ctx.agents` and wraps each concrete driver's complete `runLoop` lifetime in `agents.withInitiator(agent, ...)`. Its package-private loop, turn, step, and tool-call helpers recover the exact Agent from `ctx.agents` instead of forwarding the concrete driver through their signatures. Concurrent drivers therefore receive independent stores, a child driver shadows its parent, and the parent store returns when the child boundary settles. Creation, persistence load, and unpublished `setup(agentCtx)` remain outside the child's driver boundary: creation initiated by a parent runs under the parent identity, while `agentCtx.agent` explicitly identifies the child. Ambient identity does not replace explicit contracts. `ToolExecution.agent`, `AssembleContext.agent`, `GenerateOptions.sessionId`, task ownership, parent/child requests, `ctx.agent`, `agentCtx.agent`, approval and hook subjects, `cwd` selection, cancellation, worker/process messages, persistence records, and wire identity remain explicit. A remote boundary materializes the identity it needs into its typed request because ALS is process-local. @@ -30,9 +30,9 @@ This decision extends the [Agent registration-scope contract](2026-07-08-agent-s ## Verification -Agent service tests pin optional and required reads, exact synchronous and cross-realm Promise identity, overlapping, nested, and cleared boundaries, restoration after throws or rejection, drain ordering, and retained-reference errors. AgentLoop integration pins concurrent and nested drivers, agentless calls, AgentRegistry restart, and root teardown. Composition, module-graph, build, and runtime-closure checks keep `ctx.agents` wired through the default bundle, SDK spine, Python runtime closure, and direct AgentLoop harnesses without another provider. +Agent service tests pin optional and required reads, exact synchronous and cross-realm Promise identity, overlapping, nested, and cleared boundaries, restoration after throws or rejection, drain ordering, and retained-reference errors. AgentLoop integration pins concurrent and nested drivers, agentless calls, AgentRegistry restart, root teardown, and package-private loop and tool scheduling through the ambient lookup. Composition, module-graph, build, and runtime-closure checks keep `ctx.agents` wired through the default bundle, SDK spine, Python runtime closure, and direct AgentLoop harnesses without another provider. -Only a test-double host-aware transport consumes ambient identity; it derives `X-Harness-Session-Id` internally and verifies that tool schema and logged arguments contain no identity field. The service deliberately does not drain async work omitted from the Promise returned by the boundary operation; that work remains subject to its owner's explicit stop contract. +A test-double host-aware transport derives `X-Harness-Session-Id` internally and verifies that tool schema and logged arguments contain no identity field. The service deliberately does not drain async work omitted from the Promise returned by the boundary operation; that work remains subject to its owner's explicit stop contract. ## Alternatives considered diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md index 29ea3a47f4..e420e3cd5f 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md @@ -16,7 +16,7 @@ Harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负 `currentInitiator()` 用于可选读取，`requireInitiator()` 抛出 `no initiating agent is active`，`withInitiator(agent, operation)` 保留操作返回的同步值或 Promise 本身。`withoutInitiator(operation)` 会建立清空边界，供不得继承 Agent 的工作使用。会话仍通过 `agent.session` 推导；轮次、步骤、工具调用、`signal`、模型、`cwd`、沙箱和授权继续由现有归属方管理。 -`AgentLoop` 已经注入 `ctx.agents`，并用 `agents.withInitiator(agent, ...)` 包裹每个具体驱动的完整 `runLoop` 生命周期。因此，并发驱动使用彼此独立的存储，子驱动会遮蔽父驱动，子边界结束后父存储得到恢复。创建、持久化加载和尚未发布的 `setup(agentCtx)` 位于子驱动边界之外：由父 Agent 发起的创建使用父身份，而 `agentCtx.agent` 显式标识子 Agent。 +`AgentLoop` 已经注入 `ctx.agents`，并用 `agents.withInitiator(agent, ...)` 包裹每个具体驱动的完整 `runLoop` 生命周期。其包内私有的循环、轮次、步骤和工具调用辅助函数从 `ctx.agents` 恢复同一个 Agent，无需在函数签名中转发具体驱动。因此，并发驱动使用彼此独立的存储，子驱动会遮蔽父驱动，子边界结束后父存储得到恢复。创建、持久化加载和尚未发布的 `setup(agentCtx)` 位于子驱动边界之外：由父 Agent 发起的创建使用父身份，而 `agentCtx.agent` 显式标识子 Agent。 隐式身份不会取代显式契约。`ToolExecution.agent`、`AssembleContext.agent`、`GenerateOptions.sessionId`、任务归属、父子请求、`ctx.agent`、`agentCtx.agent`、审批与 hook 主体、`cwd` 选择、取消、worker 和进程消息、持久化记录及协议身份都保持显式传递。远程边界会把所需身份写入类型化请求，因为 ALS 只在进程内有效。 @@ -30,9 +30,9 @@ Harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负 ## 验证 -Agent 服务测试锁定可选与必需读取、同步值和跨 realm Promise 的引用身份、并发、嵌套及清空边界、同步抛错或 Promise 拒绝后的恢复、排空顺序及保留引用的错误。AgentLoop 集成测试锁定并发与嵌套驱动、无 Agent 调用、AgentRegistry 重启及根 Context 销毁。组合、模块图、构建及运行时闭包检查确保默认组合包、SDK 主干、Python 运行时闭包及直接 AgentLoop harness 通过 `ctx.agents` 完成接线，无需其他提供方。 +Agent 服务测试锁定可选与必需读取、同步值和跨 realm Promise 的引用身份、并发、嵌套及清空边界、同步抛错或 Promise 拒绝后的恢复、排空顺序及保留引用的错误。AgentLoop 集成测试锁定并发与嵌套驱动、无 Agent 调用、AgentRegistry 重启、根 Context 销毁，以及包内私有的循环和工具调度通过隐式查找完成。组合、模块图、构建及运行时闭包检查确保默认组合包、SDK 主干、Python 运行时闭包及直接 AgentLoop harness 通过 `ctx.agents` 完成接线，无需其他提供方。 -只有测试替身形式的宿主感知传输层消费隐式身份；它在内部推导 `X-Harness-Session-Id`，并验证工具 schema 与记录参数都不包含身份字段。服务有意不排空边界操作所返回 Promise 之外的异步工作；这类工作仍由所属方的显式停止契约管理。 +测试替身形式的宿主感知传输层在内部推导 `X-Harness-Session-Id`，并验证工具 schema 与记录参数都不包含身份字段。服务有意不排空边界操作所返回 Promise 之外的异步工作；这类工作仍由所属方的显式停止契约管理。 ## 考虑过的替代方案 diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index db5861f8c1..f0030845c1 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -50,7 +50,7 @@ The concrete `Agent` class, its `Inbox`, `runLoop`, and instance-bound publicati ### Loop lifecycle (`loop.ts`) -The driver owns one agent for its lifetime and runs inside `ctx.agents.withInitiator(agent, ...)`, so process-local asynchronous continuations can recover the initiating Agent. Creation, persistence load, and unpublished setup stay outside the driver boundary; explicit Agent fields remain authoritative at service, worker, process, persistence, and wire boundaries. The [agent service](../agent/README.md#initiating-agent-scope) owns propagation, teardown, and detached-work rules. +The driver owns one agent for its lifetime and runs inside `ctx.agents.withInitiator(agent, ...)`, so package-private loop, turn, step, and tool-call helpers recover the exact Agent from `ctx.agents` instead of forwarding the concrete driver through their signatures. Creation, persistence load, and unpublished setup stay outside the driver boundary; explicit Agent fields remain authoritative at service, worker, process, persistence, and wire boundaries. The [agent service](../agent/README.md#initiating-agent-scope) owns propagation, teardown, and detached-work rules. Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history. diff --git a/packages/core/agent-loop/src/agent.ts b/packages/core/agent-loop/src/agent.ts index 6ab02bfc13..61b661c082 100644 --- a/packages/core/agent-loop/src/agent.ts +++ b/packages/core/agent-loop/src/agent.ts @@ -387,7 +387,7 @@ export class ReactLoopAgent implements Agent { [startDriver](): void { if (this._status === 'disposed') return this.driverStarted = true - this.done = this.loopCtx.agents.withInitiator(this, () => runLoop(this.loopCtx, this, { + this.done = this.loopCtx.agents.withInitiator(this, () => runLoop(this.loopCtx, { inbox: this.#inbox, maxParallelToolCalls: this.maxParallelToolCalls, setStatus: (status) => { this.setStatus(status) }, diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index a153eba7e4..56211fb2e0 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -19,7 +19,6 @@ import { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import type { PromptAssembly } from '@deepseek-ai/dsh-system-prompt' import type {} from '@deepseek-ai/dsh-tools' import { executeToolCalls } from './tool-calls.ts' -import type { ReactLoopAgent } from './agent.ts' import type { Inbox } from './inbox.ts' /** An Error with an optional machine-readable code (e.g., from LlmError or a throwing plugin). */ @@ -96,11 +95,13 @@ export interface LoopHandle { /** * Drive queued batches as durable turns until disposal. Plugin failures end the * current turn without terminating the driver. - * @param ctx - the plugin context the loop reaches events (agent/…, session/flush) and services (systemPrompt, llm, tools) through. - * @param agent - the agent this invocation drives for its whole lifetime (its inbox, session, and options). + * @param ctx - the plugin context the loop reaches its initiating Agent, + * events (agent/…, session/flush), and services (systemPrompt, llm, tools) + * through. * @param handle - the bridge to the agent's mutable state: status/abort setters plus the disposal and cancel-marker reads. */ -export async function runLoop(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle): Promise { +export async function runLoop(ctx: Context, handle: LoopHandle): Promise { + const agent = ctx.agents.requireInitiator() // Per-instance prefix and request-header state; conversation history remains in the session log. const transmission = createTransmissionLog() @@ -138,7 +139,7 @@ export async function runLoop(ctx: Context, agent: ReactLoopAgent, handle: LoopH const turn = lastTurnNumber(session) + 1 let terminalStopped = false try { - terminalStopped = await runTurn(ctx, events, agent, handle, turn, transmission) + terminalStopped = await runTurn(ctx, events, handle, turn, transmission) } catch (error: unknown) { // Pre-turn failure has no durable boundary to close; report it without appending outside a turn. const err = toError(error) @@ -161,8 +162,9 @@ export async function runLoop(ctx: Context, agent: ReactLoopAgent, handle: LoopH } async function runTurn( - ctx: Context, events: AgentEventDispatch, agent: ReactLoopAgent, handle: LoopHandle, turn: number, transmission: TransmissionLog, + ctx: Context, events: AgentEventDispatch, handle: LoopHandle, turn: number, transmission: TransmissionLog, ): Promise { + const agent = ctx.agents.requireInitiator() const { session } = agent // Drain before opening the turn, but append only after `turn/start`. @@ -262,7 +264,7 @@ async function runTurn( // Steering from the previous round's continuation listeners joins before // the request. - drainSteering(agent, handle.inbox, turn) + drainSteering(session, handle.inbox, turn) // The step's AbortController exists BEFORE any async pre-step work so a // dispose() or cancel() — in a synchronous turn-start listener or an @@ -336,7 +338,7 @@ async function runTurn( let stepOutcome: { hadToolCalls: boolean; finish: FinishReason } | { error: Error } try { stepOutcome = await runStep( - ctx, events, agent, handle, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal) + ctx, events, handle, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal) } catch (error: unknown) { stepOutcome = { error: toError(error) } } finally { @@ -365,7 +367,7 @@ async function runTurn( if (stepReason) reason = stepReason // Steering that arrived during streaming/tool execution. - const steered = drainSteering(agent, handle.inbox, turn) + const steered = drainSteering(session, handle.inbox, turn) closeStep() @@ -455,10 +457,10 @@ async function runTurn( } /** Drain the steering queue into the session. Returns whether any arrived. */ -function drainSteering(agent: ReactLoopAgent, inbox: Inbox, turn: number): boolean { +function drainSteering(session: Session, inbox: Inbox, turn: number): boolean { const messages = inbox.drainSteering() for (const message of messages) { - agent.session.append('steering/message', { turn, content: message.content, source: message.source }, { surfaceOp: 'append' }) + session.append('steering/message', { turn, content: message.content, source: message.source }, { surfaceOp: 'append' }) } return messages.length > 0 } @@ -472,7 +474,6 @@ function drainSteering(agent: ReactLoopAgent, inbox: Inbox, turn: number): boole async function runStep( ctx: Context, events: AgentEventDispatch, - agent: ReactLoopAgent, handle: LoopHandle, turn: number, step: number, @@ -482,6 +483,7 @@ async function runStep( transmission: TransmissionLog, signal: AbortSignal, ): Promise<{ hadToolCalls: boolean; finish: FinishReason }> { + const agent = ctx.agents.requireInitiator() const { session, options } = agent // Seed the first request from agent options and later requests from the logged header; @@ -567,7 +569,7 @@ async function runStep( if (toolCalls.length === 0) return { hadToolCalls: false, finish: assembler.finish } return handle.withToolBatch(async (acceptContext) => { await executeToolCalls( - ctx, agent, turn, step, toolCalls, signal, handle.maxParallelToolCalls, acceptContext, + ctx, turn, step, toolCalls, signal, handle.maxParallelToolCalls, acceptContext, ) return { hadToolCalls: true, finish: assembler.finish } }) diff --git a/packages/core/agent-loop/src/tool-calls.ts b/packages/core/agent-loop/src/tool-calls.ts index 3f3581c70a..0b9f9ae26b 100644 --- a/packages/core/agent-loop/src/tool-calls.ts +++ b/packages/core/agent-loop/src/tool-calls.ts @@ -14,7 +14,6 @@ import { assertNever, type ToolCallBlock } from '@deepseek-ai/dsh-llm' import type { HookContext } from '@deepseek-ai/dsh-agent' import type { Session } from '@deepseek-ai/dsh-session' import { TOOL_REGISTRY_SCHEDULER, type ToolExecutionInput, type ToolExecutionMode, type ToolExecutionResult, type ToolRunContext } from '@deepseek-ai/dsh-tools' -import type { ReactLoopAgent } from './agent.ts' /** One tool call after argument parsing, ready to schedule. */ interface PlannedCall { @@ -35,7 +34,6 @@ interface Slot { * accepting their context into the batch FIFO owned by the caller. * * @param ctx - loop context that owns the tool registry. - * @param agent - agent and session receiving the call lifecycle. * @param turn - current turn number. * @param step - current step number. * @param toolCalls - assistant calls in model order. @@ -45,7 +43,6 @@ interface Slot { */ export async function executeToolCalls( ctx: Context, - agent: ReactLoopAgent, turn: number, step: number, toolCalls: ToolCallBlock[], @@ -53,7 +50,7 @@ export async function executeToolCalls( maxParallel: number, acceptContext: (context: HookContext) => void, ): Promise { - const { session } = agent + const agent = ctx.agents.requireInitiator() // Inputs are distinct because tools/execute wrappers may replace `exec.signal`. const planned: PlannedCall[] = toolCalls.map(block => ({ @@ -74,7 +71,7 @@ export async function executeToolCalls( const first = planned[next]! const mode = ctx.tools.executionMode(first.exec).kind const group = mode === 'parallel' ? planned.slice(next) : [first] - next += await runGroup(ctx, session, turn, step, group, mode, signal, maxParallel, acceptContext) + next += await runGroup(ctx, turn, step, group, mode, signal, maxParallel, acceptContext) } } @@ -96,7 +93,6 @@ function parseArguments(raw: string): unknown { */ async function runGroup( ctx: Context, - session: Session, turn: number, step: number, group: PlannedCall[], @@ -105,6 +101,7 @@ async function runGroup( maxParallel: number, acceptContext: (context: HookContext) => void, ): Promise { + const { session } = ctx.agents.requireInitiator() /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) const slots: (Slot | undefined)[] = group.map(() => undefined) From fc478d675c8ce32bc838f2f56a60ac0e0cdc1511 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 14:10:47 +0800 Subject: [PATCH 40/73] fix(ci): suppress bearer auth for Azure API keys --- packages/llm/llm-pi-ai/tests/adapter.spec.ts | 3 ++- packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 26fbb392de..af791bd806 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -176,7 +176,7 @@ describe('PiAiAdapter provider routing', () => { provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/api/projects/openai/openai/v1`, - headers: { 'api-key': 'test-key' }, + headers: { 'api-key': 'test-key', Authorization: '' }, maxRetries: 0, }], }) @@ -184,6 +184,7 @@ describe('PiAiAdapter provider routing', () => { expect(result.finish.kind).toBe('error') expect(server.paths).toEqual(['/api/projects/openai/openai/v1/responses']) expect(server.headers[0]?.['api-key']).toBe('test-key') + expect(server.headers[0]?.authorization).toBe('') }) it.each([ diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index a2fcc2c143..107c12d264 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -23,7 +23,9 @@ const providerCases: ProviderCase[] = [ provider: 'openai', api: 'openai-responses', model: process.env.DSH_PI_AI_OPENAI_MODEL ?? 'gpt-5.5', - ...azureOpenAIKey ? { apiKey: azureOpenAIKey, headers: { 'api-key': azureOpenAIKey } } : {}, + ...azureOpenAIKey + ? { apiKey: azureOpenAIKey, headers: { 'api-key': azureOpenAIKey, Authorization: '' } } + : {}, ...openAIBaseURL ? { baseURL: openAIBaseURL } : {}, }, { From 4791b40b269b19a3c40daae5ed412ebc5d1b6d42 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 14:18:13 +0800 Subject: [PATCH 41/73] fix(llm-pi-ai): classify usage-based context overflow Pass each resolved catalog model capacity into pi-ai stream conversion so the upstream full-message classifier can recognize provider-specific, silent, and length-stop overflow signals. Retain the harness text fallback for legacy provider wording and cover the catalog-resolution path with a mock-provider regression. --- packages/llm/llm-pi-ai/README.md | 2 +- packages/llm/llm-pi-ai/src/adapter.ts | 2 +- packages/llm/llm-pi-ai/src/stream.ts | 34 +++++++++++++++----- packages/llm/llm-pi-ai/tests/adapter.spec.ts | 26 ++++++++++++++- packages/llm/llm-pi-ai/tests/convert.spec.ts | 28 ++++++++++++++++ 5 files changed, 81 insertions(+), 11 deletions(-) diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index e2d56e28a5..1deae8f039 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -43,7 +43,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state ## Vocabulary differences - pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output. -- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks, with recognized context overflow normalized to `CONTEXT_WINDOW_EXCEEDED`. +- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks. Provider-specific error text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. - pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map. - `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers. diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index 26e28ecd1f..7f40c67da3 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -115,7 +115,7 @@ export class PiAiAdapter extends LlmAdapter { // Harness-owned and therefore win collisions. headers: requestHeaders(profile.headers), }) - yield* toStreamChunks(events) + yield* toStreamChunks(events, model.contextWindow) } finally { options.signal?.removeEventListener('abort', onCallerAbort) controller.abort('consumer stopped streaming') diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index f7a0d6ee7f..8bc41cded2 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -10,6 +10,7 @@ import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm' import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' +import { isContextOverflow } from '@earendil-works/pi-ai' import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai' import { toPiReplayState } from './replay.ts' @@ -30,10 +31,6 @@ export function mapUsage(usage: PiUsage): TokenUsage { function classifyPiAiError(message: string): string { if (/\b(?:401|403)\b/.test(message)) return 'AUTH' if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT' - // TODO: Classify the full message with pi-ai's isContextOverflow() and the - // resolved model's contextWindow so provider-specific and usage-based overflows - // reach automatic compaction. - if (isContextWindowExceededError(message)) return CONTEXT_WINDOW_EXCEEDED_CODE if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' return 'PI_AI_ERROR' @@ -42,9 +39,22 @@ function classifyPiAiError(message: string): string { /** * Map a terminal pi-ai event to the harness finish reason. * @param message - the assistant message carried by the `done` or `error` event. + * @param contextWindow - resolved catalog capacity for usage-based overflow detection. * @returns the harness reason; `error` yields `{kind: 'error'}` with a code classified from the error text. */ -export function mapStopReason(message: AssistantMessage): FinishReason { +export function mapStopReason(message: AssistantMessage, contextWindow?: number): FinishReason { + const piAiOverflow = isContextOverflow(message, contextWindow) + const harnessOverflow = message.stopReason === 'error' + && message.errorMessage !== undefined + && isContextWindowExceededError(message.errorMessage) + if (piAiOverflow || harnessOverflow) { + return { + kind: 'error', + message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`, + code: CONTEXT_WINDOW_EXCEEDED_CODE, + } + } + switch (message.stopReason) { case 'stop': return { kind: 'stop' } case 'length': return { kind: 'max-tokens' } @@ -62,10 +72,14 @@ export function mapStopReason(message: AssistantMessage): FinishReason { * mid-stream — failures arrive as `error` events, which become error/aborted * `finish` chunks (the harness protocol's other error-delivery style). * @param events - one assistant turn's pi-ai event stream. + * @param contextWindow - resolved catalog capacity for usage-based overflow detection. * @returns the harness chunks, ending with `usage` then `finish`; throws * `LlmError` (`STREAM_CLOSED`) if the source ends without a terminal event. */ -export async function* toStreamChunks(events: AsyncIterable): AsyncGenerator { +export async function* toStreamChunks( + events: AsyncIterable, + contextWindow?: number, +): AsyncGenerator { // pi-ai contentIndex ↔ our block index map 1:1 (both count blocks from 0 // in stream order), but we track ids per index for tool calls. const toolIds = new Map() @@ -128,13 +142,17 @@ export async function* toStreamChunks(events: AsyncIterable { const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) expect(result.finish).toMatchObject({ kind: 'error', code }) }) + + it('uses the resolved catalog context window for usage-based overflow detection', async () => { + const model = getModels('deepseek').find(candidate => candidate.id === 'deepseek-v4-flash') + if (model === undefined) throw new Error('deepseek-v4-flash missing from pi-ai test catalog') + const events = [ + '{"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}', + JSON.stringify({ + choices: [{ delta: {}, index: 0, finish_reason: 'stop' }], + usage: { prompt_tokens: model.contextWindow + 1, completion_tokens: 0 }, + }), + '[DONE]', + ] + const server = await mockServer([{ events }]) + const ctx = await harness(server.url) + + const result = await assemble(ctx, { model: model.id, messages: [] }) + + expect(result.finish).toEqual({ + kind: 'error', + message: `pi-ai detected context overflow for model "${model.id}"`, + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + }) }) describe('provider profile lifecycle', () => { diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index 6c58f67bb7..a2f37ce511 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -537,6 +537,34 @@ describe('mapStopReason / mapUsage', () => { }))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' }) }) + it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => { + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'prompt is too long: 213462 tokens > 200000 maximum', + }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'ThrottlingException: Too many tokens, rate limit reached', + }))).toMatchObject({ kind: 'error', code: 'RATE_LIMIT' }) + }) + + it('uses the resolved context window for silent and length-stop overflows', () => { + const silent = assistant({ stopReason: 'stop', usage: usage(101, 0) }) + expect(mapStopReason(silent)).toEqual({ kind: 'stop' }) + expect(mapStopReason(silent, 100)).toEqual({ + kind: 'error', + message: 'pi-ai detected context overflow for model "deepseek-v4-flash"', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + + const truncated = assistant({ stopReason: 'length', usage: usage(80, 0, 19) }) + expect(mapStopReason(truncated)).toEqual({ kind: 'max-tokens' }) + expect(mapStopReason(truncated, 100)).toMatchObject({ + kind: 'error', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }) + }) + it('maps cache fields only when nonzero', () => { expect(mapUsage(usage(10, 5, 8, 2))).toEqual({ inputTokens: 10, From 7a9c2ef4738ce11b8c9bdde4ce65ea14bc022f61 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 14:20:27 +0800 Subject: [PATCH 42/73] docs(core): state loop initiator precondition --- packages/core/agent-loop/src/loop.ts | 5 ++++- packages/core/agent-loop/src/tool-calls.ts | 4 +++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 56211fb2e0..4f2f768d89 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -94,11 +94,14 @@ export interface LoopHandle { /** * Drive queued batches as durable turns until disposal. Plugin failures end the - * current turn without terminating the driver. + * current turn without terminating the driver. The caller establishes the + * `ctx.agents.withInitiator()` boundary before entry; package-private helpers + * recover that exact Agent from the inherited store. * @param ctx - the plugin context the loop reaches its initiating Agent, * events (agent/…, session/flush), and services (systemPrompt, llm, tools) * through. * @param handle - the bridge to the agent's mutable state: status/abort setters plus the disposal and cancel-marker reads. + * @throws when no initiating Agent is active. */ export async function runLoop(ctx: Context, handle: LoopHandle): Promise { const agent = ctx.agents.requireInitiator() diff --git a/packages/core/agent-loop/src/tool-calls.ts b/packages/core/agent-loop/src/tool-calls.ts index 0b9f9ae26b..5bf41c4895 100644 --- a/packages/core/agent-loop/src/tool-calls.ts +++ b/packages/core/agent-loop/src/tool-calls.ts @@ -32,8 +32,10 @@ interface Slot { * Schedule one assistant step's tool calls by their live concurrency mode. * Started calls receive ordered results. Abort drains them and rethrows after * accepting their context into the batch FIFO owned by the caller. + * The committed step's AgentLoop driver boundary supplies the initiating Agent + * that becomes each explicit {@link ToolExecutionInput.agent}. * - * @param ctx - loop context that owns the tool registry. + * @param ctx - loop context that owns the tool registry and carries the initiating Agent. * @param turn - current turn number. * @param step - current step number. * @param toolCalls - assistant calls in model order. From 62fc876318dd6b06b1e70e6a769a2194b13747be Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 14:39:49 +0800 Subject: [PATCH 43/73] feat: inspect exact Cordis API JSDoc --- ...6-07-08-self-referential-cordis-toolset.md | 8 +- docs/tool-catalog.md | 6 +- .../system-prompt.golden.md | 4 +- .../tool-schemas.golden.json | 6 +- packages/cordis/tool-cordis/README.md | 6 +- .../cordis/tool-cordis/src/api-catalog.ts | 535 ++++++++++++++---- packages/cordis/tool-cordis/src/index.ts | 18 +- packages/cordis/tool-cordis/src/inspect.ts | 71 ++- packages/cordis/tool-cordis/src/present.ts | 5 +- .../cordis/tool-cordis/tests/inspect.spec.ts | 57 +- .../cordis/tool-cordis/tests/present.spec.ts | 8 + .../tool-cordis/tests/tool-cordis.spec.ts | 3 +- scripts/gen-cordis-api.ts | 35 +- 13 files changed, 616 insertions(+), 146 deletions(-) diff --git a/docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md b/docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md index 4d592af42b..94d11ee931 100644 --- a/docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md +++ b/docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md @@ -18,11 +18,11 @@ The vm isolates accidental global pollution, and the context façade hides frame | Tool | Contract | |---|---| -| `cordis_inspect` | Read-only report over the live runtime, one Markdown section per `what` value (omit `what` for all sections). Never mutates. | +| `cordis_inspect` | Read-only report over the live runtime, one Markdown section per `what` value (omit `what` for all sections). An exact `name` with `what: "api"` or `what: "events"` narrows to one source-documented target. Never mutates. | | `cordis_mount` | Evaluates `code` (the body of an async JavaScript function) in a `node:vm` sandbox; the code must `return` a cordis plugin, which is mounted as a child of the `cordis-dynamic` group fiber and tracked under a fresh id (`dyn-1`, `dyn-2`, …). | | `cordis_unmount` | Disposes one dynamic mount by id and returns only after disposal reaches quiescence — every registration the plugin made is unwound, not merely requested to stop. | -`cordis_inspect` sections: `services` (every provided ctx service and the owning fiber, non-active owners flagged), `plugins` (a flat list of every loaded plugin with its lifecycle state, from `ctx.registry` — what capabilities are loaded, deliberately not the tree shape), `tools` (what the model can call), `dynamic` (the mount table: id, name, state, provided services, awaited services), `api` (live service signatures + the type shapes they reference, from the generated catalog), and `events` (harness events with dispatch mode and signature). The model-facing tool descriptions carry the operational rules the model needs at call time; [the generated tool catalog](../../../tool-catalog.md) is their exhaustive rendering. +`cordis_inspect` sections: `services` (every provided ctx service and the owning fiber, non-active owners flagged), `plugins` (a flat list of every loaded plugin with its lifecycle state, from `ctx.registry` — what capabilities are loaded, deliberately not the tree shape), `tools` (what the model can call), `dynamic` (the mount table: id, name, state, provided services, awaited services), `api` (live service signatures + the type shapes they reference, from the generated catalog), and `events` (harness events with dispatch mode and signature). Broad `api` and `events` reports omit full JSDoc to stay compact; an exact `name` returns one service or event with its original method/declaration JSDoc. A name is invalid with other sections, unknown targets fail, and an API target must be live. The model-facing tool descriptions carry the operational rules the model needs at call time; [the generated tool catalog](../../../tool-catalog.md) is their exhaustive rendering. ### Sandbox semantics @@ -44,9 +44,9 @@ Mounts relate to each other through ordinary cordis service semantics, with thei ### The generated API catalog -`cordis_inspect` serves API and event data from a generated catalog rather than a duplicated table. The generator reuses the Cordis catalog AST scan and emits service summaries, signatures, event modes, referenced type declarations, and the inherited context surface. Ambiguous type names are omitted and oversized declarations are marked as truncated. +`cordis_inspect` serves API and event data from a generated catalog rather than a duplicated table. The generator reuses the Cordis catalog AST scan and emits service summaries, signatures, original service-method and event JSDoc, event modes, referenced type declarations, and the inherited context surface. Ambiguous type names are omitted and oversized declarations are marked as truncated. -Freshness is gated like every generated artifact: `pnpm run verify-cordis-api` (in `doc-sync`) regenerates in memory and fails on any diff, so a JSDoc edit that changes a public signature cannot ship without regenerating the catalog the model reads. At runtime the inspect tool intersects the catalog with the live runtime rather than dumping it: live catalogued services render summary + signatures, live services without a catalog entry (mount-provided ones) render name + owning fiber, catalogued services with no live provider are listed tersely, and the referenced type shapes follow. +Freshness is gated like every generated artifact: `pnpm run verify-cordis-api` (in `doc-sync`) regenerates in memory and fails on any diff, so a JSDoc or public-signature edit cannot ship without regenerating the catalog the model reads. At runtime the inspect tool intersects the catalog with the live runtime rather than dumping it: broad reports render live catalogued services as summary + signatures, live services without a catalog entry (mount-provided ones) as name + owning fiber, catalogued services with no live provider tersely, and then the referenced type shapes. Exact-name reports render one live service or event with the original JSDoc immediately before each signature; keeping that detail opt-in avoids charging its token cost on exploratory listings. ### Configuration, rendering, and observability diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index 3dacb56902..dfb97deef3 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -169,7 +169,7 @@ The bash tool is the model-facing consumer of the bash executor seam. A `run_in_ ### `cordis_inspect` -Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections. +Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections. With `what:"api"` or `what:"events"`, pass an exact `name` to narrow to one service/event and include its original source JSDoc. ```json { @@ -186,6 +186,10 @@ Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: "api", "events" ] + }, + "name": { + "type": "string", + "description": "Exact service key or event name whose original JSDoc to include; valid only with what:\"api\" or what:\"events\"." } } } diff --git a/examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md b/examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md index fb5b48c70c..b8acef973c 100644 --- a/examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md +++ b/examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md @@ -44,10 +44,12 @@ declare const tools: { /** Required with sandbox_permissions: one sentence for the user explaining why this exact command needs the wider access. */ justification?: string; }): Promise; - /** Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections. */ + /** Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections. With `what:"api"` or `what:"events"`, pass an exact `name` to narrow to one service/event and include its original source JSDoc. */ cordis_inspect(args: { /** Limit the report to one section. Omit for all sections. */ what?: "services" | "plugins" | "tools" | "dynamic" | "api" | "events"; + /** Exact service key or event name whose original JSDoc to include; valid only with what:"api" or what:"events". */ + name?: string; }): Promise; /** Mount a NEW cordis plugin into the live runtime that is running THIS agent (self-modification). `code` runs as the body of an async JavaScript function in an isolated sandbox and MUST `return` a plugin. Two forms: FUNCTION form `return (ctx) => { … }` — declares no inject, so it can register tools, listen to events, and provide services, but reaching ANY service (e.g. ctx.bash) throws; use it only when you need no services. OBJECT form `return { name?, inject: ['bash', 'llm', …], apply(ctx) { … } }` — declares dependencies, and cordis activates the plugin only after the services exist; PREFER this form. You may reach ONLY the services you list in inject: an undeclared service throws even if it exists, because an undeclared dependency would not be cleaned up if its provider is unmounted. BEFORE calling a service from your code, read cordis_inspect what:"api" — it lists method signatures AND the type shapes of their arguments/returns (do not guess a field's type; e.g. a bash run's stdout is an object, not a string). Inside `apply`, use the standard cordis API: `ctx.on(event, listener)` to observe events (see cordis_inspect what:"events"), or call `harness.registerTool(ctx, harness.defineTool({ name, description, parameters: { text: { type: 'string', required: true } }, async execute(args) { … } }))` to give yourself a new tool — it becomes callable on your NEXT step. Tool parameters: each key IS a property — { type: 'string'|'number'|'boolean'|'object'|'array', required?: true, description?, enum?, items?, properties? }; a JSON-Schema-style { type: 'object', properties, required: […] } wrapper and type 'integer' are also accepted and normalized. A tool's `execute` MUST return an ARRAY of content blocks, e.g. `return [{ type: 'text', text: someString }]` — never a bare string. Mounts can COMPOSE: one plugin may `ctx.provide('name', value)` a service and another may declare `inject: ['name']` to consume it — the consumer stays pending until the provider exists and returns to pending when the provider is unmounted. Everything registered inside `apply` is cleaned up automatically on unmount. Sandbox globals: `console` (tagged `[cordis:]`, writes through to the harness terminal), `harness.defineTool`, `harness.registerTool`, `btoa`, `atob`, `TextEncoder`, `TextDecoder`. Node APIs are DISABLED — do filesystem/network/timer work through the cordis services, never Node built-ins: `require`, `setTimeout`/`setInterval`, and `fetch` throw redirect errors; `process` and `Buffer` are undefined. Instead use inject: ['fs'] + ctx.fs for files, inject: ['web'] + ctx.web for HTTP, inject: ['bash'] + ctx.bash for processes, and inject: ['timer'] + ctx.setTimeout/ctx.setInterval for timing (fiber effects, auto-cleaned on unmount) — cordis_inspect what:"api" shows what THIS runtime provides. Write PLAIN JavaScript, not TypeScript (no `as`, no type annotations). Cautions: (1) waterfall events (e.g. tools/pre-execute) hand the listener a trailing `next` callback which MUST be called — returning without `next()` VETOES the call; prefer plain notification events unless you intend to intercept. (2) Never await something that only resolves after the current turn (your code runs INSIDE a tool call of that turn — it would deadlock). (3) Your `ctx` is a restricted façade: you can register tools, observe events, provide/consume services, and use timers, but framework internals (ctx.root, ctx.fiber, ctx.extend, ctx.plugin, …) are withheld. It is not a security boundary though — the services you inject (e.g. ctx.bash) reach the real runtime. */ cordis_mount(args: { diff --git a/examples/acp-agent/tests/snapshots/advanced-toolchain/tool-schemas.golden.json b/examples/acp-agent/tests/snapshots/advanced-toolchain/tool-schemas.golden.json index 57e78b6345..978819fa1f 100644 --- a/examples/acp-agent/tests/snapshots/advanced-toolchain/tool-schemas.golden.json +++ b/examples/acp-agent/tests/snapshots/advanced-toolchain/tool-schemas.golden.json @@ -47,7 +47,7 @@ }, { "name": "cordis_inspect", - "description": "Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections.", + "description": "Inspect the live cordis runtime that is running THIS agent. Read-only. Sections: `services` (every provided ctx service and the plugin fiber that owns it), `plugins` (a flat list of the loaded plugins with their lifecycle states), `tools` (the model-facing tools currently registered, i.e. what you can call), `dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), `api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), `events` (every harness event with its dispatch mode and exact signature — pick listener targets here). Omit `what` to get all six sections. With `what:\"api\"` or `what:\"events\"`, pass an exact `name` to narrow to one service/event and include its original source JSDoc.", "parameters": { "type": "object", "properties": { @@ -62,6 +62,10 @@ "api", "events" ] + }, + "name": { + "type": "string", + "description": "Exact service key or event name whose original JSDoc to include; valid only with what:\"api\" or what:\"events\"." } } } diff --git a/packages/cordis/tool-cordis/README.md b/packages/cordis/tool-cordis/README.md index fdc4e51e6b..c43e2fcc4d 100644 --- a/packages/cordis/tool-cordis/README.md +++ b/packages/cordis/tool-cordis/README.md @@ -4,7 +4,7 @@ The self-referential cordis toolset: three model-facing tools over the live runt ## What it does -- `cordis_inspect` — read-only report over the runtime: services, the loaded-plugin list, registered tools, the dynamic-mount table, and the catalog-backed `api` / `events` references. +- `cordis_inspect` — read-only report over the runtime: services, the loaded-plugin list, registered tools, the dynamic-mount table, and the catalog-backed `api` / `events` references. An exact `name` with `what: "api"` or `what: "events"` narrows the report and adds the original source JSDoc. - `cordis_mount` — evaluates model-written JavaScript (the body of an async function) in a `node:vm` sandbox; the code must `return` a cordis plugin, which is mounted under the `cordis-dynamic` group fiber and tracked as `dyn-`. - `cordis_unmount` — disposes one mount by id, returning only after quiescence. @@ -22,7 +22,7 @@ The sandbox isolates globals but is not a security boundary. Node globals are ab ## The generated API catalog -`src/api-catalog.ts` is generated by `scripts/gen-cordis-api.ts` from the same AST walk as [docs/cordis-catalog](../../../docs/cordis-catalog/services.md) and freshness-gated by `pnpm run verify-cordis-api` (in `doc-sync`) — never edit it by hand. `cordis_inspect` intersects it with the live service store at call time. +`src/api-catalog.ts` is generated by `scripts/gen-cordis-api.ts` from the same AST walk as [docs/cordis-catalog](../../../docs/cordis-catalog/services.md) and freshness-gated by `pnpm run verify-cordis-api` (in `doc-sync`) — never edit it by hand. `cordis_inspect` intersects it with the live service store at call time. Broad `api` / `events` reports render summaries and signatures only; an exact `name` opts into the retained method/event JSDoc, and unknown or non-running service targets fail loud. ## Rendering @@ -42,7 +42,7 @@ Namespace plugin: named exports `name` / `inject` / `Config` / `apply`, no defau ### Tool-call history and results -**What the model sees**: Inspect joins selected sections exactly as `##
` then a newline and the data-dependent body, with one blank line between sections. Mount returns `mounted (plugin "", state: )`, optionally inserting ` — waiting for service(s): (activates when provided)` before the closing parenthesis. Unmount returns `unmounted (plugin "")`; an unknown id becomes `Error: no dynamic plugin with id "" (list mounts with cordis_inspect what:"dynamic")`. The submitted mount program remains in the assistant tool-call history. +**What the model sees**: Inspect joins selected sections exactly as `##
` then a newline and the data-dependent body, with one blank line between sections. Its broad API/event reports omit JSDoc; `name` with `what: "api"` or `what: "events"` returns one exact target with its original JSDoc. Mount returns `mounted (plugin "", state: )`, optionally inserting ` — waiting for service(s): (activates when provided)` before the closing parenthesis. Unmount returns `unmounted (plugin "")`; an unknown id becomes `Error: no dynamic plugin with id "" (list mounts with cordis_inspect what:"dynamic")`. The submitted mount program remains in the assistant tool-call history. **Token effect**: Inspect output and mount code are data-dependent and resent until compaction; lifecycle acknowledgements are small. diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 7a02519b02..b274e357bf 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -4,22 +4,30 @@ * `pnpm run verify-cordis-api` in doc-sync). * * The machine-readable cordis API catalog `cordis_inspect` serves to the - * model: harness services (summary + public method signatures), harness - * events (mode + signature), and the inherited `ctx` surface. Produced by + * model: harness services (summary + public method signatures/JSDoc), + * harness events (mode + signature/JSDoc), and the inherited `ctx` surface. Produced by * the same AST walk as docs/cordis-catalog, so this data and the rendered * docs cannot diverge. * * @module @deepseek-ai/dsh-tool-cordis/api-catalog */ -/** One harness `ctx.` service: its one-line summary and public method signatures. */ +/** One public service method and its source-owned contract. */ +export interface ServiceApiMethod { + /** Public method signature with its body stripped. */ + signature: string + /** Original method JSDoc, with only container indentation removed. */ + jsDoc: string +} + +/** One harness `ctx.` service: its one-line summary and public methods. */ export interface ServiceApiEntry { /** The `ctx.` name, e.g. `tools`. */ key: string /** First sentence of the service class JSDoc. */ summary: string - /** Public method signatures, bodies stripped, in source order. */ - methods: readonly string[] + /** Public methods, bodies stripped, in source order. */ + methods: readonly ServiceApiMethod[] } /** One harness event: its dispatch mode, exact signature, and one-line summary. */ @@ -30,6 +38,8 @@ export interface EventApiEntry { mode: string /** The exact listener signature, whitespace-normalized. */ signature: string + /** Original event JSDoc, with only container indentation removed. */ + jsDoc: string /** First sentence of the event JSDoc. */ summary: string } @@ -56,239 +66,524 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ key: 'agentLoop', summary: 'Concrete agent factory and driver service.', methods: [ - 'create(id: SessionId, options: AgentOptions = {}, meta: Pick = {}): Agent', - 'async createAgent(ownerCtx: Context, options: CreateAgentOptions): Promise', - 'async resume(ownerCtx: Context, options: ResumeAgentOptions): Promise', + { + signature: 'create(id: SessionId, options: AgentOptions = {}, meta: Pick = {}): Agent', + jsDoc: '/**\n * Create an agent and session under one caller-supplied identity, owned by\n * the accessing fiber. Constructor-driven config calls mint a fresh combined\n * id before entering this boundary.\n * @param id - shared agent/session identity.\n * @param options - concrete loop options.\n * @param meta - optional fresh-session workspace metadata.\n * @returns the published running agent.\n */', + }, + { + signature: 'async createAgent(ownerCtx: Context, options: CreateAgentOptions): Promise', + jsDoc: '/**\n * Create an owned agent on a caller-supplied session id.\n * @param ownerCtx - caller context that structurally owns the transaction.\n * @param options - identities, session seed/metadata, loop options, setup, and cancellation.\n * @returns the published handle.\n */', + }, + { + signature: 'async resume(ownerCtx: Context, options: ResumeAgentOptions): Promise', + jsDoc: '/**\n * Resume an owned agent from the configured persistence service.\n * @param ownerCtx - caller context that owns load, setup, and the live lifecycle.\n * @param options - persisted identity, loop options, setup, and cancellation.\n * @returns the published handle.\n */', + }, ], }, { key: 'agents', summary: 'Agent registry (`ctx.agents`): tracks live agents so UI, hook, and orchestrator plugins can find them without depending on the concrete loop package.', methods: [ - 'setFactory(factory: AgentFactory): () => void', - 'async create(options: CreateAgentOptions): Promise', - 'async resume(options: ResumeAgentOptions): Promise', - 'register(agent: Agent): () => void', - 'enter(agent: Agent, owner: Agent | undefined): () => void', - 'announce(agent: Agent): void', - 'get(id: SessionId): Agent | undefined', - 'isOwnedBy(id: SessionId, owner: Agent): boolean', - 'list(): Agent[]', - 'roots(): Agent[]', + { + signature: 'setFactory(factory: AgentFactory): () => void', + jsDoc: '/**\n * Register the agent-creation factory (the loop calls this on construction,\n * effect-scoped). A traced Cordis service is canonicalized to its concrete\n * target; each create/resume call is then traced through that caller\'s\n * context so ownership follows the caller without stacking proxy layers.\n * Throws if a factory is already registered. Returns the disposer; on\n * dispose the factory slot is cleared.\n * @param factory - the loop-owned factory {@link create}/{@link resume} delegate to.\n * @returns the disposer that clears the factory slot. The exact\n * Cordis effect disposer (single-shot): composite (generator) effects may\n * yield it directly — exact identity nests the teardown in order.\n */', + }, + { + signature: 'async create(options: CreateAgentOptions): Promise', + jsDoc: '/**\n * Create and publish a new agent through the registered factory.\n * Distinct from {@link register} (which records an already-constructed\n * agent): this constructs the agent and its session. Rejects if no factory is\n * registered or creation/setup fails. The resolved {@link AgentHandle} lets\n * the owner tear down exactly this agent.\n * @param options - shared identity, session seed/metadata, and agent options.\n * @returns the handle after setup, rollback-covered publication, and loop start complete.\n */', + }, + { + signature: 'async resume(options: ResumeAgentOptions): Promise', + jsDoc: '/**\n * Load a persisted session and resume an agent on it through the registered\n * factory. Rejects if no factory is registered; the factory rejects if\n * session persistence is not configured or persistence/setup fails.\n * @param options - persisted identity, configuration, and optional setup.\n * @returns the handle after setup, rollback-covered publication, and loop start complete.\n */', + }, + { + signature: 'register(agent: Agent): () => void', + jsDoc: '/**\n * Register a live agent. Throws if an agent with the same id is already\n * registered. Emits `agent/created` on registration and `agent/disposed`\n * when the calling fiber is disposed — both with the agent\'s scope carrier\n * (`scopeTarget(agent, agent)`): the subject is the agent in hand, so the\n * emits are scope-filtered regardless of which context invoked `register`\n * (calling through `agent.ctx` scopes EFFECTS; dispatch scoping always\n * requires passing the carrier). Returns the disposer.\n * @param agent - the already-constructed agent to record in the store.\n * @returns the EXACT Cordis effect disposer (single-shot; a repeat call\n * returns undefined without awaiting an in-flight teardown). Exact\n * identity is load-bearing: a composite (generator) effect that owns a\n * teardown ORDER — the agent factory\'s lifecycle chain — must yield THIS\n * function so Cordis nests the unregistration at that yield position;\n * yielding a wrapper would leave it disposing as a concurrent sibling on\n * owner unload, unregistering the agent (and emitting `agent/disposed`)\n * while its final turn is still draining.\n */', + }, + { + signature: 'enter(agent: Agent, owner: Agent | undefined): () => void', + jsDoc: '/**\n * Insert an already-constructed agent without announcing it. This is the\n * advanced ordered-lifecycle primitive used by the async agent factory: it\n * first completes setup while the agent is unpublished, then assigns the\n * returned detach closure into its pre-installed composite teardown before\n * calling {@link announce}. Ordinary callers use {@link register}.\n * @param agent - the prepared, unpublished agent.\n * @param owner - live agent whose scoped context created this agent, or\n * undefined for a top-level runtime root. This is runtime ownership, not\n * the resumed session\'s durable parent lineage.\n * @returns an idempotent closure that removes this exact entry and emits\n * `agent/disposed` with listener failures contained. When called from a\n * synchronous `agent/created` listener, removal and disposal wait until\n * that creation dispatch unwinds.\n */', + }, + { + signature: 'announce(agent: Agent): void', + jsDoc: '/**\n * Announce an agent previously inserted with {@link enter}.\n * @param agent - the live inserted agent to announce.\n * @throws if `agent` is not the exact live registry entry for its id, or its\n * creation announcement already began (including a reentrant call from a\n * creation listener).\n */', + }, + { + signature: 'get(id: SessionId): Agent | undefined', + jsDoc: '/**\n * Look up a live agent.\n * @param id - the shared agent/session id to look up.\n * @returns the agent, or undefined when no live agent has that id.\n */', + }, + { + signature: 'isOwnedBy(id: SessionId, owner: Agent): boolean', + jsDoc: '/**\n * Test whether a live agent was created through one exact parent agent\'s\n * scoped context. Runtime ownership is independent of durable session\n * lineage and remains unambiguous when unrelated providers reuse an id.\n * @param id - the candidate child agent\'s shared agent/session id.\n * @param owner - the expected runtime creator agent.\n * @returns true only while the exact child entry is live under that owner.\n */', + }, + { + signature: 'list(): Agent[]', + jsDoc: '/**\n * All live agents, in registration order.\n * @returns a fresh array; mutating it does not affect the registry.\n */', + }, + { + signature: 'roots(): Agent[]', + jsDoc: '/**\n * All live top-level agents in registration order. A top-level agent was\n * created without an owning agent context; durable session lineage does not\n * affect this runtime relation, so a resumed fork may still be a root.\n * @returns a fresh array; mutating it does not affect the registry.\n */', + }, ], }, { key: 'approval', summary: 'Approval service that applies session policy before answerers and logs every ask/outcome pair to the requesting session.', methods: [ - 'async request(req: ApprovalRequest): Promise', + { + signature: 'async request(req: ApprovalRequest): Promise', + jsDoc: '/**\n * Ask the composed answerers to decide one readonly same-process request.\n * The service borrows the request, agent, session, and live signal directly.\n * The request requires an open turn because the audit pair must be enclosed\n * by the durable log\'s commit/replay boundary; an idle ask rejects before\n * appending anything. The answerer phase always produces an outcome: an\n * aborted signal yields `\'cancelled\'`, a missing or throwing answerer yields\n * `\'unavailable\'` (fail closed), and a rogue non-vocabulary return value is\n * normalized to `\'unavailable\'`. A failure that prevents either audit append\n * from committing still rejects because returning an unlogged decision would\n * violate the pair. Session contains post-commit observer failures, so an\n * authoritative append cannot reject the request or suppress its matching\n * audit event.\n * @param req - the pending decision (agent, tool identity, reason, signal).\n * @returns the closed outcome; `\'allowed-once\'` is the only grant.\n * @throws when no turn is open or either audit event fails before the session\n * append commit point.\n */', + }, ], }, { key: 'bash', summary: 'Abstract bash execution service.', methods: [ - 'abstract resolve(request: BashExecRequest): BashExecSpec', - 'abstract run(spec: BashExecSpec): Promise', - 'abstract start(spec: BashExecSpec): BashProcess', + { + signature: 'abstract resolve(request: BashExecRequest): BashExecSpec', + jsDoc: '/**\n * Apply implementation-owned defaults and caps to a request before execution.\n * @param request - the caller\'s request; omitted fields get this\n * implementation\'s defaults, capped fields are clamped.\n * @returns the fully-specified spec to hand to {@link run}/{@link start}.\n */', + }, + { + signature: 'abstract run(spec: BashExecSpec): Promise', + jsDoc: '/**\n * Run a command in the foreground; resolves when it finishes.\n * @param spec - a resolved spec from {@link resolve}, never a raw request.\n * @returns the outcome; nonzero exits, timeout kills, and abort kills\n * resolve with a descriptive result rather than reject.\n */', + }, + { + signature: 'abstract start(spec: BashExecSpec): BashProcess', + jsDoc: '/**\n * Start a background process and return its handle immediately.\n * @param spec - a resolved spec from {@link resolve}, never a raw request.\n * @returns the live process handle (reads, kill, quiescence promise).\n */', + }, ], }, { key: 'bashEnv', summary: 'Registry (`ctx.bashEnv`) for trusted, per-execution `DSH_*` variables.', methods: [ - 'register(contributor: BashEnvContributor): () => void', - 'collect(execution: ToolExecution): DshEnvironment', - 'list(): BashEnvVariableInfo[]', + { + signature: 'register(contributor: BashEnvContributor): () => void', + jsDoc: '/**\n * Register one environment contributor. Names and keys are unique; built-in\n * keys are reserved. Registration is disposed with the calling plugin fiber.\n * @param contributor - declared key ownership and per-execution resolver.\n * @returns the disposer that unregisters the contribution.\n */', + }, + { + signature: 'collect(execution: ToolExecution): DshEnvironment', + jsDoc: '/**\n * Build the trusted `DSH_*` snapshot for one bash tool execution.\n * @param execution - the current tool execution.\n * @returns an immutable environment overlay containing built-ins and current contributions.\n */', + }, + { + signature: 'list(): BashEnvVariableInfo[]', + jsDoc: '/**\n * Enumerate plugin-contributed variables without executing their resolvers.\n * @returns declarations sorted by environment variable name.\n */', + }, ], }, { key: 'codeRuntime', summary: 'Registers one `ctx.codeRuntime` implementation.', methods: [ - 'abstract run(request: CodeRunRequest): Promise', + { + signature: 'abstract run(request: CodeRunRequest): Promise', + jsDoc: '/**\n * Execute one program against the request\'s bindings and capture what it\n * emitted. See the class doc for the resolution contract (error is a result\n * field; rejection means seam misuse only).\n * @param request - the program, its bindings, and the abort signal; the\n * request carries everything the runtime acts on, with no hidden defaults.\n * @returns the run\'s outcome: completion value (when transferable), the\n * ordered log capture, and the failure (if any).\n */', + }, ], }, { key: 'compact', summary: 'Abstract compaction service.', methods: [ - 'abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise', - 'abstract compactRegion( start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise', + { + signature: 'abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise', + jsDoc: '/**\n * Check token pressure and compact if the conversation is too large.\n * Estimate the next request, including its session prefix, derived history,\n * and system prompt. Above threshold, compact a head-anchored range ending at\n * a balanced tool boundary and reconsolidate any prior automatic checkpoint.\n * Return `null` when no compaction is needed or an open tail leaves no safe\n * cutoff. A single oversized retained unit or prefix cannot be repaired here.\n *\n * @param agent - agent context owning the session surface and model options.\n * @param fullSystemPrompt - assembled system prompt, counted toward the estimate.\n * @param sessionPrefix - the instance\'s composed session prefix, counted toward the\n * estimate.\n * @param signal - cancellation signal; model-backed implementations must forward it.\n * @returns the compaction result, or `null` if no compaction was needed.\n */', + }, + { + signature: 'abstract compactRegion( start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise', + jsDoc: '/**\n * Forcibly compact a range of surface nodes into a single summary node.\n * `start` and `end` name an inclusive span by surface position, not numeric seq\n * order; replacements can make visible seqs non-monotonic. Both edges must be\n * balanced so assistant tool calls remain paired with their results. A model-\n * backed implementation forwards cancellation and rejects active, missing,\n * reversed, or unbalanced ranges. The target session is `agent.session`.\n * Use {@link toolPairingBalancedBefore} and {@link toolPairingBalancedAfter}\n * for the edge checks.\n *\n * @param start - first surface seq, inclusive.\n * @param end - last surface seq, inclusive.\n * @param agent - context whose session is mutated and whose routing options guide summarization.\n * @param signal - optional cancellation; model-backed implementations must forward it.\n * @throws when compaction is active or the range is missing, reversed, or unbalanced.\n * @returns the appended event seqs, summary, replaced range, and token accounting.\n */', + }, ], }, { key: 'fs', summary: 'Abstract filesystem provider.', methods: [ - 'abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise', - 'abstract stat(target: FsTarget, signal?: AbortSignal): Promise', - 'abstract lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise', - 'abstract readText(target: FsTarget, signal?: AbortSignal): Promise', - 'abstract streamText(target: FsTarget, signal?: AbortSignal): Promise>', - 'abstract listDir(target: FsTarget, signal?: AbortSignal): Promise', - 'abstract writeText(target: FsTarget, content: string, expected?: FsWriteIntent, signal?: AbortSignal): Promise', - 'abstract editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise', + { + signature: 'abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise', + jsDoc: '/**\n * Resolve a model/plugin-supplied path into a stable {@link FsTarget}. May perform I/O (a\n * remote/sandboxed backend may need a round-trip to map a path to a stable identity), hence\n * async even though the local backend only normalizes + realpaths.\n *\n * @param path - the path to resolve; relative paths resolve against `opts.cwd`.\n * @param opts - optional cwd override and cancellation signal.\n * @returns the stable target; the same file yields the same `targetKey`.\n */', + }, + { + signature: 'abstract stat(target: FsTarget, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Return target metadata, or `undefined` when the target does not exist.\n * @param target - the resolved target to stat.\n * @param signal - aborts the metadata round-trip.\n * @returns metadata only, never content; undefined for an absent target.\n */', + }, + { + signature: 'abstract lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Return path metadata without following the final path component when it is a\n * symbolic link. This is intentionally path-shaped, not target-shaped:\n * {@link resolve} follows symlinks to produce the stable identity used by\n * normal reads/writes, while `lstat` lets a consumer reject the path itself\n * before that follow happens.\n *\n * `opts.cwd` follows {@link resolve}\'s cwd rules. `undefined` means the path is\n * absent.\n * @param path - the path to inspect; relative paths resolve against `opts.cwd`.\n * @param opts - `cwd` overrides the backend\'s default base for relative paths.\n * @param signal - aborts the metadata round-trip.\n * @returns metadata only, never content; undefined for an absent path.\n */', + }, + { + signature: 'abstract readText(target: FsTarget, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Read the whole regular text file as a single decoded string.\n * @param target - the resolved target to read.\n * @param signal - aborts the read.\n * @returns the full decoded UTF-8 content.\n */', + }, + { + signature: 'abstract streamText(target: FsTarget, signal?: AbortSignal): Promise>', + jsDoc: '/**\n * Stream the whole regular text file as decoded text chunks (same text\n * semantics as {@link readText}, for large files). The backend owns\n * cross-chunk UTF-8 decoding and binary rejection so the policy layer never\n * touches raw bytes.\n * @param target - the resolved target to read.\n * @param signal - aborts the stream, including between chunks.\n * @returns the chunk iterable, decoded and validated like {@link readText}.\n */', + }, + { + signature: 'abstract listDir(target: FsTarget, signal?: AbortSignal): Promise', + jsDoc: '/**\n * List direct children of a directory in stable name order. Returns resolved\n * child targets plus cheap metadata only; never reads file contents.\n * @param target - the resolved directory target.\n * @param signal - aborts the listing.\n * @returns one entry per direct child, in stable name order.\n */', + }, + { + signature: 'abstract writeText(target: FsTarget, content: string, expected?: FsWriteIntent, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Atomically create or replace UTF-8 text. `expected` guards intent and\n * staleness; omission allows unconditional overwrite.\n * @param target - the resolved target to write.\n * @param content - the full new file content.\n * @param expected - the write intent guarding the write; omit for unconditional.\n * @param signal - aborts before the atomic rename takes effect.\n * @returns the outcome, including the version the write produced.\n */', + }, + { + signature: 'abstract editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Atomically edit literal text. When supplied, the version guard is checked\n * before matching so stale content reports `FS_STALE_VERSION`; omission edits\n * the current content without a freshness precondition.\n * @param target - the resolved target to edit.\n * @param edit - the literal search/replace request.\n * @param expected - the version guard; omit for an unconditional edit.\n * @param signal - aborts before the atomic rename takes effect.\n * @returns the outcome, including the version the edit produced.\n */', + }, ], }, { key: 'llm', summary: 'The abstract `llm` service: an adapter registry plus a streaming model-call surface, interceptable via the `llm/stream` waterfall.', methods: [ - 'registerAdapter(providers: string[], adapter: LlmAdapter): () => void', - 'listProviders(): LlmProviderInfo[]', - 'async listModels(provider: string): Promise', - 'stream(options: GenerateOptions): AsyncIterable', + { + signature: 'registerAdapter(providers: string[], adapter: LlmAdapter): () => void', + jsDoc: '/**\n * Register an adapter for the given provider routes. Throws `LlmError` with code\n * `DUPLICATE_ADAPTER` if any provider already has an adapter (all-or-nothing).\n * Disposed with the fiber.\n * @param providers - every provider route this adapter should serve.\n * @param adapter - the adapter that streams calls for those providers.\n * @returns the disposer that unregisters all of them.\n */', + }, + { + signature: 'listProviders(): LlmProviderInfo[]', + jsDoc: '/**\n * Describe provider routes with a registered adapter.\n * @returns detached provider metadata in registration order.\n */', + }, + { + signature: 'async listModels(provider: string): Promise', + jsDoc: '/**\n * Discover models advertised by one registered provider. Catalog membership\n * is advisory and never changes routing or request validation.\n * @param provider - registered provider route to inspect.\n * @returns detached model metadata in adapter-preferred order.\n */', + }, + { + signature: 'stream(options: GenerateOptions): AsyncIterable', + jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Dispatches\n * through the `llm/stream` waterfall.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */', + }, ], }, { key: 'permission', summary: 'Owns the deployment\'s permission presets and their write path.', methods: [ - 'current(events: readonly SessionEvent[]): string', - 'resolve(name: string): PresetSpec', - 'optionOf(name: string): PresetOption', - 'set(session: Session, name: string): void', + { + signature: 'current(events: readonly SessionEvent[]): string', + jsDoc: '/**\n * Resolve the preset matching the effective knob values. A still-matching\n * last selection wins shared-bundle ties; otherwise the first table match\n * wins, or {@link CUSTOM_PRESET} when no entry matches.\n * @param events - the session\'s events in log order.\n * @returns the effective preset name, or `custom` when nothing matches.\n */', + }, + { + signature: 'resolve(name: string): PresetSpec', + jsDoc: '/**\n * Resolve a preset\'s knob bundle.\n * @param name - the preset name to resolve.\n * @returns the configured bundle.\n * @throws when `name` is not in the table.\n */', + }, + { + signature: 'optionOf(name: string): PresetOption', + jsDoc: '/**\n * Build the client option for a table entry or {@link CUSTOM_PRESET}. A\n * missing label falls back to the table key.\n * @param name - a table key, or `custom`.\n * @returns the option a client renders.\n * @throws when `name` is neither a table key nor `custom`.\n */', + }, + { + signature: 'set(session: Session, name: string): void', + jsDoc: '/**\n * Record a changed preset, then update each changed knob through its own\n * setter. Selecting the effective preset again appends nothing.\n * @param session - the session the switch belongs to.\n * @param name - the preset to switch to; unknown names throw.\n */', + }, ], }, { key: 'sandbox', summary: 'Abstract process-sandbox service.', methods: [ - 'abstract confine(argv: readonly string[], policy: SandboxPolicy): ConfinedArgv', + { + signature: 'abstract confine(argv: readonly string[], policy: SandboxPolicy): ConfinedArgv', + jsDoc: '/**\n * Wrap `argv` so it executes confined under `policy` on this host; the\n * caller spawns the returned argv in place of its own.\n * @param argv - the exact argv the caller is about to spawn (program plus\n * arguments), NOT a shell string — a shell-shaped consumer passes\n * `[\'bash\', \'-c\', command]`.\n * @param policy - the file-effect policy this execution runs under,\n * carried per call (see {@link SandboxPolicy}).\n * @returns the argv to spawn instead, plus the enforcement completeness\n * the selected backend achieves for it.\n */', + }, ], }, { key: 'sessionPersistence', summary: 'Durable append-only session storage.', methods: [ - 'abstract locate(meta: SessionHeader): SessionLocation | undefined', - 'abstract create(meta: SessionHeader): Promise', - 'abstract append(id: SessionId, events: readonly SessionEvent[]): Promise', - 'abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>', - 'abstract list(): Promise', + { + signature: 'abstract locate(meta: SessionHeader): SessionLocation | undefined', + jsDoc: '/**\n * Resolve this backend\'s independent local artifact for a session without\n * reading, creating, flushing, or otherwise materializing it. Backends such\n * as SQLite that do not own one artifact per session return `undefined`.\n * @param meta - the immutable session header whose artifact is requested.\n * @returns the backend-specific absolute location, when one exists.\n */', + }, + { + signature: 'abstract create(meta: SessionHeader): Promise', + jsDoc: '/**\n * Register a new session\'s metadata. A backend MAY defer the physical write\n * until the first {@link append} (lazy materialization), in which case a\n * created-but-never-appended session is absent from {@link list}\n * — abandoned sessions leave nothing behind.\n * @param meta - the immutable header (id, version, cwd, lineage) to record.\n */', + }, + { + signature: 'abstract append(id: SessionId, events: readonly SessionEvent[]): Promise', + jsDoc: '/**\n * Durably persist a batch of events (called from the write-behind drain at\n * the `session/flush` checkpoint). Honors the append-only and contiguous-seq\n * contracts: the first event\'s `seq` MUST equal the stored next-seq (after\n * `load` has durably closed any interrupted turn). Rejects non-JSON-\n * serializable `event.data` with an error naming the offending event type.\n * @param id - the session the batch belongs to.\n * @param events - the contiguous batch to persist, in seq order.\n */', + }, + { + signature: 'abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>', + jsDoc: '/**\n * Load a header and balanced contiguous log. A complete interrupted final\n * turn is preserved and durably closed with missing tool errors plus any open\n * step and turn boundaries; only a torn final record is discarded. Unknown\n * versions and corruption in the committed prefix reject.\n * @param id - the persisted session to reload.\n * @returns the header and a log ending on a balanced `turn/end`.\n */', + }, + { + signature: 'abstract list(): Promise', + jsDoc: '/**\n * Lightweight listing from metadata, without a full-log parse.\n * @returns one header per materialized session.\n */', + }, ], }, { key: 'sessionQuery', summary: 'Live-preferred logical-corpus exact-read and relationship-tracing service.', methods: [ - 'listSessions(): Promise', - 'async listEvents(sessionId: SessionId): Promise', - 'async traceSession(sessionId: SessionId): Promise', - 'async traceEvent(request: SessionEventTraceRequest): Promise', - 'async readEvent(request: SessionEventReadRequest): Promise', + { + signature: 'listSessions(): Promise', + jsDoc: '/**\n * List the complete logical corpus using live-preferred records.\n * @returns deterministic newest-first cloned session records.\n */', + }, + { + signature: 'async listEvents(sessionId: SessionId): Promise', + jsDoc: '/**\n * List lightweight raw-log event records for one logical session.\n * @param sessionId - live-preferred session id to read.\n * @returns event records in ascending seq order.\n */', + }, + { + signature: 'async traceSession(sessionId: SessionId): Promise', + jsDoc: '/**\n * Trace known ancestry and descendants from one corpus observation.\n * @param sessionId - logical session id to trace.\n * @returns a complete lineage or an explicit unresolved parent boundary.\n * @throws when corpus resolution fails, the target is absent, or its known ancestry cycles.\n */', + }, + { + signature: 'async traceEvent(request: SessionEventTraceRequest): Promise', + jsDoc: '/**\n * Trace one event\'s direct positional and provenance relationships.\n * @param request - target session id and event seq.\n * @returns direct links plus the target\'s positional replacement chain.\n * @throws when source resolution fails, the target is absent, or surface/provenance validation fails.\n */', + }, + { + signature: 'async readEvent(request: SessionEventReadRequest): Promise', + jsDoc: '/**\n * Read one full event plus a bounded raw-log context window.\n * @param request - target session/seq and context sizes.\n * @returns cloned target and neighboring events.\n */', + }, ], }, { key: 'sessions', summary: 'In-memory session store (`ctx.sessions`).', methods: [ - 'create(id?: SessionId, options?: CreateSessionOptions): Session', - 'prepare(id?: SessionId, options?: CreateSessionOptions): Session', - 'enter(session: Session): () => void', - 'announce(session: Session): void', - 'async flush(session: Session): Promise', - 'get(id: SessionId): Session | undefined', - 'list(): Session[]', - 'fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session', + { + signature: 'create(id?: SessionId, options?: CreateSessionOptions): Session', + jsDoc: '/**\n * Create a session owned by the calling fiber: disposing that fiber stops\n * event notification and removes the session from the store. `options.seed`\n * populates the session with a copy of those events (replay/fork);\n * `options.meta` attaches creation metadata (validated absolute `cwd`,\n * `parentSession` lineage) as the immutable {@link SessionHeader} (the store\n * fills `version`/`id`/`createdAt`).\n *\n * For an agent whose session must be torn down IN ORDER with its loop (so the\n * loop\'s final flush is captured before the store attachment ends), do NOT use this\n * — fold the session lifecycle into the agent\'s own effect via\n * {@link prepare} + {@link enter} + {@link announce} (see\n * `dsh-agent-loop`\'s creation transaction).\n *\n * @param id - the session id; omitted, the store mints `session-`.\n * @param options - seed events and/or creation metadata for the header.\n * @returns the live session, already entered and announced.\n * @throws if a session with `id` already exists, metadata is not a plain\n * lossless-JSON record with valid scalar fields, or `meta.cwd` is a\n * non-absolute path (storage backends key directories off it).\n */', + }, + { + signature: 'prepare(id?: SessionId, options?: CreateSessionOptions): Session', + jsDoc: '/**\n * Build a session WITHOUT entering it into the store — validate the id/cwd and\n * construct the {@link Session} (with its immutable {@link SessionHeader}).\n * Pairs with {@link enter} + {@link announce}: a caller that owns a composite\n * `ctx.effect` (the agent factory) folds the session lifecycle into that ONE\n * effect so a fiber unload tears the session + agent down as a single ORDERED\n * chain rather than as racing sibling effects — which would remove the publication hooks\n * before the loop\'s closing `session/flush`, dropping the closing events.\n *\n * @param id - the session id; omitted, the store mints `session-`.\n * @param options - seed events and/or creation metadata for the header.\n * @returns the constructed session, NOT yet in the store.\n * @throws if a session with `id` already exists, metadata is not a plain\n * lossless-JSON record with valid scalar fields, or `meta.cwd` is a\n * non-absolute path.\n */', + }, + { + signature: 'enter(session: Session): () => void', + jsDoc: '/**\n * Enter a {@link prepare}d session into the store: install the module-private\n * append publication hooks and add it to the store. Returns the DETACH\n * disposer (hooks + store removal). Does NOT emit `session/created` —\n * the caller yields this disposer inside its effect and THEN calls\n * {@link announce}, so a throwing `session/created` listener rolls the attach\n * back instead of leaking it.\n *\n * Re-checks the id for a duplicate: `prepare` and `enter` are public\n * cross-package primitives and a caller may interleave arbitrary work (or\n * another create) between them, so a stale prepared session must NOT overwrite\n * a live store entry of the same id — its detach disposer would later delete\n * the REAL session. The {@link create} convenience and the agent factory call\n * the two back-to-back so they never trip this, but the public seam cannot\n * assume that.\n *\n * @param session - a {@link prepare}d session not yet in the store.\n * @returns the detach disposer (publication hooks + store removal). When called from\n * a synchronous `session/created` listener, removal and disposal wait until\n * that creation dispatch unwinds.\n * @throws if a session with this id is already in the store.\n */', + }, + { + signature: 'announce(session: Session): void', + jsDoc: '/** Emit `session/created` exactly once for an {@link enter}ed session (with\n * the carrier {@link enter} captured). Separate from {@link enter} so the\n * caller can yield the detach disposer first (rollback safety — see\n * {@link enter}).\n * @param session - the entered session to announce to listeners.\n * @throws if the session is not live or its announcement already began,\n * including a reentrant call from a creation listener. */', + }, + { + signature: 'async flush(session: Session): Promise', + jsDoc: '/**\n * Dispatch the awaited `session/flush` durability checkpoint for `session`,\n * with the carrier captured at {@link enter}. THE flush entry point: the\n * store owns the carrier, so callers (the loop\'s turn-end checkpoint, idle\n * injection, teardown drains) must come through here rather than dispatch a\n * raw `ctx.parallel(\'session/flush\', …)` — one owner, one spelling, and the\n * scoped-dispatch invariant can pin it.\n * @param session - the session whose buffered events must reach durable storage.\n * @returns resolves when every flush listener has settled; after all settle,\n * rejects with the first registered listener failure if any listener failed.\n */', + }, + { + signature: 'get(id: SessionId): Session | undefined', + jsDoc: '/**\n * Look up a live session.\n * @param id - the session id to look up.\n * @returns the session, or undefined when no live session has that id.\n */', + }, + { + signature: 'list(): Session[]', + jsDoc: '/**\n * All live sessions, in creation order.\n * @returns a fresh array; mutating it does not affect the store.\n */', + }, + { + signature: 'fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session', + jsDoc: '/**\n * Create a live child session from a turn-enclosed prefix of a live source.\n * `boundary` is an inclusive source event seq; omitted means the source\'s\n * current last event. A non-empty selected slice must end at `turn/end`.\n *\n * @param source - Live source session object or id.\n * @param boundary - Inclusive source event seq to fork through; omitted means\n * the source\'s current last event, and omitted on an empty source forks an\n * empty child.\n * @param childSessionId - Optional child session id; omitted delegates to\n * `SessionStore`\'s id policy.\n * @returns The created live child session.\n */', + }, ], }, { key: 'skills', summary: 'Registry of skill providers.', methods: [ - 'registerProvider(provider: SkillProvider): () => void', - 'register(skill: SkillRegistration): () => void', - 'async list(options: SkillLookupOptions = {}): Promise', - 'async get(name: string, options: SkillLookupOptions = {}): Promise', + { + signature: 'registerProvider(provider: SkillProvider): () => void', + jsDoc: '/**\n * Register a borrowed same-process provider synchronously during plugin apply. Duplicate and\n * reserved names throw; remote initialization belongs in `list()`. Fiber disposal unregisters\n * the provider and invalidates catalog caches.\n * @param provider - the provider to register by `provider.name`.\n * @returns the exact Cordis effect disposer that unregisters this provider;\n * composite effects may yield it directly to preserve teardown ordering.\n */', + }, + { + signature: 'register(skill: SkillRegistration): () => void', + jsDoc: '/**\n * Register a borrowed readonly runtime skill. Project entries outrank runtime entries, which\n * outrank user entries. Same-name runtime entries are first-wins; a duplicate logs a warning and\n * receives a no-op disposer so it cannot remove the winner.\n * @param skill - the complete skill definition to expose for discovery.\n * @returns the exact Cordis effect disposer, preserving composite teardown order and invalidating caches.\n */', + }, + { + signature: 'async list(options: SkillLookupOptions = {}): Promise', + jsDoc: '/**\n * List model-invocable skill summaries for a workspace. Lookup options and\n * provider candidates are readonly same-process values borrowed throughout\n * discovery.\n * @param options - lookup options; `cwd` selects project roots and `signal` cancels discovery.\n * @returns sorted summaries, excluding skills disabled for model invocation.\n */', + }, + { + signature: 'async get(name: string, options: SkillLookupOptions = {}): Promise', + jsDoc: '/**\n * Load and validate the winning candidate, passing its opaque discovery locator back to the\n * provider. Cancellation is rechecked after selection, including cache hits, and raced against\n * loading so an uncooperative provider cannot hang the caller.\n * @param name - kebab-case skill name.\n * @param options - lookup options; `cwd` selects workspace-sensitive skills and `signal` cancels work.\n * @returns the full skill, including body content, or `undefined`.\n */', + }, ], }, { key: 'spillStore', summary: 'Abstract spill storage service.', methods: [ - 'abstract saveText(input: SaveTextSpill): Promise', + { + signature: 'abstract saveText(input: SaveTextSpill): Promise', + jsDoc: '/**\n * Persist `input.content` to a session-scoped spill artifact.\n * @param input - the owner, provenance, suggested name, and full text to save.\n * @returns the saved artifact\'s {@link SpillRef}; rejects on a storage failure.\n */', + }, ], }, { key: 'subagents', summary: 'Named provider registry and capability-checked start surface.', methods: [ - 'registerProvider(provider: SubagentProvider): () => void', - 'getProvider(name: string): SubagentProvider | undefined', - 'list(): string[]', - 'async start(name: string, request: SubagentStartRequest): Promise', + { + signature: 'registerProvider(provider: SubagentProvider): () => void', + jsDoc: '/**\n * Register a provider under its name. Registration is effect-scoped and HMR\n * safe; removing a provider blocks new starts but does not revoke runs that\n * were already returned to their holders.\n * @param provider - the trusted provider implementation.\n * @returns the exact Cordis effect disposer.\n */', + }, + { + signature: 'getProvider(name: string): SubagentProvider | undefined', + jsDoc: '/**\n * Look up a provider by name.\n * @param name - the provider name.\n * @returns the provider, or undefined when absent.\n */', + }, + { + signature: 'list(): string[]', + jsDoc: '/**\n * List registered provider names in insertion order.\n * @returns the registered names.\n */', + }, + { + signature: 'async start(name: string, request: SubagentStartRequest): Promise', + jsDoc: '/**\n * Establish a ready child on the named provider. Capability and semantic\n * checks run before delegation. Provider ownership lasts until its promise\n * fulfills; a rejection therefore has no run for the caller to dispose and\n * emits no run lifecycle events.\n * @param name - the provider to use.\n * @param request - child prompt, parent, signal, and optional capabilities.\n * @returns the ready holder-owned run.\n */', + }, ], }, { key: 'systemPrompt', summary: 'Registry service for the prompt inputs assembled before each model step.', methods: [ - 'section(section: PromptSection): () => void', - 'tools(provider: (context: AssembleContext) => ToolProviderResult): () => void', - 'variable(name: string, provider: (context: AssembleContext) => string | undefined): () => void', - 'async assemble(context: AssembleContext = {}): Promise', + { + signature: 'section(section: PromptSection): () => void', + jsDoc: '/**\n * Register an ordered prompt section in the calling context\'s scope. A scoped\n * section shadows a global section with the same name; duplicates within one\n * layer and non-finite orders throw. Registration and disposal emit\n * `system-prompt/change`.\n * @param section - the section to register.\n * @returns the exact Cordis effect disposer.\n */', + }, + { + signature: 'tools(provider: (context: AssembleContext) => ToolProviderResult): () => void', + jsDoc: '/**\n * Register a tool-schema provider in the calling context\'s scope. Global and\n * matching scoped providers both contribute; returning the reserved\n * {@link TOOL_ORDER_REST} name makes assembly fail.\n * @param provider - evaluated for each assembly with its context.\n * @returns the exact Cordis effect disposer.\n */', + }, + { + signature: 'variable(name: string, provider: (context: AssembleContext) => string | undefined): () => void', + jsDoc: '/**\n * Register a prompt variable in the calling context\'s scope. Scoped values\n * shadow globals; invalid or duplicate names throw. A provider may return\n * `undefined`, but rendering a section that references that value then fails.\n * @param name - the `[a-z][a-z0-9_]*` reference name.\n * @param provider - evaluated for each assembly.\n * @returns the exact Cordis effect disposer.\n */', + }, + { + signature: 'async assemble(context: AssembleContext = {}): Promise', + jsDoc: '/**\n * Assemble global and scoped providers, detach tool parameters, apply\n * canonical ordering, then run the assembly waterfall. Scoped sections and\n * variables shadow globals; the returned waterfall value is authoritative.\n * @param context - the optional scope and plugin-defined assembly fields.\n * @returns the authoritative post-waterfall assembly.\n */', + }, ], }, { key: 'tasks', summary: 'The `tasks` service: the runtime-global background task registry.', methods: [ - 'start(spec: TaskStart): TaskId', - 'list(caller?: Agent): TaskSnapshot[]', - 'get(id: TaskId, caller?: Agent): TaskSnapshot', - 'read(id: TaskId, caller?: Agent): TaskRead', - 'kill(id: TaskId, caller?: Agent, reason?: string): \'requested\' | \'already-finished\'', - 'async wait(id: TaskId, timeoutMs: number, caller?: Agent, signal?: AbortSignal): Promise', - 'onTaskDone(listener: TaskDoneListener): () => void', - 'attachSurface(name: string): () => void', + { + signature: 'start(spec: TaskStart): TaskId', + jsDoc: '/**\n * Preflight access, validation, and owner cleanup before starting and\n * atomically registering work. A throwing starter leaves nothing registered;\n * after it returns, registration cannot fail. Settlement records the outcome,\n * notifies listeners, and releases waiters.\n * @param spec - task identity, owner, and synchronous starter.\n * @returns the registry-issued `-N` id.\n */', + }, + { + signature: 'list(caller?: Agent): TaskSnapshot[]', + jsDoc: '/**\n * List caller-owned and unowned tasks in registration order without exposing\n * another session\'s labels.\n * @param caller - reading agent; a non-agent caller sees only unowned tasks.\n * @returns fresh snapshots.\n */', + }, + { + signature: 'get(id: TaskId, caller?: Agent): TaskSnapshot', + jsDoc: '/**\n * Return a non-consuming snapshot without changing its read cursor or notice\n * state. Throws for an unknown or foreign task.\n * @param id - task to look up.\n * @param caller - reading agent checked against the owner.\n * @returns a fresh snapshot.\n */', + }, + { + signature: 'read(id: TaskId, caller?: Agent): TaskRead', + jsDoc: '/**\n * Read the next stream delta, or the idempotent final output after settlement.\n * A terminal read marks the task reported. Throws for an unknown or foreign\n * task.\n * @param id - task to read.\n * @param caller - reading agent checked against the owner.\n * @returns output text and the post-read snapshot.\n */', + }, + { + signature: 'kill(id: TaskId, caller?: Agent, reason?: string): \'requested\' | \'already-finished\'', + jsDoc: '/**\n * Request cancellation, then mark the task stopping and reported. A producer\n * throw propagates without changing task state. Throws for an unknown or\n * foreign task.\n * @param id - task to cancel.\n * @param caller - killing agent checked against the owner.\n * @param reason - logged reason forwarded to the producer.\n * @returns `requested` for live work, otherwise `already-finished`.\n */', + }, + { + signature: 'async wait(id: TaskId, timeoutMs: number, caller?: Agent, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Wait for settlement or timeout without cancelling the task. Caller abort\n * rejects only while the task is live; after settlement it returns the\n * terminal snapshot so a notice suppressed for this waiter is still delivered.\n * Timed-out and aborted waits detach their resolvers. Throws for invalid,\n * unknown, or foreign input.\n * @param id - task to wait for.\n * @param timeoutMs - positive finite wait bound in milliseconds.\n * @param caller - waiting agent checked against the owner.\n * @param signal - optional cancellation of the wait itself.\n * @returns snapshot at settlement or timeout.\n */', + }, + { + signature: 'onTaskDone(listener: TaskDoneListener): () => void', + jsDoc: '/**\n * Register an effect-scoped completion listener. Each listener is contained;\n * returned promises are observed but not awaited. No listener runs after\n * service disposal.\n * @param listener - receives each terminal snapshot and its exact owner.\n * @returns disposer that unregisters the listener.\n */', + }, + { + signature: 'attachSurface(name: string): () => void', + jsDoc: '/**\n * Attach an effect-scoped surface that can read and stop tasks. {@link start}\n * refuses work while none is attached.\n * @param name - diagnostic label; duplicate names remain independent.\n * @returns disposer that detaches this surface.\n */', + }, ], }, { key: 'tokenMeter', summary: 'Replay owner for one service-wide estimator and isolated per-session folds.', methods: [ - 'measure(session: Session, requestHeader?: EpochHeader): TokenMeasurement', - 'estimateMessage(message: Message): number', + { + signature: 'measure(session: Session, requestHeader?: EpochHeader): TokenMeasurement', + jsDoc: '/**\n * Measure current request pressure and surface through the durable tail.\n *\n * Provider usage is reused only when the latest successful call\'s canonical\n * request envelope matches `requestHeader` and its total is no lower than\n * that call\'s full heuristic anchor; otherwise the complete envelope and\n * surface are heuristically repriced.\n *\n * `requestHeader` affects request pressure only; surface fields always\n * describe the current session surface. Every call clones those positional\n * nodes, so measurement is O(surface).\n *\n * @param session - session to replay through its current durable tail.\n * @param requestHeader - optional effective request envelope replacing the latest logged header.\n * @returns a detached deeply immutable pressure and surface measurement.\n */', + }, + { + signature: 'estimateMessage(message: Message): number', + jsDoc: '/**\n * Heuristically price one model-visible message.\n * @param message - message to price without mutation.\n * @returns content and role-framing tokens under the fixed service heuristic.\n */', + }, ], }, { key: 'tools', summary: 'Tool registry and execution pipeline.', methods: [ - 'register(definition: ToolDefinition): () => void', - 'restrict(filter: ToolRestriction): () => void', - 'guard(guard: ToolGuard): () => void', - 'get(name: string, scope?: ScopeKey): ToolDefinition | undefined', - 'schemas(scope?: ScopeKey): ToolSchema[]', - 'executionMode(exec: ToolExecutionInput): ToolExecutionMode', - 'async execute(exec: ToolExecutionInput): Promise', + { + signature: 'register(definition: ToolDefinition): () => void', + jsDoc: '/**\n * Register globally or in the calling agent scope. Scoped tools shadow\n * globals; duplicates within one layer and the reserved `run_code` name fail.\n * @param definition - the tool schema, execution, and optional presentation functions.\n * @returns the exact disposer that unregisters the tool.\n */', + }, + { + signature: 'restrict(filter: ToolRestriction): () => void', + jsDoc: '/**\n * Restrict global tools for the calling agent scope. Empty filters, unknown\n * names, scope-local names, and reserved transport names fail. Restrictions\n * intersect; scoped registrations remain visible.\n * @param filter - global-surface mask: `allow` (keep only) and/or `deny` (remove).\n * @returns the exact disposer that lifts this restriction.\n */', + }, + { + signature: 'guard(guard: ToolGuard): () => void', + jsDoc: '/**\n * Register a monotonic guard after the extensible `tools/pre-execute`\n * waterfall. A plain-context guard applies globally; one registered through\n * `agent.ctx` applies only to that agent. Any matching guard may deny by\n * returning a reason, while no guard can force-allow a call another guard\n * denied. The exact effect disposer is returned for ordered ownership and\n * HMR cleanup.\n * @param guard - synchronous check; a returned string denies the execution.\n * @returns the exact disposer that unregisters the guard.\n */', + }, + { + signature: 'get(name: string, scope?: ScopeKey): ToolDefinition | undefined', + jsDoc: '/**\n * Look up a tool as one scope sees it (scoped\n * shadows global; a restricted-away global reads as absent). Presenters pass\n * the calling agent so the rendered card matches the definition that\n * actually executed.\n * @param name - the tool name as registered.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns the definition the scope resolves, or undefined when none is visible.\n */', + }, + { + signature: 'schemas(scope?: ScopeKey): ToolSchema[]', + jsDoc: '/**\n * Project visible definitions onto the allowlisted model-facing schema fields,\n * excluding execution and presentation callbacks.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns one deep-cloned schema per visible tool.\n */', + }, + { + signature: 'executionMode(exec: ToolExecutionInput): ToolExecutionMode', + jsDoc: '/**\n * Classify a pending call through the caller\'s visible tool definition. Only\n * an exact `true` is parallel; unknown, hidden, undeclared, invalid, or\n * throwing classifiers are exclusive.\n * @param exec - call name, parsed arguments, and optional agent scope.\n * @returns the fail-closed scheduling mode.\n */', + }, + { + signature: 'async execute(exec: ToolExecutionInput): Promise', + jsDoc: '/**\n * Execute through pre-policy, guards, around-dispatch, post-policy, and final\n * notification. Tool and listener failures resolve as materialized error\n * results; an invisible tool reports `UNKNOWN_TOOL`. The returned outcome is\n * the same lossless, frozen snapshot final observers receive.\n * @param exec - the typed same-process call input. The registry assigns its\n * correlation token before policy begins.\n * @returns the materialized final result.\n */', + }, ], }, { key: 'userInteraction', summary: '`ctx.userInteraction`: one active UI provider plus an `ask()` surface.', methods: [ - 'registerProvider(provider: UserInteractionProvider): () => void', - 'async ask(request: AskUserQuestionRequest): Promise', + { + signature: 'registerProvider(provider: UserInteractionProvider): () => void', + jsDoc: '/**\n * Register the UI provider. Only one provider may be active in a context.\n *\n * @param provider UI-side implementation that collects answers.\n * @returns Disposer that unregisters this provider.\n */', + }, + { + signature: 'async ask(request: AskUserQuestionRequest): Promise', + jsDoc: '/**\n * Ask the active UI provider and wait for the user\'s answer.\n *\n * @param request Questions, owner agent, and abort signal.\n * @returns The answer chosen or typed by the human.\n */', + }, ], }, { key: 'web', summary: 'The web access service.', methods: [ - 'registerSearchProvider(provider: WebSearchProvider): () => void', - 'registerFetchProvider(provider: WebFetchProvider): () => void', - 'async search(request: WebSearchRequest, signal?: AbortSignal): Promise', - 'async fetch(request: WebFetchRequest, signal?: AbortSignal): Promise', + { + signature: 'registerSearchProvider(provider: WebSearchProvider): () => void', + jsDoc: '/**\n * Register a search provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER`\n * if its id is already registered for search. Returns a disposer; disposed\n * with the calling fiber.\n * @param provider - the provider; its `id` is the registry key.\n * @returns the disposer that unregisters the provider.\n */', + }, + { + signature: 'registerFetchProvider(provider: WebFetchProvider): () => void', + jsDoc: '/**\n * Register a fetch provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER`\n * if its id is already registered for fetch. Returns a disposer; disposed\n * with the calling fiber.\n * @param provider - the provider; its `id` is the registry key.\n * @returns the disposer that unregisters the provider.\n */', + }, + { + signature: 'async search(request: WebSearchRequest, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Run one search through the selected provider. Resolves the provider at call\n * time with the selection rules above; throws {@link WebError} when the\n * capability cannot run. The seam enforces `request.maxResults` on the result:\n * if the provider over-returns, `sources[]` is truncated and `truncated` set.\n * @param request - the query plus result-shaping options.\n * @param signal - optional cancellation signal forwarded to the provider.\n * @returns the provider\'s results, capped to `request.maxResults`.\n */', + }, + { + signature: 'async fetch(request: WebFetchRequest, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Retrieve one URL through the selected provider. Resolves the provider at\n * call time with the selection rules above; throws {@link WebError} when the\n * capability cannot run. A non-2xx response is a result, not a throw.\n * @param request - the URL plus retrieval options.\n * @param signal - optional cancellation signal forwarded to the provider.\n * @returns the retrieval outcome; non-2xx responses resolve descriptively.\n */', + }, ], }, { key: 'workflows', summary: 'Workflow execution seam.', methods: [ - 'abstract start(request: WorkflowStartRequest): WorkflowRun', + { + signature: 'abstract start(request: WorkflowStartRequest): WorkflowRun', + jsDoc: '/**\n * Parse and execute a workflow script.\n * @param request - the script, its `args`, the parent agent, and an\n * optional cancel signal.\n * @returns the live run; its `result` resolves when the script settles.\n */', + }, ], }, ] @@ -299,240 +594,280 @@ export const EVENT_API: readonly EventApiEntry[] = [ name: 'agent-loop/config-start-failed', mode: 'emit', signature: '\'agent-loop/config-start-failed\'(sessionId: SessionId, error: unknown): void', + jsDoc: '/**\n * A declarative agent entry failed before it could publish a live agent.\n * Consumers that buffer work for the configured identity use this\n * transient signal to reject that work instead of waiting forever. Normal\n * factory teardown suppresses failures from the cancelled startup attempt.\n * @param sessionId - exact shared agent/session identity that failed startup.\n * @param error - persistence, setup, or publication failure.\n * @mode emit\n */', summary: 'A declarative agent entry failed before it could publish a live agent.', }, { name: 'agent/created', mode: 'emit', signature: '\'agent/created\'(this: Scoped, agent: Agent): void', + jsDoc: '/**\n * A fully configured agent and live session were published. Setup is\n * composition-only; `agent/session-start` is the first startup-driving seam.\n * Synchronous listener failure vetoes publication, while returned-promise\n * rejection is reported. Detach requested during dispatch waits until every\n * creation listener has observed the stable entry.\n * @param agent - the newly registered agent with its live session and completed setup.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'A fully configured agent and live session were published.', }, { name: 'agent/disposed', mode: 'emit', signature: '\'agent/disposed\'(this: Scoped, agent: Agent): void', + jsDoc: '/**\n * An agent left the registry; AgentLoop emits this after driver quiescence\n * but before session detachment and scoped-registration unwind. Custom\n * registry users own their driver-ordering contract.\n * @param agent - the exact agent removed from the registry.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'An agent left the registry; AgentLoop emits this after driver quiescence but before session detachment and scoped-registration unwind.', }, { name: 'agent/error', mode: 'emit', signature: '\'agent/error\'(this: Scoped, agent: Agent, turn: number, step: number, error: Error): void', + jsDoc: '/**\n * A step or turn errored. The loop reports a failure here (plus the logger)\n * even when the error has no in-turn position for a session `error` event.\n * @param agent - the agent whose turn errored.\n * @param turn - the turn in which the failure surfaced.\n * @param step - the step at which the failure surfaced.\n * @param error - the failure, verbatim.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'A step or turn errored.', }, { name: 'agent/pre-step', mode: 'serial', signature: '\'agent/pre-step\'(this: Scoped, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise | void', + jsDoc: '/**\n * Awaited serial checkpoint for session-surface mutation after prompt\n * assembly and before `step/start`; appends land outside the pending step.\n * The loop derives history once afterward, so compaction records and\n * replacements are included without rewriting an assembled request. The\n * prompt and prefix are the exact pressure inputs for that request, and\n * `signal` cancels listener work.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent opening the step.\n * @param turn - the open turn number.\n * @param step - the pending step number.\n * @param fullSystemPrompt - the assembled prompt.\n * @param sessionPrefix - the frozen request prefix.\n * @param signal - the turn abort signal.\n * @mode serial\n */', summary: 'Awaited serial checkpoint for session-surface mutation after prompt assembly and before `step/start`; appends land outside the pending step.', }, { name: 'agent/prompt-submit', mode: 'waterfall', signature: '\'agent/prompt-submit\'(this: Scoped, agent: Agent, content: ContentBlock[], source: MessageSource, next: () => Promise): Promise', + jsDoc: '/**\n * Allow, rewrite, or block one drained prompt before it becomes a user\n * message. Call `next()` for the unchanged default.\n * @param agent - the agent draining its inbox.\n * @param content - the drained message\'s blocks, as queued.\n * @param source - the message\'s resolved source.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', summary: 'Allow, rewrite, or block one drained prompt before it becomes a user message.', }, { name: 'agent/queued', mode: 'emit', signature: '\'agent/queued\'(this: Scoped, agent: Agent, content: ContentBlock[], info: { source: MessageSource; steering: boolean }): void', + jsDoc: '/**\n * Detached, frozen content entered the agent\'s inbox. Source defaults have\n * already been applied, so these are the exact values retained for the log.\n * @param agent - the agent whose inbox received the message.\n * @param content - the accepted content blocks retained by the inbox.\n * @param info - the accepted source plus whether it entered as steering.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'Detached, frozen content entered the agent\'s inbox.', }, { name: 'agent/request', mode: 'waterfall', signature: '\'agent/request\'(this: Scoped, agent: Agent, turn: number, step: number, config: LlmCallConfig, next: () => Promise): Promise', + jsDoc: '/**\n * Replace the frozen call configuration. Model-visible content must use\n * logged channels; this seam cannot mutate messages. Injection here joins\n * the next request because the current step boundary is already fixed.\n * @param agent - the agent making the model call.\n * @param turn - the open turn number.\n * @param step - the step whose request this is.\n * @param config - the config the loop would use (frozen); return a replacement to switch.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', summary: 'Replace the frozen call configuration.', }, { name: 'agent/session-prefix', mode: 'waterfall', signature: '\'agent/session-prefix\'(this: Scoped, agent: Agent, prefix: Message[], signal: AbortSignal, next: () => Promise): Promise', + jsDoc: '/**\n * Compose request-only messages placed before derived history. The frozen\n * result is computed once per loop instance, logged on its anchoring request\n * header, and reused so the provider prefix remains stable. Interrupted\n * composition is discarded. Composition precedes the first `agent/pre-step`\n * and request boundary, so listener appends join the current request and\n * pressure accounting sees the composed prefix. Changing context belongs in\n * history; contributors should prepend to `await next()` to preserve registration order.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent whose session prefix is being composed.\n * @param prefix - the frozen seed; return an extended replacement.\n * @param signal - aborts composition when the step is torn down.\n * @mode waterfall\n */', summary: 'Compose request-only messages placed before derived history.', }, { name: 'agent/session-start', mode: 'emit', signature: '\'agent/session-start\'(this: Scoped, agent: Agent, source: SessionStartSource): void', + jsDoc: '/**\n * The session lifecycle began, once before the first turn. Use\n * `agent.inject()` to seed model-facing context. This is a notification, not\n * a veto; disposal requested by a lifecycle owner is rechecked before the\n * driver starts.\n * @param agent - the agent whose session lifecycle began.\n * @param source - why the session started (fresh startup, resume, …).\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'The session lifecycle began, once before the first turn.', }, { name: 'agent/status', mode: 'emit', signature: '\'agent/status\'(this: Scoped, agent: Agent, status: AgentStatus): void', + jsDoc: '/**\n * Agent status changed (`idle` ⇄ `running`, or → `disposed`). `send()` does\n * not enter `running` synchronously; drive lifecycle from this event.\n * @param agent - the agent whose status flipped.\n * @param status - the status just entered (the transition\'s destination).\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */', summary: 'Agent status changed (`idle` ⇄ `running`, or → `disposed`).', }, { name: 'agent/step-result', mode: 'waterfall', signature: '\'agent/step-result\'(this: Scoped, agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise', + jsDoc: '/**\n * Waterfall: post-process the assembled assistant {@link Message} before\n * tool dispatch (validation, content rewriting, …).\n * @param agent - the agent that received the step\'s response.\n * @param turn - the open turn number.\n * @param step - the step that produced the message.\n * @param message - the assistant message as assembled from the stream.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', summary: 'Waterfall: post-process the assembled assistant Message before tool dispatch (validation, content rewriting, …).', }, { name: 'agent/turn-continuation', mode: 'waterfall', signature: '\'agent/turn-continuation\'(this: Scoped, agent: Agent, turn: number, defaultDecision: ContinuationDecision, next: () => Promise): Promise', + jsDoc: '/**\n * Override whether the turn continues. The default continues after tool\n * calls or steering and stops otherwise; a continue reason becomes steering.\n * @param agent - the agent deciding whether to run another step.\n * @param turn - the turn being continued or stopped.\n * @param defaultDecision - what the loop would do absent an override.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', summary: 'Override whether the turn continues.', }, { name: 'agent/turn-stop', mode: 'serial', signature: '\'agent/turn-stop\'(this: Scoped, agent: Agent, turn: number): ContinuationStop | undefined', + jsDoc: '/**\n * Monotonic terminal-stop checkpoint after continuation and steering are\n * folded; a stop remains authoritative through turn close and flush:\n * steering queued in that window is discarded, while ordinary sends survive.\n * @param agent - the agent whose composed continuation outcome may be stopped.\n * @param turn - the turn at its terminal-stop checkpoint.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode serial\n */', summary: 'Monotonic terminal-stop checkpoint after continuation and steering are folded; a stop remains authoritative through turn close and flush: steering queued in that window is discarded, while ordinary sends survive.', }, { name: 'approval/request', mode: 'waterfall', signature: '\'approval/request\'(this: Scoped, req: ApprovalRequest, next: () => Promise): Promise', + jsDoc: '/**\n * Ask composed answerers for one decision. Return an outcome to claim the\n * request or call `next()`; failure yields the fail-closed default.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param req - the pending decision (agent, tool identity, reason, signal).\n * @mode waterfall\n */', summary: 'Ask composed answerers for one decision.', }, { name: 'fs/edit-intent', mode: 'waterfall', signature: '\'fs/edit-intent\'(target: FsTarget, actor: object | undefined, next: () => { version: FsVersion } | undefined | Promise<{ version: FsVersion } | undefined>): Promise<{ version: FsVersion } | undefined>', + jsDoc: '/**\n * Single-slot decision for the next {@link FileSystem.editText}. Calling\n * `next()` yields an unconditional edit; the first returned guard wins.\n * @param target - the resolved target about to be edited.\n * @param actor - the opaque tool-execution context the decider keys off.\n * @mode waterfall\n */', summary: 'Single-slot decision for the next FileSystem.editText.', }, { name: 'fs/observed', mode: 'emit', signature: '\'fs/observed\'(target: FsTarget, version: FsVersion, actor: object | undefined): void', + jsDoc: '/**\n * Record a successful observation. Listeners must be synchronous recorders:\n * throws fail the tool call and returned promises are not awaited.\n * @param target - the target that was read/written/edited.\n * @param version - the version the actor now holds as its observation.\n * @param actor - the observing tool-execution context; undefined records nothing useful.\n * @mode emit\n */', summary: 'Record a successful observation.', }, { name: 'fs/write-intent', mode: 'waterfall', signature: '\'fs/write-intent\'(target: FsTarget, actor: object | undefined, next: () => FsWriteIntent | undefined | Promise): Promise', + jsDoc: '/**\n * Single-slot decision for the next {@link FileSystem.writeText}. Calling\n * `next()` yields the bare provider\'s unconditional write; the first listener\n * that returns an intent owns the decision rather than composing with peers.\n * @param target - the resolved target about to be written.\n * @param actor - the opaque tool-execution context the decider keys off.\n * @mode waterfall\n */', summary: 'Single-slot decision for the next FileSystem.writeText.', }, { name: 'llm/stream', mode: 'waterfall', signature: '\'llm/stream\'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable', + jsDoc: '/**\n * Waterfall around every streaming model call (retry, replay, routing).\n * Bound to the {@link LlmService}; call `next()` to reach the resolved\n * adapter\'s stream, or yield your own chunks to short-circuit.\n * @param options - the full request. A LOOP-built request arrives\n * deep-frozen (mutation throws): its content is a pure function of the\n * session log (the reconstructability RFC), so listeners read it, never\n * rewrite it. A hand-built one-shot (compaction summarize) is the\n * caller\'s own object and stays mutable here.\n * @mode waterfall\n */', summary: 'Waterfall around every streaming model call (retry, replay, routing).', }, { name: 'session/created', mode: 'emit', signature: '\'session/created\'(this: Scoped, session: Session): void', + jsDoc: '/**\n * Creation announcement during session publication. A synchronous throw vetoes and rolls\n * back with a paired disposal; detach requested during dispatch is deferred.\n * A returned-promise rejection is logged but cannot retroactively veto this\n * synchronous boundary.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners\n * receive only sessions entered through that agent\'s context.\n * @param session - the session just entered and announced.\n * @dshScopeScan unsupported\n * @mode emit\n */', summary: 'Creation announcement during session publication.', }, { name: 'session/disposed', mode: 'emit', signature: '\'session/disposed\'(this: Scoped, session: Session): void', + jsDoc: '/**\n * Emitted once when an announced session leaves the store, including\n * publication rollback, but never for an entry whose creation announcement\n * did not begin. Listener failures are logged and contained.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`) reuses the owner scope.\n * @param session - the session that is no longer live in the store.\n * @dshScopeScan unsupported\n * @mode emit\n */', summary: 'Emitted once when an announced session leaves the store, including publication rollback, but never for an entry whose creation announcement did not begin.', }, { name: 'session/event', mode: 'emit', signature: '\'session/event\'(this: Scoped, session: Session, event: SessionEvent): void', + jsDoc: '/**\n * Post-commit, fire-and-forget append feed. The listener snapshot resolves\n * before the log push, but callbacks run after it; observer failures are\n * logged and contained without making the committed append fail.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners\n * receive only events from sessions entered through that agent\'s context.\n * @param session - the session whose log grew.\n * @param event - the appended event, exactly as recorded.\n * @dshScopeScan unsupported\n * @mode emit\n */', summary: 'Post-commit, fire-and-forget append feed.', }, { name: 'session/flush', mode: 'parallel', signature: '\'session/flush\'(this: Scoped, session: Session): Promise | void', + jsDoc: '/**\n * Awaited parallel durability checkpoint: every listener runs and the\n * caller awaits all of them, with no waterfall veto. Dispatch through\n * {@link SessionStore.flush}. Scope-filtered dispatch\n * (`@deepseek-ai/dsh-scope`) reuses the session\'s owner scope.\n * @param session - the session whose buffered events must reach durable storage.\n * @dshScopeScan unsupported\n * @mode parallel\n */', summary: 'Awaited parallel durability checkpoint: every listener runs and the caller awaits all of them, with no waterfall veto.', }, { name: 'subagent/end', mode: 'emit', signature: '\'subagent/end\'(this: Scoped, info: SubagentRunEndInfo): void', + jsDoc: '/**\n * A ready child settled. Scope-filtered dispatch uses the same delegating\n * parent carrier as `subagent/start`, so the lifecycle pair reaches the\n * same scoped audience.\n * @param info - the run identity and terminal outcome.\n * @dshScopeScan unsupported\n * @mode emit\n */', summary: 'A ready child settled.', }, { name: 'subagent/provider-added', mode: 'emit', signature: '\'subagent/provider-added\'(provider: SubagentProvider): void', + jsDoc: '/**\n * A provider became resolvable in the registry.\n * @param provider - the registered provider.\n * @mode emit\n */', summary: 'A provider became resolvable in the registry.', }, { name: 'subagent/provider-removed', mode: 'emit', signature: '\'subagent/provider-removed\'(name: string): void', + jsDoc: '/**\n * A provider left the registry. Accepted runs remain holder-owned.\n * @param name - the provider name that no longer resolves.\n * @mode emit\n */', summary: 'A provider left the registry.', }, { name: 'subagent/start', mode: 'emit', signature: '\'subagent/start\'(this: Scoped, info: SubagentRunInfo): void', + jsDoc: '/**\n * A provider established a ready child. For in-process providers,\n * `ctx.agents.get(info.id)` resolves during this notification.\n * Scope-filtered dispatch keys the carrier by the delegating parent, so a\n * parent-scoped listener observes only its own delegations. Paired with\n * `subagent/end`.\n * @param info - the provider and ready child identity.\n * @dshScopeScan unsupported\n * @mode emit\n */', summary: 'A provider established a ready child.', }, { name: 'system-prompt/assemble', mode: 'waterfall', signature: '\'system-prompt/assemble\'(this: Scoped, assembly: PromptAssembly, context: AssembleContext, next: () => Promise): Promise', + jsDoc: '/**\n * Expert waterfall over the assembled sections, tools, and variables.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): scoped listeners\n * receive only that scope\'s assemblies. The returned value is authoritative.\n * @param assembly - the mutable assembly built from registered providers.\n * @param context - the caller\'s per-assembly context.\n * @mode waterfall\n */', summary: 'Expert waterfall over the assembled sections, tools, and variables.', }, { name: 'system-prompt/change', mode: 'emit', signature: '\'system-prompt/change\'(): void', + jsDoc: '/**\n * Emitted when any prompt provider changes. This registry notification is\n * unfiltered because a global change affects every scope.\n * @mode emit\n */', summary: 'Emitted when any prompt provider changes.', }, { name: 'tools/change', mode: 'emit', signature: '\'tools/change\'(): void', + jsDoc: '/**\n * A tool was registered or unregistered, or a scoped restriction changed\n * (the available tool set changed — possibly for one scope only). An\n * UNFILTERED registry-subject notification, deliberately not scope-filtered\n * dispatch: a global change concerns every agent\'s next assembly, so a\n * scoped listener subscribing here sees every change, not just its own\n * scope\'s.\n * @mode emit\n */', summary: 'A tool was registered or unregistered, or a scoped restriction changed (the available tool set changed — possibly for one scope only).', }, { name: 'tools/execute', mode: 'waterfall', signature: '\'tools/execute\'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise', + jsDoc: '/**\n * Around-dispatch waterfall for timeout, retry, or metrics. `next()` returns\n * a normalized result; wrappers may change only `exec.signal`, while call\n * identity remains immutable.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent\'s calls.\n * @param exec - the allowed call about to dispatch (name, parsed arguments, caller agent, signal).\n * @mode waterfall\n */', summary: 'Around-dispatch waterfall for timeout, retry, or metrics.', }, { name: 'tools/post-execute', mode: 'waterfall', signature: '\'tools/post-execute\'(this: Scoped, exec: ToolExecution, result: Readonly, next: () => Promise): Promise', + jsDoc: '/**\n * Accept, replace, enrich, or block a normalized dispatch result. `next()`\n * accepts it unchanged; thrown tools still reach this seam as errors.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent\'s calls.\n * @param exec - the call that just ran (name, parsed arguments, caller agent).\n * @param result - the dispatch outcome a listener may accept, replace, or block.\n * @mode waterfall\n */', summary: 'Accept, replace, enrich, or block a normalized dispatch result.', }, { name: 'tools/pre-execute', mode: 'waterfall', signature: '\'tools/pre-execute\'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise', + jsDoc: '/**\n * Allow, deny, or ask before dispatch. `next()` delegates to allow; missing\n * approval support turns `ask` into denial.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent\'s calls.\n * @param exec - the pending call (name, parsed arguments, caller agent).\n * @mode waterfall\n */', summary: 'Allow, deny, or ask before dispatch.', }, { name: 'tools/result', mode: 'emit', signature: '\'tools/result\'(this: Scoped, exec: Readonly, result: Readonly): undefined', + jsDoc: '/**\n * Observe the frozen, lossless-JSON final outcome. Listener failures are contained.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): keyed by `exec.agent`.\n * @param exec - the execution object that traversed the pipeline.\n * @param result - a deep-frozen snapshot of the final returned result.\n * @mode emit\n */', summary: 'Observe the frozen, lossless-JSON final outcome.', }, { name: 'workflow/agent-end', mode: 'emit', signature: '\'workflow/agent-end\'(info: WorkflowRunInfo, agent: WorkflowAgentEndInfo): void', + jsDoc: '/**\n * One `agent()` call settled (clean result, child failure, or run\n * cancellation). Paired with {@link Events[\'workflow/agent-start\']} by\n * `agent.seq`, exactly once per started call on every stop path — on an\n * engine termination path (a worker killed past its grace) the end is\n * engine-synthesized with outcome `\'cancelled\'`.\n * @param info - the run\'s identity snapshot.\n * @param agent - the call identity plus its outcome.\n * @mode emit\n */', summary: 'One `agent()` call settled (clean result, child failure, or run cancellation).', }, { name: 'workflow/agent-start', mode: 'emit', signature: '\'workflow/agent-start\'(info: WorkflowRunInfo, agent: WorkflowAgentInfo): void', + jsDoc: '/**\n * One `agent()` call established a ready child run. Paired with\n * {@link Events[\'workflow/agent-end\']} by `agent.seq`. A call that never\n * receives a ready run from the provider emits neither\n * event in this pair.\n * @param info - the run\'s identity snapshot.\n * @param agent - the call\'s sequence number, label, phase, and child id.\n * @mode emit\n */', summary: 'One `agent()` call established a ready child run.', }, { name: 'workflow/end', mode: 'emit', signature: '\'workflow/end\'(info: WorkflowRunInfo, result: WorkflowResultInfo): void', + jsDoc: '/**\n * A workflow run settled (any stop reason). Fired when\n * {@link WorkflowRun.result} resolves. Paired with\n * {@link Events[\'workflow/start\']}.\n * @param info - the run\'s identity snapshot.\n * @param result - the outcome data (stop reason, error, agent count) —\n * deliberately WITHOUT the result value (see {@link WorkflowResultInfo}).\n * @mode emit\n */', summary: 'A workflow run settled (any stop reason).', }, { name: 'workflow/log', mode: 'emit', signature: '\'workflow/log\'(info: WorkflowRunInfo, message: string): void', + jsDoc: '/**\n * The script emitted a narration line (a `log(message)` call).\n * @param info - the run\'s identity snapshot.\n * @param message - the logged message, verbatim.\n * @mode emit\n */', summary: 'The script emitted a narration line (a `log(message)` call).', }, { name: 'workflow/phase', mode: 'emit', signature: '\'workflow/phase\'(info: WorkflowRunInfo, title: string): void', + jsDoc: '/**\n * The script entered a phase (a `phase(title)` call) — progress grouping\n * for observers; no execution semantics.\n * @param info - the run\'s identity snapshot.\n * @param title - the phase title, verbatim.\n * @mode emit\n */', summary: 'The script entered a phase (a `phase(title)` call) — progress grouping for observers; no execution semantics.', }, { name: 'workflow/start', mode: 'emit', signature: '\'workflow/start\'(info: WorkflowRunInfo): void', + jsDoc: '/**\n * A workflow run started — the script\'s meta block validated, the body\n * about to execute. Paired with {@link Events[\'workflow/end\']}.\n * @param info - the run\'s identity snapshot (id + meta).\n * @mode emit\n */', summary: 'A workflow run started — the script\'s meta block validated, the body about to execute.', }, ] diff --git a/packages/cordis/tool-cordis/src/index.ts b/packages/cordis/tool-cordis/src/index.ts index 321546c8c3..fd15d75624 100644 --- a/packages/cordis/tool-cordis/src/index.ts +++ b/packages/cordis/tool-cordis/src/index.ts @@ -12,9 +12,9 @@ import z from 'schemastery' import { defineTool } from '@deepseek-ai/dsh-tools' import { STATE_LABELS } from './fiber-state.ts' import { isPlugin, pluginName } from './guard.ts' +import { EVENT_API, INHERITED_CTX_API, SERVICE_API, TYPE_API } from './api-catalog.ts' import { describeApi, describeDynamic, describeEvents, describePlugins, describeServices, describeTools } from './inspect.ts' -import { missingServices, mountDynamic } from './mount.ts' -import type { DynamicMount } from './mount.ts' +import { missingServices, mountDynamic, type DynamicMount } from './mount.ts' import { presentInspectCall, presentMountCall, presentUnmountCall } from './present.ts' import { createSandbox, evaluateMountCode } from './sandbox.ts' @@ -63,15 +63,23 @@ export function apply(ctx: Context, config: Config): void { + '`dynamic` (plugins you mounted via cordis_mount: id, name, state, provided services, awaited services), ' + '`api` (method signatures AND argument/return type shapes for every LIVE service — read this before writing plugin code that calls a service), ' + '`events` (every harness event with its dispatch mode and exact signature — pick listener targets here). ' - + 'Omit `what` to get all six sections.', + + 'Omit `what` to get all six sections. With `what:"api"` or `what:"events"`, pass an exact `name` ' + + 'to narrow to one service/event and include its original source JSDoc.', parameters: { what: { type: 'string', enum: ['services', 'plugins', 'tools', 'dynamic', 'api', 'events'], description: 'Limit the report to one section. Omit for all sections.', }, + name: { + type: 'string', + description: 'Exact service key or event name whose original JSDoc to include; valid only with what:"api" or what:"events".', + }, }, execute(args, exec): Promise<{ type: 'text'; text: string }[]> { + if (args.name !== undefined && args.what !== 'api' && args.what !== 'events') { + throw new Error('name is valid only with what:"api" or what:"events"') + } const sections: [heading: string, body: () => string[]][] = [ ['services', () => describeServices(ctx)], ['plugins', () => describePlugins(ctx)], @@ -79,8 +87,8 @@ export function apply(ctx: Context, config: Config): void { // globals absent — "what you can call", not the global registry. ['tools', () => describeTools(ctx, exec.agent)], ['dynamic', () => describeDynamic(ctx, mounts)], - ['api', () => describeApi(ctx)], - ['events', () => describeEvents()], + ['api', () => describeApi(ctx, SERVICE_API, INHERITED_CTX_API, TYPE_API, args.name)], + ['events', () => describeEvents(EVENT_API, args.name)], ] const selected = sections.filter(([heading]) => args.what === undefined || args.what === heading) const text = selected diff --git a/packages/cordis/tool-cordis/src/inspect.ts b/packages/cordis/tool-cordis/src/inspect.ts index b483b13712..7196f370ce 100644 --- a/packages/cordis/tool-cordis/src/inspect.ts +++ b/packages/cordis/tool-cordis/src/inspect.ts @@ -1,7 +1,8 @@ /** * Read-only renderers over the live runtime for `cordis_inspect`: the service list, the flat * plugin list, the registered tools, the dynamic-mount table (with per-mount provides/waits), - * and the catalog-backed `api` / `events` sections. + * and the catalog-backed `api` / `events` sections. Exact-name lookups add the + * original source JSDoc without inflating the default reports. * @module @deepseek-ai/dsh-tool-cordis/inspect */ @@ -126,6 +127,18 @@ function typeClosure(seeds: string[], types: readonly TypeApiEntry[]): TypeApiEn return [...included.values()].sort((a, b) => a.name.localeCompare(b.name)) } +/** Render one catalogued service, optionally including source-owned method JSDoc. */ +function serviceLines(entry: ServiceApiEntry, detailed: boolean): string[] { + const lines = [`- ${entry.key} — ${entry.summary}`] + for (const method of entry.methods) { + if (detailed) { + for (const docLine of method.jsDoc.split('\n')) lines.push(` ${docLine}`) + } + lines.push(` ${method.signature}`) + } + return lines +} + /** * Render the generated catalog against the live runtime: live catalogued services with methods, * uncatalogued live services with owners, absent loadable services, referenced type shapes, and @@ -134,6 +147,7 @@ function typeClosure(seeds: string[], types: readonly TypeApiEntry[]): TypeApiEn * @param api - generated service entries, replaceable in tests. * @param inherited - inherited `ctx` entries, replaceable in tests. * @param types - public type shapes, replaceable in tests. + * @param name - exact live service key whose methods should include original JSDoc; omitted for the compact catalog. * @returns the section lines. */ export function describeApi( @@ -141,25 +155,33 @@ export function describeApi( api: readonly ServiceApiEntry[] = SERVICE_API, inherited: readonly InheritedApiEntry[] = INHERITED_CTX_API, types: readonly TypeApiEntry[] = TYPE_API, + name?: string, ): string[] { const live = new Map() for (const impl of liveImpls(ctx)) live.set(impl.name, impl.fiber.name) const lines: string[] = [] const liveMethodTexts: string[] = [] - for (const entry of api) { - if (!live.has(entry.key)) continue - lines.push(`- ${entry.key} — ${entry.summary}`) + let selected = api.filter(entry => live.has(entry.key)) + if (name !== undefined) { + const entry = api.find(candidate => candidate.key === name) + if (!entry) throw new Error(`no catalogued service named "${name}"`) + if (!live.has(name)) throw new Error(`catalogued service "${name}" is not running`) + selected = [entry] + } + for (const entry of selected) { + lines.push(...serviceLines(entry, name !== undefined)) for (const method of entry.methods) { - lines.push(` ${method}`) - liveMethodTexts.push(method) + liveMethodTexts.push(method.signature) } } - const catalogued = new Set(api.map(entry => entry.key)) - for (const [name, fiber] of [...live].sort(([a], [b]) => a.localeCompare(b))) { - if (!catalogued.has(name)) lines.push(`- ${name} (provided by ${fiber}, no catalog entry)`) + if (name === undefined) { + const catalogued = new Set(api.map(entry => entry.key)) + for (const [liveName, fiber] of [...live].sort(([a], [b]) => a.localeCompare(b))) { + if (!catalogued.has(liveName)) lines.push(`- ${liveName} (provided by ${fiber}, no catalog entry)`) + } + const notRunning = api.filter(entry => !live.has(entry.key)).map(entry => entry.key) + if (notRunning.length > 0) lines.push(`not running (loadable services with no live provider): ${notRunning.join(', ')}`) } - const notRunning = api.filter(entry => !live.has(entry.key)).map(entry => entry.key) - if (notRunning.length > 0) lines.push(`not running (loadable services with no live provider): ${notRunning.join(', ')}`) const shapes = typeClosure(liveMethodTexts, types) if (shapes.length > 0) { lines.push('type shapes (referenced by the signatures above — read these before assuming a field is a string):') @@ -167,8 +189,10 @@ export function describeApi( for (const declLine of shape.declaration.split('\n')) lines.push(` ${declLine}`) } } - lines.push('inherited ctx API:') - for (const entry of inherited) lines.push(`- ${entry.name} — ${entry.summary}`) + if (name === undefined) { + lines.push('inherited ctx API:') + for (const entry of inherited) lines.push(`- ${entry.name} — ${entry.summary}`) + } return lines } @@ -176,13 +200,24 @@ export function describeApi( * The `events` section: every harness event with its dispatch mode, one-line * summary, and exact signature, closed by the waterfall caution. * @param events - the event catalog (the generated one by default; injectable for tests). + * @param name - exact event name whose signature should include original JSDoc; omitted for the compact catalog. * @returns the section lines. */ -export function describeEvents(events: readonly EventApiEntry[] = EVENT_API): string[] { - const lines = events.flatMap(event => [ - `- ${event.name} [${event.mode}] — ${event.summary}`, - ` ${event.signature}`, - ]) +export function describeEvents(events: readonly EventApiEntry[] = EVENT_API, name?: string): string[] { + let selected = events + if (name !== undefined) { + const event = events.find(candidate => candidate.name === name) + if (!event) throw new Error(`no catalogued event named "${name}"`) + selected = [event] + } + const lines = selected.flatMap((event) => { + const entry = [`- ${event.name} [${event.mode}] — ${event.summary}`] + if (name !== undefined) { + for (const docLine of event.jsDoc.split('\n')) entry.push(` ${docLine}`) + } + entry.push(` ${event.signature}`) + return entry + }) lines.push('waterfall listeners receive a trailing next() and MUST call it to delegate — returning without next() vetoes the chain.') return lines } diff --git a/packages/cordis/tool-cordis/src/present.ts b/packages/cordis/tool-cordis/src/present.ts index 614b070193..e13570cf43 100644 --- a/packages/cordis/tool-cordis/src/present.ts +++ b/packages/cordis/tool-cordis/src/present.ts @@ -15,11 +15,12 @@ import type { GenericCallView } from '@deepseek-ai/dsh-tools' * @param args - the validated call arguments. * @returns the generic card the ACP bridge renders. */ -export function presentInspectCall(args: { what?: string }): GenericCallView { +export function presentInspectCall(args: { what?: string; name?: string }): GenericCallView { + const target = args.name === undefined ? args.what : `${args.what}: ${args.name}` return { card: 'generic', kind: 'read', - title: args.what === undefined ? 'Inspect cordis runtime' : `Inspect cordis runtime: ${args.what}`, + title: target === undefined ? 'Inspect cordis runtime' : `Inspect cordis runtime: ${target}`, } } diff --git a/packages/cordis/tool-cordis/tests/inspect.spec.ts b/packages/cordis/tool-cordis/tests/inspect.spec.ts index 1a8c39467a..4d986d4f3e 100644 --- a/packages/cordis/tool-cordis/tests/inspect.spec.ts +++ b/packages/cordis/tool-cordis/tests/inspect.spec.ts @@ -64,6 +64,24 @@ describe('cordis_inspect', () => { // The inherited ctx surface closes the section. expect(report).toContain('inherited ctx API:') expect(report).toContain('- ctx.effect — ') + // The broad report stays compact; exact-name lookup owns full JSDoc. + expect(report).not.toContain('/**') + expect(report).not.toContain('@param definition') + }) + + it('adds original method JSDoc only for an exact live api name', async () => { + const ctx = await setup() + const report = text(await call(ctx, 'cordis_inspect', { what: 'api', name: 'tools' })) + expect(report).toContain('## api') + expect(report).toContain('- tools — Tool registry and execution pipeline.') + expect(report).toContain('/**') + expect(report).toContain('Register globally or in the calling agent scope.') + expect(report).toContain('@param definition - the tool schema') + expect(report).toContain('@returns the exact disposer') + expect(report).toContain('register(definition: ToolDefinition)') + expect(report).toContain('type shapes (referenced by the signatures above') + expect(report).not.toContain('not running (loadable services') + expect(report).not.toContain('inherited ctx API:') }) it('renders the events section with mode badges, signatures, and the waterfall caution', async () => { @@ -73,6 +91,39 @@ describe('cordis_inspect', () => { expect(report).toContain('- tools/pre-execute [waterfall]') expect(report).toMatch(/'agent\/status'\(/) expect(report).toContain('returning without next() vetoes the chain') + expect(report).not.toContain('/**') + expect(report).not.toContain('@mode waterfall') + }) + + it('adds original event JSDoc only for an exact event name', async () => { + const ctx = await setup() + const report = text(await call(ctx, 'cordis_inspect', { what: 'events', name: 'tools/pre-execute' })) + expect(report).toContain('## events') + expect(report).toContain('- tools/pre-execute [waterfall]') + expect(report).toContain('/**') + expect(report).toContain('Allow, deny, or ask before dispatch.') + expect(report).toContain('@param exec - the pending call') + expect(report).toContain('@mode waterfall') + expect(report).not.toContain('- tools/change [emit]') + }) + + it('fails loud for incompatible, unknown, and non-running names', async () => { + const ctx = await setup() + const incompatible = await call(ctx, 'cordis_inspect', { what: 'tools', name: 'tools' }) + expect(incompatible.isError).toBe(true) + expect(text(incompatible)).toContain('name is valid only with what:"api" or what:"events"') + + const unknownService = await call(ctx, 'cordis_inspect', { what: 'api', name: 'not-a-service' }) + expect(unknownService.isError).toBe(true) + expect(text(unknownService)).toContain('no catalogued service named "not-a-service"') + + const nonRunning = await call(ctx, 'cordis_inspect', { what: 'api', name: 'bash' }) + expect(nonRunning.isError).toBe(true) + expect(text(nonRunning)).toContain('catalogued service "bash" is not running') + + const unknownEvent = await call(ctx, 'cordis_inspect', { what: 'events', name: 'not/an-event' }) + expect(unknownEvent.isError).toBe(true) + expect(text(unknownEvent)).toContain('no catalogued event named "not/an-event"') }) }) @@ -102,7 +153,11 @@ describe('inspect renderers (direct)', () => { it('describeApi omits the not-running line and type shapes when nothing applies', async () => { const ctx = await setup() - const lines = describeApi(ctx, [{ key: 'tools', summary: 'The registry.', methods: ['register(x): void'] }], [], []) + const lines = describeApi(ctx, [{ + key: 'tools', + summary: 'The registry.', + methods: [{ signature: 'register(x): void', jsDoc: '/** Register x. */' }], + }], [], []) expect(lines[0]).toBe('- tools — The registry.') expect(lines[1]).toBe(' register(x): void') expect(lines.join('\n')).not.toContain('not running') diff --git a/packages/cordis/tool-cordis/tests/present.spec.ts b/packages/cordis/tool-cordis/tests/present.spec.ts index d8f380439f..ed45fd1518 100644 --- a/packages/cordis/tool-cordis/tests/present.spec.ts +++ b/packages/cordis/tool-cordis/tests/present.spec.ts @@ -11,6 +11,11 @@ describe('presenters', () => { it('cordis_inspect renders a generic read card titled with the section', () => { expect(presentInspectCall({})).toEqual({ card: 'generic', kind: 'read', title: 'Inspect cordis runtime' }) expect(presentInspectCall({ what: 'api' })).toEqual({ card: 'generic', kind: 'read', title: 'Inspect cordis runtime: api' }) + expect(presentInspectCall({ what: 'events', name: 'tools/change' })).toEqual({ + card: 'generic', + kind: 'read', + title: 'Inspect cordis runtime: events: tools/change', + }) }) it('cordis_mount renders a generic execute card carrying the code as raw input', () => { @@ -33,6 +38,9 @@ describe('presenters', () => { kind: 'read', title: 'Inspect cordis runtime: tools', }) + expect(ctx.tools.get('cordis_inspect')!.presentCall!({ what: 'api', name: 'tools' })).toMatchObject({ + title: 'Inspect cordis runtime: api: tools', + }) expect(ctx.tools.get('cordis_mount')!.presentCall!({ code: 'return 1' })).toMatchObject({ kind: 'execute' }) expect(ctx.tools.get('cordis_unmount')!.presentCall!({ id: 'dyn-2' })).toMatchObject({ title: 'Unmount dyn-2' }) // Soft validation: presenter args that fail the schema render as no card, never a throw. diff --git a/packages/cordis/tool-cordis/tests/tool-cordis.spec.ts b/packages/cordis/tool-cordis/tests/tool-cordis.spec.ts index 8953b5da94..d305a92883 100644 --- a/packages/cordis/tool-cordis/tests/tool-cordis.spec.ts +++ b/packages/cordis/tool-cordis/tests/tool-cordis.spec.ts @@ -32,8 +32,9 @@ describe('tool registration', () => { const names = ctx.tools.schemas().map(schema => schema.name) expect(names).toEqual(expect.arrayContaining(['cordis_inspect', 'cordis_mount', 'cordis_unmount'])) const inspect = ctx.tools.schemas().find(schema => schema.name === 'cordis_inspect')! - const props = (inspect.parameters as { properties: Record }).properties + const props = (inspect.parameters as { properties: Record }).properties expect(props.what?.enum).toEqual(['services', 'plugins', 'tools', 'dynamic', 'api', 'events']) + expect(props.name?.type).toBe('string') }) }) diff --git a/scripts/gen-cordis-api.ts b/scripts/gen-cordis-api.ts index 5d8a70fba4..7c61b9395d 100644 --- a/scripts/gen-cordis-api.ts +++ b/scripts/gen-cordis-api.ts @@ -1,8 +1,9 @@ /** * Generate the model-facing Cordis API data module from the same event/service - * collector as the documentation catalogs. It emits first-sentence docs, raw - * signatures, transitive public type shapes, and inherited context entries, - * without source pointers; output is deterministic and `--check` verifies it. + * collector as the documentation catalogs. It emits original declaration + * JSDoc, first-sentence summaries, raw signatures, transitive public type + * shapes, and inherited context entries, without source pointers; output is + * deterministic and `--check` verifies it. */ import { globSync, readFileSync, writeFileSync } from 'node:fs' @@ -88,22 +89,30 @@ function render(): string { ' * `pnpm run verify-cordis-api` in doc-sync).', ' *', ' * The machine-readable cordis API catalog `cordis_inspect` serves to the', - ' * model: harness services (summary + public method signatures), harness', - ' * events (mode + signature), and the inherited `ctx` surface. Produced by', + ' * model: harness services (summary + public method signatures/JSDoc),', + ' * harness events (mode + signature/JSDoc), and the inherited `ctx` surface. Produced by', ' * the same AST walk as docs/cordis-catalog, so this data and the rendered', ' * docs cannot diverge.', ' *', ' * @module @deepseek-ai/dsh-tool-cordis/api-catalog', ' */', '', - '/** One harness `ctx.` service: its one-line summary and public method signatures. */', + '/** One public service method and its source-owned contract. */', + 'export interface ServiceApiMethod {', + ' /** Public method signature with its body stripped. */', + ' signature: string', + ' /** Original method JSDoc, with only container indentation removed. */', + ' jsDoc: string', + '}', + '', + '/** One harness `ctx.` service: its one-line summary and public methods. */', 'export interface ServiceApiEntry {', ' /** The `ctx.` name, e.g. `tools`. */', ' key: string', ' /** First sentence of the service class JSDoc. */', ' summary: string', - ' /** Public method signatures, bodies stripped, in source order. */', - ' methods: readonly string[]', + ' /** Public methods, bodies stripped, in source order. */', + ' methods: readonly ServiceApiMethod[]', '}', '', '/** One harness event: its dispatch mode, exact signature, and one-line summary. */', @@ -114,6 +123,8 @@ function render(): string { ' mode: string', ' /** The exact listener signature, whitespace-normalized. */', ' signature: string', + ' /** Original event JSDoc, with only container indentation removed. */', + ' jsDoc: string', ' /** First sentence of the event JSDoc. */', ' summary: string', '}', @@ -145,7 +156,12 @@ function render(): string { lines.push(' methods: [],') } else { lines.push(' methods: [') - for (const method of service.methods) lines.push(` ${quote(method.signature)},`) + for (const method of service.methods) { + lines.push(' {') + lines.push(` signature: ${quote(method.signature)},`) + lines.push(` jsDoc: ${quote(method.jsDoc)},`) + lines.push(' },') + } lines.push(' ],') } lines.push(' },') @@ -161,6 +177,7 @@ function render(): string { lines.push(` name: ${quote(event.name)},`) lines.push(` mode: ${quote(event.mode)},`) lines.push(` signature: ${quote(event.signature)},`) + lines.push(` jsDoc: ${quote(event.jsDoc)},`) lines.push(` summary: ${quote(firstSentence(event.doc))},`) lines.push(' },') } From 3956118c18badbacf0eec7e434e57a95c0335185 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 14:53:39 +0800 Subject: [PATCH 44/73] fix(dsbench): adapt evaluation config to master runtime --- docs/config-catalog.md | 2 +- examples/dsbench-coding-agent/cordis.yml | 3 --- examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts | 2 +- examples/package.json | 3 +++ packages/ui/jsonrpc/src/index.ts | 4 +++- pnpm-lock.yaml | 9 +++++++++ 6 files changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 5fe8ffe568..542728293c 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -159,7 +159,7 @@ export interface SkillConfig { Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts) -Source: [`packages/examples/agent-spine-demo/src/index.ts:57`](../packages/examples/agent-spine-demo/src/index.ts) +Source: [`packages/examples/agent-spine-demo/src/index.ts:59`](../packages/examples/agent-spine-demo/src/index.ts) ## `@deepseek-ai/dsh-bash-local` diff --git a/examples/dsbench-coding-agent/cordis.yml b/examples/dsbench-coding-agent/cordis.yml index 67e1604b07..4d970c8b47 100644 --- a/examples/dsbench-coding-agent/cordis.yml +++ b/examples/dsbench-coding-agent/cordis.yml @@ -11,9 +11,6 @@ config: apiKey: !!js process.env.DEEPSEEK_API_KEY baseURL: !!js process.env.DEEPSEEK_BASE_URL - models: - - deepseek-v4-flash - - deepseek-v4-pro - id: bash name: '@deepseek-ai/dsh-bash-local' diff --git a/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts b/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts index 9bbc07a829..f5e2d7b938 100644 --- a/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts +++ b/examples/dsbench-coding-agent/tests/keyless-smoke.e2e.ts @@ -97,7 +97,7 @@ describe('dsbench-coding-agent keyless smoke', () => { jsonrpc: '2.0', id: 1, method: 'initialize', - params: { cwd: root, model: 'deepseek-v4-pro' }, + params: { cwd: root, provider: 'deepseek', model: 'deepseek-v4-pro' }, })}\n`) const initialized = await waitForLine(lines, value => value.id === 1, () => stderr) expect(initialized).toMatchObject({ diff --git a/examples/package.json b/examples/package.json index 8d77731a05..01faf631ba 100644 --- a/examples/package.json +++ b/examples/package.json @@ -8,6 +8,7 @@ "@cordisjs/plugin-hmr": "workspace:*", "@cordisjs/plugin-include": "workspace:*", "@deepseek-ai/dsh-acp-demo": "workspace:*", + "@deepseek-ai/dsh-agent-spine-demo": "workspace:*", "@deepseek-ai/dsh-bash-local": "workspace:*", "@deepseek-ai/dsh-bash-sandbox": "workspace:*", "@deepseek-ai/dsh-code-runtime-worker": "workspace:*", @@ -16,12 +17,14 @@ "@deepseek-ai/dsh-fs-policy": "workspace:*", "@deepseek-ai/dsh-hooks-claude": "workspace:*", "@deepseek-ai/dsh-hooks-codex": "workspace:*", + "@deepseek-ai/dsh-jsonrpc": "workspace:*", "@deepseek-ai/dsh-llm": "workspace:*", "@deepseek-ai/dsh-llm-deepseek": "workspace:*", "@deepseek-ai/dsh-llm-replay": "workspace:*", "@deepseek-ai/dsh-permission": "workspace:*", "@deepseek-ai/dsh-repeat-tool-guard": "workspace:*", "@deepseek-ai/dsh-sandbox-local": "workspace:*", + "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:*", "@deepseek-ai/dsh-spill-local": "workspace:*", "@deepseek-ai/dsh-spill-policy": "workspace:*", "@deepseek-ai/dsh-stdio-demo": "workspace:*", diff --git a/packages/ui/jsonrpc/src/index.ts b/packages/ui/jsonrpc/src/index.ts index fe782c934f..099ef6008f 100644 --- a/packages/ui/jsonrpc/src/index.ts +++ b/packages/ui/jsonrpc/src/index.ts @@ -45,6 +45,8 @@ export const Config: Schema = Schema.object({ * owns root-context disposal for EOF and signals. */ export function apply(ctx: Context, config: JsonRpcConfig): void { + // Cordis applies the schema default before invoking the plugin. + const resolvedConfig = config as JsonRpcConfig & { maxTokensAsSuccess: boolean } // The later transport callback must dispose this plugin's fiber, not its ambient context. const fiber = ctx.fiber /* v8 ignore next -- production stdio wiring; tests always inject the runtime seams */ @@ -56,7 +58,7 @@ export function apply(ctx: Context, config: JsonRpcConfig): void { const transport = new JsonRpcLineTransport(input, output) const server = new HarnessSdkServer(ctx, transport, { - maxTokensAsSuccess: config.maxTokensAsSuccess ?? false, + maxTokensAsSuccess: resolvedConfig.maxTokensAsSuccess, }) // Share one exit task and attempt flush and disposal independently before exiting. diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6c7e4a3d05..b363f3e88a 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -101,6 +101,9 @@ importers: '@deepseek-ai/dsh-acp-demo': specifier: workspace:* version: link:../packages/examples/acp-demo + '@deepseek-ai/dsh-agent-spine-demo': + specifier: workspace:* + version: link:../packages/examples/agent-spine-demo '@deepseek-ai/dsh-bash-local': specifier: workspace:* version: link:../packages/bash/bash-local @@ -125,6 +128,9 @@ importers: '@deepseek-ai/dsh-hooks-codex': specifier: workspace:* version: link:../packages/hooks/hooks-codex + '@deepseek-ai/dsh-jsonrpc': + specifier: workspace:* + version: link:../packages/ui/jsonrpc '@deepseek-ai/dsh-llm': specifier: workspace:* version: link:../packages/llm/llm @@ -143,6 +149,9 @@ importers: '@deepseek-ai/dsh-sandbox-local': specifier: workspace:* version: link:../packages/sandbox/sandbox-local + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:* + version: link:../packages/session-persistence/session-persistence-jsonl '@deepseek-ai/dsh-spill-local': specifier: workspace:* version: link:../packages/spill/spill-local From 350d1b268830d76def9fd8dae53759d1953bb098 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 14:57:52 +0800 Subject: [PATCH 45/73] docs: verify Cordis signature type links --- docs/config-catalog.md | 4 +- docs/cordis-catalog/events.md | 68 ++++-- docs/cordis-catalog/services.md | 42 ++-- .../2026-06-20-generated-cordis-catalog.md | 6 +- .../agent/tests/gen-cordis-catalog.spec.ts | 36 +++- scripts/gen-cordis-catalog.ts | 198 ++++++++++++++++-- 6 files changed, 294 insertions(+), 60 deletions(-) diff --git a/docs/config-catalog.md b/docs/config-catalog.md index f77334b442..9aa0d48dc6 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -99,7 +99,7 @@ export interface Config { } ``` -Depends on: [`AgentOptions`](../packages/core/agent/src/index.ts) · [`SessionId`](../packages/core/session/src/index.ts) +Depends on: [`AgentOptions`](core-data-structures/core.md) · [`SessionId`](core-data-structures/core.md) Source: [`packages/core/agent-loop/src/index.ts:369`](../packages/core/agent-loop/src/index.ts) @@ -1108,7 +1108,7 @@ export interface Config { } ``` -Depends on: [`AgentOptions`](../packages/core/agent/src/index.ts) +Depends on: [`AgentOptions`](core-data-structures/core.md) Source: [`packages/subagent/tool-subagent/src/index.ts:23`](../packages/subagent/tool-subagent/src/index.ts) diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 12f877ad48..f522790fd0 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -31,7 +31,7 @@ A fully configured agent and live session were published. Setup is composition-o 'agent/created'(this: Scoped, agent: Agent): void ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:141`](../../packages/core/agent/src/types.ts) @@ -51,7 +51,7 @@ An agent left the registry; AgentLoop emits this after driver quiescence but bef 'agent/disposed'(this: Scoped, agent: Agent): void ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:150`](../../packages/core/agent/src/types.ts) @@ -73,7 +73,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w 'agent/error'(this: Scoped, agent: Agent, turn: number, step: number, error: Error): void ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:285`](../../packages/core/agent/src/types.ts) @@ -101,7 +101,7 @@ Awaited serial checkpoint for session-surface mutation after prompt assembly and 'agent/pre-step'(this: Scoped, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise | void ``` -Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:204`](../../packages/core/agent/src/types.ts) @@ -122,7 +122,7 @@ Allow, rewrite, or block one drained prompt before it becomes a user message. Ca 'agent/prompt-submit'(this: Scoped, agent: Agent, content: ContentBlock[], source: MessageSource, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) · [PromptDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:214`](../../packages/core/agent/src/types.ts) @@ -143,7 +143,7 @@ Detached, frozen content entered the agent's inbox. Source defaults have already 'agent/queued'(this: Scoped, agent: Agent, content: ContentBlock[], info: { source: MessageSource; steering: boolean }): void ``` -Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:169`](../../packages/core/agent/src/types.ts) @@ -166,7 +166,7 @@ Replace the frozen call configuration. Model-visible content must use logged cha 'agent/request'(this: Scoped, agent: Agent, turn: number, step: number, config: LlmCallConfig, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:226`](../../packages/core/agent/src/types.ts) @@ -192,7 +192,7 @@ Compose request-only messages placed before derived history. The frozen result i 'agent/session-prefix'(this: Scoped, agent: Agent, prefix: Message[], signal: AbortSignal, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:241`](../../packages/core/agent/src/types.ts) @@ -214,7 +214,7 @@ The session lifecycle began, once before the first turn. Use `agent.inject()` to 'agent/session-start'(this: Scoped, agent: Agent, source: SessionStartSource): void ``` -Types: [Agent](../core-data-structures/core.md) · [SessionStartSource](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) · [SessionStartSource](../core-data-structures/core.md) Source: [`packages/core/agent/src/types.ts:182`](../../packages/core/agent/src/types.ts) @@ -234,7 +234,7 @@ Agent status changed (`idle` ⇄ `running`, or → `disposed`). `send()` does no 'agent/status'(this: Scoped, agent: Agent, status: AgentStatus): void ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [AgentStatus](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:159`](../../packages/core/agent/src/types.ts) @@ -256,7 +256,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va 'agent/step-result'(this: Scoped, agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:252`](../../packages/core/agent/src/types.ts) @@ -277,7 +277,7 @@ Override whether the turn continues. The default continues after tool calls or s 'agent/turn-continuation'(this: Scoped, agent: Agent, turn: number, defaultDecision: ContinuationDecision, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [ContinuationDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:262`](../../packages/core/agent/src/types.ts) @@ -298,7 +298,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a 'agent/turn-stop'(this: Scoped, agent: Agent, turn: number): ContinuationStop | undefined ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [ContinuationStop](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/core/agent/src/types.ts:272`](../../packages/core/agent/src/types.ts) @@ -321,6 +321,8 @@ A declarative agent entry failed before it could publish a live agent. Consumers 'agent-loop/config-start-failed'(sessionId: SessionId, error: unknown): void ``` +Types: [SessionId](../core-data-structures/core.md) + Source: [`packages/core/agent-loop/src/index.ts:362`](../../packages/core/agent-loop/src/index.ts) ## `approval/*` @@ -340,7 +342,7 @@ Ask composed answerers for one decision. Return an outcome to claim the request 'approval/request'(this: Scoped, req: ApprovalRequest, next: () => Promise): Promise ``` -Types: [ApprovalOutcome](../core-data-structures/approval.md) · [ApprovalRequest](../core-data-structures/approval.md) +Types: [ApprovalOutcome](../core-data-structures/approval.md) · [ApprovalRequest](../core-data-structures/approval.md) · [ApprovalService](../core-data-structures/approval.md) · [Scoped](../core-data-structures/scope.md) Source: [`packages/ui/user-approval/src/index.ts:31`](../../packages/ui/user-approval/src/index.ts) @@ -426,7 +428,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t 'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable ``` -Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) +Types: [GenerateOptions](../core-data-structures/core.md) · [LlmService](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md) Source: [`packages/llm/llm/src/index.ts:40`](../../packages/llm/llm/src/index.ts) @@ -451,6 +453,8 @@ Creation announcement during session publication. A synchronous throw vetoes and 'session/created'(this: Scoped, session: Session): void ``` +Types: [Scoped](../core-data-structures/scope.md) · [Session](../core-data-structures/session.md) + Source: [`packages/core/session/src/index.ts:47`](../../packages/core/session/src/index.ts) ### `session/disposed` — emit @@ -470,6 +474,8 @@ Emitted once when an announced session leaves the store, including publication r 'session/disposed'(this: Scoped, session: Session): void ``` +Types: [Scoped](../core-data-structures/scope.md) · [Session](../core-data-structures/session.md) + Source: [`packages/core/session/src/index.ts:57`](../../packages/core/session/src/index.ts) ### `session/event` — emit @@ -491,7 +497,7 @@ Post-commit, fire-and-forget append feed. The listener snapshot resolves before 'session/event'(this: Scoped, session: Session, event: SessionEvent): void ``` -Types: [SessionEvent](../core-data-structures/core.md) +Types: [Scoped](../core-data-structures/scope.md) · [Session](../core-data-structures/session.md) · [SessionEvent](../core-data-structures/core.md) Source: [`packages/core/session/src/index.ts:69`](../../packages/core/session/src/index.ts) @@ -512,6 +518,8 @@ Awaited parallel durability checkpoint: every listener runs and the caller await 'session/flush'(this: Scoped, session: Session): Promise | void ``` +Types: [Scoped](../core-data-structures/scope.md) · [Session](../core-data-structures/session.md) + Source: [`packages/core/session/src/index.ts:79`](../../packages/core/session/src/index.ts) ## `subagent/*` @@ -532,6 +540,8 @@ A ready child settled. Scope-filtered dispatch uses the same delegating parent c 'subagent/end'(this: Scoped, info: SubagentRunEndInfo): void ``` +Types: [Scoped](../core-data-structures/scope.md) · [SubagentService](../core-data-structures/subagent.md) + Source: [`packages/subagent/subagent/src/index.ts:112`](../../packages/subagent/subagent/src/index.ts) ### `subagent/provider-added` — emit @@ -547,6 +557,8 @@ A provider became resolvable in the registry. 'subagent/provider-added'(provider: SubagentProvider): void ``` +Types: [SubagentProvider](../core-data-structures/subagent.md) + Source: [`packages/subagent/subagent/src/index.ts:86`](../../packages/subagent/subagent/src/index.ts) ### `subagent/provider-removed` — emit @@ -582,6 +594,8 @@ A provider established a ready child. For in-process providers, `ctx.agents.get( 'subagent/start'(this: Scoped, info: SubagentRunInfo): void ``` +Types: [Scoped](../core-data-structures/scope.md) · [SubagentService](../core-data-structures/subagent.md) + Source: [`packages/subagent/subagent/src/index.ts:103`](../../packages/subagent/subagent/src/index.ts) ## `system-prompt/*` @@ -602,6 +616,8 @@ Expert waterfall over the assembled sections, tools, and variables. Scope-filter 'system-prompt/assemble'(this: Scoped, assembly: PromptAssembly, context: AssembleContext, next: () => Promise): Promise ``` +Types: [AssembleContext](../core-data-structures/system-prompt.md) · [Scoped](../core-data-structures/scope.md) · [SystemPrompt](../core-data-structures/system-prompt.md) + Source: [`packages/core/system-prompt/src/index.ts:27`](../../packages/core/system-prompt/src/index.ts) ### `system-prompt/change` — emit @@ -656,7 +672,7 @@ Around-dispatch waterfall for timeout, retry, or metrics. `next()` returns a nor 'tools/execute'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise ``` -Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) +Types: [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:89`](../../packages/core/tools/src/index.ts) @@ -676,7 +692,7 @@ Accept, replace, enrich, or block a normalized dispatch result. `next()` accepts 'tools/post-execute'(this: Scoped, exec: ToolExecution, result: Readonly, next: () => Promise): Promise ``` -Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) +Types: [PostToolDecision](../core-data-structures/tools.md) · [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:98`](../../packages/core/tools/src/index.ts) @@ -695,7 +711,7 @@ Allow, deny, or ask before dispatch. `next()` delegates to allow; missing approv 'tools/pre-execute'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise ``` -Types: [ToolExecution](../core-data-structures/tools.md) +Types: [PreToolDecision](../core-data-structures/tools.md) · [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:80`](../../packages/core/tools/src/index.ts) @@ -714,7 +730,7 @@ Observe the frozen, lossless-JSON final outcome. Listener failures are contained 'tools/result'(this: Scoped, exec: Readonly, result: Readonly): undefined ``` -Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) +Types: [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:106`](../../packages/core/tools/src/index.ts) @@ -738,6 +754,8 @@ One `agent()` call settled (clean result, child failure, or run cancellation). P 'workflow/agent-end'(info: WorkflowRunInfo, agent: WorkflowAgentEndInfo): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:81`](../../packages/workflow/workflow/src/index.ts) ### `workflow/agent-start` — emit @@ -757,6 +775,8 @@ One `agent()` call established a ready child run. Paired with Events['workflow/a 'workflow/agent-start'(info: WorkflowRunInfo, agent: WorkflowAgentInfo): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:70`](../../packages/workflow/workflow/src/index.ts) ### `workflow/end` — emit @@ -776,6 +796,8 @@ A workflow run settled (any stop reason). Fired when WorkflowRun.result resolves 'workflow/end'(info: WorkflowRunInfo, result: WorkflowResultInfo): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:91`](../../packages/workflow/workflow/src/index.ts) ### `workflow/log` — emit @@ -792,6 +814,8 @@ The script emitted a narration line (a `log(message)` call). 'workflow/log'(info: WorkflowRunInfo, message: string): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:60`](../../packages/workflow/workflow/src/index.ts) ### `workflow/phase` — emit @@ -809,6 +833,8 @@ The script entered a phase (a `phase(title)` call) — progress grouping for obs 'workflow/phase'(info: WorkflowRunInfo, title: string): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:53`](../../packages/workflow/workflow/src/index.ts) ### `workflow/start` — emit @@ -825,6 +851,8 @@ A workflow run started — the script's meta block validated, the body about to 'workflow/start'(info: WorkflowRunInfo): void ``` +Types: [WorkflowRunInfo](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:45`](../../packages/workflow/workflow/src/index.ts) ## Inherited events (cordis core + loader/hmr/timer) diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 78ec4c82c0..02207c52dd 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -42,7 +42,7 @@ async createAgent(ownerCtx: Context, options: CreateAgentOptions): Promise ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [AgentOptions](../core-data-structures/core.md) · [SessionHeader](../core-data-structures/persistence.md) · [SessionId](../core-data-structures/core.md) Source: [`packages/core/agent-loop/src/index.ts:407`](../../packages/core/agent-loop/src/index.ts) @@ -163,7 +163,7 @@ list(): Agent[] roots(): Agent[] ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [SessionId](../core-data-structures/core.md) Source: [`packages/core/agent/src/index.ts:201`](../../packages/core/agent/src/index.ts) @@ -233,7 +233,7 @@ abstract run(spec: BashExecSpec): Promise abstract start(spec: BashExecSpec): BashProcess ``` -Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) +Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashProcess](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) Source: [`packages/bash/bash/src/index.ts:49`](../../packages/bash/bash/src/index.ts) @@ -264,7 +264,7 @@ collect(execution: ToolExecution): DshEnvironment list(): BashEnvVariableInfo[] ``` -Types: [ToolExecution](../core-data-structures/tools.md) +Types: [DshEnvironment](../core-data-structures/bash.md) · [ToolExecution](../core-data-structures/tools.md) Source: [`packages/bash/tool-bash/src/index.ts:102`](../../packages/bash/tool-bash/src/index.ts) @@ -331,7 +331,7 @@ abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, abstract compactRegion( start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise ``` -Types: [Message](../core-data-structures/core.md) +Types: [CompactionResult](../core-data-structures/compaction.md) · [Message](../core-data-structures/core.md) Source: [`packages/compact/compact/src/index.ts:38`](../../packages/compact/compact/src/index.ts) @@ -427,7 +427,7 @@ abstract writeText(target: FsTarget, content: string, expected?: FsWriteIntent, abstract editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise ``` -Types: [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsInfo](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) +Types: [FsDirEntry](../core-data-structures/filesystem.md) · [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsInfo](../core-data-structures/filesystem.md) · [FsPathInfo](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) Source: [`packages/fs/fs/src/index.ts:80`](../../packages/fs/fs/src/index.ts) @@ -472,7 +472,7 @@ async listModels(provider: string): Promise stream(options: GenerateOptions): AsyncIterable ``` -Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) +Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) Source: [`packages/llm/llm/src/index.ts:94`](../../packages/llm/llm/src/index.ts) @@ -516,7 +516,7 @@ optionOf(name: string): PresetOption set(session: Session, name: string): void ``` -Types: [SessionEvent](../core-data-structures/core.md) +Types: [Session](../core-data-structures/session.md) · [SessionEvent](../core-data-structures/core.md) Source: [`packages/ui/permission/src/index.ts:94`](../../packages/ui/permission/src/index.ts) @@ -594,7 +594,7 @@ abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEven abstract list(): Promise ``` -Types: [SessionEvent](../core-data-structures/core.md) +Types: [SessionEvent](../core-data-structures/core.md) · [SessionHeader](../core-data-structures/persistence.md) · [SessionId](../core-data-structures/core.md) · [SessionLocation](../core-data-structures/persistence.md) Source: [`packages/session-persistence/session-persistence/src/index.ts:42`](../../packages/session-persistence/session-persistence/src/index.ts) @@ -640,6 +640,8 @@ async traceEvent(request: SessionEventTraceRequest): Promise async readEvent(request: SessionEventReadRequest): Promise ``` +Types: [SessionEventReadRequest](../core-data-structures/session-query.md) · [SessionEventRecord](../core-data-structures/session-query.md) · [SessionEventTrace](../core-data-structures/session-query.md) · [SessionEventTraceRequest](../core-data-structures/session-query.md) · [SessionEventWindow](../core-data-structures/session-query.md) · [SessionId](../core-data-structures/core.md) · [SessionLineageTrace](../core-data-structures/session-query.md) · [SessionRecord](../core-data-structures/session-query.md) + Source: [`packages/session-query/session-query/src/index.ts:38`](../../packages/session-query/session-query/src/index.ts) ## `ctx.sessions` — `SessionStore` @@ -765,6 +767,8 @@ list(): Session[] fork(source: SessionForkSource, boundary?: number, childSessionId?: SessionId): Session ``` +Types: [CreateSessionOptions](../core-data-structures/persistence.md) · [Session](../core-data-structures/session.md) · [SessionId](../core-data-structures/core.md) + Source: [`packages/core/session/src/index.ts:577`](../../packages/core/session/src/index.ts) ## `ctx.skills` — `SkillService` @@ -811,6 +815,8 @@ async list(options: SkillLookupOptions = {}): Promise async get(name: string, options: SkillLookupOptions = {}): Promise ``` +Types: [SkillDefinition](../core-data-structures/skills.md) · [SkillLookupOptions](../core-data-structures/skills.md) · [SkillProvider](../core-data-structures/skills.md) · [SkillRegistration](../core-data-structures/skills.md) · [SkillSummary](../core-data-structures/skills.md) + Source: [`packages/skill/skill/src/index.ts:141`](../../packages/skill/skill/src/index.ts) ## `ctx.spillStore` — `SpillStore` (abstract seam) @@ -832,6 +838,8 @@ Semantics every implementation must honor: abstract saveText(input: SaveTextSpill): Promise ``` +Types: [SaveTextSpill](../core-data-structures/spill.md) · [SpillRef](../core-data-structures/spill.md) + Source: [`packages/spill/spill/src/index.ts:45`](../../packages/spill/spill/src/index.ts) ## `ctx.subagents` — `SubagentService` @@ -873,6 +881,8 @@ list(): string[] async start(name: string, request: SubagentStartRequest): Promise ``` +Types: [SubagentProvider](../core-data-structures/subagent.md) · [SubagentRun](../core-data-structures/subagent.md) · [SubagentStartRequest](../core-data-structures/subagent.md) + Source: [`packages/subagent/subagent/src/index.ts:153`](../../packages/subagent/subagent/src/index.ts) ## `ctx.systemPrompt` — `SystemPrompt` @@ -919,6 +929,8 @@ variable(name: string, provider: (context: AssembleContext) => string | undefine async assemble(context: AssembleContext = {}): Promise ``` +Types: [AssembleContext](../core-data-structures/system-prompt.md) · [PromptSection](../core-data-structures/system-prompt.md) · [ToolProviderResult](../core-data-structures/system-prompt.md) + Source: [`packages/core/system-prompt/src/index.ts:209`](../../packages/core/system-prompt/src/index.ts) ## `ctx.tasks` — `TaskService` @@ -1006,7 +1018,7 @@ onTaskDone(listener: TaskDoneListener): () => void attachSurface(name: string): () => void ``` -Types: [Agent](../core-data-structures/core.md) +Types: [Agent](../core-data-structures/core.md) · [TaskDoneListener](../core-data-structures/tasks.md) · [TaskId](../core-data-structures/tasks.md) · [TaskRead](../core-data-structures/tasks.md) · [TaskSnapshot](../core-data-structures/tasks.md) · [TaskStart](../core-data-structures/tasks.md) Source: [`packages/tasks/tasks/src/index.ts:76`](../../packages/tasks/tasks/src/index.ts) @@ -1041,7 +1053,7 @@ measure(session: Session, requestHeader?: EpochHeader): TokenMeasurement estimateMessage(message: Message): number ``` -Types: [Message](../core-data-structures/core.md) +Types: [EpochHeader](../core-data-structures/session.md) · [Message](../core-data-structures/core.md) · [Session](../core-data-structures/session.md) · [TokenMeasurement](../core-data-structures/token-meter.md) Source: [`packages/llm/token-meter/src/index.ts:106`](../../packages/llm/token-meter/src/index.ts) @@ -1119,7 +1131,7 @@ executionMode(exec: ToolExecutionInput): ToolExecutionMode async execute(exec: ToolExecutionInput): Promise ``` -Types: [ToolDefinition](../core-data-structures/tools.md) · [ToolExecutionInput](../core-data-structures/tools.md) · [ToolExecutionMode](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) +Types: [ScopeKey](../core-data-structures/scope.md) · [ToolDefinition](../core-data-structures/tools.md) · [ToolExecutionInput](../core-data-structures/tools.md) · [ToolExecutionMode](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolGuard](../core-data-structures/tools.md) · [ToolRestriction](../core-data-structures/tools.md) · [ToolSchema](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:438`](../../packages/core/tools/src/index.ts) @@ -1145,6 +1157,8 @@ registerProvider(provider: UserInteractionProvider): () => void async ask(request: AskUserQuestionRequest): Promise ``` +Types: [AskUserQuestionAnswer](../core-data-structures/user-interaction.md) · [AskUserQuestionRequest](../core-data-structures/user-interaction.md) · [UserInteractionProvider](../core-data-structures/user-interaction.md) + Source: [`packages/ui/user-interaction/src/index.ts:82`](../../packages/ui/user-interaction/src/index.ts) ## `ctx.web` — `WebService` @@ -1201,6 +1215,8 @@ async search(request: WebSearchRequest, signal?: AbortSignal): Promise ``` +Types: [WebFetchProvider](../core-data-structures/web.md) · [WebFetchRequest](../core-data-structures/web.md) · [WebFetchResult](../core-data-structures/web.md) · [WebSearchProvider](../core-data-structures/web.md) · [WebSearchRequest](../core-data-structures/web.md) · [WebSearchResult](../core-data-structures/web.md) + Source: [`packages/web/web/src/index.ts:74`](../../packages/web/web/src/index.ts) ## `ctx.workflows` — `WorkflowService` (abstract seam) @@ -1217,6 +1233,8 @@ Workflow execution seam. Invalid requests throw before publication; a live run i abstract start(request: WorkflowStartRequest): WorkflowRun ``` +Types: [WorkflowRun](../core-data-structures/workflow.md) · [WorkflowStartRequest](../core-data-structures/workflow.md) + Source: [`packages/workflow/workflow/src/index.ts:159`](../../packages/workflow/workflow/src/index.ts) ## Inherited `ctx` members (cordis core + loader/hmr/timer) diff --git a/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md b/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md index c4c57d36d6..879f8de701 100644 --- a/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md +++ b/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md @@ -20,7 +20,7 @@ Specific choices: - **`@mode` tag, cross-checked.** Each harness event's JSDoc carries an explicit `@mode emit|waterfall|parallel|serial` tag; the generator hard-errors on a missing tag. Where the signature shape is conclusive — a trailing `next: () => …` parameter is structurally a waterfall — it asserts the tag agrees and hard-errors on a contradiction. The emit/parallel/serial distinction is not structurally visible (`session/flush` returns `Promise | void` with no `next`, as does the ordered `agent/pre-step` checkpoint), so it is trusted from the tag. The authoring rule lives in [AGENTS.md](../../../../AGENTS.md). - **Tiered scope.** The harness tier (the 8 `@deepseek-ai/dsh-*` services + their events) is rendered in full from source. The inherited tier (cordis-core `ctx.on/emit/effect/provide/…` + the `internal/*` events + loader/hmr/timer) is pinned vendor source a plugin also sees; it is rendered tersely (name + one-line + source pointer) from a curated table in the generator, NOT walked from the vendor AST — the cordis-core `Context` mixes true ctx members with non-service fields (`root`, `baseUrl`, `logger`), and the vendor surface changes only on a deliberate vendor sync. -- **Cross-links to the data-structure catalog.** A type name in a signature (`GenerateOptions`, `StreamChunk`, `ToolDefinition`, …) links to the core-data-structures page that documents it. The map is a small hand-curated const in the generator — NOT `type-equiv.manifest.json`, which documents the `…Map` symbols while signatures reference the derived union names, and lists a few symbols on two pages. +- **Cross-links to the data-structure catalog.** Every repository-owned type name in a signature (`GenerateOptions`, `StreamChunk`, `ToolDefinition`, …) links to its primary core-data-structures page through a curated map. The AST walk is fail-closed: each parameter, generic constraint/default, and return-type reference must be mapped, be the signature's own type parameter, be a named TypeScript/Cordis foundation type, or carry a named exception with its non-catalog documentation owner. Violations aggregate with source pointers and name the appropriate owning lists. The map does NOT reuse `type-equiv.manifest.json`, which documents `…Map` symbols while signatures reference derived union names and lists some symbols on multiple pages. - **A dedicated fence.** Signature blocks use a ` ```ts cordis-catalog ` info string and place the original event or public-method JSDoc immediately before its declaration. `doc-typecheck` recognizes and skips the bare fragments, excluding them from the opt-out ratio — the same treatment `type-equiv` blocks get. This **supersedes the event-taxonomy half** of [doc-sync enforcement](2026-06-11-doc-sync-enforcement.md): `verify-event-taxonomy` and its `docs/architecture.md` table are retired (the architecture.md heading stays, its body now points at the catalog; the Service-map role table stays as curated prose). doc-typecheck, verify-md-wrap, verify-md-links, and verify-type-equiv are unchanged. @@ -29,11 +29,11 @@ This **supersedes the event-taxonomy half** of [doc-sync enforcement](2026-06-11 - **Verify-don't-generate, as the retired taxonomy check did** — reversed *for this surface only*: the data here is mechanically complete, so generation is strictly stronger (full signatures, cannot drift, catches undocumented events) than a name-set check of a hand-maintained table. - **Walking the vendor AST for the inherited tier** — rejected for the curated table: the cordis-core `Context` mixes true ctx members with non-service fields, and the pinned vendor surface changes only on a deliberate sync. -- **Reusing `type-equiv.manifest.json` as the signature cross-link map** — rejected for a small hand-curated const: the manifest documents the `…Map` symbols while signatures reference the derived union names, and it lists a few symbols on two pages. +- **Reusing `type-equiv.manifest.json` as the signature cross-link map** — rejected for a complete curated const plus fail-closed coverage: the manifest documents `…Map` symbols while signatures reference derived union names, and it lists some symbols on multiple pages. The explicit map makes each rendered destination and each non-catalog exception a reviewable decision. ## Consequences -- The catalog cannot drift: a source change that the committed file doesn't reflect fails `verify-cordis-catalog` in the pre-push hook and CI. A new event with no `@mode` tag, or a tag that contradicts its signature, fails the generator outright. +- The catalog cannot drift: a source change that the committed file doesn't reflect fails `verify-cordis-catalog` in the pre-push hook and CI. A new event with no `@mode` tag, a tag that contradicts its signature, or an unclassified signature type fails the generator outright. - Event and service-method contracts have a single home — the JSDoc at the declaration. The catalog repeats that original JSDoc inside its generated signature block and uses its description portion as entry prose, so thin source documentation yields a thin catalog entry. - The inherited tier is hand-summarized, so a vendor sync that adds/renames a cordis-core event or `ctx` member needs a matching edit to the curated table in `gen-cordis-catalog.ts`. This is the deliberate cost of not walking pinned vendor source; it changes rarely and is called out in the generator. - `verify-event-taxonomy.ts` is deleted and the `docs/architecture.md` event table is gone; anyone who linked to a specific table row now lands on the generated catalog instead. diff --git a/packages/core/agent/tests/gen-cordis-catalog.spec.ts b/packages/core/agent/tests/gen-cordis-catalog.spec.ts index 24841c8c8e..23c558bd95 100644 --- a/packages/core/agent/tests/gen-cordis-catalog.spec.ts +++ b/packages/core/agent/tests/gen-cordis-catalog.spec.ts @@ -1,5 +1,6 @@ /** - * Negative-path tests for the cordis catalog generator (`scripts/gen-cordis-catalog.ts`). + * Contract and negative-path tests for the cordis catalog generator + * (`scripts/gen-cordis-catalog.ts`). */ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs' @@ -76,6 +77,33 @@ describe('gen-cordis-catalog collectEvents', () => { expect(events[0]?.mode).toBe('parallel') }) + it('accepts linked, foundation, generic-parameter, and explicitly exempt signature types', () => { + const events = collectEvents(make( + ' /**\n * Carry linked and foundation types.\n * @param value - the linked value.\n * @param preset - deployment metadata outside the core catalog.\n * @param signal - cancellation.\n * @mode parallel\n */\n \'fix/typed\'(value: Readonly, preset: PresetSpec, signal: AbortSignal): Promise', + )) + expect(events).toHaveLength(1) + expect(renderEvents(events)).toContain('Types: [SessionEvent](../core-data-structures/core.md)') + expect(renderEvents(events)).not.toContain('[PresetSpec]') + }) + + it('aggregates every unclassified signature type with its source and remediation', () => { + const expected = new RegExp([ + '2 signature type-link coverage violation\\(s\\)', + 'fix/one', + 'packages/group/fix/src/index.ts', + 'MissingOne', + 'fix/two', + 'packages/group/fix/src/index.ts', + 'missingTwo', + 'Add it to LINK_MAP', + 'FOUNDATION_TYPE_NAMES', + 'TYPE_LINK_EXEMPTIONS', + ].join('[\\s\\S]*')) + expect(() => collectEvents(make( + ' /**\n * First.\n * @param value - first value.\n * @mode emit\n */\n \'fix/one\'(value: MissingOne): void\n /**\n * Second.\n * @param value - second value.\n * @mode emit\n */\n \'fix/two\'(value: missingTwo): void', + ))).toThrow(expected) + }) + it('hard-errors when an event is missing its @mode tag', () => { expect(() => collectEvents(make( ' /** No mode here. */\n \'fix/untagged\'(): void', @@ -167,6 +195,12 @@ export class FixService { expect(renderServices(services)).toContain('```ts cordis-catalog\n/**\n * Do the thing.\n * @param id - which thing to do.\n * @returns the outcome of doing it.\n */\nrun(id: string): string\n\n/** Fire and forget (void needs no @returns). */\npoke(): void') }) + it('hard-errors on an unclassified service-method signature type', () => { + expect(() => collectServices(makeService( + '/** Fixture service. */\nexport class FixService {\n /**\n * Use an unknown value.\n * @param value - the value.\n */\n run(value: MissingServiceType): void {}\n}', + ))).toThrow(/service method ctx\.fix\.run .* references unclassified type 'MissingServiceType'/) + }) + it('hard-errors on a public method with no JSDoc at all', () => { expect(() => collectServices(makeService( '/** Fixture service. */\nexport class FixService {\n run(id: string): string { return id }\n}', diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 08f04947da..5176c8e330 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -1,8 +1,8 @@ /** * Generate the Cordis event and service catalogs from static declarations. - * The walk enforces event modes plus JSDoc parameter/return completeness; - * inherited Cordis services come from the curated table below. `--check` - * verifies both committed artifacts. + * The walk enforces event modes, JSDoc parameter/return completeness, and + * signature type-link coverage; inherited Cordis services come from the + * curated table below. `--check` verifies both committed artifacts. */ import { globSync, readFileSync, writeFileSync } from 'node:fs' @@ -20,48 +20,196 @@ const OUT_SERVICES = 'docs/cordis-catalog/services.md' const FENCE = 'ts cordis-catalog' /** - * One primary core-data-structures page per signature type, shared by the - * Cordis and config catalogs; union names intentionally do not reuse the - * type-equivalence manifest's map-symbol entries. + * One primary core-data-structures page per project type used by a generated + * signature. This stays curated because union names intentionally do not + * reuse the type-equivalence manifest's map-symbol entries and some symbols + * appear on more than one page. */ -// TODO(catalog-type-links): verify or generate link-map coverage. export const LINK_MAP: Record = { Agent: 'core.md', + AgentOptions: 'core.md', + AgentStatus: 'core.md', ContentBlock: 'core.md', - Message: 'core.md', - MessageSource: 'core.md', + ContinuationDecision: 'core.md', + ContinuationStop: 'core.md', GenerateOptions: 'core.md', LlmCallConfig: 'core.md', + LlmModelInfo: 'core.md', + LlmProviderInfo: 'core.md', + Message: 'core.md', + MessageSource: 'core.md', + PromptDecision: 'core.md', SessionEvent: 'core.md', + SessionId: 'core.md', SessionStartSource: 'core.md', - StreamChunk: 'llm-streaming.md', - TurnEndReason: 'session.md', - ToolDefinition: 'tools.md', - ToolExecution: 'tools.md', - ToolExecutionMode: 'tools.md', - ToolExecutionInput: 'tools.md', - ToolExecutionResult: 'tools.md', - ToolExecutionToken: 'tools.md', ApprovalOutcome: 'approval.md', ApprovalPolicy: 'approval.md', ApprovalRequest: 'approval.md', + ApprovalService: 'approval.md', BashExecRequest: 'bash.md', BashExecSpec: 'bash.md', + BashProcess: 'bash.md', BashRunResult: 'bash.md', - ConfinedArgv: 'sandbox.md', - SandboxMode: 'sandbox.md', - SandboxPolicy: 'sandbox.md', + DshEnvironment: 'bash.md', CodeRunRequest: 'code-runtime.md', CodeRunResult: 'code-runtime.md', + CompactionResult: 'compaction.md', + FileReadOutcome: 'filesystem.md', + FsDirEntry: 'filesystem.md', FsEditOutcome: 'filesystem.md', FsEditRequest: 'filesystem.md', FsInfo: 'filesystem.md', + FsPathInfo: 'filesystem.md', + FsPolicyExec: 'filesystem.md', FsTarget: 'filesystem.md', FsVersion: 'filesystem.md', FsWriteIntent: 'filesystem.md', FsWriteOutcome: 'filesystem.md', - FsPolicyExec: 'filesystem.md', - FileReadOutcome: 'filesystem.md', + LlmAdapter: 'llm-streaming.md', + LlmService: 'llm-streaming.md', + StreamChunk: 'llm-streaming.md', + CreateSessionOptions: 'persistence.md', + SessionHeader: 'persistence.md', + SessionLocation: 'persistence.md', + ConfinedArgv: 'sandbox.md', + SandboxMode: 'sandbox.md', + SandboxPolicy: 'sandbox.md', + ScopeKey: 'scope.md', + Scoped: 'scope.md', + EpochHeader: 'session.md', + Session: 'session.md', + TurnEndReason: 'session.md', + SessionEventReadRequest: 'session-query.md', + SessionEventRecord: 'session-query.md', + SessionEventTrace: 'session-query.md', + SessionEventTraceRequest: 'session-query.md', + SessionEventWindow: 'session-query.md', + SessionLineageTrace: 'session-query.md', + SessionRecord: 'session-query.md', + SkillDefinition: 'skills.md', + SkillLookupOptions: 'skills.md', + SkillProvider: 'skills.md', + SkillRegistration: 'skills.md', + SkillSummary: 'skills.md', + SaveTextSpill: 'spill.md', + SpillRef: 'spill.md', + SubagentProvider: 'subagent.md', + SubagentRun: 'subagent.md', + SubagentService: 'subagent.md', + SubagentStartRequest: 'subagent.md', + AssembleContext: 'system-prompt.md', + PromptSection: 'system-prompt.md', + SystemPrompt: 'system-prompt.md', + ToolProviderResult: 'system-prompt.md', + TaskDoneListener: 'tasks.md', + TaskId: 'tasks.md', + TaskRead: 'tasks.md', + TaskSnapshot: 'tasks.md', + TaskStart: 'tasks.md', + TokenMeasurement: 'token-meter.md', + PostToolDecision: 'tools.md', + PreToolDecision: 'tools.md', + ToolDefinition: 'tools.md', + ToolExecution: 'tools.md', + ToolExecutionInput: 'tools.md', + ToolExecutionMode: 'tools.md', + ToolExecutionResult: 'tools.md', + ToolExecutionToken: 'tools.md', + ToolGuard: 'tools.md', + ToolRegistry: 'tools.md', + ToolRestriction: 'tools.md', + ToolSchema: 'tools.md', + AskUserQuestionAnswer: 'user-interaction.md', + AskUserQuestionRequest: 'user-interaction.md', + UserInteractionProvider: 'user-interaction.md', + WebFetchProvider: 'web.md', + WebFetchRequest: 'web.md', + WebFetchResult: 'web.md', + WebSearchProvider: 'web.md', + WebSearchRequest: 'web.md', + WebSearchResult: 'web.md', + WorkflowRun: 'workflow.md', + WorkflowRunInfo: 'workflow.md', + WorkflowStartRequest: 'workflow.md', +} + +/** TypeScript lib and pinned framework types that have no repository-owned data page. */ +const FOUNDATION_TYPE_NAMES = new Set([ + 'AbortSignal', + 'AsyncIterable', + 'Context', + 'Error', + 'Pick', + 'Promise', + 'Readonly', +]) + +/** Project types deliberately documented outside the core-data catalog. */ +const TYPE_LINK_EXEMPTIONS: Readonly> = { + AgentFactory: 'agent creation seam is owned by packages/core/agent/README.md', + AgentHandle: 'agent ownership handle is owned by packages/core/agent/README.md', + BashEnvContributor: 'service-local extension type is owned by packages/bash/tool-bash/src/index.ts', + BashEnvVariableInfo: 'service-local metadata type is owned by packages/bash/tool-bash/src/index.ts', + CompactAgentContext: 'compaction service input is owned by packages/compact/compact/src/index.ts', + CreateAgentOptions: 'agent creation contract is owned by packages/core/agent/README.md', + PresetOption: 'deployment menu metadata is owned by packages/ui/permission/README.md', + PresetSpec: 'deployment preset composition is owned by packages/ui/permission/README.md', + PromptAssembly: 'assembly result is owned by packages/core/system-prompt/README.md', + ResumeAgentOptions: 'agent resume contract is owned by packages/core/agent/README.md', + SessionForkSource: 'service-local fork input is owned by packages/core/session/src/index.ts', + SubagentRunEndInfo: 'event-local snapshot is owned by packages/subagent/subagent/src/index.ts', + SubagentRunInfo: 'event-local snapshot is owned by packages/subagent/subagent/src/index.ts', + WorkflowAgentEndInfo: 'event-local snapshot is owned by packages/workflow/workflow/src/index.ts', + WorkflowAgentInfo: 'event-local snapshot is owned by packages/workflow/workflow/src/index.ts', + WorkflowResultInfo: 'event-local snapshot is owned by packages/workflow/workflow/src/index.ts', +} + +/** Collect named references from parameter, generic-constraint/default, and return types. */ +function signatureTypeNames(member: ts.MethodSignature | ts.MethodDeclaration, sf: ts.SourceFile): string[] { + const declared = new Set(member.typeParameters?.map(parameter => parameter.name.text) ?? []) + const referenced = new Set() + const visit = (node: ts.Node): void => { + if (ts.isTypeReferenceNode(node)) referenced.add(node.typeName.getText(sf)) + if (ts.isTypeQueryNode(node)) referenced.add(node.exprName.getText(sf)) + ts.forEachChild(node, visit) + } + for (const parameter of member.typeParameters ?? []) { + if (parameter.constraint) visit(parameter.constraint) + if (parameter.default) visit(parameter.default) + } + for (const parameter of member.parameters) { + if (parameter.type) visit(parameter.type) + } + if (member.type) visit(member.type) + return [...referenced].filter(name => !declared.has(name)).sort() +} + +/** Append fail-closed signature type-link violations with actionable ownership choices. */ +function checkTypeLinks( + where: string, + member: ts.MethodSignature | ts.MethodDeclaration, + sf: ts.SourceFile, + violations: string[], +): void { + for (const name of signatureTypeNames(member, sf)) { + if (Object.hasOwn(LINK_MAP, name) + || FOUNDATION_TYPE_NAMES.has(name) + || Object.hasOwn(TYPE_LINK_EXEMPTIONS, name)) continue + violations.push( + `${where} references unclassified type '${name}'. Add it to LINK_MAP with its core-data-structures page, ` + + 'to FOUNDATION_TYPE_NAMES if TypeScript or Cordis owns it, or to TYPE_LINK_EXEMPTIONS with ' + + 'the non-catalog documentation owner.', + ) + } +} + +/** Throw one aggregated diagnostic for every unclassified signature type. */ +function reportTypeLinkViolations(gate: string, violations: string[]): void { + if (violations.length === 0) return + throw new Error( + `${gate}: ${violations.length} signature type-link coverage violation(s):\n` + + violations.map(violation => ` ${violation}`).join('\n'), + ) } /** One harness event, extracted from an `interface Events` block. */ @@ -148,6 +296,7 @@ function jsDocText(text: string, sf: ts.SourceFile, node: ts.Node): string { export function collectEvents(scanRoot: string = root): EventEntry[] { const entries: EventEntry[] = [] const violations: string[] = [] + const typeLinkViolations: string[] = [] for (const rel of globSync('packages/*/*/src/*.ts', { cwd: scanRoot }).map(s => s.split(sep).join('/')).sort()) { const abs = resolve(scanRoot, rel) const text = readFileSync(abs, 'utf8') @@ -161,6 +310,7 @@ export function collectEvents(scanRoot: string = root): EventEntry[] { const { doc, mode } = parseJsDoc(raw) const src = pointer(rel, sf, member) const where = `event '${name}' (${src})` + checkTypeLinks(where, member, sf, typeLinkViolations) if (!mode) { violations.push(`${where} is missing an @mode tag. Add '@mode emit|waterfall|parallel|serial' to its JSDoc (see AGENTS.md).`) } @@ -185,6 +335,7 @@ export function collectEvents(scanRoot: string = root): EventEntry[] { } } reportViolations('gen-cordis-catalog', violations) + reportTypeLinkViolations('gen-cordis-catalog', typeLinkViolations) return entries } @@ -197,6 +348,7 @@ export function collectEvents(scanRoot: string = root): EventEntry[] { export function collectServices(scanRoot: string = root): ServiceEntry[] { const entries: ServiceEntry[] = [] const violations: string[] = [] + const typeLinkViolations: string[] = [] for (const rel of globSync('packages/*/*/src/index.ts', { cwd: scanRoot }).map(s => s.split(sep).join('/')).sort()) { const abs = resolve(scanRoot, rel) const text = readFileSync(abs, 'utf8') @@ -220,6 +372,7 @@ export function collectServices(scanRoot: string = root): ServiceEntry[] { const memberName = member.name.getText(sf) if (memberName.startsWith('[')) continue // computed/symbol members const where = `service method ctx.${key}.${memberName} (${pointer(rel, sf, member)})` + checkTypeLinks(where, member, sf, typeLinkViolations) const raw = rawJsDoc(text, member) methods.push({ signature: memberSignature(member, sf), jsDoc: jsDocText(text, sf, member) }) if (!raw) { violations.push(`${where} has no JSDoc.`); continue } @@ -243,6 +396,7 @@ export function collectServices(scanRoot: string = root): ServiceEntry[] { } } reportViolations('gen-cordis-catalog', violations) + reportTypeLinkViolations('gen-cordis-catalog', typeLinkViolations) return entries.sort((a, b) => a.key.localeCompare(b.key)) } From 98e224e45e04308b99751eae4bc227df1c2ba763 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 15:02:11 +0800 Subject: [PATCH 46/73] refactor(agent): exhaust recovery and compaction decisions Dispatch RequestErrorDecision and CompactionTrigger through explicit discriminant switches. End each closed union with assertNever so new variants fail compilation instead of silently inheriting fail or pressure behavior. This preserves the current retry, fail, pressure, and overflow semantics while aligning the new recovery seams with the repository closed-union contract. --- packages/compact/compact-basic/src/index.ts | 19 +++++++++++++------ packages/core/agent-loop/src/loop.ts | 16 +++++++++++----- 2 files changed, 24 insertions(+), 11 deletions(-) diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index b77ebebc76..5d325d57ba 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -9,7 +9,7 @@ import z from 'schemastery' import { CompactService } from '@deepseek-ai/dsh-compact' import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact' import type { Session } from '@deepseek-ai/dsh-session' -import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' +import { CONTEXT_WINDOW_EXCEEDED_CODE, assertNever } from '@deepseek-ai/dsh-llm' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { Agent } from '@deepseek-ai/dsh-agent' import { resolveConfig } from './config.ts' @@ -152,11 +152,18 @@ export class BasicCompactService extends CompactService { const model = routedModel(agent.session) if (model === undefined) return null const meter = this.ctx.tokenMeter - if (trigger === 'context-overflow') { - const measurement = meter.measure(agent.session) - const range = selectCompactableRange(agent.session, measurement, 0) - if (range === null) return null - return this.compactRegion(range.start, range.end, agent, signal) + switch (trigger) { + case 'context-overflow': { + const measurement = meter.measure(agent.session) + const range = selectCompactableRange(agent.session, measurement, 0) + if (range === null) return null + return this.compactRegion(range.start, range.end, agent, signal) + } + case 'pressure': + break + /* v8 ignore next -- closed-union exhaustiveness guard */ + default: + assertNever(trigger, 'compaction trigger') } const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio) diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 54cbbc6bdd..08c1ab9412 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -8,7 +8,7 @@ import type { Context } from 'cordis' import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, Message } from '@deepseek-ai/dsh-llm' import { isDeepStrictEqual } from 'node:util' -import { BlockAssembler, HarnessError, deepFreeze, isLlmAdapterFailure } from '@deepseek-ai/dsh-llm' +import { BlockAssembler, HarnessError, assertNever, deepFreeze, isLlmAdapterFailure } from '@deepseek-ai/dsh-llm' import { agentEvents, assembleContextFor } from '@deepseek-ai/dsh-agent' import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent' import { canonicalHeader } from '@deepseek-ai/dsh-session' @@ -392,11 +392,17 @@ async function runTurn( : { kind: 'aborted', reason: String(abort.signal.reason) } break } - if (recoveryDecision.action === 'retry') { - requestRetryAttempt += 1 - continue + switch (recoveryDecision.action) { + case 'retry': + requestRetryAttempt += 1 + continue + case 'fail': + failTurn(stepOutcome.requestError) + break + /* v8 ignore next -- closed-union exhaustiveness guard */ + default: + assertNever(recoveryDecision, 'agent request-error decision') } - failTurn(stepOutcome.requestError) break } From 8eb9beb31909ae83639c9b5c5df1664c996be0b5 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 15:03:15 +0800 Subject: [PATCH 47/73] docs(rfc): correct recovery translation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Use the repository-mandated 回放 terminology for replayable pressure and replace the literal cancellation-owned rendering with idiomatic Chinese that preserves boundedness, cancellation authority, and monotonic retry behavior. The corresponding English clauses remain accurate. Re-record the English and Chinese pair after checking the corrected text against the implemented recovery contract. --- ...r-call-compaction-pressure-and-overflow-recovery.i18n.yaml | 2 +- ...after-call-compaction-pressure-and-overflow-recovery.zh.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml index 9f3f9fcebf..aa45a7ccb2 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write 2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: abc374d98fed8cac039f10d930fec332515c846e -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 5f6661c9fdc905048aaec56d9b0831a9a5a4548e +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: ef13c64fc4c728e5b4fe9718ae8ef8b8ef9ecdd9 diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md index 5f6661c9fd..ef13c64fc4 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md @@ -8,7 +8,7 @@ Status: implemented `agent/pre-step` 运行在最终请求路由之前，也早于 assistant 输出、工具结果、缓冲上下文与 steering 的产生。即使它接收已装配提示词与会话前缀，压力视图仍是临时的，因为 `agent/request` 还可以改变路由或调用配置，工具 schema 也没有与这些输入一同冻结。增加字段无法让调用前状态描述已完成调用，还会把通用 seam 与压缩耦合。 -成功调用也不是唯一的压力信号。提供方可能在返回 usage 之前就因上下文窗口超限拒绝请求，一些成功调用也不提供 usage。因此，系统需要可重放的调用后压力，以及一条狭窄的失败恢复路径；当压缩无法证明取得有效进展时，必须保留原始提供方错误。 +成功调用也不是唯一的压力信号。提供方可能在返回 usage 之前就因上下文窗口超限拒绝请求，一些成功调用也不提供 usage。因此，系统需要可回放的调用后压力，以及一条狭窄的失败恢复路径；当压缩无法证明取得有效进展时，必须保留原始提供方错误。 ## 决策 @@ -54,7 +54,7 @@ Status: implemented ## 后果 -Post-step 压力描述已完成的路由请求，包括持久工具结果与仅请求前缀字段。当成功 usage 锚点不存在时，规范化溢出提供兜底路径。恢复有上限、受取消所有，并保持单调：只有模型可见的表层 generation 变化后才重试。 +Post-step 压力描述已完成的路由请求，包括持久工具结果与仅请求前缀字段。当成功 usage 锚点不存在时，规范化溢出提供兜底路径。恢复有明确上限、以取消为准，并保持单调：只有模型可见的表层 generation 变化后才重试。 代价是成功 step 增加一个串行检查点，并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况，也不能拆分单个不可分割的超大消息或工具单元。 From 761796347482382699baa71f36b9c8705c75fc01 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 15:04:51 +0800 Subject: [PATCH 48/73] docs(llm-pi-ai): describe usage overflow mapping Document every context-window path returned by mapStopReason: recognized provider error wording, successful stop usage beyond the resolved window, and zero-output length stops that fill the window. This keeps the public return contract aligned with usage-based overflow classification without duplicating the provider-specific detection table owned by pi-ai. --- packages/llm/llm-pi-ai/src/stream.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index 8bc41cded2..c1a85addf0 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -40,7 +40,9 @@ function classifyPiAiError(message: string): string { * Map a terminal pi-ai event to the harness finish reason. * @param message - the assistant message carried by the `done` or `error` event. * @param contextWindow - resolved catalog capacity for usage-based overflow detection. - * @returns the harness reason; `error` yields `{kind: 'error'}` with a code classified from the error text. + * @returns the mapped harness reason. Recognized error text, `stop` usage above + * `contextWindow`, and zero-output `length` usage that fills the window map + * to `CONTEXT_WINDOW_EXCEEDED`. */ export function mapStopReason(message: AssistantMessage, contextWindow?: number): FinishReason { const piAiOverflow = isContextOverflow(message, contextWindow) From bf93605f8cffea1c977c8957873f5085557b0504 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 15:12:29 +0800 Subject: [PATCH 49/73] docs: keep architecture within its budget Condense the app-bundle sentence after combining the current-master CLI wiring with the recovery architecture changes. The text still identifies the TUI, line-oriented, headless, and ACP front doors and their output contracts. This restores the enforced architecture.md word ceiling without raising the budget or relocating an architecture-level seam fact. --- docs/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture.md b/docs/architecture.md index 415d3b2aee..f65a901315 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -156,7 +156,7 @@ Some seams bend the template deliberately: LLM combines interface and consumer b ### Bundles And Apps -`dsh-agent-spine-demo` bundles the default spine ([README](../packages/examples/agent-spine-demo/README.md)). `dsh-stdio-demo` adds a terminal front door that selects `dsh-tui` for interactive terminals and line-oriented `dsh-stdio` for pipes; `dsh-cli-demo` runs one persisted headless turn with format-pure stdout; and `dsh-acp-demo` adds stdout-pure ACP over JSON-RPC ([ui/](../packages/ui/README.md)). `dsh-jsonrpc-agent` boots external `cordis.yml`; the Python SDK supplies its default only without an explicit config channel and drives `dsh-jsonrpc` over line-delimited JSON-RPC ([Python SDK](../python/README.md)). Deployments remain thin leaves with swappable backends and optional product tools ([examples/](../examples/AGENTS.md), [runnable wirings](cookbook/extension-cookbook.md#runnable-wirings), [graph atlas](graph-atlas.md)). +`dsh-agent-spine-demo` bundles the default spine ([README](../packages/examples/agent-spine-demo/README.md)). `dsh-stdio-demo` selects `dsh-tui` for interactive terminals and line-oriented `dsh-stdio` for pipes; `dsh-cli-demo` runs one persisted headless turn with format-pure stdout; `dsh-acp-demo` adds stdout-pure ACP over JSON-RPC ([ui/](../packages/ui/README.md)). `dsh-jsonrpc-agent` boots external `cordis.yml`; the Python SDK supplies its default only without an explicit config channel and drives `dsh-jsonrpc` over line-delimited JSON-RPC ([Python SDK](../python/README.md)). Deployments remain thin leaves with swappable backends and optional product tools ([examples/](../examples/AGENTS.md), [runnable wirings](cookbook/extension-cookbook.md#runnable-wirings), [graph atlas](graph-atlas.md)). ### Where New Behavior Goes From d2672460fed59aecc8ddcfd7325589d3afb7a8b6 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 15:36:48 +0800 Subject: [PATCH 50/73] fix(cli-demo): accept disabled task controls --- packages/examples/cli-demo/src/index.ts | 2 +- packages/examples/cli-demo/tests/cli-demo.spec.ts | 15 +++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/packages/examples/cli-demo/src/index.ts b/packages/examples/cli-demo/src/index.ts index 1308209681..e5c77af9ed 100644 --- a/packages/examples/cli-demo/src/index.ts +++ b/packages/examples/cli-demo/src/index.ts @@ -61,7 +61,7 @@ export const Config: z = z.object({ toolOrder: z.array(z.string()).default(undefined as unknown as string[]), tools: ToolRegistry.Config, toolBash: agentCore.ToolBashConfigSchema, - toolTasks: agentCore.ToolTasksConfigSchema, + toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), }) /* jscpd:ignore-end */ diff --git a/packages/examples/cli-demo/tests/cli-demo.spec.ts b/packages/examples/cli-demo/tests/cli-demo.spec.ts index 343d6184d7..2111ff6aa8 100644 --- a/packages/examples/cli-demo/tests/cli-demo.spec.ts +++ b/packages/examples/cli-demo/tests/cli-demo.spec.ts @@ -146,6 +146,21 @@ describe('dsh-cli-demo app composition', () => { expect(wait).toHaveBeenCalledWith(id, 7, undefined, undefined) }) + it('accepts false to keep task services without model-facing task controls', async () => { + const ctx = await mount({ + provider: 'mock', + model: 'mock', + skills: { enabled: false }, + toolTasks: false, + workspaceContext: false, + }) + + expect(ctx.get('tasks')).toBeDefined() + expect(ctx.get('tools')?.get('task_output')).toBeUndefined() + expect(ctx.get('tools')?.get('task_list')).toBeUndefined() + expect(ctx.get('tools')?.get('task_kill')).toBeUndefined() + }) + it('exposes the Loader-safe namespace plugin shape and schema', () => { expect(cliDemo.name).toBe('cli-demo') expect(cliDemo.Config).toBeDefined() From ee6b9d081a254279a21452e3f0aa76b3ea907a63 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 16:14:58 +0800 Subject: [PATCH 51/73] fix(llm): scope adapter failures to model calls Replace the process-wide adapter-failure WeakSet with a per-call scope bound to the exact AsyncIterable returned by LlmService.stream(). Give every call a unique wrapper so waterfall middleware can reuse an iterable without sharing provenance. Move agent-loop recovery classification to the model-stream boundary. Only the final adapter behind that exact call can become an agent/request-error; nested llm/stream calls remain ordinary outer middleware failures while preserving the original Error. Cover nested calls, reused middleware iterables, and end-to-end agent-loop recovery. Update the package and RFC contracts, bilingual pairing record, and generated API and catalog references. --- docs/cordis-catalog/events.md | 2 +- docs/cordis-catalog/services.md | 7 +- docs/event-producer-consumer.md | 2 +- ...n-pressure-and-overflow-recovery.i18n.yaml | 4 +- ...mpaction-pressure-and-overflow-recovery.md | 2 +- ...ction-pressure-and-overflow-recovery.zh.md | 2 +- packages/core/agent-loop/src/loop.ts | 24 ++-- .../agent-loop/tests/request-recovery.spec.ts | 36 +++++ packages/llm/llm/README.md | 2 +- packages/llm/llm/src/adapter-failure.ts | 49 +++++-- packages/llm/llm/src/index.ts | 21 ++- packages/llm/llm/tests/service.spec.ts | 124 ++++++++++++++++-- website/zh-CN/api/harness/events.md | 2 +- website/zh-CN/api/harness/llm.md | 12 +- 14 files changed, 233 insertions(+), 56 deletions(-) diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 1acfb39a3d..20a92bffb8 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -471,7 +471,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:42`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:43`](../../packages/llm/llm/src/index.ts) ## `session/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 60b6fcbd4e..d07d0c2164 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -461,8 +461,9 @@ async listModels(provider: string): Promise * `options.provider`. Replay state is retained only when the same adapter * instance owns its historical provider and the target provider. Final * adapter selection, dispatch, and iteration failures retain their original - * Error identity and are tagged for narrow agent-loop request recovery; - * middleware failures remain untagged. + * Error identity and are tagged in a call-local scope for narrow agent-loop + * request recovery; middleware and nested-call failures remain untagged for + * the outer call. * @param options - the full request; `options.provider` selects the adapter. * @returns the chunk stream, possibly wrapped by `llm/stream` listeners. */ @@ -471,7 +472,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:96`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:97`](../../packages/llm/llm/src/index.ts) ## `ctx.permission` — `PermissionService` diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index b10fd76897..5b5a899b44 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -27,7 +27,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:61`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:70`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:53`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:42`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:43`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | | `session/created` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/event` | `emit` | [`packages/core/session/src/index.ts:69`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/ui/acp), [`cli-demo`](../packages/examples/cli-demo), [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio`](../packages/ui/stdio), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`workspace-context`](../packages/context/workspace-context) | diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml index aa45a7ccb2..c54a0b344d 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: abc374d98fed8cac039f10d930fec332515c846e -2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: ef13c64fc4c728e5b4fe9718ae8ef8b8ef9ecdd9 +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: d88d7aaea8ccec30b10bfeb17f1312cfe87a0ce7 +2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 42e6114304de9c8022ef8f1341035858c0c7d9ec diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md index abc374d98f..d88d7aaea8 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md @@ -22,7 +22,7 @@ The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after ### Request recovery is limited to the final model boundary -`RequestError`, `RequestErrorDecision`, and the `agent/request-error` waterfall represent failures after the final adapter has been selected. Private `WeakSet` tagging preserves the original thrown error identity across dispatch, iterator construction, and iteration. Terminal in-band `error` or `aborted` finishes enter the same path. Prompt assembly, request middleware, request logging, result processing, tools, post-step listeners, and cleanup remain ordinary failures. +`RequestError`, `RequestErrorDecision`, and the `agent/request-error` waterfall represent failures after the final adapter has been selected. Each returned stream handle owns a private failure set that preserves the original thrown error identity across dispatch, iterator construction, and iteration without leaking nested-call provenance into an outer call. Terminal in-band `error` or `aborted` finishes enter the same path. Prompt assembly, request middleware, request logging, result processing, tools, post-step listeners, and cleanup remain ordinary failures. The failed step closes before recovery runs. A retry opens the next numbered step and rebuilds the request from the durable log; consecutive recovery attempts reset only after a successful provider request. Both DeepSeek adapters normalize recognized provider context-limit failures to `CONTEXT_WINDOW_EXCEEDED`. diff --git a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md index ef13c64fc4..42e6114304 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md @@ -22,7 +22,7 @@ Status: implemented ### 请求恢复只覆盖最终模型边界 -`RequestError`、`RequestErrorDecision` 与 `agent/request-error` waterfall 表示最终适配器已经选定之后的失败。私有 `WeakSet` 标记在分发、异步迭代器构造与迭代过程中保留原始抛出错误的身份。终止性的带内 `error` 或 `aborted` finish 进入同一路径。提示词装配、请求中间件、请求日志、结果处理、工具、post-step 监听器与清理仍属于普通失败。 +`RequestError`、`RequestErrorDecision` 与 `agent/request-error` waterfall 表示最终适配器已经选定之后的失败。每个返回的流句柄都绑定一个私有失败集合；该集合在分发、异步迭代器构造与迭代过程中保留原始抛出错误的身份，同时防止把嵌套调用的错误来源误归到外层调用。终止性的带内 `error` 或 `aborted` finish 进入同一路径。提示词装配、请求中间件、请求日志、结果处理、工具、post-step 监听器与清理仍属于普通失败。 恢复运行前，失败 step 已经关闭。重试会打开下一个编号 step，并从持久日志重建请求；连续恢复尝试计数只在提供方请求成功后重置。两个 DeepSeek 适配器都把识别出的提供方上下文限制错误规范化为 `CONTEXT_WINDOW_EXCEEDED`。 diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 08c1ab9412..3d34026df0 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -27,7 +27,7 @@ function toError(error: unknown): RequestError { return error instanceof Error ? error : new HarnessError(String(error), 'UNKNOWN', { cause: error }) } -/** Distinguishes a terminal failure finish from failures in later step processing. */ +/** Distinguishes final model-request failures from failures in later step processing. */ class TerminalModelRequestFailure extends Error { constructor(readonly requestError: RequestError) { super(requestError.message, { cause: requestError }) @@ -347,9 +347,7 @@ async function runTurn( stepOutcome = await runStep( ctx, events, agent, handle, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal) } catch (error: unknown) { - if (isLlmAdapterFailure(error)) { - stepOutcome = { requestError: error } - } else if (error instanceof TerminalModelRequestFailure) { + if (error instanceof TerminalModelRequestFailure) { stepOutcome = { requestError: error.requestError } } else { stepOutcome = { error: toError(error) } @@ -624,12 +622,18 @@ async function runStep( // --- Model call (streaming-first; raw chunks are the replay record) --- const assembler = new BlockAssembler() const chunkSeqs: number[] = [] - for await (const chunk of ctx.llm.stream(request)) { - /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ - if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) - const chunkEvent = session.append('assistant/chunk', { turn, step, chunk }) - chunkSeqs.push(chunkEvent.seq) - assembler.push(chunk) + const stream = ctx.llm.stream(request) + try { + for await (const chunk of stream) { + /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ + if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) + const chunkEvent = session.append('assistant/chunk', { turn, step, chunk }) + chunkSeqs.push(chunkEvent.seq) + assembler.push(chunk) + } + } catch (error: unknown) { + if (isLlmAdapterFailure(stream, error)) throw new TerminalModelRequestFailure(error) + throw error } // Normalize failure finish chunks into the same path as thrown stream errors. diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts index 442fba7565..bfbcad23ba 100644 --- a/packages/core/agent-loop/tests/request-recovery.spec.ts +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -307,6 +307,42 @@ describe('agent post-step and request-error lifecycle', () => { expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'error' } } }) }) + it('does not offer a nested model-call failure as the outer request failure', async () => { + const outer = new FailureScriptAdapter([textResponse('outer adapter must not run')]) + const nested = new FailureScriptAdapter([contextError('nested overflow')]) + const ctx = await harness(outer) + ctx.llm.registerAdapter(['nested'], nested) + ctx.on('llm/stream', (options, next) => { + if (options.provider !== 'mock') return next() + return (async function* () { + yield* ctx.llm.stream({ + provider: 'nested', + model: 'nested', + messages: [], + ...options.signal === undefined ? {} : { signal: options.signal }, + }) + yield* next() + })() + }) + const agent = ctx.agentLoop.create(SessionId('nested-stream-not-recoverable'), { provider: 'mock', model: 'mock' }) + let recoveries = 0 + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + recoveries += 1 + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(nested.requests).toHaveLength(1) + expect(outer.requests).toHaveLength(0) + expect(recoveries).toBe(0) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'error', message: 'nested overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } }, + }) + }) + it.each(['prompt-submit', 'prompt-assembly', 'pre-step', 'request'] as const)( 'does not offer %s middleware failures to request recovery', async (boundary) => { diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index 83f5259f8d..44e7d80a05 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -13,7 +13,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa - `ctx.llm.listModels(provider: string): Promise` Discover the models one registered provider currently advertises. - `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`. -`LlmService` preserves and privately tags errors from final adapter selection, synchronous dispatch, iterator construction, and iteration. `isLlmAdapterFailure(value)` exposes that provenance without classifying `llm/stream` middleware or downstream consumer failures as provider failures, and without replacing the adapter's original coded `Error`. +`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`. Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`. diff --git a/packages/llm/llm/src/adapter-failure.ts b/packages/llm/llm/src/adapter-failure.ts index 240f934c39..745cbbdc64 100644 --- a/packages/llm/llm/src/adapter-failure.ts +++ b/packages/llm/llm/src/adapter-failure.ts @@ -5,30 +5,63 @@ */ import { HarnessError } from './error.ts' +import type { StreamChunk } from './types.ts' -/** Errors proven to originate in final adapter dispatch or iteration. */ -const adapterFailures = new WeakSet() +/** Errors proven to originate in one model call's final adapter boundary. */ +export type AdapterFailureScope = WeakSet + +/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */ +const adapterFailureScopes = new WeakMap, AdapterFailureScope>() + +/** + * Bind one call's adapter-failure scope to a unique returned stream handle. + * @param stream - the waterfall-selected stream for this call. + * @param failures - errors tagged by this call's final adapter boundary. + * @returns a unique stream handle that delegates iteration to `stream`. + * @internal + */ +export function bindAdapterFailureScope( + stream: AsyncIterable, + failures: AdapterFailureScope, +): AsyncIterable { + const call = { + [Symbol.asyncIterator](): AsyncIterator { + return stream[Symbol.asyncIterator]() + }, + } + adapterFailureScopes.set(call, failures) + return call +} /** * Preserve an adapter's Error identity while tagging its provider origin. + * @param failures - the call-local final-adapter failure scope. * @param value - arbitrary value thrown by adapter dispatch or iteration. * @returns the original Error, or a coded Error wrapping a non-Error throw. * @internal */ -export function markLlmAdapterFailure(value: unknown): Error & { code?: string } { +export function markLlmAdapterFailure( + failures: AdapterFailureScope, + value: unknown, +): Error & { code?: string } { const error = value instanceof Error ? value as Error & { code?: string } : new HarnessError(String(value), 'UNKNOWN', { cause: value }) - adapterFailures.add(error) + failures.add(error) return error } /** * Whether a failure came from final adapter dispatch, iterator construction, - * or iteration rather than from an `llm/stream` waterfall listener. + * or iteration for the call represented by the exact returned stream handle. + * @param stream - the exact stream returned by the model call being classified. * @param value - arbitrary failure caught by a model-call consumer. - * @returns true only for errors tagged at the final adapter boundary. + * @returns true only for errors tagged at that call's final adapter boundary. */ -export function isLlmAdapterFailure(value: unknown): value is Error & { code?: string } { - return value instanceof Error && adapterFailures.has(value) +export function isLlmAdapterFailure( + stream: AsyncIterable, + value: unknown, +): value is Error & { code?: string } { + const failures = adapterFailureScopes.get(stream) + return value instanceof Error && failures !== undefined && failures.has(value) } diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index 135b620f45..ac31738a61 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -10,7 +10,8 @@ import { Context, Service } from 'cordis' import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts' import { deepFreeze } from './call-config.ts' import { HarnessError } from './error.ts' -import { markLlmAdapterFailure } from './adapter-failure.ts' +import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts' +import type { AdapterFailureScope } from './adapter-failure.ts' export * from './attribution.ts' export * from './brand.ts' @@ -206,14 +207,17 @@ export class LlmService extends Service { * so it cannot suppress the primary provider error. A downstream close awaits * adapter cleanup, whose failures remain ordinary untagged work. */ - private async * adapterStream(options: GenerateOptions): AsyncGenerator { + private async * adapterStream( + options: GenerateOptions, + failures: AdapterFailureScope, + ): AsyncGenerator { let iterator: AsyncIterator try { const adapter = this.registration(options.provider).adapter const stream = adapter.stream(this.forAdapter(options, adapter)) iterator = stream[Symbol.asyncIterator]() } catch (error: unknown) { - throw markLlmAdapterFailure(error) + throw markLlmAdapterFailure(failures, error) } let completed = false @@ -230,7 +234,7 @@ export class LlmService extends Service { value = item.value } catch (error: unknown) { iterationFailed = true - throw markLlmAdapterFailure(error) + throw markLlmAdapterFailure(failures, error) } // End the adapter-owned try before yielding: consumer/middleware // failures resumed into this generator must remain untagged. @@ -251,13 +255,16 @@ export class LlmService extends Service { * `options.provider`. Replay state is retained only when the same adapter * instance owns its historical provider and the target provider. Final * adapter selection, dispatch, and iteration failures retain their original - * Error identity and are tagged for narrow agent-loop request recovery; - * middleware failures remain untagged. + * Error identity and are tagged in a call-local scope for narrow agent-loop + * request recovery; middleware and nested-call failures remain untagged for + * the outer call. * @param options - the full request; `options.provider` selects the adapter. * @returns the chunk stream, possibly wrapped by `llm/stream` listeners. */ stream(options: GenerateOptions): AsyncIterable { - return this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options)) + const failures: AdapterFailureScope = new WeakSet() + const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures)) + return bindAdapterFailureScope(stream, failures) } } diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index 0431d4faba..6e14d749ba 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -30,6 +30,16 @@ class RecordingAdapter extends ScriptedAdapter { } } +class ThrowingAdapter extends LlmAdapter { + constructor(private readonly failure: Error) { + super() + } + + stream(_options: GenerateOptions): AsyncIterable { + throw this.failure + } +} + class CatalogAdapter extends ScriptedAdapter { constructor( private readonly provider: LlmProviderInfo, @@ -83,16 +93,17 @@ describe('LlmService', () => { it('throws NO_ADAPTER for unregistered providers', async () => { const ctx = new Context() await ctx.plugin(LlmService) + const stream = ctx.llm.stream({ provider: 'nope', model: 'any-model', messages: [] }) let caught: unknown try { - for await (const _ of ctx.llm.stream({ provider: 'nope', model: 'any-model', messages: [] })) { /* drain */ } + for await (const _ of stream) { /* drain */ } } catch (error: unknown) { caught = error } expect(caught).toBeInstanceOf(LlmError) expect((caught as LlmError).code).toBe('NO_ADAPTER') expect((caught as LlmError).message).toContain('no adapter registered') - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(isLlmAdapterFailure(stream, caught)).toBe(true) }) it.each(['done', 'value'] as const)('tags a throwing IteratorResult.%s getter without replacing its Error', async (field) => { @@ -122,15 +133,16 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ } + for await (const _chunk of stream) { /* drain */ } } catch (error: unknown) { caught = error } expect(caught).toBe(original) - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(isLlmAdapterFailure(stream, caught)).toBe(true) expect(cleanupLookups).toBe(0) }) @@ -146,15 +158,91 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ } + for await (const _chunk of stream) { /* drain */ } } catch (error: unknown) { caught = error } expect(caught).toBe(original) - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(isLlmAdapterFailure(stream, caught)).toBe(true) + }) + + it('keeps a nested adapter failure scoped to the nested model call', async () => { + const original = new LlmError('nested provider failed', 'NESTED_FAILED') + const outer = new RecordingAdapter(SCRIPT) + const nested = new ThrowingAdapter(original) + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['outer'], outer) + ctx.llm.registerAdapter(['nested'], nested) + let nestedStream: AsyncIterable | undefined + ctx.on('llm/stream', (options, next) => { + if (options.provider !== 'outer') return next() + return (async function* () { + nestedStream = ctx.llm.stream({ provider: 'nested', model: 'nested', messages: [] }) + yield * nestedStream + })() + }) + + const outerStream = ctx.llm.stream({ provider: 'outer', model: 'outer', messages: [] }) + let caught: unknown + try { + for await (const _chunk of outerStream) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(nestedStream).toBeDefined() + expect(isLlmAdapterFailure(nestedStream!, caught)).toBe(true) + expect(isLlmAdapterFailure(outerStream, caught)).toBe(false) + expect(outer.lastOptions).toBeUndefined() + }) + + it('keeps call scopes distinct when middleware reuses an iterable', async () => { + const firstFailure = new LlmError('first provider failed', 'FIRST_FAILED') + const secondFailure = new LlmError('second provider failed', 'SECOND_FAILED') + const delegates: AsyncIterable[] = [] + const shared: AsyncIterable = { + [Symbol.asyncIterator](): AsyncIterator { + const delegate = delegates.shift() + if (delegate === undefined) throw new Error('shared stream has no call delegate') + return delegate[Symbol.asyncIterator]() + }, + } + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['first'], new ThrowingAdapter(firstFailure)) + ctx.llm.registerAdapter(['second'], new ThrowingAdapter(secondFailure)) + ctx.on('llm/stream', (_options, next) => { + delegates.push(next()) + return shared + }) + + const firstStream = ctx.llm.stream({ provider: 'first', model: 'first', messages: [] }) + const secondStream = ctx.llm.stream({ provider: 'second', model: 'second', messages: [] }) + const catchFailure = async (stream: AsyncIterable): Promise => { + try { + for await (const _chunk of stream) { /* drain */ } + } catch (error: unknown) { + return error + } + return new Error('expected adapter to fail') + } + + expect(firstStream).not.toBe(secondStream) + const firstCaught = await catchFailure(firstStream) + expect(firstCaught).toBe(firstFailure) + expect(isLlmAdapterFailure(firstStream, firstCaught)).toBe(true) + expect(isLlmAdapterFailure(secondStream, firstCaught)).toBe(false) + const secondCaught = await catchFailure(secondStream) + expect(secondCaught).toBe(secondFailure) + expect(isLlmAdapterFailure(secondStream, secondCaught)).toBe(true) + expect(isLlmAdapterFailure(firstStream, secondCaught)).toBe(false) + expect(delegates).toHaveLength(0) }) it('propagates a rejected next promptly without awaiting a non-settling return', async () => { @@ -179,9 +267,10 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) const failure = (async (): Promise => { try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ } + for await (const _chunk of stream) { /* drain */ } } catch (error: unknown) { return error } @@ -195,7 +284,7 @@ describe('LlmService', () => { if (timer !== undefined) clearTimeout(timer) expect(caught).toBe(original) - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(isLlmAdapterFailure(stream, caught)).toBe(true) expect(cleanupCalls).toBe(0) }) @@ -221,15 +310,16 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) break + for await (const _chunk of stream) break } catch (error: unknown) { caught = error } expect(caught).toBe(cleanup) - expect(isLlmAdapterFailure(caught)).toBe(false) + expect(isLlmAdapterFailure(stream, caught)).toBe(false) expect(cleanupCalls).toBe(1) }) @@ -272,16 +362,17 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], adapter) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ } + for await (const _chunk of stream) { /* drain */ } } catch (error: unknown) { caught = error } expect(caught).toBeInstanceOf(HarnessError) expect(caught).toMatchObject({ code: 'UNKNOWN', cause: 'plain provider failure' }) - expect(isLlmAdapterFailure(caught)).toBe(true) + expect(isLlmAdapterFailure(stream, caught)).toBe(true) }) it('does not tag a failure thrown downstream while consuming adapter output', async () => { @@ -290,15 +381,20 @@ describe('LlmService', () => { await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT)) + const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] }) let caught: unknown try { - for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) throw downstream + for await (const _chunk of stream) throw downstream } catch (error: unknown) { caught = error } expect(caught).toBe(downstream) - expect(isLlmAdapterFailure(caught)).toBe(false) + expect(isLlmAdapterFailure(stream, caught)).toBe(false) + expect(isLlmAdapterFailure(new ScriptedAdapter(SCRIPT).stream({ + provider: 'unbound', model: 'unbound', messages: [], + }), caught)).toBe(false) + expect(isLlmAdapterFailure(stream, 'consumer failed')).toBe(false) }) it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => { diff --git a/website/zh-CN/api/harness/events.md b/website/zh-CN/api/harness/events.md index bc2a9e7e4b..b7b19dd0e2 100644 --- a/website/zh-CN/api/harness/events.md +++ b/website/zh-CN/api/harness/events.md @@ -342,7 +342,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t - `options` — the full request. A LOOP-built request arrives deep-frozen (mutation throws): its content is a pure function of the session log (the reconstructability RFC), so listeners read it, never rewrite it. A hand-built one-shot (compaction summarize) is the caller's own object and stays mutable here. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L42) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L43) ## session/* diff --git a/website/zh-CN/api/harness/llm.md b/website/zh-CN/api/harness/llm.md index 0aebd9afe4..288839eca4 100644 --- a/website/zh-CN/api/harness/llm.md +++ b/website/zh-CN/api/harness/llm.md @@ -6,7 +6,7 @@ The abstract `llm` service: an adapter registry plus a streaming model-call surface, interceptable via the `llm/stream` waterfall. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L96) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L97) ### ctx.llm.registerAdapter(providers, adapter) @@ -21,7 +21,7 @@ Register an adapter for the given provider routes. Throws `LlmError` with code ` **Returns** the disposer that unregisters all of them. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L111) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L112) ### ctx.llm.listProviders() @@ -33,7 +33,7 @@ Describe provider routes with a registered adapter. **Returns** detached provider metadata in registration order. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L142) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L143) ### ctx.llm.listModels(provider) @@ -47,7 +47,7 @@ Discover models advertised by one registered provider. Catalog membership is adv **Returns** detached model metadata in adapter-preferred order. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L152) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L153) ### ctx.llm.stream(options) @@ -55,10 +55,10 @@ Discover models advertised by one registered provider. Catalog membership is adv stream(options: GenerateOptions): AsyncIterable ``` -Stream one model call as raw chunks (token-level deltas). Throws `LlmError` with code `NO_ADAPTER` if no adapter is registered for `options.provider`. Replay state is retained only when the same adapter instance owns its historical provider and the target provider. Final adapter selection, dispatch, and iteration failures retain their original Error identity and are tagged for narrow agent-loop request recovery; middleware failures remain untagged. +Stream one model call as raw chunks (token-level deltas). Throws `LlmError` with code `NO_ADAPTER` if no adapter is registered for `options.provider`. Replay state is retained only when the same adapter instance owns its historical provider and the target provider. Final adapter selection, dispatch, and iteration failures retain their original Error identity and are tagged in a call-local scope for narrow agent-loop request recovery; middleware and nested-call failures remain untagged for the outer call. - `options` — the full request; `options.provider` selects the adapter. **Returns** the chunk stream, possibly wrapped by `llm/stream` listeners. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L259) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L264) From 36dbe686314393efd70a7de3001f587f02d581a0 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 16:22:09 +0800 Subject: [PATCH 52/73] test(cordis): snapshot targeted JSDoc inspection --- examples/acp-agent/tests/acp.snapshot.ts | 7 ++++ .../snapshots/cordis-inspect-jsdoc/input.json | 7 ++++ .../cordis-inspect-jsdoc/session.jsonl | 33 +++++++++++++++++++ .../cordis-inspect-jsdoc/stdout.golden.jsonl | 8 +++++ 4 files changed, 55 insertions(+) create mode 100644 examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/input.json create mode 100644 examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl create mode 100644 examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.golden.jsonl diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts index 124117afef..8f749923ef 100644 --- a/examples/acp-agent/tests/acp.snapshot.ts +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -122,6 +122,13 @@ const SCENARIOS: Scenario[] = [ headerClass: 'advanced', configPath: ADVANCED_CONFIG, }, + { + name: 'cordis-inspect-jsdoc', + hasModelTurn: true, + recorded: false, + headerClass: 'advanced', + configPath: ADVANCED_CONFIG, + }, // Prompt-submit blocks are authored keylessly: they persist a rejected turn // and hook events without starting a model step, so their logs still compare. { name: 'hook-cc-promptsubmit-block', hasModelTurn: false, comparesLog: true, recorded: false }, diff --git a/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/input.json b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/input.json new file mode 100644 index 0000000000..62df391622 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "Inspect the exact tools service API and tools/pre-execute event with cordis_inspect, then reply with exactly CORDIS_INSPECT_JSDOC_OK." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl new file mode 100644 index 0000000000..94ea8a8fec --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl @@ -0,0 +1,33 @@ +{"type":"session","version":0,"id":"22222222-2222-4222-8222-222222222222","createdAt":1783951000000,"cwd":"/tmp/cordis-inspect-jsdoc"} +{"type":"turn/start","seq":0,"time":1784449176717,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1784449176718,"data":{"content":[{"type":"text","text":"Inspect the exact tools service API and tools/pre-execute event with cordis_inspect, then reply with exactly CORDIS_INSPECT_JSDOC_OK."}],"source":{"kind":"user"}},"surfaceOp":"append"} +{"type":"step/start","seq":2,"time":1784449176720,"data":{"turn":1,"step":1}} +{"type":"request/header","seq":3,"time":1784449176720,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"assistant/chunk","seq":4,"time":1783951000005,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":5,"time":1783951000006,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":0,"id":"inspect-tools-api","name":"cordis_inspect","argumentsDelta":"{\"what\":\"api\",\"name\":\"tools\"}"}}} +{"type":"assistant/chunk","seq":6,"time":1783951000007,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"inspect-tools-api","name":"cordis_inspect","arguments":"{\"what\":\"api\",\"name\":\"tools\"}"}}}} +{"type":"assistant/chunk","seq":7,"time":1783951000008,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":8,"time":1783951000009,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":9,"time":1784449176722,"data":{"turn":1,"step":1,"content":[{"type":"tool-call","id":"inspect-tools-api","name":"cordis_inspect","arguments":"{\"what\":\"api\",\"name\":\"tools\"}"}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[4,5,6,7,8],"surfaceOp":"append"} +{"type":"tool/call","seq":10,"time":1784449176722,"data":{"turn":1,"step":1,"callId":"inspect-tools-api","name":"cordis_inspect","arguments":"{\"what\":\"api\",\"name\":\"tools\"}"}} +{"type":"tool/result","seq":11,"time":1784449176732,"data":{"turn":1,"step":1,"callId":"inspect-tools-api","content":[{"type":"text","text":"## api\n- tools — Tool registry and execution pipeline.\n /**\n * Register globally or in the calling agent scope. Scoped tools shadow\n * globals; duplicates within one layer and the reserved `run_code` name fail.\n * @param definition - the tool schema, execution, and optional presentation functions.\n * @returns the exact disposer that unregisters the tool.\n */\n register(definition: ToolDefinition): () => void\n /**\n * Restrict global tools for the calling agent scope. Empty filters, unknown\n * names, scope-local names, and reserved transport names fail. Restrictions\n * intersect; scoped registrations remain visible.\n * @param filter - global-surface mask: `allow` (keep only) and/or `deny` (remove).\n * @returns the exact disposer that lifts this restriction.\n */\n restrict(filter: ToolRestriction): () => void\n /**\n * Register a monotonic guard after the extensible `tools/pre-execute`\n * waterfall. A plain-context guard applies globally; one registered through\n * `agent.ctx` applies only to that agent. Any matching guard may deny by\n * returning a reason, while no guard can force-allow a call another guard\n * denied. The exact effect disposer is returned for ordered ownership and\n * HMR cleanup.\n * @param guard - synchronous check; a returned string denies the execution.\n * @returns the exact disposer that unregisters the guard.\n */\n guard(guard: ToolGuard): () => void\n /**\n * Look up a tool as one scope sees it (scoped\n * shadows global; a restricted-away global reads as absent). Presenters pass\n * the calling agent so the rendered card matches the definition that\n * actually executed.\n * @param name - the tool name as registered.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns the definition the scope resolves, or undefined when none is visible.\n */\n get(name: string, scope?: ScopeKey): ToolDefinition | undefined\n /**\n * Project visible definitions onto the allowlisted model-facing schema fields,\n * excluding execution and presentation callbacks.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns one deep-cloned schema per visible tool.\n */\n schemas(scope?: ScopeKey): ToolSchema[]\n /**\n * Classify a pending call through the caller's visible tool definition. Only\n * an exact `true` is parallel; unknown, hidden, undeclared, invalid, or\n * throwing classifiers are exclusive.\n * @param exec - call name, parsed arguments, and optional agent scope.\n * @returns the fail-closed scheduling mode.\n */\n executionMode(exec: ToolExecutionInput): ToolExecutionMode\n /**\n * Execute through pre-policy, guards, around-dispatch, post-policy, and final\n * notification. Tool and listener failures resolve as materialized error\n * results; an invisible tool reports `UNKNOWN_TOOL`. The returned outcome is\n * the same lossless, frozen snapshot final observers receive.\n * @param exec - the typed same-process call input. The registry assigns its\n * correlation token before policy begins.\n * @returns the materialized final result.\n */\n async execute(exec: ToolExecutionInput): Promise\ntype shapes (referenced by the signatures above — read these before assuming a field is a string):\n export interface Agent {\n readonly id: SessionId;\n readonly options: AgentOptions;\n readonly session: Session;\n readonly status: AgentStatus;\n readonly ctx: Context;\n send(content: ContentBlock[], options?: SendOptions): void;\n steer(content: ContentBlock[], options?: SendOptions): void;\n inject(content: ContentBlock[], options?: InjectOptions): void;\n cancel(reason?: string): void;\n whenIdle(): Promise;\n }\n export interface AgentOptions {\n provider?: string;\n model?: string;\n }\n export type AgentStatus = 'idle' | 'running' | 'disposed';\n export type Branded = string & {\n readonly [BRAND]: B;\n };\n export type CallId = Branded<'CallId'>;\n export type ContextEnvelope = 'context' | 'raw';\n export interface DiffCallView {\n card: 'diff';\n title: string;\n diffs: FileDiff[];\n locations?: FileLocation[];\n }\n export interface DiffResultView {\n card: 'diff';\n title?: string;\n diffs: FileDiff[];\n }\n export interface FileDiff {\n path: string;\n oldText: string | null;\n newText: string;\n }\n export interface FileLocation {\n path: string;\n line?: number;\n }\n export interface GenericCallView {\n card: 'generic';\n title: string;\n kind?: ToolCallKind;\n rawInput?: unknown;\n content?: ContentBlock[];\n locations?: FileLocation[];\n }\n export interface GenericResultView {\n card: 'generic';\n title?: string;\n content?: ContentBlock[];\n }\n export interface HookContext {\n content: ContentBlock[];\n source: MessageSource;\n envelope?: ContextEnvelope;\n meta?: JsonValue;\n }\n export interface InjectOptions extends SendOptions {\n envelope?: ContextEnvelope;\n meta?: JsonValue;\n }\n export type JsonValue = null | boolean | number | string | JsonValue[] | {\n [key: string]: JsonValue;\n };\n export type MessageSource = MessageSourceMap[keyof MessageSourceMap];\n export interface MessageSourceMap {\n user: {\n kind: 'user';\n };\n plugin: {\n kind: 'plugin';\n plugin: string;\n };\n }\n export type ScopeKey = object;\n export interface SendOptions {\n source?: MessageSource;\n }\n export type SessionId = Branded<'SessionId'>;\n export interface TerminalCallView {\n card: 'terminal';\n title: string;\n description?: string;\n cwd?: string;\n }\n export interface TerminalResultView {\n card: 'terminal';\n title?: string;\n output?: string;\n exitCode?: number;\n signal?: string;\n }\n export type ToolCallKind = 'read' | 'edit' | 'delete' | 'move' | 'search' | 'execute' | 'fetch' | 'other';\n export type ToolCallView = GenericCallView | TerminalCallView | DiffCallView;\n export interface ToolDefinition extends ToolSchema {\n execute(args: unknown, exec: ToolRunContext): Promise;\n timeoutMs?: number;\n isConcurrencySafe?(args: unknown): boolean;\n presentCall?(args: unknown): ToolCallView | undefined;\n presentResult?(args: unknown, result: ToolResult): ToolResultView | undefined;\n }\n export interface ToolErrorInfo {\n name: string;\n code: string;\n }\n export type ToolExecuteReturn = ContentBlock[] | {\n content: ContentBlock[];\n meta?: unknown;\n };\n export interface ToolExecution extends ToolExecutionInput {\n readonly token: ToolExecutionToken;\n }\n export interface ToolExecutionInput {\n readonly callId: CallId;\n readonly name: string;\n readonly arguments: unknown;\n readonly agent?: Agent;\n readonly parent?: ToolExecutionToken;\n signal?: AbortSignal;\n }\n export type ToolExecutionMode = {\n kind: 'parallel';\n } | {\n kind: 'exclusive';\n };\n export interface ToolExecutionResult {\n content: ContentBlock[];\n isError: boolean;\n error?: ToolErrorInfo;\n additionalContexts?: HookContext[];\n meta?: unknown;\n }\n export type ToolExecutionToken = symbol & {\n readonly [toolExecutionTokenBrand]: true;\n };\n export type ToolGuard = (execution: Readonly) => string | undefined;\n export interface ToolRestriction {\n readonly allow?: readonly string[];\n readonly deny?: readonly string[];\n }\n export interface ToolResult {\n content: ContentBlock[];\n isError: boolean;\n meta?: unknown;\n }\n export type ToolResultView = GenericResultView | TerminalResultView | DiffResultView;\n export interface ToolRunContext extends ToolExecution {\n deferContext(context: HookContext): void;\n }\n export interface ToolSchema {\n name: string;\n description: string;\n parameters: Record;\n }"}],"isError":false},"sourceEventSeqs":[10],"surfaceOp":"append"} +{"type":"step/end","seq":12,"time":1784449176732,"data":{"turn":1,"step":1}} +{"type":"step/start","seq":13,"time":1784449176733,"data":{"turn":1,"step":2}} +{"type":"assistant/chunk","seq":14,"time":1783951000015,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":15,"time":1784449176734,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":0,"id":"inspect-tools-event","name":"cordis_inspect","argumentsDelta":"{\"what\":\"events\",\"name\":\"tools/pre-execute\"}"}}} +{"type":"assistant/chunk","seq":16,"time":1784449176734,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"inspect-tools-event","name":"cordis_inspect","arguments":"{\"what\":\"events\",\"name\":\"tools/pre-execute\"}"}}}} +{"type":"assistant/chunk","seq":17,"time":1784449176734,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":18,"time":1784449176734,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":19,"time":1784449176734,"data":{"turn":1,"step":2,"content":[{"type":"tool-call","id":"inspect-tools-event","name":"cordis_inspect","arguments":"{\"what\":\"events\",\"name\":\"tools/pre-execute\"}"}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[14,15,16,17,18],"surfaceOp":"append"} +{"type":"tool/call","seq":20,"time":1784449176734,"data":{"turn":1,"step":2,"callId":"inspect-tools-event","name":"cordis_inspect","arguments":"{\"what\":\"events\",\"name\":\"tools/pre-execute\"}"}} +{"type":"tool/result","seq":21,"time":1784449176734,"data":{"turn":1,"step":2,"callId":"inspect-tools-event","content":[{"type":"text","text":"## events\n- tools/pre-execute [waterfall] — Allow, deny, or ask before dispatch.\n /**\n * Allow, deny, or ask before dispatch. `next()` delegates to allow; missing\n * approval support turns `ask` into denial.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent's calls.\n * @param exec - the pending call (name, parsed arguments, caller agent).\n * @mode waterfall\n */\n 'tools/pre-execute'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise\nwaterfall listeners receive a trailing next() and MUST call it to delegate — returning without next() vetoes the chain."}],"isError":false},"sourceEventSeqs":[20],"surfaceOp":"append"} +{"type":"step/end","seq":22,"time":1784449176734,"data":{"turn":1,"step":2}} +{"type":"step/start","seq":23,"time":1784449176735,"data":{"turn":1,"step":3}} +{"type":"assistant/chunk","seq":24,"time":1784449176735,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","seq":25,"time":1784449176735,"data":{"turn":1,"step":3,"chunk":{"type":"text-delta","index":0,"text":"CORDIS_INSPECT_JSDOC_OK"}}} +{"type":"assistant/chunk","seq":26,"time":1784449176735,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"CORDIS_INSPECT_JSDOC_OK"}}}} +{"type":"assistant/chunk","seq":27,"time":1784449176735,"data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":28,"time":1784449176735,"data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":29,"time":1784449176735,"data":{"turn":1,"step":3,"content":[{"type":"text","text":"CORDIS_INSPECT_JSDOC_OK"}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[24,25,26,27,28],"surfaceOp":"append"} +{"type":"step/end","seq":30,"time":1784449176735,"data":{"turn":1,"step":3}} +{"type":"turn/end","seq":31,"time":1784449176735,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.golden.jsonl new file mode 100644 index 0000000000..321c5499a2 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.golden.jsonl @@ -0,0 +1,8 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","configOptions":[{"id":"model","name":"Model","description":"Sets this session's provider and model.","category":"model","type":"select","currentValue":"[\"deepseek\",\"deepseek-v4-flash\"]","options":[{"value":"[\"deepseek\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"}]},{"id":"permission","name":"Permissions","description":"The session permission preset: each choice bundles a sandbox mode and an approval policy.","category":"mode","type":"select","currentValue":"danger-full-access","options":[{"value":"workspace-write","name":"workspace-write","description":"Write inside the workspace and permitted temporary directories; wider retries require approval."},{"value":"danger-full-access","name":"danger-full-access","description":"Full file access without approval prompts."}]}]}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"inspect-tools-api","title":"Inspect cordis runtime: api: tools","kind":"read","status":"in_progress"}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"inspect-tools-api","status":"completed","content":[{"type":"content","content":{"type":"text","text":"## api\n- tools — Tool registry and execution pipeline.\n /**\n * Register globally or in the calling agent scope. Scoped tools shadow\n * globals; duplicates within one layer and the reserved `run_code` name fail.\n * @param definition - the tool schema, execution, and optional presentation functions.\n * @returns the exact disposer that unregisters the tool.\n */\n register(definition: ToolDefinition): () => void\n /**\n * Restrict global tools for the calling agent scope. Empty filters, unknown\n * names, scope-local names, and reserved transport names fail. Restrictions\n * intersect; scoped registrations remain visible.\n * @param filter - global-surface mask: `allow` (keep only) and/or `deny` (remove).\n * @returns the exact disposer that lifts this restriction.\n */\n restrict(filter: ToolRestriction): () => void\n /**\n * Register a monotonic guard after the extensible `tools/pre-execute`\n * waterfall. A plain-context guard applies globally; one registered through\n * `agent.ctx` applies only to that agent. Any matching guard may deny by\n * returning a reason, while no guard can force-allow a call another guard\n * denied. The exact effect disposer is returned for ordered ownership and\n * HMR cleanup.\n * @param guard - synchronous check; a returned string denies the execution.\n * @returns the exact disposer that unregisters the guard.\n */\n guard(guard: ToolGuard): () => void\n /**\n * Look up a tool as one scope sees it (scoped\n * shadows global; a restricted-away global reads as absent). Presenters pass\n * the calling agent so the rendered card matches the definition that\n * actually executed.\n * @param name - the tool name as registered.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns the definition the scope resolves, or undefined when none is visible.\n */\n get(name: string, scope?: ScopeKey): ToolDefinition | undefined\n /**\n * Project visible definitions onto the allowlisted model-facing schema fields,\n * excluding execution and presentation callbacks.\n * @param scope - the viewing scope (the agent); omitted = the global view.\n * @returns one deep-cloned schema per visible tool.\n */\n schemas(scope?: ScopeKey): ToolSchema[]\n /**\n * Classify a pending call through the caller's visible tool definition. Only\n * an exact `true` is parallel; unknown, hidden, undeclared, invalid, or\n * throwing classifiers are exclusive.\n * @param exec - call name, parsed arguments, and optional agent scope.\n * @returns the fail-closed scheduling mode.\n */\n executionMode(exec: ToolExecutionInput): ToolExecutionMode\n /**\n * Execute through pre-policy, guards, around-dispatch, post-policy, and final\n * notification. Tool and listener failures resolve as materialized error\n * results; an invisible tool reports `UNKNOWN_TOOL`. The returned outcome is\n * the same lossless, frozen snapshot final observers receive.\n * @param exec - the typed same-process call input. The registry assigns its\n * correlation token before policy begins.\n * @returns the materialized final result.\n */\n async execute(exec: ToolExecutionInput): Promise\ntype shapes (referenced by the signatures above — read these before assuming a field is a string):\n export interface Agent {\n readonly id: SessionId;\n readonly options: AgentOptions;\n readonly session: Session;\n readonly status: AgentStatus;\n readonly ctx: Context;\n send(content: ContentBlock[], options?: SendOptions): void;\n steer(content: ContentBlock[], options?: SendOptions): void;\n inject(content: ContentBlock[], options?: InjectOptions): void;\n cancel(reason?: string): void;\n whenIdle(): Promise;\n }\n export interface AgentOptions {\n provider?: string;\n model?: string;\n }\n export type AgentStatus = 'idle' | 'running' | 'disposed';\n export type Branded = string & {\n readonly [BRAND]: B;\n };\n export type CallId = Branded<'CallId'>;\n export type ContextEnvelope = 'context' | 'raw';\n export interface DiffCallView {\n card: 'diff';\n title: string;\n diffs: FileDiff[];\n locations?: FileLocation[];\n }\n export interface DiffResultView {\n card: 'diff';\n title?: string;\n diffs: FileDiff[];\n }\n export interface FileDiff {\n path: string;\n oldText: string | null;\n newText: string;\n }\n export interface FileLocation {\n path: string;\n line?: number;\n }\n export interface GenericCallView {\n card: 'generic';\n title: string;\n kind?: ToolCallKind;\n rawInput?: unknown;\n content?: ContentBlock[];\n locations?: FileLocation[];\n }\n export interface GenericResultView {\n card: 'generic';\n title?: string;\n content?: ContentBlock[];\n }\n export interface HookContext {\n content: ContentBlock[];\n source: MessageSource;\n envelope?: ContextEnvelope;\n meta?: JsonValue;\n }\n export interface InjectOptions extends SendOptions {\n envelope?: ContextEnvelope;\n meta?: JsonValue;\n }\n export type JsonValue = null | boolean | number | string | JsonValue[] | {\n [key: string]: JsonValue;\n };\n export type MessageSource = MessageSourceMap[keyof MessageSourceMap];\n export interface MessageSourceMap {\n user: {\n kind: 'user';\n };\n plugin: {\n kind: 'plugin';\n plugin: string;\n };\n }\n export type ScopeKey = object;\n export interface SendOptions {\n source?: MessageSource;\n }\n export type SessionId = Branded<'SessionId'>;\n export interface TerminalCallView {\n card: 'terminal';\n title: string;\n description?: string;\n cwd?: string;\n }\n export interface TerminalResultView {\n card: 'terminal';\n title?: string;\n output?: string;\n exitCode?: number;\n signal?: string;\n }\n export type ToolCallKind = 'read' | 'edit' | 'delete' | 'move' | 'search' | 'execute' | 'fetch' | 'other';\n export type ToolCallView = GenericCallView | TerminalCallView | DiffCallView;\n export interface ToolDefinition extends ToolSchema {\n execute(args: unknown, exec: ToolRunContext): Promise;\n timeoutMs?: number;\n isConcurrencySafe?(args: unknown): boolean;\n presentCall?(args: unknown): ToolCallView | undefined;\n presentResult?(args: unknown, result: ToolResult): ToolResultView | undefined;\n }\n export interface ToolErrorInfo {\n name: string;\n code: string;\n }\n export type ToolExecuteReturn = ContentBlock[] | {\n content: ContentBlock[];\n meta?: unknown;\n };\n export interface ToolExecution extends ToolExecutionInput {\n readonly token: ToolExecutionToken;\n }\n export interface ToolExecutionInput {\n readonly callId: CallId;\n readonly name: string;\n readonly arguments: unknown;\n readonly agent?: Agent;\n readonly parent?: ToolExecutionToken;\n signal?: AbortSignal;\n }\n export type ToolExecutionMode = {\n kind: 'parallel';\n } | {\n kind: 'exclusive';\n };\n export interface ToolExecutionResult {\n content: ContentBlock[];\n isError: boolean;\n error?: ToolErrorInfo;\n additionalContexts?: HookContext[];\n meta?: unknown;\n }\n export type ToolExecutionToken = symbol & {\n readonly [toolExecutionTokenBrand]: true;\n };\n export type ToolGuard = (execution: Readonly) => string | undefined;\n export interface ToolRestriction {\n readonly allow?: readonly string[];\n readonly deny?: readonly string[];\n }\n export interface ToolResult {\n content: ContentBlock[];\n isError: boolean;\n meta?: unknown;\n }\n export type ToolResultView = GenericResultView | TerminalResultView | DiffResultView;\n export interface ToolRunContext extends ToolExecution {\n deferContext(context: HookContext): void;\n }\n export interface ToolSchema {\n name: string;\n description: string;\n parameters: Record;\n }"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"inspect-tools-event","title":"Inspect cordis runtime: events: tools/pre-execute","kind":"read","status":"in_progress"}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"inspect-tools-event","status":"completed","content":[{"type":"content","content":{"type":"text","text":"## events\n- tools/pre-execute [waterfall] — Allow, deny, or ask before dispatch.\n /**\n * Allow, deny, or ask before dispatch. `next()` delegates to allow; missing\n * approval support turns `ask` into denial.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent's calls.\n * @param exec - the pending call (name, parsed arguments, caller agent).\n * @mode waterfall\n */\n 'tools/pre-execute'(this: Scoped, exec: ToolExecution, next: () => Promise): Promise\nwaterfall listeners receive a trailing next() and MUST call it to delegate — returning without next() vetoes the chain."}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"CORDIS_INSPECT_JSDOC_OK"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} From 5e2607df8c077c1dbab63f8ca080e4bb941ec983 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 16:28:31 +0800 Subject: [PATCH 53/73] refactor(core): capture session inside loop operations --- docs/architecture.md | 2 +- ...2026-07-15-agent-initiator-scope.i18n.yaml | 4 +- .../2026-07-15-agent-initiator-scope.md | 4 +- .../2026-07-15-agent-initiator-scope.zh.md | 4 +- packages/core/agent-loop/README.md | 2 +- packages/core/agent-loop/src/loop.ts | 131 +++++++----------- packages/core/agent-loop/src/tool-calls.ts | 57 ++++---- 7 files changed, 84 insertions(+), 120 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 2644e787b6..c181e4b951 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -120,7 +120,7 @@ Every live agent owns a scoped `agent.ctx`. Its registrations shadow globals, re ### Initiating Agent Scope -`AgentLoop` runs each process-local driver inside `ctx.agents.withInitiator()`; the [decision](rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md) owns boundary and explicit-identity rules. +`AgentLoop` runs each driver inside `ctx.agents.withInitiator()`; private code derives `agent.session`, while other identities stay explicit ([decision](rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md)). ## State diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml index 247dfac57c..52128ea344 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-15-agent-initiator-scope.md: adae5d943faa6df436014ef3657b56adfbc6ea60 -2026-07-15-agent-initiator-scope.zh.md: a494c561c99adad9d1c44159e112b7220ea8468f +2026-07-15-agent-initiator-scope.md: a9df15beb8744216e020c259934db9fdf8b28b79 +2026-07-15-agent-initiator-scope.zh.md: 4198f066ef27042bda0d12fbcaf86f143482d596 diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md index adae5d943f..a9df15beb8 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md @@ -16,7 +16,9 @@ The mandatory `ctx.agents` service uses Node `AsyncLocalStorage` to carry the in `currentInitiator()` reads optionally, `requireInitiator()` throws `no initiating agent is active`, and `withInitiator(agent, operation)` preserves the operation's exact synchronous value or Promise. `withoutInitiator(operation)` establishes a clearing boundary for work that must not inherit an Agent. Session remains derived as `agent.session`; turn, step, tool call, `signal`, model, `cwd`, sandbox, and authorization stay with their existing owners. -`AgentLoop` already injects `ctx.agents` and wraps each concrete driver's complete `runLoop` lifetime in `agents.withInitiator(agent, ...)`. Its package-private loop, turn, step, and tool-call helpers recover the exact Agent from `ctx.agents` instead of forwarding the concrete driver through their signatures. Concurrent drivers therefore receive independent stores. A child driver's continuations carry the child, while the caller resumes in its prior store as soon as `withInitiator()` returns; active-run tracking keeps the returned Promise in the teardown drain until it settles. Creation, persistence load, and unpublished `setup(agentCtx)` remain outside the child's driver boundary: creation initiated by a parent runs under the parent identity, while `agentCtx.agent` explicitly identifies the child. +`AgentLoop` already injects `ctx.agents` and wraps each concrete driver's complete `runLoop` lifetime in `agents.withInitiator(agent, ...)`. Its package-private loop, turn, step, and tool-call orchestration entries recover the exact Agent from `ctx.agents`, derive `agent.session` once, and let operation-local helpers capture it instead of forwarding the concrete driver or `Session` through shallow interfaces. A leaf helper keeps a narrow `Session` parameter when that is its actual interface rather than accepting a broader `Context` only for an ambient lookup. + +Concurrent drivers receive independent stores. A child driver's continuations carry the child, while the caller resumes in its prior store as soon as `withInitiator()` returns; active-run tracking keeps the returned Promise in the teardown drain until it settles. Creation, persistence load, and unpublished `setup(agentCtx)` remain outside the child's driver boundary: creation initiated by a parent runs under the parent identity, while `agentCtx.agent` explicitly identifies the child. Ambient identity does not replace explicit contracts. `ToolExecution.agent`, `AssembleContext.agent`, `GenerateOptions.sessionId`, task ownership, parent/child requests, `ctx.agent`, `agentCtx.agent`, approval and hook subjects, `cwd` selection, cancellation, worker/process messages, persistence records, and wire identity remain explicit. A remote boundary materializes the identity it needs into its typed request because ALS is process-local. diff --git a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md index a494c561c9..4198f066ef 100644 --- a/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.zh.md @@ -16,7 +16,9 @@ Harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负 `currentInitiator()` 用于可选读取，`requireInitiator()` 抛出 `no initiating agent is active`，`withInitiator(agent, operation)` 保留操作返回的同步值或 Promise 本身。`withoutInitiator(operation)` 会建立清空边界，供不得继承 Agent 的工作使用。会话仍通过 `agent.session` 推导；轮次、步骤、工具调用、`signal`、模型、`cwd`、沙箱和授权继续由现有归属方管理。 -`AgentLoop` 已经注入 `ctx.agents`，并用 `agents.withInitiator(agent, ...)` 包裹每个具体驱动的完整 `runLoop` 生命周期。其包内私有的循环、轮次、步骤和工具调用辅助函数从 `ctx.agents` 恢复同一个 Agent，无需在函数签名中转发具体驱动。因此，并发驱动使用彼此独立的存储。子驱动的异步延续携带子 Agent；`withInitiator()` 返回后，调用方立即恢复之前的存储，而活动运行计数仍持续跟踪返回的 Promise，直到其结束。创建、持久化加载和尚未发布的 `setup(agentCtx)` 位于子驱动边界之外：由父 Agent 发起的创建使用父身份，而 `agentCtx.agent` 显式标识子 Agent。 +`AgentLoop` 已经注入 `ctx.agents`，并用 `agents.withInitiator(agent, ...)` 包裹每个具体驱动的完整 `runLoop` 生命周期。循环、轮次、步骤和工具调用的包内私有入口从 `ctx.agents` 恢复同一个 Agent，一次推导 `agent.session`，再由操作内辅助函数捕获该值，避免在浅层接口中转发具体驱动或 `Session`。若 `Session` 本身就是底层辅助函数的实际接口，该函数会保留狭窄的 `Session` 参数，而不会只为隐式查找而接收更宽泛的 `Context`。 + +因此，并发驱动使用彼此独立的存储。子驱动的异步延续携带子 Agent；`withInitiator()` 返回后，调用方立即恢复之前的存储，而活动运行计数仍持续跟踪返回的 Promise，直到其结束。创建、持久化加载和尚未发布的 `setup(agentCtx)` 位于子驱动边界之外：由父 Agent 发起的创建使用父身份，而 `agentCtx.agent` 显式标识子 Agent。 隐式身份不会取代显式契约。`ToolExecution.agent`、`AssembleContext.agent`、`GenerateOptions.sessionId`、任务归属、父子请求、`ctx.agent`、`agentCtx.agent`、审批与 hook 主体、`cwd` 选择、取消、worker 和进程消息、持久化记录及协议身份都保持显式传递。远程边界会把所需身份写入类型化请求，因为 ALS 只在进程内有效。 diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index f0030845c1..6d373a303a 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -50,7 +50,7 @@ The concrete `Agent` class, its `Inbox`, `runLoop`, and instance-bound publicati ### Loop lifecycle (`loop.ts`) -The driver owns one agent for its lifetime and runs inside `ctx.agents.withInitiator(agent, ...)`, so package-private loop, turn, step, and tool-call helpers recover the exact Agent from `ctx.agents` instead of forwarding the concrete driver through their signatures. Creation, persistence load, and unpublished setup stay outside the driver boundary; explicit Agent fields remain authoritative at service, worker, process, persistence, and wire boundaries. The [agent service](../agent/README.md#initiating-agent-scope) owns propagation, teardown, and detached-work rules. +The driver owns one agent for its lifetime and runs inside `ctx.agents.withInitiator(agent, ...)`. Package-private orchestration entry points recover the exact Agent, derive `agent.session` once, and let operation-local helpers capture it instead of forwarding the concrete driver or per-operation `Session` through shallow interfaces. A helper keeps an explicit `Session` when that is its actual interface, while creation, persistence load, unpublished setup, services, workers, processes, persistence, and wire protocols retain their explicit identities. The [agent service](../agent/README.md#initiating-agent-scope) owns propagation, teardown, and detached-work rules. Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history. diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 4f2f768d89..ce95af3d13 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -95,8 +95,8 @@ export interface LoopHandle { /** * Drive queued batches as durable turns until disposal. Plugin failures end the * current turn without terminating the driver. The caller establishes the - * `ctx.agents.withInitiator()` boundary before entry; package-private helpers - * recover that exact Agent from the inherited store. + * `ctx.agents.withInitiator()` boundary before entry; package-private + * orchestration recovers that exact Agent and captures its Session locally. * @param ctx - the plugin context the loop reaches its initiating Agent, * events (agent/…, session/flush), and services (systemPrompt, llm, tools) * through. @@ -169,6 +169,13 @@ async function runTurn( ): Promise { const agent = ctx.agents.requireInitiator() const { session } = agent + const drainSteering = (): boolean => { + const messages = handle.inbox.drainSteering() + for (const message of messages) { + session.append('steering/message', { turn, content: message.content, source: message.source }, { surfaceOp: 'append' }) + } + return messages.length > 0 + } // Drain before opening the turn, but append only after `turn/start`. const queued = handle.inbox.drainQueued() @@ -267,7 +274,7 @@ async function runTurn( // Steering from the previous round's continuation listeners joins before // the request. - drainSteering(session, handle.inbox, turn) + drainSteering() // The step's AbortController exists BEFORE any async pre-step work so a // dispose() or cancel() — in a synchronous turn-start listener or an @@ -370,7 +377,7 @@ async function runTurn( if (stepReason) reason = stepReason // Steering that arrived during streaming/tool execution. - const steered = drainSteering(session, handle.inbox, turn) + const steered = drainSteering() closeStep() @@ -459,15 +466,6 @@ async function runTurn( return terminalStopped } -/** Drain the steering queue into the session. Returns whether any arrived. */ -function drainSteering(session: Session, inbox: Inbox, turn: number): boolean { - const messages = inbox.drainSteering() - for (const message of messages) { - session.append('steering/message', { turn, content: message.content, source: message.source }, { surfaceOp: 'append' }) - } - return messages.length > 0 -} - /** * Run one committed step: transform call config, log the request header, build * the request from the cached prefix plus the step-boundary snapshot, stream and @@ -543,15 +541,47 @@ async function runStep( const stepError = finishError(assembler.finish) if (stepError) throw stepError + const recordAssistantMessage = ( + assembledContent: ContentBlock[], + message: Message, + preserveReplayState = true, + ): void => { + session.append( + 'assistant/message', + { + turn, + step, + content: message.content, + provenance: assistantProvenance( + header.config, + assembler.replayState, + preserveReplayState && isDeepStrictEqual(message.content, assembledContent), + ), + ...assembler.usage === undefined ? {} : { usage: assembler.usage }, + }, + { surfaceOp: 'append', sourceEventSeqs: chunkSeqs }, + ) + } + + // A rejected result still records the successful provider call without retaining rejected output. + const processStepResult = async (assembledContent: ContentBlock[], message: Message): Promise => { + try { + return await events.waterfall( + 'agent/step-result', turn, step, message, () => Promise.resolve(message), + ) + } catch (error: unknown) { + recordAssistantMessage(assembledContent, { ...message, content: [] }, false) + throw error + } + } + if (assembler.finish.kind === 'max-tokens') { const assembled = assembler.message() const assembledContent = structuredClone(assembled.content) let message: Message = withoutToolCalls(assembled) - message = withoutToolCalls(await processStepResult( - events, session, turn, step, header.config, assembledContent, message, assembler, chunkSeqs, - )) + message = withoutToolCalls(await processStepResult(assembledContent, message)) // Preserve usage even when max-token truncation produced no content. - recordAssistantMessage(session, turn, step, header.config, assembledContent, message, assembler, chunkSeqs) + recordAssistantMessage(assembledContent, message) return { hadToolCalls: false, finish: assembler.finish } } @@ -559,13 +589,11 @@ async function runStep( const assembled = assembler.message() const assembledContent = structuredClone(assembled.content) let message: Message = assembled - message = await processStepResult( - events, session, turn, step, header.config, assembledContent, message, assembler, chunkSeqs, - ) + message = await processStepResult(assembledContent, message) // Every successful call records its completion anchor, including explicit // empty chunk provenance for a contentless, usage-less provider response. - recordAssistantMessage(session, turn, step, header.config, assembledContent, message, assembler, chunkSeqs) + recordAssistantMessage(assembledContent, message) // Dispatch may overlap; policy, durable results, and result context stay model-ordered. const toolCalls = message.content.filter(block => block.type === 'tool-call') @@ -578,67 +606,6 @@ async function runStep( }) } -/** Preserve successful-call accounting without retaining output that result processing rejected. */ -async function processStepResult( - events: AgentEventDispatch, - session: Session, - turn: number, - step: number, - config: LlmCallConfig, - assembledContent: ContentBlock[], - message: Message, - assembler: BlockAssembler, - chunkSeqs: number[], -): Promise { - try { - return await events.waterfall( - 'agent/step-result', turn, step, message, () => Promise.resolve(message), - ) - } catch (error: unknown) { - recordAssistantMessage( - session, - turn, - step, - config, - assembledContent, - { ...message, content: [] }, - assembler, - chunkSeqs, - false, - ) - throw error - } -} - -/** Record one content-or-usage assistant message with replay-safe provenance. */ -function recordAssistantMessage( - session: Session, - turn: number, - step: number, - config: LlmCallConfig, - assembledContent: ContentBlock[], - message: Message, - assembler: BlockAssembler, - chunkSeqs: number[], - preserveReplayState = true, -): void { - session.append( - 'assistant/message', - { - turn, - step, - content: message.content, - provenance: assistantProvenance( - config, - assembler.replayState, - preserveReplayState && isDeepStrictEqual(message.content, assembledContent), - ), - ...assembler.usage === undefined ? {} : { usage: assembler.usage }, - }, - { surfaceOp: 'append', sourceEventSeqs: chunkSeqs }, - ) -} - /** Build durable assistant provenance, dropping replay state after any content rewrite. */ function assistantProvenance(config: LlmCallConfig, replayState: unknown, contentUnchanged: boolean): NonNullable { return { diff --git a/packages/core/agent-loop/src/tool-calls.ts b/packages/core/agent-loop/src/tool-calls.ts index 5bf41c4895..5ec1ceb4c2 100644 --- a/packages/core/agent-loop/src/tool-calls.ts +++ b/packages/core/agent-loop/src/tool-calls.ts @@ -12,7 +12,6 @@ import type { Context } from 'cordis' import { assertNever, type ToolCallBlock } from '@deepseek-ai/dsh-llm' import type { HookContext } from '@deepseek-ai/dsh-agent' -import type { Session } from '@deepseek-ai/dsh-session' import { TOOL_REGISTRY_SCHEDULER, type ToolExecutionInput, type ToolExecutionMode, type ToolExecutionResult, type ToolRunContext } from '@deepseek-ai/dsh-tools' /** One tool call after argument parsing, ready to schedule. */ @@ -104,6 +103,29 @@ async function runGroup( acceptContext: (context: HookContext) => void, ): Promise { const { session } = ctx.agents.requireInitiator() + const appendToolCall = (block: ToolCallBlock): number => { + return session.append('tool/call', { + turn, + step, + callId: block.id, + name: block.name, + arguments: block.arguments, + }).seq + } + const appendToolResult = (block: ToolCallBlock, result: ToolExecutionResult, callSeq: number): void => { + session.append('tool/result', { + turn, + step, + // Correlation stays with the loop's authoritative model-transcript call id; + // registry results deliberately do not duplicate it. + callId: block.id, + content: result.content, + isError: result.isError, + ...result.error ? { error: result.error } : {}, + // Persist presentation payloads so UI bridges reproduce result cards on replay. + ...result.meta !== undefined ? { meta: result.meta } : {}, + }, { surfaceOp: 'append', sourceEventSeqs: [callSeq] }) + } /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) const slots: (Slot | undefined)[] = group.map(() => undefined) @@ -124,7 +146,7 @@ async function runGroup( ? await ctx.tools[TOOL_REGISTRY_SCHEDULER].finalize(slot.exec, slot.result) : ctx.tools[TOOL_REGISTRY_SCHEDULER].finish(slot.exec, slot.result) // eslint-disable-next-line @typescript-eslint/no-non-null-assertion -- bounded index - appendToolResult(session, turn, step, call!.block, result, callSeqs[committed]!) + appendToolResult(call!.block, result, callSeqs[committed]!) for (const context of result.additionalContexts ?? []) acceptContext(context) committed++ } @@ -135,7 +157,7 @@ async function runGroup( const startCall = async (index: number): Promise => { // eslint-disable-next-line @typescript-eslint/no-non-null-assertion -- bounded index const call = group[index]! - callSeqs[index] = appendToolCall(session, turn, step, call.block) + callSeqs[index] = appendToolCall(call.block) started++ const prepared = await ctx.tools[TOOL_REGISTRY_SCHEDULER].prepare(call.exec) switch (prepared.kind) { @@ -197,32 +219,3 @@ async function runGroup( if (committed !== started) throw new Error('tool-call scheduler: uncommitted settled calls') return started } - -/** Append a started call and return its provenance sequence. */ -function appendToolCall(session: Session, turn: number, step: number, block: ToolCallBlock): number { - const event = session.append('tool/call', { turn, step, callId: block.id, name: block.name, arguments: block.arguments }) - return event.seq -} - -/** Append a model-ordered result linked to its call event. */ -function appendToolResult( - session: Session, - turn: number, - step: number, - block: ToolCallBlock, - result: ToolExecutionResult, - callSeq: number, -): void { - session.append('tool/result', { - turn, step, - // Correlation stays with the loop's authoritative model-transcript call id; - // registry results deliberately do not duplicate it. - callId: block.id, - content: result.content, - isError: result.isError, - ...result.error ? { error: result.error } : {}, - // The tool's private presentation payload (e.g. a result-time diff), - // persisted so a UI bridge reproduces the card on replay. - ...result.meta !== undefined ? { meta: result.meta } : {}, - }, { surfaceOp: 'append', sourceEventSeqs: [callSeq] }) -} From 5f9fe1415623d8fe7643a4c9a0bc764d6f7c2c2e Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sun, 19 Jul 2026 16:39:58 +0800 Subject: [PATCH 54/73] docs(bash-local): clarify spill cleanup semantics --- packages/bash/bash-local/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/bash/bash-local/README.md b/packages/bash/bash-local/README.md index 8150119acd..159effec64 100644 --- a/packages/bash/bash-local/README.md +++ b/packages/bash/bash-local/README.md @@ -38,6 +38,6 @@ Indirectly, through `dsh-tool-bash`, which renders this executor's bounded stdou - **No persistent shell or PTY** — every call starts a fresh non-login `bash -c`; cwd-only persistence and interactive terminal sessions remain deferred until a real workflow requires them. - **POSIX-only** — the `bash` binary, detached process groups, group kills, and SIGTERM→SIGKILL escalation are hardcoded; Windows is unsupported. - **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. -- **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are deleted immediately. +- **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. The raw process handling lives in `src/run.ts`; `src/index.ts` is the service wiring. From a1de4d00714556c3a0d00ab0ef6728cc431c4d0d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 16:44:24 +0800 Subject: [PATCH 55/73] docs: codify initiator-owned identity plumbing --- packages/AGENTS.md | 1 + scripts/doc-budgets.manifest.json | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/AGENTS.md b/packages/AGENTS.md index 766a188105..9ab0327cf9 100644 --- a/packages/AGENTS.md +++ b/packages/AGENTS.md @@ -6,6 +6,7 @@ These package-specific rules supplement the repo-wide [conventions](../AGENTS.md - **Optional services use `ctx.get(name)`.** Reserve `ctx.` for declared injections; the property proxy is topology-sensitive, while strict `ctx.get` reads the global service store ([postmortem](../docs/postmortem/0001-acp-default-export-drops-inject.md)). - **Product-visible plugins require a non-unit REAL-composition test.** Hand-built `ctx.plugin(...)` suites are insufficient. Boot test-only `cordis.yml` through the Loader and app/process; mock only external/nondeterministic boundaries and assert model-visible, durable, or user-visible output. Keep opt-ins out of shipped defaults. [Policy](../docs/testing.md). - **Typed same-process service and plugin calls are contracts, not serialization boundaries.** Prefer readonly borrowed values; materialize or defensively validate only at parser/config, queued, model/tool JSON, durable/file, worker, process, or wire boundaries. +- **Initiator-owned private chains derive, then capture.** Under `ctx.agents.withInitiator()`, recover the Agent at each orchestration entry, derive `agent.session`, and let operation-local helpers close over it. Keep `Agent` and `Session` explicit at lifecycle, session-log, service, authority, worker/process, persistence, and wire interfaces; do not widen a leaf helper from `Session` to `Context` merely to hide a parameter ([rationale](../docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md)). - **Represent one asynchronous operation with one lifecycle controller or transaction.** Separate readiness, cancellation, disposal, reservation, or sentinel state requires an independent owner or settlement boundary; otherwise fold it while preserving rollback, callback containment, and quiescence. Naming notes: diff --git a/scripts/doc-budgets.manifest.json b/scripts/doc-budgets.manifest.json index 3b99ad454a..b4d2bca159 100644 --- a/scripts/doc-budgets.manifest.json +++ b/scripts/doc-budgets.manifest.json @@ -6,6 +6,6 @@ "docs/defensive-patterns.md": 550, "docs/testing.md": 960, "examples/AGENTS.md": 310, - "packages/AGENTS.md": 290, + "packages/AGENTS.md": 370, "packages/README.md": 760 } From d5350cd42e9c3c352567accefff480d2453c590d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 17:08:00 +0800 Subject: [PATCH 56/73] docs(review): distill human review principles --- .agents/skills/dsh-code-review/SKILL.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/.agents/skills/dsh-code-review/SKILL.md b/.agents/skills/dsh-code-review/SKILL.md index 2b562cc321..ca42b97323 100644 --- a/.agents/skills/dsh-code-review/SKILL.md +++ b/.agents/skills/dsh-code-review/SKILL.md @@ -29,19 +29,20 @@ description: Use when reviewing a pull request in the deepseek-harness repo — - **Intent and seam contracts:** trace both sides of every changed interface. Confirm the implementation matches the PR and any RFC, including errors, cancellation, ownership, and disposal. - **Lifecycle and concurrency:** for async setup, callbacks, processes, or teardown, apply [defensive-patterns.md](../../../docs/defensive-patterns.md). Check races before publication, cancellation during awaits, independent error reporting, callback containment, ownership before reentry, complete detach cleanup, and quiescent disposal. -- **Capability shape:** a swappable capability follows the interface / implementation / consumer split. Consumers depend on the interface, not a backend. -- **Scope, ownership, and necessity:** tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer. Challenge unrelated features, speculative generality, and behavior placed outside its owning plugin or service. -- **Configuration:** deployment-varying timeouts, caps, models, URLs, paths, and retry counts are validated `Config` fields, not literals or `DEFAULT_*` constants. +- **Capability and consumer fit:** a swappable capability follows the interface / implementation / consumer split. Shape the interface for all current consumers, not one tool schema, loader, UI, transport, or backend; keep consumer-specific behavior in the consumer or adapter. +- **Scope, ownership, and necessity:** tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer. Challenge unrelated features, speculative generality, and behavior placed outside its owning plugin or service. Review whether the PR is one coherent change; move an independently meaningful feature or design decision into a separate or stacked PR when combining it obscures ownership, intent, or verification. +- **Configuration and public choices:** deployment-varying tunables are validated `Config` fields. Do not invent a default, public operation set, format, or borrowed external concept merely because it can be configurable; require evidence from a current consumer or relevant prior art, otherwise keep the choice explicit or defer it. +- **Model perspective:** inspect the exact prompts, tool schemas, results, and diagnostics the model receives across affected modes. Flag UI, transport, or implementation concepts irrelevant to the model's task. Verify stable model-visible text verbatim and dynamic behavior through snapshots or end-to-end coverage. - **Enforcement boundaries:** hidden schema fields, filtered prompts, facades, wrappers, and listener ordering are not authoritative enforcement when direct or alternate callers can bypass them. Exercise denial paths at the boundary that actually executes the operation. - **Borrowed and derived state:** determine whether retained caller-owned values are borrowed or snapshotted by contract; do not demand copies at typed same-process seams. Materialize mutable values that cross queues, model/tool JSON, durable logs or files, workers, processes, or wire boundaries. Commit notifications and derived state only at the documented success boundary, and trace caches, prompts, UI echoes, replay, and query views to one authoritative source. - **Bounds cover the final operation:** verify byte, token, item, and time limits at the boundary that owns the complete emitted or retained result, including wrappers and metadata. Probe tiny limits, exact thresholds, oversized single chunks, and multibyte text for byte limits. - **Real entry path:** tests exercise the shipped Loader, bin, worker, ACP bridge, or subprocess where relevant. A hand-mounted plugin does not catch Loader export-shape failures; a function plugin must named-export its namespace and have no default export. - **Test strength:** assertions fail on the intended regression and verify external state, logs, events, or disposal rather than restating the implementation or trusting an agent's report. Coverage is necessary but not evidence that the scenario is correct. -- **Changed checks have a negative control:** a new automated check, or a changed acceptance path in one, has a deliberately invalid case that reaches the real top-level runner and fails for the intended rule; a green happy path does not prove the check is wired. +- **Mechanized invariants and negative controls:** when a repeatable invariant is mechanically checkable, require it to be wired into an executed top-level gate. A new check, or a changed acceptance path in one, also needs a deliberately invalid case that reaches the real runner and fails for the intended rule. - **Implemented RFCs match shipped reality:** when a PR implements a proposed RFC, move and rewrite it as present-tense shipped state in the same diff, then verify paths, names, and mechanisms against the implementation. - **Transcript changes:** editor-visible or model-visible changes update snapshots or explain why no snapshot applies. Review golden diffs as behavior changes, not formatting noise. - **Bilingual changes:** compare meaning and terminology on both sides; a green pairing hash does not prove translation quality. ## Reporting findings -State the defect, location, impact, and evidence. Separate blockers from suggestions and omit issues already enforced by a green gate. Use the existing GitHub review thread for replies. When receiving review, verify each claim and fix or rebut it on technical grounds without performative agreement. +State the defect, location, impact, and evidence. Place a localized defect inline on the tightest relevant diff range; use a PR-level comment for cross-cutting architecture, scope, or review-wide synthesis. Separate blockers from suggestions and omit issues already enforced by a green gate. Use the existing GitHub review thread for replies. When receiving review, verify each claim and fix or rebut it on technical grounds without performative agreement. From 4ab5a76317f6763e682f7525d913e08fb205fdd4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 17:35:43 +0800 Subject: [PATCH 57/73] docs(agents): promote review principles --- .agents/skills/dsh-code-review/SKILL.md | 20 ++++++++++---------- AGENTS.md | 6 +++--- docs/AGENTS.md | 2 +- packages/AGENTS.md | 8 ++++++++ scripts/doc-budgets.manifest.json | 6 +++--- 5 files changed, 25 insertions(+), 17 deletions(-) diff --git a/.agents/skills/dsh-code-review/SKILL.md b/.agents/skills/dsh-code-review/SKILL.md index ca42b97323..cabc6211ff 100644 --- a/.agents/skills/dsh-code-review/SKILL.md +++ b/.agents/skills/dsh-code-review/SKILL.md @@ -9,7 +9,7 @@ description: Use when reviewing a pull request in the deepseek-harness repo — ## Sources of truth -- [AGENTS.md](../../../AGENTS.md) and [packages/AGENTS.md](../../../packages/AGENTS.md): repository and package rules. +- [AGENTS.md](../../../AGENTS.md) and [packages/AGENTS.md](../../../packages/AGENTS.md): standing repository and package authoring contracts. - [docs/defensive-patterns.md](../../../docs/defensive-patterns.md): subprocess, callback, async-state, and disposal bug classes. - [docs/AGENTS.md](../../../docs/AGENTS.md): documentation placement and prose discipline. - [dsh-prose-standard](../dsh-prose-standard/SKILL.md): required coverage and editorial judgment for comments, docs, prompts, and visible strings. @@ -22,23 +22,23 @@ description: Use when reviewing a pull request in the deepseek-harness repo — 1. **New prose receives semantic review.** Use [dsh-prose-standard](../dsh-prose-standard/SKILL.md) to critically review every added or changed Markdown passage, JSDoc, comment, prompt, description, diagnostic, and visible string. Verify required coverage, accuracy, placement, and editorial quality against the owning code or behavior; automated checks do not establish those properties. 2. **Docs match the code.** Config, defaults, errors, wire fields, events, and public behavior update the package README and JSDoc in the same diff. Comments state non-obvious contracts; flag implementation narration, test walkthroughs, review history, and duplicated rationale for deletion or a link to their one home. 3. **Core type docs match.** Changes to spine or seam vocabulary update the appropriate [core-data-structures](../../../docs/core-data-structures/core.md) page and any `type-equiv` entry. Internal types need no catalog entry. -4. **Registrations clean up.** A new registry contribution has a test that disposes its owner and observes removal. +4. **Registrations clean up.** Verify each new registry contribution satisfies the disposal-test contract in [packages/AGENTS.md](../../../packages/AGENTS.md). 5. **Required gates pass.** Trust the [current readiness sequence](../../../AGENTS.md#run-the-ci-gates-locally-before-marking-a-pr-ready) and `pnpm run check:pre-push` for their enforced inventory; review the semantic gaps they cannot detect. ## Manual checks - **Intent and seam contracts:** trace both sides of every changed interface. Confirm the implementation matches the PR and any RFC, including errors, cancellation, ownership, and disposal. - **Lifecycle and concurrency:** for async setup, callbacks, processes, or teardown, apply [defensive-patterns.md](../../../docs/defensive-patterns.md). Check races before publication, cancellation during awaits, independent error reporting, callback containment, ownership before reentry, complete detach cleanup, and quiescent disposal. -- **Capability and consumer fit:** a swappable capability follows the interface / implementation / consumer split. Shape the interface for all current consumers, not one tool schema, loader, UI, transport, or backend; keep consumer-specific behavior in the consumer or adapter. -- **Scope, ownership, and necessity:** tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer. Challenge unrelated features, speculative generality, and behavior placed outside its owning plugin or service. Review whether the PR is one coherent change; move an independently meaningful feature or design decision into a separate or stacked PR when combining it obscures ownership, intent, or verification. -- **Configuration and public choices:** deployment-varying tunables are validated `Config` fields. Do not invent a default, public operation set, format, or borrowed external concept merely because it can be configurable; require evidence from a current consumer or relevant prior art, otherwise keep the choice explicit or defer it. -- **Model perspective:** inspect the exact prompts, tool schemas, results, and diagnostics the model receives across affected modes. Flag UI, transport, or implementation concepts irrelevant to the model's task. Verify stable model-visible text verbatim and dynamic behavior through snapshots or end-to-end coverage. -- **Enforcement boundaries:** hidden schema fields, filtered prompts, facades, wrappers, and listener ordering are not authoritative enforcement when direct or alternate callers can bypass them. Exercise denial paths at the boundary that actually executes the operation. -- **Borrowed and derived state:** determine whether retained caller-owned values are borrowed or snapshotted by contract; do not demand copies at typed same-process seams. Materialize mutable values that cross queues, model/tool JSON, durable logs or files, workers, processes, or wire boundaries. Commit notifications and derived state only at the documented success boundary, and trace caches, prompts, UI echoes, replay, and query views to one authoritative source. -- **Bounds cover the final operation:** verify byte, token, item, and time limits at the boundary that owns the complete emitted or retained result, including wrappers and metadata. Probe tiny limits, exact thresholds, oversized single chunks, and multibyte text for byte limits. +- **Capability and consumer fit:** trace every current consumer, then flag consumer-specific behavior leaking into the interface under [the package contract](../../../packages/AGENTS.md). +- **Scope, ownership, and necessity:** map each abstraction, state machine, option, defensive copy, and compatibility path to its current contract, production consumer, and owning plugin or service. Challenge unrelated features and speculative generality, then test the PR's coherence against [the root contract](../../../AGENTS.md#conventions). +- **Configuration and public choices:** ask what current-consumer evidence or prior art supports each default, public operation set, format, or imported external concept. Require an explicit choice or deferral when that evidence is absent. +- **Model perspective:** inspect the exact prompts, tool schemas, results, and diagnostics the model receives across affected modes. Flag concepts outside the model's task, then verify stable text verbatim and dynamic behavior through snapshots or end-to-end coverage. +- **Enforcement boundaries:** follow every denial path to the operation that executes it; exercise direct and alternate callers that can bypass schemas, prompts, facades, wrappers, or listener ordering. +- **Borrowed and derived state:** classify each retained value under the package boundary contract, then trace notifications and every cache, prompt, UI echo, replay, and query view to the documented success point and authoritative source. +- **Bounds cover the final operation:** locate the owner of the complete emitted or retained result, including wrappers and metadata. Probe tiny and exact limits, oversized single chunks, and multibyte text for byte limits. - **Real entry path:** tests exercise the shipped Loader, bin, worker, ACP bridge, or subprocess where relevant. A hand-mounted plugin does not catch Loader export-shape failures; a function plugin must named-export its namespace and have no default export. - **Test strength:** assertions fail on the intended regression and verify external state, logs, events, or disposal rather than restating the implementation or trusting an agent's report. Coverage is necessary but not evidence that the scenario is correct. -- **Mechanized invariants and negative controls:** when a repeatable invariant is mechanically checkable, require it to be wired into an executed top-level gate. A new check, or a changed acceptance path in one, also needs a deliberately invalid case that reaches the real runner and fails for the intended rule. +- **Mechanized invariants and negative controls:** trace each new or changed check through the executed top-level gate and its deliberately invalid case; confirm the real runner fails for the intended rule. - **Implemented RFCs match shipped reality:** when a PR implements a proposed RFC, move and rewrite it as present-tense shipped state in the same diff, then verify paths, names, and mechanisms against the implementation. - **Transcript changes:** editor-visible or model-visible changes update snapshots or explain why no snapshot applies. Review golden diffs as behavior changes, not formatting noise. - **Bilingual changes:** compare meaning and terminology on both sides; a green pairing hash does not prove translation quality. diff --git a/AGENTS.md b/AGENTS.md index efaa29bbac..27b5a655e2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,7 +110,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Plugins, not loop changes**: new behavior goes on the documented extension seams; changing `agent-loop` requires updating docs/architecture.md. - **Capability seams are three packages** — interface / implementation / consumer; don't split preemptively. - **Explicit > implicit at package seams**: defaulting is an explicit `resolve(request): Spec` step in the owning implementation, never a hidden `?? default` inside `run()` (the `dsh-bash` request/spec split is the template). -- **No hardcoded tunables in plugins**: deployment choices are defaulted, validated `Config` fields changeable from cordis.yml; a `DEFAULT_*` constant or test seam is not configurability. Protocol constants, external specs, and security invariants stay fixed. +- **No hardcoded tunables in plugins**: deployment-varying choices are validated `Config` fields changeable from cordis.yml; a `DEFAULT_*` constant or test seam is not configurability. Protocol constants, external specs, and security invariants stay fixed. - **Misconfiguration fails loud** at load when self-contained, otherwise at the earliest resolvable point; never silently skip a missing referent. - **Opaque cross-boundary ids are branded** (`Branded` from `dsh-brand`), never bare `string`. - **An empty `catch` names what it swallows** and why nothing else can reach it; keep the `try` to one statement. @@ -120,7 +120,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Testing policy** — [docs/testing.md](docs/testing.md). Transcript changes need snapshots or a PR note. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's ACP render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces, and schedule any missing harness support before implementation. -- **Merge PRs with merge commits**, never squash/rebase or rewrite pushed branches. Put a review fix on its introducing PR, then merge down the stack ([guide](docs/cookbook/responding-to-pr-review-on-a-stack.md)). +- **Keep PRs coherent and merge with merge commits.** Split an independently meaningful feature or design decision into a separate or stacked PR when combining it obscures ownership, intent, or verification. Never squash/rebase or rewrite pushed branches; put a review fix on its introducing PR, then merge down the stack ([guide](docs/cookbook/responding-to-pr-review-on-a-stack.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --check` (pre-push) gates it. @@ -132,7 +132,7 @@ Read [docs/defensive-patterns.md](docs/defensive-patterns.md) before lifecycle, Everything compiles under `strict: true` with `noImplicitAny`; every remaining `any` explains why a narrower type is infeasible. Every module and export has concise JSDoc for its non-obvious contract; function-like exports include `@param`/`@returns`, as enforced by `verify-export-jsdoc`. Heritage-declared members, plugin-protocol slots, and constructors keep their docs at the declaring seam, protocol, or class. -Comments and docs preserve complete contracts and non-obvious orientation, not reasoning transcripts. Do not narrate control flow or tests, preserve review history, or restate code. Keep factual clauses affecting behavior, failure, timing, ownership, or safe use; link aggressively to owning rationale. Use [dsh-prose-standard](.agents/skills/dsh-prose-standard/SKILL.md) for prose decisions. Encode enforceable invariants in checks, using narrow justified exceptions rather than disabling a rule globally. +Comments and docs preserve complete contracts and non-obvious orientation, not reasoning transcripts. Do not narrate control flow or tests, preserve review history, or restate code. Keep factual clauses affecting behavior, failure, timing, ownership, or safe use; link aggressively to owning rationale. Use [dsh-prose-standard](.agents/skills/dsh-prose-standard/SKILL.md) for prose decisions. Wire mechanically checkable invariants into an executed top-level gate and prove each new or changed acceptance path rejects an invalid case. Use narrow justified exceptions instead of disabling a rule globally. Docs are part of every change: code changes update their README and JSDoc in the SAME change; a bilingual-pair edit updates the counterpart and re-records ([i18n contract](docs/i18n/README.md)). The writing rules — document the current state never the history, one physical line per paragraph, one home per fact — and the word-budget gate live in [docs/AGENTS.md](docs/AGENTS.md). diff --git a/docs/AGENTS.md b/docs/AGENTS.md index f7635c659b..a0c30c5545 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -43,7 +43,7 @@ When the gate goes red: 2. **Condense** content that belongs here but can be shorter. 3. **Raise** the ceiling only when the words truly need the space; justify the manifest diff in the PR. A too-low ceiling is a budget bug. -Ceilings are guardrails, not reduction targets. Retain at least 5% headroom; lower a ceiling only when the document's durable contract still has room, and raise it when necessary content would otherwise be deleted. Targets: root `AGENTS.md` ≤ 1,500 words; `architecture.md` ≤ 1,800; each subtree `AGENTS.md` ≤ 600, except this file ≤ 1,250; `packages/README.md` ≤ 600. Review and the slop checklist govern unbudgeted tiers. +Ceilings are guardrails, not reduction targets. Retain at least 5% headroom; lower a ceiling only when the document's durable contract still has room, and raise it when necessary content would otherwise be deleted. Targets: root `AGENTS.md` ≤ 1,600 words; `architecture.md` ≤ 1,800; each subtree `AGENTS.md` ≤ 600, except `packages/AGENTS.md` ≤ 650 and this file ≤ 1,250; `packages/README.md` ≤ 600. Review and the slop checklist govern unbudgeted tiers. ## The slop checklist diff --git a/packages/AGENTS.md b/packages/AGENTS.md index 9ab0327cf9..e4b5ded6a3 100644 --- a/packages/AGENTS.md +++ b/packages/AGENTS.md @@ -8,6 +8,14 @@ These package-specific rules supplement the repo-wide [conventions](../AGENTS.md - **Typed same-process service and plugin calls are contracts, not serialization boundaries.** Prefer readonly borrowed values; materialize or defensively validate only at parser/config, queued, model/tool JSON, durable/file, worker, process, or wire boundaries. - **Initiator-owned private chains derive, then capture.** Under `ctx.agents.withInitiator()`, recover the Agent at each orchestration entry, derive `agent.session`, and let operation-local helpers close over it. Keep `Agent` and `Session` explicit at lifecycle, session-log, service, authority, worker/process, persistence, and wire interfaces; do not widen a leaf helper from `Session` to `Context` merely to hide a parameter ([rationale](../docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md)). - **Represent one asynchronous operation with one lifecycle controller or transaction.** Separate readiness, cancellation, disposal, reservation, or sentinel state requires an independent owner or settlement boundary; otherwise fold it while preserving rollback, callback containment, and quiescence. +- **Shape capability interfaces around all current consumers.** Keep tool-schema, Loader, UI, transport, and backend-specific behavior in the consumer or adapter; do not let one consumer dictate the interface ([capability-seam rationale](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)). +- **Require a current owner and need.** Tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer, and keep behavior in its owning plugin or service. +- **Require evidence for public choices.** Configurability does not justify an unsupported default, public operation set, format, or imported external concept. Use current-consumer evidence or relevant prior art; otherwise require an explicit value or defer the choice. +- **Write model-facing contracts from the model's perspective.** Prompts, tool schemas, results, and diagnostics contain only task-relevant concepts, not UI, transport, or implementation vocabulary. Pin stable model-visible text verbatim and dynamic behavior through snapshots or end-to-end coverage. +- **Enforce at the operation boundary that owns the decision.** Schema omission, prompt filtering, facades, wrappers, and listener order are not enforcement when direct or alternate callers can bypass them; test denial through the executor. +- **Publish state only at its commit point.** Emit each notification and update derived state only after the success boundary that makes it true; derive caches, prompts, UI echoes, replay, and query views from one authoritative source. +- **Apply bounds to the complete result.** Enforce byte, token, item, and time limits where the complete emitted or retained value, including wrappers and metadata, is known; test tiny and exact limits, oversized single chunks, and multibyte byte limits. +- **Registry contributions prove disposal.** Add the HMR-safety test required by the [testing policy](../docs/testing.md): dispose the contributing fiber and observe removal. Naming notes: diff --git a/scripts/doc-budgets.manifest.json b/scripts/doc-budgets.manifest.json index b4d2bca159..d849248189 100644 --- a/scripts/doc-budgets.manifest.json +++ b/scripts/doc-budgets.manifest.json @@ -1,11 +1,11 @@ { - "AGENTS.md": 1500, - "docs/AGENTS.md": 1100, + "AGENTS.md": 1600, + "docs/AGENTS.md": 1150, "docs/architecture.md": 1790, "docs/cordis-primer.md": 600, "docs/defensive-patterns.md": 550, "docs/testing.md": 960, "examples/AGENTS.md": 310, - "packages/AGENTS.md": 370, + "packages/AGENTS.md": 650, "packages/README.md": 760 } From bcc920369c5db08fdbd5ca0e40fac377955f25bf Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 19 Jul 2026 17:39:50 +0800 Subject: [PATCH 58/73] docs: document package KV cache effects --- docs/cookbook/adding-a-package.i18n.yaml | 4 +- docs/cookbook/adding-a-package.md | 8 ++- docs/cookbook/adding-a-package.zh.md | 8 ++- ...07-12-package-model-experience-contract.md | 14 ++--- packages/AGENTS.md | 2 +- packages/bash/bash-local/README.md | 2 + packages/bash/bash-sandbox/README.md | 6 ++ packages/bash/bash/README.md | 2 + packages/bash/tool-bash/README.md | 10 ++++ .../code-runtime-worker/README.md | 2 + packages/code-runtime/code-runtime/README.md | 2 + packages/compact/compact-basic/README.md | 6 ++ packages/compact/compact/README.md | 4 ++ packages/context/time-context/README.md | 2 + packages/context/workspace-context/README.md | 6 ++ packages/cordis/tool-cordis/README.md | 6 ++ packages/core/agent-loop/README.md | 6 ++ packages/core/agent/README.md | 4 ++ packages/core/session/README.md | 6 ++ packages/core/system-prompt/README.md | 4 ++ packages/core/tools/README.md | 6 ++ packages/examples/acp-demo/README.md | 2 + packages/examples/agent-spine-demo/README.md | 2 + packages/examples/cli-demo/README.md | 2 + packages/examples/jsonrpc-demo/README.md | 2 + packages/examples/stdio-demo/README.md | 4 ++ packages/fs/fs-local/README.md | 2 + packages/fs/fs-policy/README.md | 2 + packages/fs/fs/README.md | 2 + packages/fs/tool-fs-search/README.md | 8 +++ packages/fs/tool-fs/README.md | 10 ++++ packages/guard/repeat-tool-guard/README.md | 4 ++ packages/hooks/hook-protocol/README.md | 2 + packages/hooks/hooks-claude/README.md | 4 ++ packages/hooks/hooks-codex/README.md | 4 ++ packages/llm/llm-deepseek/README.md | 4 ++ packages/llm/llm-pi-ai/README.md | 4 ++ packages/llm/llm/README.md | 2 + packages/llm/token-meter/README.md | 2 + packages/mcp/mcp-client/README.md | 4 ++ packages/sandbox/sandbox-local/README.md | 2 + packages/sandbox/sandbox/README.md | 2 + packages/sdk/create-sdk/README.md | 2 + packages/sdk/helper/README.md | 2 + packages/sdk/scripts/README.md | 2 + .../session-persistence-jsonl/README.md | 2 + .../session-persistence-sqlite/README.md | 2 + .../session-persistence/README.md | 2 + .../session-query/session-query/README.md | 2 + packages/skill/skill-local/README.md | 2 + packages/skill/skill/README.md | 2 + packages/skill/tool-skill/README.md | 8 +++ packages/spill/spill-local/README.md | 2 + packages/spill/spill-policy/README.md | 2 + packages/spill/spill/README.md | 2 + packages/subagent/subagent-acp/README.md | 4 ++ packages/subagent/subagent-fork/README.md | 4 ++ .../subagent/subagent-inprocess/README.md | 8 +++ packages/subagent/subagent-spawn/README.md | 4 ++ .../subagent/subagent-subprocess/README.md | 2 + packages/subagent/subagent/README.md | 2 + packages/subagent/tool-subagent/README.md | 6 ++ packages/support/acp-snapshot/README.md | 2 + packages/support/agent-loop-testkit/README.md | 2 + packages/support/invariants/README.md | 2 + packages/support/llm-replay/README.md | 2 + packages/support/loader-smoke/README.md | 2 + packages/tasks/tasks/README.md | 2 + packages/tasks/tool-tasks/README.md | 6 ++ packages/timeout/timeout-policy/README.md | 2 + packages/todo/tool-todo/README.md | 4 ++ packages/ui/acp/README.md | 10 ++++ packages/ui/app-boot/README.md | 2 + packages/ui/jsonrpc/README.md | 2 + packages/ui/permission/README.md | 2 + packages/ui/stdio/README.md | 4 ++ packages/ui/tool-ask-user/README.md | 4 ++ packages/ui/tui/README.md | 4 ++ packages/ui/user-approval/README.md | 4 ++ packages/ui/user-interaction/README.md | 2 + packages/util/home/README.md | 2 + packages/util/retention/README.md | 2 + packages/util/timeout/README.md | 2 + packages/web/tool-web/README.md | 10 ++++ packages/web/web-fetch-local/README.md | 2 + packages/web/web-search-deepseek/README.md | 4 ++ packages/web/web-search-exa/README.md | 2 + packages/web/web-search-perplexity/README.md | 4 ++ packages/web/web/README.md | 2 + packages/workflow/tool-workflow/README.md | 6 ++ .../workflow/workflow-workerthread/README.md | 4 ++ packages/workflow/workflow/README.md | 2 + .../verify-package-readme-model-experience.ts | 58 +++++++++++++------ 93 files changed, 366 insertions(+), 34 deletions(-) diff --git a/docs/cookbook/adding-a-package.i18n.yaml b/docs/cookbook/adding-a-package.i18n.yaml index 27c31ba1ef..692c7448ca 100644 --- a/docs/cookbook/adding-a-package.i18n.yaml +++ b/docs/cookbook/adding-a-package.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -adding-a-package.md: 2930cee9ab64b382f6211335ae639bce45629d1d -adding-a-package.zh.md: 10e906c320203c3658c103fc8936540f72697d65 +adding-a-package.md: 2e4374caff589199288e67bbca58604231a17966 +adding-a-package.zh.md: 35b5b9a0705dd0b752d68093769a65c339f6b4e3 diff --git a/docs/cookbook/adding-a-package.md b/docs/cookbook/adding-a-package.md index 2930cee9ab..2e4374caff 100644 --- a/docs/cookbook/adding-a-package.md +++ b/docs/cookbook/adding-a-package.md @@ -16,7 +16,7 @@ packages/// src/index.ts # service default export or plugin (name/inject/apply/Config) tests/.spec.ts README.md # service API, events, extension points, design notes, - # + gated Model Experience context blocks or short sentence + # + gated Model Experience context blocks or short form # + the gated "Known Limitations and Deferred Work" section # (or a whitelist entry in scripts/verify-package-readme-limitations.ts) ``` @@ -55,6 +55,8 @@ Keep package-specific service API, config, events, extension points, and design **Token effect**: Fixed, conditional, retained, replaced, capped, or zero-direct token effect. +**KV Cache effect**: Append-only, prefix-stable, replacing, or independent behavior, including the exact conditions that may invalidate reuse. + #### Verbatim text for this context surface, when needed ```markdown @@ -66,9 +68,9 @@ Stable system-prompt prose of any length, or another long non-generated literal, - **Consumer-visible gap** — exact boundary, consequence, or maintainer constraint. ```` -Fill Model Experience from the implementation. Use one H3 per direct, conditional, capped, lifetime, or auxiliary-model surface, with the two fields shown above. Quote stable text owned by the package: system-prompt prose goes in a titled H4 plus `markdown` fence, other short literals stay inline with named placeholders, and other long literals use the same nested form. Summarize only data-dependent or provider-owned text. A tool-schema surface links its anchored section in the generated [tool catalog](../tool-catalog.md) and states only deltas absent there. Keep prompt and schema surfaces separate when scoping can hide one without the other. The [prose standard](../../.agents/skills/dsh-prose-standard/SKILL.md) governs completeness and ownership; the verifier enforces the mechanical shape. +Fill Model Experience from the implementation. Use one H3 per direct, conditional, capped, lifetime, or auxiliary-model surface, with the three fields shown above. Quote stable text owned by the package: system-prompt prose goes in a titled H4 plus `markdown` fence, other short literals stay inline with named placeholders, and other long literals use the same nested form. Summarize only data-dependent or provider-owned text. A tool-schema surface links its anchored section in the generated [tool catalog](../tool-catalog.md) and states only deltas absent there. Keep prompt and schema surfaces separate when scoping can hide one without the other. In `KV Cache effect`, distinguish append-only growth, a stable repeated prefix, replacement of earlier request tokens, and an independent model request, then name the package-owned changes that can invalidate reuse. “Does not invalidate” means the package preserves an already-reusable prefix; provider cache availability and eviction remain outside the package contract. The [prose standard](../../.agents/skills/dsh-prose-standard/SKILL.md) governs completeness and ownership; the verifier enforces the mechanical shape. -A package with no context effect or one consumer-owned path uses the audited `None, as ` or `Indirectly, through ` sentence in [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts); a model-agnostic generic package may instead join `NO_MODEL_EXPERIENCE_SECTION`. Do not expand either case into a description of another package's work. The limitations [allowlist](../../scripts/verify-package-readme-limitations.ts) is independent. The [Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) records the rationale. +A package with no context effect or one consumer-owned path uses the audited `None, as ` or `Indirectly, through ` sentence in [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts), followed by a non-empty `**KV Cache effect**` field; a model-agnostic generic package may instead join `NO_MODEL_EXPERIENCE_SECTION`. Do not expand either case into a description of another package's work. The limitations [allowlist](../../scripts/verify-package-readme-limitations.ts) is independent. The [Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) records the rationale. ## 5. Verify diff --git a/docs/cookbook/adding-a-package.zh.md b/docs/cookbook/adding-a-package.zh.md index 10e906c320..35b5b9a070 100644 --- a/docs/cookbook/adding-a-package.zh.md +++ b/docs/cookbook/adding-a-package.zh.md @@ -16,7 +16,7 @@ packages/// src/index.ts # service default export or plugin (name/inject/apply/Config) tests/.spec.ts README.md # service API, events, extension points, design notes, - # + gated Model Experience context blocks or short sentence + # + gated Model Experience context blocks or short form # + the gated "Known Limitations and Deferred Work" section # (or a whitelist entry in scripts/verify-package-readme-limitations.ts) ``` @@ -55,6 +55,8 @@ package.json 不变式（由 `pnpm run constraints` / `scripts/check-workspace-c **Token effect**: Fixed, conditional, retained, replaced, capped, or zero-direct token effect. +**KV Cache effect**: Append-only, prefix-stable, replacing, or independent behavior, including the exact conditions that may invalidate reuse. + #### Verbatim text for this context surface, when needed ```markdown @@ -66,9 +68,9 @@ Stable system-prompt prose of any length, or another long non-generated literal, - **Consumer-visible gap** — exact boundary, consequence, or maintainer constraint. ```` -根据实现填写 Model Experience。每个直接、条件、上限、生命周期或辅助模型的 surface 使用一个 H3，包含上述两个字段。引用包拥有的稳定文本：系统提示词放在带标题的 H4 加 `markdown` 围栏中，其他短文本以命名占位符内联，其他长文本使用相同的嵌套形式。仅概述数据依赖或提供方拥有的文本。tool-schema surface 链接到生成的[工具目录](../tool-catalog.md)中对应的锚定章节，仅说明该处缺失的差异。当作用域可以隐藏 prompt 或 schema 其中之一而不影响另一个时，将二者分开。[行文标准](../../.agents/skills/dsh-prose-standard/SKILL.md)约束完整性与归属；验证器强制执行机械形状。 +根据实现填写 Model Experience。每个直接、条件、上限、生命周期或辅助模型的 surface 使用一个 H3，包含上述三个字段。引用包拥有的稳定文本：系统提示词放在带标题的 H4 加 `markdown` 围栏中，其他短文本以命名占位符内联，其他长文本使用相同的嵌套形式。仅概述数据依赖或提供方拥有的文本。tool-schema surface 链接到生成的[工具目录](../tool-catalog.md)中对应的锚定章节，仅说明该处缺失的差异。当作用域可以隐藏 prompt 或 schema 其中之一而不影响另一个时，将二者分开。填写 `KV Cache effect` 时，应区分仅追加增长、稳定重复的前缀、替换既有请求 token 和独立模型请求，并列出会使缓存复用失效、且由本包拥有的变化。“不使缓存失效”仅表示本包保留了已有的可复用前缀；缓存是否可用以及何时淘汰不属于本包契约。[行文标准](../../.agents/skills/dsh-prose-standard/SKILL.md)约束完整性与归属；验证器强制执行机械形状。 -没有上下文效果或仅有消费方拥有路径的包使用 [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts) 中经过审计的 `None, as ` 或 `Indirectly, through ` 语句；与模型无关的通用包可以改为加入 `NO_MODEL_EXPERIENCE_SECTION`。两种情况都不要展开为对另一个包工作的描述。limitations [allowlist](../../scripts/verify-package-readme-limitations.ts) 独立管理。[Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) 记录了设计动机。 +没有上下文效果或仅有消费方拥有路径的包使用 [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts) 中经过审计的 `None, as ` 或 `Indirectly, through ` 语句，随后添加非空的 `**KV Cache effect**` 字段；与模型无关的通用包可以改为加入 `NO_MODEL_EXPERIENCE_SECTION`。两种情况都不要展开为对另一个包工作的描述。limitations [allowlist](../../scripts/verify-package-readme-limitations.ts) 独立管理。[Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) 记录了设计动机。 ## 5. 验证 diff --git a/docs/rfc/implemented/process/2026-07-12-package-model-experience-contract.md b/docs/rfc/implemented/process/2026-07-12-package-model-experience-contract.md index 0f9b0b02a0..5fd5d2b7c2 100644 --- a/docs/rfc/implemented/process/2026-07-12-package-model-experience-contract.md +++ b/docs/rfc/implemented/process/2026-07-12-package-model-experience-contract.md @@ -4,28 +4,28 @@ Status: implemented ## Problem -A package README can explain APIs and runtime mechanics without answering the question that dominates an agent harness's behavior and cost: what from this package reaches a model request, under which conditions, and how long those tokens remain. The omission is especially hard to audit in a plugin architecture. A consumer may turn a backend result into a tool message, a policy plugin may replace success with an error, compaction may remove old history, and an agent-scoped registration may change one agent's prompt or schemas while leaving every other agent unchanged. Reading only the nominally model-facing packages therefore misses real context effects, while reading source across every dependency is too expensive for routine review. +A package README can explain APIs and runtime mechanics without answering the questions that dominate an agent harness's behavior and cost: what from this package reaches a model request, under which conditions, how long those tokens remain, and whether later requests preserve a reusable KV-cache prefix. The omission is especially hard to audit in a plugin architecture. A consumer may turn a backend result into a tool message, a policy plugin may replace success with an error, compaction may remove old history, and an agent-scoped registration may change one agent's prompt or schemas while leaving every other agent unchanged. Reading only the nominally model-facing packages therefore misses real context effects, while reading source across every dependency is too expensive for routine review. ## Decision Every workspace package README with a model-facing or model-adjacent contract ends with the canonical [Model Experience section](../../../cookbook/adding-a-package.md#4-write-the-package-readme), immediately before `## Known Limitations and Deferred Work`; a package on the no-limitations allowlist ends with Model Experience itself. An audited model-agnostic generic package omits the section through `NO_MODEL_EXPERIENCE_SECTION`. -Packages with direct, conditional, capped, lifetime, multi-surface, or auxiliary-model effects use one H3 per context surface. Each names what the relevant model receives and when, then classifies the token effect. Stable package-owned text is quoted exactly: system-prompt prose and other long literals use a nested H4 plus `markdown` fence, while short literals stay inline with named interpolation placeholders. Tool-schema surfaces link their anchored section in the generated [tool catalog](../../../tool-catalog.md) and state only composition or configuration deltas; runtime-only definitions explain why the catalog omits them. Data-dependent and provider-owned text is summarized. Agent-scoped visibility is explicit, and prompt and schema surfaces remain separate when scoping can hide one without the other. +Packages with direct, conditional, capped, lifetime, multi-surface, or auxiliary-model effects use one H3 per context surface. Each names what the relevant model receives and when, classifies the token effect, and states the KV-cache effect. The cache field distinguishes append-only growth, a stable repeated prefix, replacement of earlier tokens, and an independent model request; it names every package-owned configuration, scope, lifecycle, compaction, or routing change that can alter the request before newly appended content. “Does not invalidate” means the package preserves an already-reusable prefix, not that a provider promises a cache hit or retention period. Stable package-owned text is quoted exactly: system-prompt prose and other long literals use a nested H4 plus `markdown` fence, while short literals stay inline with named interpolation placeholders. Tool-schema surfaces link their anchored section in the generated [tool catalog](../../../tool-catalog.md) and state only composition or configuration deltas; runtime-only definitions explain why the catalog omits them. Data-dependent and provider-owned text is summarized. Agent-scoped visibility is explicit, and prompt and schema surfaces remain separate when scoping can hide one without the other. -A package with no model-context effect, or one path rendered entirely by another package, uses the verifier's audited one-sentence form: `None, as ` or `Indirectly, through `. Pure transport and keyless test-support packages use the none form when they create no model-bound content. Provider backends use the indirect form even when they cap or filter data, and wiring bundles use it when named children own every effect. These sentences locate the contribution without restating the consumer. Structured sections likewise document only package-owned inputs, transformations, and deltas. +A package with no model-context effect, or one path rendered entirely by another package, uses the verifier's audited short form: one sentence beginning `None, as ` or `Indirectly, through ` followed by a `**KV Cache effect**` field. Pure transport and keyless test-support packages use the none form when they create no model-bound content. Provider backends use the indirect form even when they cap or filter data, and wiring bundles use it when named children own every effect. These sections locate the contribution and disclaim direct cache invalidation without restating the consumer. Structured sections likewise document only package-owned inputs, transformations, and deltas. -`verify-package-readme-model-experience` discovers package manifests and validates the three classifications, canonical final-section order, required fields, concrete literal evidence, nested verbatim blocks, and anchored tool-catalog links. It runs in `doc-sync` and the parallel gate runner. Review still owns coverage, link relevance, and factual accuracy. +`verify-package-readme-model-experience` discovers package manifests and validates the three classifications, canonical final-section order, required model-view, token, and KV-cache fields, concrete literal evidence, nested verbatim blocks, and anchored tool-catalog links. It runs in `doc-sync` and the parallel gate runner. Review still owns coverage, link relevance, and factual accuracy. ## Alternatives considered - **Document only packages that register prompts or tools** — rejected because backends, policy plugins, adapters, persistence, scoping, and compaction change the content or lifetime of tokens without owning a model-facing schema. - **Generate one central context-cost catalog from source** — rejected because an AST can find registrations but cannot infer semantic conditions such as history retention, output truncation, parent-versus-child visibility, or an auxiliary model boundary. The package README is the implementation-local contract; a central copy would add another drift surface. - **Require numeric token counts** — rejected because exact counts depend on the selected model tokenizer, adapter serialization, configuration, and runtime data. The stable contract is the growth shape: fixed per request, conditional per call, retained, replaced, capped, or zero-direct. -- **Use a three-column table** — rejected because exact source text and conditional result shapes make cells dense and difficult to scan. Repeated subsections give each context surface readable vertical space while preserving the same fields. +- **Use a table** — rejected because exact source text and conditional result shapes make cells dense and difficult to scan. Repeated subsections give each context surface readable vertical space while preserving the same fields. - **Allow every zero-impact package to omit the section** — rejected because unconstrained absence is ambiguous between an audited zero and forgotten documentation. Omission is reserved for model-agnostic generic packages named with a reason in the verifier; model-adjacent zero-impact packages keep one explicit sentence. -- **Require the full structured form for audited zero or simple indirect packages** — rejected because it repeats labels around one fact. A gated sentence preserves explicit coverage without the ceremony. +- **Require the full structured form for audited zero or simple indirect packages** — rejected because it repeats labels around one fact. A gated sentence plus cache field preserves explicit coverage without the ceremony. - **Convention without a gate** — rejected because a repo-wide contract must also cover every future package; review memory cannot reliably detect an omitted README section. ## Consequences -A reviewer can start at any model-facing or model-adjacent package and see its contribution to the conversation model, child models, and auxiliary calls without reconstructing the full plugin graph. Token-budget work can distinguish repeated request overhead from data-dependent history, and agent-scoped changes have an explicit documentation checkpoint. Package authors maintain one or more compact context-surface blocks or one classified sentence whenever model-visible behavior changes; audited generic packages carry no irrelevant model boilerplate. The structured fields do not promise provider-exact token counts; measurements remain model- and workload-specific, while the documented growth and visibility contract stays stable. +A reviewer can start at any model-facing or model-adjacent package and see its contribution to the conversation model, child models, and auxiliary calls without reconstructing the full plugin graph. Token-budget work can distinguish repeated request overhead from data-dependent history, while cache-sensitive work can identify append-only paths and the earliest package-owned prefix mutation. Agent-scoped changes have an explicit documentation checkpoint. Package authors maintain one or more compact context-surface blocks or one classified short form whenever model-visible behavior changes; audited generic packages carry no irrelevant model boilerplate. The structured fields do not promise provider-exact token counts or cache hits; measurements remain model-, provider-, and workload-specific, while the documented growth, visibility, and prefix-stability contract stays stable. diff --git a/packages/AGENTS.md b/packages/AGENTS.md index 9ab0327cf9..38713e2e58 100644 --- a/packages/AGENTS.md +++ b/packages/AGENTS.md @@ -14,5 +14,5 @@ Naming notes: - `src/types.ts` contains only types — no runtime code. - Tests live at package level under `tests/`, not `src/__tests__/`. - A package's README and JSDoc are part of the change: altered behavior (config keys, defaults, error codes, wire fields) updates them in the same commit. `doc-sync` gates what it can; apply [dsh-prose-standard](../.agents/skills/dsh-prose-standard/SKILL.md) for complete, concise prose and verify accuracy against code. -- Package READMEs document model/token effects using the [canonical Model Experience format](../docs/cookbook/adding-a-package.md#4-write-the-package-readme). +- Package READMEs document model, token, and KV-cache effects using the [canonical Model Experience format](../docs/cookbook/adding-a-package.md#4-write-the-package-readme). - Package READMEs put durable consumer gaps and non-obvious maintainer constraints under `## Known Limitations and Deferred Work`; ordinary cleanup stays in its TODO or RFC. Packages with none use a justified [allowlist entry](../scripts/verify-package-readme-limitations.ts) ([rationale](../docs/rfc/implemented/process/2026-07-10-readme-known-limitations-gate.md)). diff --git a/packages/bash/bash-local/README.md b/packages/bash/bash-local/README.md index afa5000767..cf9f86575e 100644 --- a/packages/bash/bash-local/README.md +++ b/packages/bash/bash-local/README.md @@ -31,6 +31,8 @@ Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi; Indirectly, through `dsh-tool-bash`, which renders this executor's bounded stdout/stderr tails, background-process deltas, spill-file paths, and infrastructure failures. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Unconfined by itself** — this executor always runs commands with the harness process's authority; deployments needing confinement compose [`dsh-bash-sandbox`](../bash-sandbox/README.md), while per-call allow/deny/ask policy belongs on `tools/pre-execute`. diff --git a/packages/bash/bash-sandbox/README.md b/packages/bash/bash-sandbox/README.md index 347aefb679..ab8b6c62dd 100644 --- a/packages/bash/bash-sandbox/README.md +++ b/packages/bash/bash-sandbox/README.md @@ -42,18 +42,24 @@ The keyless consumer-integration proofs are `tests/bwrap.e2e.ts`, `tests/landloc **Token effect**: Small fixed schema increment on requests where `bash` is visible; mode switches add no context tokens. +**KV Cache effect**: Prefix-stable while the executor advertises the same sandbox capabilities. Changing those capabilities alters the `bash` schema and may invalidate reuse from that definition; per-session mode switches do not. + ### Bash tool result, indirectly **What the model sees**: After ordinary bounded output, a denied call appends exactly `[sandbox: file access denied under mode]`. When escalation is available it next appends `[sandbox: escalation available — retry this exact command once with sandbox_permissions (the narrowest wider mode that suffices) + justification; the approval prompt asks the user]`. A settled background runner failure instead appends `[sandbox: the sandbox runner itself failed under mode — the command did not run; this is a sandbox problem, not a command failure]`. **Token effect**: Zero additional tokens on an unremarkable allowed run beyond ordinary output. Denial or failure adds the quoted conditional marker, retained until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Bash tool error, indirectly **What the model sees**: If no runner can enforce a confined mode, the foreground call propagates the [`SANDBOX_UNAVAILABLE` error owned by `dsh-sandbox`](../../sandbox/sandbox/README.md#confinement-error-indirectly). For an execution-time runner failure, this backend supplies the first stderr line as its detail. **Token effect**: Conditional error text is visible for that call and retained in history until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Confinement covers file effects only** — network access and process visibility are unchanged, so the modes are not a general-purpose security sandbox. diff --git a/packages/bash/bash/README.md b/packages/bash/bash/README.md index e4b5bf1952..fffc206bd4 100644 --- a/packages/bash/bash/README.md +++ b/packages/bash/bash/README.md @@ -37,6 +37,8 @@ The seam also owns the per-session mode override vocabulary: the log-only `'bash Indirectly, through `dsh-tool-bash`, which turns executor output and sandbox facts into guidance and retained tool-result tokens. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **No interactive-input vocabulary** — `stdin` is written once at spawn and closed; the seam has no channel to feed a running task and no PTY session concept. diff --git a/packages/bash/tool-bash/README.md b/packages/bash/tool-bash/README.md index d5e65f1a39..eece3249eb 100644 --- a/packages/bash/tool-bash/README.md +++ b/packages/bash/tool-bash/README.md @@ -77,6 +77,8 @@ For sandboxing executors, each call resolves mode as one-shot escalation, then s **Token effect**: Small fixed input cost per request while the plugin is active, unchanged by sandbox mode or mode switches. +**KV Cache effect**: Prefix-stable while the registration scope and prompt text are unchanged. Plugin activation or disposal may invalidate reuse from this prompt section; sandbox mode switches do not. + #### Bash guidance ```markdown @@ -89,24 +91,32 @@ Check the [exit code: N] marker on every bash result; investigate failures befor **Token effect**: Fixed schema cost on every request where the tools are visible; sandbox support adds the escalation fields and its conditional description paragraph. +**KV Cache effect**: Prefix-stable while visibility, background support, and executor sandbox capabilities are unchanged. A restriction, config change, or executor change may invalidate reuse from the first changed tool definition. + ### Foreground result **What the model sees**: The renderer emits the data-dependent stdout tail, then optional `[stderr]` and the stderr tail. With no output it emits exactly `(no output)`. Conditional lines are exactly `[output truncated; full output: ]`, `[sandbox: file access denied under mode]`, `[timed out after ms]`, `[killed by signal: ]`, and `[exit code: ]`; the sandbox escalation and runner-failure lines are quoted in [`dsh-bash-sandbox`](../bash-sandbox/README.md). **Token effect**: Zero result tokens before a call. Output is bounded per stream, while each emitted line remains in history until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Background task context and results **What the model sees**: Start returns exactly `started background task `. This producer supplies incremental process output, optional `[some output was dropped from memory; full output: ]`, sandbox facts, and terminal detail such as `exit code: ` or `signal: ` to the generic task runtime. [`dsh-tool-tasks`](../../tasks/tool-tasks/README.md) owns the visible status line, completion notice, listing, and cancellation response. **Token effect**: The start acknowledgement is small and retained; collected output is data-dependent and bounded by the executor's stream buffers. Consuming reads do not repeat prior output. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Tool errors **What the model sees**: Validation and policy failures are normalized as `Error: `. This package's stable messages are `invalid command: expected a non-empty string`, `invalid description: expected a non-empty string`, `invalid timeoutMs: expected a positive number, got `, `invalid escalation: sandbox_permissions requires a justification`, `invalid escalation: justification is only valid together with sandbox_permissions`, `invalid justification: expected a non-empty sentence`, `background execution is disabled for this bash tool`, `background tasks unavailable: load @deepseek-ai/dsh-tasks and @deepseek-ai/dsh-tool-tasks`, `sandbox_permissions is not available in this composition (no sandboxing executor to escalate)`, `sandbox escalation to "" is not strictly wider than this call's current "" mode`, the approval-availability/rejection/cancellation variants, and `command aborted`. **Token effect**: Only the failing call adds these retained tokens; a rejected escalation does not add command output because the command does not run. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Replay exit pills parse from result text** — output whose final line happens to be exactly `[exit code: N]` / `[killed by signal: …]` shows a wrong pill on session replay; a display-only known residual. diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 342799dabb..ab39d027ba 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -37,6 +37,8 @@ The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. Th Indirectly, through Code Mode in [`dsh-tools`](../../core/tools/README.md), which renders this worker's capped printed or returned data and exact `[dsh-code-runtime-worker] log capture truncated at bytes` and `… [truncated]` markers into a retained `run_code` result. Binding traffic and worker internals stay outside context. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **OS processes a program spawns survive termination** — `worker.terminate()` ends the thread only, weaker than bash-local's process-group kill; orphan cleanup is a deployment concern until a container backend exists. diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index 1f680741a1..49b79e1f52 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -22,6 +22,8 @@ Semantics every implementation must honor (contract details in the class JSDoc): Indirectly, through Code Mode in `dsh-tools`, which exposes `run_code` and returns program logs, values, or failures as retained tool-result tokens. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. diff --git a/packages/compact/compact-basic/README.md b/packages/compact/compact-basic/README.md index 69dd0c5036..77a1e49a5d 100644 --- a/packages/compact/compact-basic/README.md +++ b/packages/compact/compact-basic/README.md @@ -60,6 +60,8 @@ Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it c **Token effect**: The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget. +**KV Cache effect**: Replacing rather than append-only. Each checkpoint invalidates reuse from the first replaced history token; the unchanged request prefix before that range remains reusable. + #### Conversation checkpoint preamble ```markdown @@ -72,12 +74,16 @@ This is an automatically generated checkpoint condensing an earlier span of the **Token effect**: This is a separate model call with data-dependent input and `maxTokens`-capped output. Convergence retries can pay this cost more than once. +**KV Cache effect**: Independent of the conversation request cache. An auxiliary call can reuse an exact transcript prefix, while a different selected range or rendering invalidates reuse from its first changed token. + ### Auxiliary summarizer system prompt **What the model sees**: The summarization model receives the checkpoint-writing instruction below. **Token effect**: Fixed auxiliary input cost plus the data-dependent transcript on every summarization attempt. +**KV Cache effect**: Prefix-stable for auxiliary calls while this instruction and the summarizer route are unchanged. Changing either starts a different prefix; transcript changes occur after the instruction. + #### Auxiliary summarizer system prompt ```markdown diff --git a/packages/compact/compact/README.md b/packages/compact/compact/README.md index 55abc2b12e..65f89ff4a1 100644 --- a/packages/compact/compact/README.md +++ b/packages/compact/compact/README.md @@ -65,12 +65,16 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and **Token effect**: Zero direct tokens from this interface. A backend trades many retained history tokens for one summary and leaves the recent tail unchanged. +**KV Cache effect**: A successful backend replacement invalidates reuse from the first shadowed history token; the seam itself does not alter a request. + ### Transcript supplied to a compaction consumer **What the model sees**: `renderTranscript()` joins entries with one blank line and renders them exactly as `User: `, `Assistant: `, `Tool result (call ): `, `Tool error (call ): `, `[Context: ]`, or `[Steering: ]`. Non-text blocks render exactly as `[reasoning: ]`, `[tool-call: ()]`, `[tool-result: ]`, `[tool-result]`, or `[]`. **Token effect**: Data-dependent input tokens are paid only by the auxiliary model or consumer that requests this transcript; the conversation model does not receive a duplicate transcript. +**KV Cache effect**: No conversation-cache invalidation. A consumer's auxiliary request can reuse only the exact prefix produced by this rendering; changed or compacted entries invalidate reuse from their first difference. + ## Known Limitations and Deferred Work - **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener. diff --git a/packages/context/time-context/README.md b/packages/context/time-context/README.md index fea92e726f..e5d462e75a 100644 --- a/packages/context/time-context/README.md +++ b/packages/context/time-context/README.md @@ -36,6 +36,8 @@ The time reading stays in derived conversation history until a later compaction **Token effect**: Each injected two-line message accumulates until compaction shadows it. A positive interval reduces additions; omission or `0` adds one for every eligible preparation attempt. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### First step ```markdown diff --git a/packages/context/workspace-context/README.md b/packages/context/workspace-context/README.md index 430ffe8a41..a2e4a70596 100644 --- a/packages/context/workspace-context/README.md +++ b/packages/context/workspace-context/README.md @@ -82,6 +82,8 @@ Instruction content is read through `streamText()` under `maxSourceBytes`, even **Token effect**: The rendered baseline is frozen and resent on every request in that loop instance. `maxBytes` bounds the complete message, broader files are omitted before the most-specific file is truncated, and an empty chain contributes zero tokens. +**KV Cache effect**: Prefix-stable within one loop instance because the baseline is frozen. A new or resumed instance recomposes it, so instruction, precedence, cwd, candidate, or byte-budget changes may invalidate reuse from the first changed baseline token. + #### Baseline instruction template ```markdown @@ -104,6 +106,8 @@ Instructions from: AGENTS.md **Token effect**: Each discovered scope adds bounded history tokens until compaction. Unchanged content is suppressed by visible session state plus version/digest comparison, and Code Mode defers the same message until after the outer `run_code` result. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### Additional instruction template ```markdown @@ -122,6 +126,8 @@ These instructions apply to work under `packages/app`. Use them as guidance when **Token effect**: Each confirmed change or removal is one retained history message bounded by `maxBytes`. Provider failures add no message, and an update omitted by the budget remains eligible for a later filesystem touch. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### Removal notice ```markdown diff --git a/packages/cordis/tool-cordis/README.md b/packages/cordis/tool-cordis/README.md index c43e2fcc4d..73672048af 100644 --- a/packages/cordis/tool-cordis/README.md +++ b/packages/cordis/tool-cordis/README.md @@ -40,18 +40,24 @@ Namespace plugin: named exports `name` / `inject` / `Config` / `apply`, no defau **Token effect**: Fixed schema cost on every request in that tool view. +**KV Cache effect**: Prefix-stable while this tool view is unchanged. Scoping or plugin lifecycle changes that hide these definitions may invalidate reuse from the first changed schema token. + ### Tool-call history and results **What the model sees**: Inspect joins selected sections exactly as `##
` then a newline and the data-dependent body, with one blank line between sections. Its broad API/event reports omit JSDoc; `name` with `what: "api"` or `what: "events"` returns one exact target with its original JSDoc. Mount returns `mounted (plugin "", state: )`, optionally inserting ` — waiting for service(s): (activates when provided)` before the closing parenthesis. Unmount returns `unmounted (plugin "")`; an unknown id becomes `Error: no dynamic plugin with id "" (list mounts with cordis_inspect what:"dynamic")`. The submitted mount program remains in the assistant tool-call history. **Token effect**: Inspect output and mount code are data-dependent and resent until compaction; lifecycle acknowledgements are small. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Later requests after a mount **What the model sees**: A mounted plugin may register tools, prompt contributions, or listeners that change later requests for the scopes it targets; unmount removes those contributions after quiescence. **Token effect**: Indirect token impact equals the mounted plugin's contributions and lasts only for the mount lifetime. +**KV Cache effect**: Mounting or unmounting a prompt or tool contribution changes later request prefixes and may invalidate reuse from the first changed contribution; an unchanged mount set remains prefix-stable. + ## Known Limitations and Deferred Work - **The sandbox is containment for honest code, not a security boundary** — host-realm helpers on the sandbox global are reachable, so mount code can reach Node; load this plugin as deliberately as you would grant a bash tool (see § Trust stance). diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index d604e2b16f..1045d26a62 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -76,18 +76,24 @@ Everything that goes beyond "call the model, run the tools, repeat" belongs to p **Token effect**: System text, schemas, and prefix are paid again on every step. Per-agent scoping chooses the initial contributions, while the authoritative assembly waterfall can alter the final request and makes its listener responsible for protocol coherence. +**KV Cache effect**: Append-only only while system text, schemas, session prefix, and earlier history remain byte-identical under the same provider and model route. A token-bearing assembly rewrite or composition change may invalidate reuse from the first altered request token. + ### Retained message history **What the model sees**: Accepted user messages, assistant messages, tool calls and results, injected context, and steering are logged and sent on later steps. Raw stream chunks, lifecycle boundaries, and other log-only events are excluded. **Token effect**: Input grows with every surface message until a compaction replacement shadows older nodes; a multi-step tool turn resends the accumulated prefix and history each step. +**KV Cache effect**: Ordinary history growth is append-only and preserves reusable entries. A surface replacement or compaction invalidates reuse from the first shadowed history token. + ### Undispatched calls after cancellation **What the model sees**: If a later request replays an aborted step, each tool call that cancellation prevented from dispatching has the error result text `Error: tool call skipped because the step was aborted before execution`. **Token effect**: One fixed error result per skipped call remains in history until compaction shadows it. +**KV Cache effect**: Append-only; each synthetic result follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Classification is unary** — calls whose safety depends on comparing siblings or resources must remain exclusive ([rationale](../../../docs/rfc/implemented/feature/2026-07-10-parallel-tool-call-execution.md)). diff --git a/packages/core/agent/README.md b/packages/core/agent/README.md index bf5e563443..0443061e09 100644 --- a/packages/core/agent/README.md +++ b/packages/core/agent/README.md @@ -75,12 +75,16 @@ The handle every plugin programs against: **Token effect**: Accepted content becomes retained history or a repeated session prefix; blocked content contributes no request tokens. Size is caller- and plugin-dependent. +**KV Cache effect**: Accepted history and steering are append-only; a blocked submission sends no request. A session prefix remains stable within its loop instance, while a new or resumed instance may establish a different prefix. + ### Agent-scoped request composition **What the model sees**: Registrations through `agent.ctx` can shadow prompt sections or tools and can install agent-only interceptors during unpublished setup. **Token effect**: The package adds zero tokens itself; scoped contributions affect only that agent and disappear on disposal. +**KV Cache effect**: Prefix-stable while an agent's scoped registrations are unchanged. Setup or reload that changes prompt sections, tool definitions, or request listeners may invalidate reuse from the first affected request token. + ## Known Limitations and Deferred Work - **Initiator scope is process-local** — workers, child processes, HTTP, durable queues, and restarts materialize any required identity explicitly. diff --git a/packages/core/session/README.md b/packages/core/session/README.md index ba4e8ea94a..13d859e5f7 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -89,18 +89,24 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata) **Token effect**: Appended surface entries are resent on later steps. A `replace` surface operation removes the shadowed entries from future inputs without deleting their raw log records. +**KV Cache effect**: Appended surface entries preserve reusable prefixes. A `replace` operation invalidates reuse from the first shadowed message even though the underlying event log stays append-only. + ### Crash-repair result **What the model sees**: If a persisted turn ended with unanswered tool calls, each synthetic error result contains exactly `Tool call interrupted by a crash; no result was recorded.` **Token effect**: Zero tokens in an intact session. Each repaired call adds this retained error text on resume. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Logged request header **What the model sees**: The session reconstructs the system prompt, tool schemas, call config, and session prefix that the loop actually sent. Header events do not add a second copy to message history; the prefix is prepended outside `deriveMessages()`. **Token effect**: Zero duplicate tokens from logging. The reconstructed prefix, system text, and schemas still incur their normal per-request cost. +**KV Cache effect**: Logging causes no invalidation, and exact reconstruction preserves request-prefix identity. A later header with changed prefix, prompt, or schemas may invalidate reuse from its first difference. + ## Known Limitations and Deferred Work - **Session branching/tree** (pi-style entry tree) — deferred unless needed beyond boundary-based `fork()`. diff --git a/packages/core/system-prompt/README.md b/packages/core/system-prompt/README.md index e8975bf17e..05cc92aaf2 100644 --- a/packages/core/system-prompt/README.md +++ b/packages/core/system-prompt/README.md @@ -48,6 +48,8 @@ Design rationale: [the prompt-variables RFC](../../../docs/rfc/implemented/archi **Token effect**: Identity is a fixed per-request cost. Persona and plugin text are repeated per request and scale with their rendered content. +**KV Cache effect**: Prefix-stable while identity, persona, variables, section text, and order render identically. Any change may invalidate reuse from the first changed system-prompt token. + #### Harness identity ```markdown @@ -60,6 +62,8 @@ You are an AI agent powered by the DeepSeek Harness SDK. **Token effect**: Schema tokens repeat on every request. Restricting a tool removes its entire schema cost for that agent but not a separate prompt section; reordering changes cache shape but not semantic content. +**KV Cache effect**: Prefix-stable while the visible schema set, rendering, and order are unchanged. Registration, restriction, or reordering may invalidate reuse from the first changed schema token. + ## Known Limitations and Deferred Work - **Deployment-authored prompt text is config/composition only** — this plugin owns the global persona default, creator plugins may register agent-scoped shadows, and other sections come from the plugin that owns the fact; there is no end-user prompt-editing API. diff --git a/packages/core/tools/README.md b/packages/core/tools/README.md index bfd75fdcb6..b0b79d04cb 100644 --- a/packages/core/tools/README.md +++ b/packages/core/tools/README.md @@ -123,12 +123,16 @@ The agent loop groups consecutive `parallel` calls into a bounded rolling pool a **Token effect**: Fixed per-request cost proportional to the visible definitions. Restrictions that hide tools remove their entire schema cost for that agent. +**KV Cache effect**: Prefix-stable while visible definitions and their order are unchanged. Registration, disposal, or scoped restriction may invalidate reuse from the first changed schema token. + ### Code Mode schema and system prompt **What the model sees**: Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact `declare const tools` block. `both` exposes normal schemas and this Code Mode surface. **Token effect**: Fixed per-request cost proportional to the visible definitions. Code Mode trades end-tool schemas for generated SDK text plus one transport schema rather than promising a universal reduction. +**KV Cache effect**: Prefix-stable while the Code Mode selection, generated SDK, transport schema, and visible tool set are unchanged. Mode or filter changes may invalidate reuse from the first changed prompt or schema token. + #### Code Mode SDK instructions ```markdown @@ -150,6 +154,8 @@ The available tools: **Token effect**: Arguments, results, and additional context are data-dependent and resent until compaction. Restrictions that hide tools also remove their schemas before the model can call them. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Concurrency policy is not an event seam** — `executionMode()` reads the resolved tool definition directly; plugins can only declare a classifier on definitions they own. diff --git a/packages/examples/acp-demo/README.md b/packages/examples/acp-demo/README.md index 1dbda9a08a..98b62fc9bf 100644 --- a/packages/examples/acp-demo/README.md +++ b/packages/examples/acp-demo/README.md @@ -55,6 +55,8 @@ All diagnostics go to **stderr** — stdout is the protocol. Indirectly, through `dsh-agent-spine-demo` and `dsh-acp`, which compose each ACP agent's prompt, tools, and message history; this app bundle adds no model-bound content itself. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **JSONL persistence is baked in** — config chooses its root but cannot select a different backend; that requires a sibling entry or differently composed app package. diff --git a/packages/examples/agent-spine-demo/README.md b/packages/examples/agent-spine-demo/README.md index 409aef5ef2..e2b3c47abe 100644 --- a/packages/examples/agent-spine-demo/README.md +++ b/packages/examples/agent-spine-demo/README.md @@ -56,6 +56,8 @@ A YAML include can deduplicate config but cannot own a bin or provide front-door Indirectly, through `dsh-system-prompt`, `dsh-tool-skill`, `dsh-tool-bash`, and `dsh-tools`, which this bundle mounts without adding model-bound wrapper content. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **The spine set is fixed in code** — `apply()` mounts every child unconditionally (including `tool-bash`); no config excludes or replaces one, so swapping the loop or dropping a spine member means composing a different bundle. diff --git a/packages/examples/cli-demo/README.md b/packages/examples/cli-demo/README.md index c938f6c583..40790885be 100644 --- a/packages/examples/cli-demo/README.md +++ b/packages/examples/cli-demo/README.md @@ -59,6 +59,8 @@ The headless-agent leaf supplies local bash, filesystem, skill, subagent, workfl **Token effect**: The task, prompt sections, tool schemas, assistant output, and tool results consume tokens on each model step. JSON event streaming and final rendering add no model tokens; delegated child work has its own model usage and is not included in the parent result's `usage` total. +**KV Cache effect**: Tool-round history is append-only while the one-shot agent's prompt, schemas, model route, and session prefix remain fixed. Changing that composition establishes a different request prefix; JSON output mode has no cache effect. + ## Known Limitations and Deferred Work - **One fresh top-level session per process** — its workspace cwd is the launch directory; there is no resume, second prompt, stdin context, or concurrent top-level session in this app. diff --git a/packages/examples/jsonrpc-demo/README.md b/packages/examples/jsonrpc-demo/README.md index 39cb4cd917..f7c9a71bea 100644 --- a/packages/examples/jsonrpc-demo/README.md +++ b/packages/examples/jsonrpc-demo/README.md @@ -20,6 +20,8 @@ stdout carries only JSON-RPC frames. The bin and boot guards diagnose on stderr, Indirectly, through the plugins loaded from the external `cordis.yml`, which own every model-bound prompt, schema, message, and result; this bin adds none of its own. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **The bin cannot prove that the config serves JSON-RPC** — a valid config with no `dsh-jsonrpc` entry boots successfully and serves nothing. diff --git a/packages/examples/stdio-demo/README.md b/packages/examples/stdio-demo/README.md index ba4fc10101..4a50b5ccbd 100644 --- a/packages/examples/stdio-demo/README.md +++ b/packages/examples/stdio-demo/README.md @@ -83,12 +83,16 @@ Swap `llm-deepseek` for a `mock-llm` leaf plugin and you have the echo demo — **Token effect**: Child prompt and schema costs repeat per request; user input and tool history grow until compaction. Terminal banners, logger output, cards, and rendered transcripts add zero model tokens. +**KV Cache effect**: User and tool history is append-only while the composed prompt, schemas, child model route, and session prefix remain fixed. A composition change or compaction may invalidate reuse from its first changed token; terminal rendering has no cache effect. + ### Human-answer result **What the model sees**: Through `dsh-tool-ask-user`, successful terminal answers use that package's exact compact JSON shape. Interruption becomes exactly `Error: ask_user_question was interrupted before the user answered`; a closed stdin becomes `Error: ask_user_question cannot be answered because stdin is closed`. **Token effect**: Only a completed or failed tool call adds retained result tokens; prompts printed while waiting are terminal-only. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **One pre-created `main` agent drives the selected terminal UI** — there is no multi-session or concurrent-agent surface in this app; a run is one conversation. diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 65ed76efce..3403ef59fd 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -25,6 +25,8 @@ The package-root SDK surface is the default/named `LocalFileSystem` class plus ` Indirectly, through [`dsh-tool-fs`](../tool-fs/README.md), which renders this provider's line-windowed UTF-8 content, mutation acknowledgements, and exact provider messages in capped retained results while versions, atomic-write mechanics, and directory metadata remain internal. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)). diff --git a/packages/fs/fs-policy/README.md b/packages/fs/fs-policy/README.md index bd85be88cf..d509e63645 100644 --- a/packages/fs/fs-policy/README.md +++ b/packages/fs/fs-policy/README.md @@ -55,6 +55,8 @@ Because the plugin influences the world only through events, removing it does no **Token effect**: Zero tokens on allowed operations beyond the ordinary tool result. A denial adds the small retained error result and avoids any success payload. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Observed state does not survive a session resume** — persistence of the `WeakMap` record is deferred, so a resumed session must re-read files before guarded writes/edits. diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 6dae32d235..b67efe5fad 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -48,6 +48,8 @@ This package declares three events (see the generated [events catalog](../../../ Indirectly, through `dsh-tool-fs`, which renders provider text and errors as bounded, retained filesystem tool results. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Text-only by contract** — backends reject binary/non-UTF-8 content with `FS_NOT_TEXT`; binary-safe operations are a deliberate deferral of [the tool-schemas RFC](../../../docs/rfc/implemented/feature/2026-06-17-filesystem-tool-schemas.md). diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index 29afbebfec..4075890ba2 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -53,6 +53,8 @@ Search failures carry the package-owned `SearchError` (a `HarnessError` subclass **Token effect**: Fixed guidance cost per request while the plugin is active. +**KV Cache effect**: Prefix-stable while the plugin scope and guidance text are unchanged. Activation or disposal may invalidate reuse from this prompt section. + #### Glob guidance ```markdown @@ -71,18 +73,24 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read **Token effect**: Fixed schema cost on every request where the tools are visible. +**KV Cache effect**: Prefix-stable while tool visibility and definitions are unchanged. Registration lifecycle or scoped restrictions may invalidate reuse from the first changed schema token. + ### Results and spill notices **What the model sees**: `glob` returns one path per line; `grep` groups `Line : ` matches beneath each path. Empty searches return `No files found` or `No matches found`. A capped result ends with its omission count plus the spill locator and backend retrieval hint, or says the complete result could not be saved. **Token effect**: Inline paths and matches are bounded by `globMaxResults`, `grepMaxMatches`, and `grepMaxLineBytes`; the call and retained result remain in history until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Tool errors **What the model sees**: Failures are normalized as `Error: ` with structured `SEARCH_INVALID_PATTERN`, `SEARCH_FAILED`, `SEARCH_RAW_OUTPUT_OVERFLOW`, or `SEARCH_ABORTED` metadata for callers. **Token effect**: Only a failing call adds these retained tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the bash workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation. diff --git a/packages/fs/tool-fs/README.md b/packages/fs/tool-fs/README.md index 2f116a2b71..5bd59f8888 100644 --- a/packages/fs/tool-fs/README.md +++ b/packages/fs/tool-fs/README.md @@ -58,6 +58,8 @@ The package root exports only the Cordis plugin contract (`name`, `inject`, `Con **Token effect**: Fixed guidance cost per request while the plugin is active, even when a restriction hides one or more tools. +**KV Cache effect**: Prefix-stable while the plugin scope and guidance text are unchanged. Tool restrictions do not remove this section, but plugin activation or disposal may invalidate reuse from it. + #### Read guidance ```markdown @@ -82,24 +84,32 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces **Token effect**: Fixed schema cost on every request in that tool view. +**KV Cache effect**: Prefix-stable while the visible tool definitions and order are unchanged. Registration lifecycle or scoped restrictions may invalidate reuse from the first changed schema token. + ### Read result **What the model sees**: A successful read is exactly ``, newline, `file`, newline, ``, numbered lines as `: `, a blank line, one footer, and ``. The footer is exactly `(Output capped. Showing lines -. Use offset= to continue.)`, `(Showing lines - of . Use offset= to continue.)`, or `(End of file - total lines)`. A long line ends exactly `... (line truncated to chars)`. **Token effect**: Read output is capped by `readLimit`, `readMaxLineLength`, and `readMaxBytes`; the retained call and result are resent until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Write and edit results **What the model sees**: Write returns the exact five-line envelope ``, `file`, ``, `Created file` or `Updated file`, then ``. Edit returns exactly `The file has been updated successfully.` or, for `replace_all`, `The file has been updated. All occurrences were successfully replaced.` The full write or replacement text remains in the assistant tool-call arguments. **Token effect**: Success text is small, but large mutation arguments and any result are resent until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Tool errors **What the model sees**: Failures are normalized as `Error: `. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to `, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "": not found`, `cannot read "": not a regular file`, and `offset is out of range for "" ( lines)`; provider and policy templates are quoted in their package READMEs. **Token effect**: Only a failing call adds these retained tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **No model-facing directory listing ships** — `ctx.fs.listDir` serves provider code such as skill discovery, while the sibling [`dsh-tool-fs-search`](../tool-fs-search/) package supplies bash-backed `glob` and `grep` rather than extending the filesystem seam. diff --git a/packages/guard/repeat-tool-guard/README.md b/packages/guard/repeat-tool-guard/README.md index 2927e3f20a..2ca3edc46a 100644 --- a/packages/guard/repeat-tool-guard/README.md +++ b/packages/guard/repeat-tool-guard/README.md @@ -44,6 +44,8 @@ Unit suites drive a real agent loop against a mock adapter (no network) and cove **Token effect**: Zero tokens before the threshold. The reminder is retained history for that agent. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### First-threshold reminder ```markdown @@ -56,6 +58,8 @@ You are repeating the exact same tool call with identical arguments. Carefully a **Token effect**: Each reminder is retained history; `argumentsPreviewChars` bounds its data-dependent argument text, while agents keep independent counters. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### Later-threshold reminder ```markdown diff --git a/packages/hooks/hook-protocol/README.md b/packages/hooks/hook-protocol/README.md index 96a423fea9..6513745173 100644 --- a/packages/hooks/hook-protocol/README.md +++ b/packages/hooks/hook-protocol/README.md @@ -33,6 +33,8 @@ Like every event they must sit inside an open turn. The mid-turn points (`PreToo Indirectly, through `dsh-hooks-claude` and `dsh-hooks-codex`, which can turn parsed hook output into prompt context, blocked outcomes, or continuation feedback. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **`HookOutput.updatedInput` is parsed but not honored** — input rewrite is a deferred consistency-design problem ([the pre-tool-input-rewrite RFC](../../../docs/rfc/proposed/feature/2026-06-30-pre-tool-input-rewrite.md)); a bridge logs + warns when a hook sets it. See `src/types.ts` for the full contracts. diff --git a/packages/hooks/hooks-claude/README.md b/packages/hooks/hooks-claude/README.md index 4430f5511a..2498d0ac58 100644 --- a/packages/hooks/hooks-claude/README.md +++ b/packages/hooks/hooks-claude/README.md @@ -60,12 +60,16 @@ Injected context carries an explicit `{ kind: 'plugin', plugin: 'hooks-claude' } **Token effect**: No cost when hooks return no context. Hook text is data-dependent, logged, and resent in later conversation requests until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Blocked prompt or tool outcome **What the model sees**: Provider-supplied reasons pass through verbatim. When absent, a blocked prompt uses exactly `blocked by UserPromptSubmit hook`, a denied tool becomes `Error: blocked by PreToolUse hook`, blocked post-tool feedback is exactly `blocked by PostToolUse hook`, and a blocking stop adds steering exactly `continue: blocked by Stop hook`. `systemMessage` and `updatedInput` are logged or warned but are not model-visible in this implementation. **Token effect**: Blocking a prompt removes that prompt's request tokens; denial or feedback adds the retained fallback or provider text; forced continuation pays another full request. +**KV Cache effect**: A blocked prompt sends no request and invalidates nothing. Denial, feedback, and forced-continuation context append after the reusable prefix without rewriting it. + ## Known Limitations and Deferred Work - **Unsupported hook events (23 of Claude Code's current 30):** `Setup`, `InstructionsLoaded`, `UserPromptExpansion`, `MessageDisplay`, `PermissionRequest`, `PostToolUseFailure`, `PostToolBatch`, `PermissionDenied`, `Notification`, `TaskCreated`, `TaskCompleted`, `StopFailure`, `TeammateIdle`, `ConfigChange`, `CwdChanged`, `FileChanged`, `WorktreeCreate`, `WorktreeRemove`, `PreCompact`, `PostCompact`, `SessionEnd`, `Elicitation`, and `ElicitationResult`. Config for these events is parsed but never dispatched. The comparison baseline is Claude Code's [official hook-event reference](https://code.claude.com/docs/en/hooks#hook-events). diff --git a/packages/hooks/hooks-codex/README.md b/packages/hooks/hooks-codex/README.md index 0784857286..698fcc7a29 100644 --- a/packages/hooks/hooks-codex/README.md +++ b/packages/hooks/hooks-codex/README.md @@ -64,12 +64,16 @@ Injected context carries an explicit `{ kind: 'plugin', plugin: 'hooks-codex' }` **Token effect**: No cost when hooks return no context. Hook text is data-dependent, logged, and resent until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Blocked prompt or tool outcome **What the model sees**: Provider-supplied reasons pass through verbatim. When absent, a blocked prompt uses exactly `blocked by UserPromptSubmit hook`, a denied tool becomes `Error: blocked by PreToolUse hook`, blocked post-tool feedback is exactly `blocked by PostToolUse hook`, and a blocking stop adds steering exactly `continue: blocked by Stop hook`. Codex `systemMessage` is not surfaced. **Token effect**: Blocking a prompt removes its request tokens; denial or feedback adds the retained fallback or provider text; forced continuation pays another full request. +**KV Cache effect**: A blocked prompt sends no request and invalidates nothing. Denial, feedback, and forced-continuation context append after the reusable prefix without rewriting it. + ## Known Limitations and Deferred Work - **Unsupported hook events (5 of Codex's current 10):** `PermissionRequest`, `PreCompact`, `PostCompact`, `SubagentStart`, and `SubagentStop`. Config for these events is silently dropped during parsing. The comparison baseline is Codex's [official hook reference](https://learn.chatgpt.com/docs/hooks). diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index b58da7bf57..019284a872 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -56,12 +56,16 @@ Unit suites run against a local `node:http` mock SSE server (no network). Real-A **Token effect**: Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available. +**KV Cache effect**: An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips. + ### DeepSeek response **What the model sees**: Reasoning, text, and raw-string tool arguments are translated into harness chunks for the loop to log and assemble. **Token effect**: Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input. +**KV Cache effect**: Loop-retained response blocks append to the next request and preserve its earlier reusable prefix; dropped blocks have no later cache effect. Changing the provider or model selects a different cache domain. + ## Known Limitations and Deferred Work - **`tool_choice` is not mapped** — not part of the core vocabulary (MVP cut, shared with the pi-ai twin). diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 1deae8f039..d295b9cfcf 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -67,12 +67,16 @@ Unit tests use pi-ai catalog models redirected to local mock servers and cover p **Token effect**: Provider tokenization governs exact input. Conversion adds no model-visible text; replay metadata may let a native API reuse provider-side state. +**KV Cache effect**: Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference. + ### Provider response **What the model sees**: pi-ai events become harness reasoning, text, tool-call, usage, and finish chunks. Parsed tool arguments cross the harness boundary as raw JSON strings. **Token effect**: Generated content affects later inputs only after the loop records it. pi-ai folds reasoning tokens into output usage when the provider does not report them separately. +**KV Cache effect**: Recorded response content appends to the next request and does not invalidate its earlier reusable prefix. Unrecorded transport metadata and usage accounting do not affect cache identity. + ## Known Limitations and Deferred Work - **Catalog membership is required** — custom model ids that are absent from the installed pi-ai catalog fail with `UNKNOWN_MODEL`, even when a provider profile supplies a custom endpoint. diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index 44e7d80a05..2332bc0b0d 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -58,6 +58,8 @@ Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-l None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message. +**KV Cache effect**: Pass-through; the registry preserves the assembled request prefix, while the selected adapter and provider own cache reuse and routing boundaries. + ## Known Limitations and Deferred Work - **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. diff --git a/packages/llm/token-meter/README.md b/packages/llm/token-meter/README.md index 262fc8d152..8db289a05a 100644 --- a/packages/llm/token-meter/README.md +++ b/packages/llm/token-meter/README.md @@ -42,6 +42,8 @@ Both plugins have usable defaults. A deployment with a different capacity config Indirectly, through consumers such as `dsh-compact-basic`; the service itself adds no prompt, message, schema, tool, or model call. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **The fixed heuristic is approximate** — content without reusable provider usage is priced by character count plus structural overhead, not an exact provider tokenizer or request serializer. diff --git a/packages/mcp/mcp-client/README.md b/packages/mcp/mcp-client/README.md index e8c4f54f5f..9968b2b6ff 100644 --- a/packages/mcp/mcp-client/README.md +++ b/packages/mcp/mcp-client/README.md @@ -74,12 +74,16 @@ Every MCP tool has two names: the raw MCP name (sent on the wire in `tools/call` **Token effect**: Data-dependent schema cost is paid on every request while the tools are registered. Re-sync replaces rather than accumulates schemas, and the server-qualified name adds tokens to every tool definition and call. +**KV Cache effect**: Prefix-stable while the discovered tool set and schemas are unchanged. A re-sync that adds, removes, renames, or changes a tool replaces definitions and may invalidate reuse from the first changed schema token. + ### Tool-call history and results **What the model sees**: The public tool name and JSON arguments remain in assistant history. Text result blocks are joined with newlines into one retained text result; image, audio, resource, and unsupported blocks become short placeholders, and MCP `isError` results follow the registry's model-visible error path. **Token effect**: Arguments and mapped text are retained until compaction. Binary and resource payloads are discarded rather than added to context. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Initial discovery is asynchronous** — plugin load does not wait for connection and `listTools()`, so a turn started immediately after boot or HMR can assemble before the MCP tools are registered. diff --git a/packages/sandbox/sandbox-local/README.md b/packages/sandbox/sandbox-local/README.md index cbd765bbf9..723df60f17 100644 --- a/packages/sandbox/sandbox-local/README.md +++ b/packages/sandbox/sandbox-local/README.md @@ -25,6 +25,8 @@ Consumers: [`@deepseek-ai/dsh-bash-sandbox`](../../bash/bash-sandbox/); see [the Indirectly, through [`dsh-bash-sandbox`](../../bash/bash-sandbox/README.md) and [`dsh-tool-bash`](../../bash/tool-bash/README.md), which render this provider's enforcement and denial facts while the [`dsh-sandbox`](../sandbox/README.md) seam owns the `SANDBOX_UNAVAILABLE` text and runner selection and profiles stay outside context. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Windows has no runner** — `win32` fails closed with `SANDBOX_UNAVAILABLE`; an AppContainer-family backend is deferred. diff --git a/packages/sandbox/sandbox/README.md b/packages/sandbox/sandbox/README.md index 93e274b485..9a56b53e8a 100644 --- a/packages/sandbox/sandbox/README.md +++ b/packages/sandbox/sandbox/README.md @@ -18,6 +18,8 @@ Implementations: [`@deepseek-ai/dsh-sandbox-local`](../sandbox-local/) (Linux: ` **Token effect**: Conditional error text is visible for that call and retained in history until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### Exact error ```markdown diff --git a/packages/sdk/create-sdk/README.md b/packages/sdk/create-sdk/README.md index 66cddc030a..93d0bffe28 100644 --- a/packages/sdk/create-sdk/README.md +++ b/packages/sdk/create-sdk/README.md @@ -14,6 +14,8 @@ The provider choice is DeepSeek or a custom endpoint backed by `llm-pi-ai`. Deep Indirectly, through the generated project composition and its selected runtime plugins. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **TTY-only creation** — flags prefill questions, but the wizard still requires an interactive terminal before it writes a project. diff --git a/packages/sdk/helper/README.md b/packages/sdk/helper/README.md index 5b5608cdf0..9eb46d978e 100644 --- a/packages/sdk/helper/README.md +++ b/packages/sdk/helper/README.md @@ -18,6 +18,8 @@ The package root explicitly exports only the objects consumed by `create-sdk` an None, as the project domain edits files and never mounts a live agent or model request. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **Commit is not transactional across files** — external edits are detected before each write, but a later failure does not roll back files already written. diff --git a/packages/sdk/scripts/README.md b/packages/sdk/scripts/README.md index e87c375a4f..63f417efbb 100644 --- a/packages/sdk/scripts/README.md +++ b/packages/sdk/scripts/README.md @@ -25,6 +25,8 @@ The root library exports `startSDK`, `runSDK`, and the `SdkBootArgs`/`SdkBootCon Indirectly, through the project `cordis.yml` tree loaded by `start` or `dev`. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Launcher arguments are schema-free** — `start` and `dev` preserve Node `parseArgs()` output rather than validating project-specific flags. diff --git a/packages/session-persistence/session-persistence-jsonl/README.md b/packages/session-persistence/session-persistence-jsonl/README.md index 9fcf537123..dffad31ddb 100644 --- a/packages/session-persistence/session-persistence-jsonl/README.md +++ b/packages/session-persistence/session-persistence-jsonl/README.md @@ -40,6 +40,8 @@ The plugin buffers frozen session events and drains them on flush or disposal. A **Token effect**: Zero live-request tokens. A resumed agent pays for retained history and its current envelope, plus the quoted repair result for each interrupted call. +**KV Cache effect**: JSONL storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append. + ## Known Limitations and Deferred Work - **Only the current `SESSION_FORMAT_VERSION` (v0) loads** — the on-disk format is pre-release/unstable: a breaking format change is absorbed at v0 and non-current logs are rejected; there is no migration. diff --git a/packages/session-persistence/session-persistence-sqlite/README.md b/packages/session-persistence/session-persistence-sqlite/README.md index 11fef96dad..748eb3e851 100644 --- a/packages/session-persistence/session-persistence-sqlite/README.md +++ b/packages/session-persistence/session-persistence-sqlite/README.md @@ -41,6 +41,8 @@ Like the JSONL backend, the plugin also installs the `session/event` → buffer **Token effect**: Zero live-request tokens. Resume restores retained history and pays the current envelope, plus the quoted repair result for each interrupted call. +**KV Cache effect**: SQLite storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append. + ## Known Limitations and Deferred Work - **`DatabaseSync` is synchronous** — every append transaction blocks the event loop for its duration; acceptable for local stores, a throughput ceiling for busy multi-session servers. diff --git a/packages/session-persistence/session-persistence/README.md b/packages/session-persistence/session-persistence/README.md index 3390b2cc30..dbfa61ef1f 100644 --- a/packages/session-persistence/session-persistence/README.md +++ b/packages/session-persistence/session-persistence/README.md @@ -61,6 +61,8 @@ Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `ve **Token effect**: Zero tokens during ordinary persistence. Resume restores retained history cost and pays the current request envelope normally; each repaired call adds the quoted retained error text. +**KV Cache effect**: Persistence does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append without rewriting earlier history. + ## Known Limitations and Deferred Work - **No deletion or retention surface** — the seam is `create`/`append`/`load`/`list` only; pruning stored sessions is out-of-band backend maintenance. diff --git a/packages/session-query/session-query/README.md b/packages/session-query/session-query/README.md index 269ba51e3e..8968a94d80 100644 --- a/packages/session-query/session-query/README.md +++ b/packages/session-query/session-query/README.md @@ -26,6 +26,8 @@ Persistence is optional and may mount or unmount dynamically. Cross-corpus listi None, as this trusted query service returns cloned session records only to its callers and registers no model-facing prompt, schema, tool, or message. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **No caller authorization** — this is trusted context-wide infrastructure; a future model tool or UI must constrain which sessions its caller may inspect. diff --git a/packages/skill/skill-local/README.md b/packages/skill/skill-local/README.md index 8034441272..7fb8e3cdbe 100644 --- a/packages/skill/skill-local/README.md +++ b/packages/skill/skill-local/README.md @@ -40,6 +40,8 @@ Skills can be single-level directory bundles (`/SKILL.md`) or flat Markdow Indirectly, through `dsh-tool-skill`, which renders this provider's invocable names and capped descriptions into the session-prefix catalog and a selected instruction body plus resource-base guidance into retained tool history while paths, provider ranks, and disabled skills remain hidden. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Discovery is one level deep** — only `//SKILL.md` and `/.md` are recognized; nested skill trees and package manifests are ignored. diff --git a/packages/skill/skill/README.md b/packages/skill/skill/README.md index 8edcd71ef0..98e51287f2 100644 --- a/packages/skill/skill/README.md +++ b/packages/skill/skill/README.md @@ -39,6 +39,8 @@ The registry does not render model guidance or register model-facing tools. [`@d Indirectly, through `dsh-tool-skill`, which renders provider summaries into the session prefix and loaded instructions into retained tool results. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Completed catalogs have no TTL or watcher invalidation** — a provider's underlying files or remote data can change without a registration revision, so a cached cwd stays stale until eviction or provider/runtime reload. diff --git a/packages/skill/tool-skill/README.md b/packages/skill/tool-skill/README.md index a2c7ff5c40..5af1131ccf 100644 --- a/packages/skill/tool-skill/README.md +++ b/packages/skill/tool-skill/README.md @@ -32,6 +32,8 @@ The tool does not call `agent.inject()` in v1. Its result is already recorded as **Token effect**: Repeated input cost scales with skill count and `catalogDescriptionMaxLength`; no catalog tokens are sent when the list is empty or the tool is hidden or shadowed. +**KV Cache effect**: Prefix-stable within a loop instance once the session prefix is composed. A new or resumed instance with different providers, skills, descriptions, visibility, or catalog limits may invalidate reuse from the first changed catalog token. + #### Skill catalog template ```markdown @@ -52,12 +54,16 @@ If the user names a skill, or the task clearly matches a skill's description, ca **Token effect**: Fixed schema cost per request where the tool is visible. +**KV Cache effect**: Prefix-stable while the tool definition and visibility are unchanged. Shadowing, restrictions, or plugin lifecycle changes may invalidate reuse from this schema. + ### Tool result **What the model sees**: A successful call uses the result template and the provider-managed, directory, URL, or opaque resource guidance below. **Token effect**: Loaded instructions are data-dependent tool-result tokens, resent on later steps until compaction; no duplicate `agent.inject()` copy is made. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + #### Skill result template ```markdown @@ -106,6 +112,8 @@ Load referenced resources only as needed. **Token effect**: Only a failing call adds these retained tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **The catalog omits `whenToUse`, source, and provider metadata** — routing is based only on name and a capped description; `whenToUse` remains provider metadata and is not rendered by the loaded wrapper either. diff --git a/packages/spill/spill-local/README.md b/packages/spill/spill-local/README.md index 59d23b8a46..e19bc7e50e 100644 --- a/packages/spill/spill-local/README.md +++ b/packages/spill/spill-local/README.md @@ -22,6 +22,8 @@ Files land at `/session-/​-`: Indirectly, through spill consumers that render the local path and `read`/`grep` retrieval guidance. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Local spill files persist until external cleanup** — the backend has no session-lifecycle deletion or age-based retention policy, because persisted, resumed, and forked sessions may still reference a path. diff --git a/packages/spill/spill-policy/README.md b/packages/spill/spill-policy/README.md index bb20ac9dfc..a197c82179 100644 --- a/packages/spill/spill-policy/README.md +++ b/packages/spill/spill-policy/README.md @@ -40,6 +40,8 @@ The policy sees only the FINAL formatted tool result — not a tool's internal r **Token effect**: A successful replacement is at most `maxInlineBytes` UTF-8 bytes and remains in history until compaction; the full spill text is not resent to the model. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Only final plain-text results are spillable** — mixed-content results, blocked feedback, and `read` pass through; provider truncation or tool-owned retention that happened earlier cannot be recovered here. diff --git a/packages/spill/spill/README.md b/packages/spill/spill/README.md index c80277339b..a790b73d5f 100644 --- a/packages/spill/spill/README.md +++ b/packages/spill/spill/README.md @@ -30,6 +30,8 @@ See the [tool output spill RFC](../../../docs/rfc/implemented/architecture/2026- Indirectly, through spill consumers that render a backend locator and retrieval guidance. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **The seam has no retrieval or deletion API** — consumers can only render the backend's locator and guidance; lifecycle and access semantics remain backend-specific. diff --git a/packages/subagent/subagent-acp/README.md b/packages/subagent/subagent-acp/README.md index a620051506..9fd9db3ce6 100644 --- a/packages/subagent/subagent-acp/README.md +++ b/packages/subagent/subagent-acp/README.md @@ -67,12 +67,16 @@ Keyless tests drive a scripted ACP subprocess over real stdio. The with-key e2e **Token effect**: The child pays for an independent full context and its own multi-step history. These tokens never enter the parent's context. +**KV Cache effect**: Independent of the parent request cache. Each ACP child can reuse only prefixes identical under its own provider, model, composition, and history; child steps otherwise grow append-only. + ### Parent tool result, indirectly **What the model sees**: Through `dsh-tool-subagent`, the parent receives only the child's final streamed assistant text or that consumer's exact stop-reason error, not intermediate messages or tool traffic. A request already cancelled before publication becomes exactly `Error: subagent request was aborted before the ACP child started`; other start failures pass through as `Error: `. **Token effect**: Parent input grows only by the final result or error, which is data-dependent and retained until compaction. This provider adds no parent schema itself. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **A fresh process per run** — persistent-process pooling is a future optimization ([the seam RFC](../../../docs/rfc/implemented/feature/2026-06-21-subagent-capability-seam.md)). diff --git a/packages/subagent/subagent-fork/README.md b/packages/subagent/subagent-fork/README.md index 244811ab88..8ed1c38cd9 100644 --- a/packages/subagent/subagent-fork/README.md +++ b/packages/subagent/subagent-fork/README.md @@ -31,12 +31,16 @@ See [`dsh-subagent-spawn`](../subagent-spawn/README.md) for the run lifecycle, m **Token effect**: Forking duplicates retained completed history into separate child requests; the child then accumulates its own tokens independently. Persona changes repeated prompt cost, filtering changes schema or generated SDK cost, and a first-turn fork has no inherited history. +**KV Cache effect**: The child may reuse the inherited byte-identical prefix under the same provider and model. Persona, tool-filter, generated-SDK, or route changes may invalidate reuse before inherited history; later child history is append-only. + ### Parent tool result, indirectly **What the model sees**: The parent receives only the child's own final output through `dsh-tool-subagent`, not the inherited prefix or intermediate work. **Token effect**: Parent input grows by one data-dependent final result retained until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Runs expose no `sendMessage`/`resume`** — the optional runtime capabilities are absent on in-process runs. diff --git a/packages/subagent/subagent-inprocess/README.md b/packages/subagent/subagent-inprocess/README.md index ca54dab7c0..a709df0058 100644 --- a/packages/subagent/subagent-inprocess/README.md +++ b/packages/subagent/subagent-inprocess/README.md @@ -48,12 +48,16 @@ A clean turn that never commits the required structured value reports `error`; t **Token effect**: Child input is isolated from the parent and grows through the child's own steps. A persona changes repeated prompt text; filtering changes schema or generated SDK cost but not independently registered guidance. +**KV Cache effect**: Independent of the parent request cache. The child's later history is append-only, while persona, tool-filter, generated-SDK, provider, or model changes establish a different child prefix. + ### Structured-output system prompt, schema, and results **What the model sees**: A structured run adds the structured-output instruction below. It also adds a child-scoped `structured_output` definition with exact description `Report your final structured result. Call this exactly once, when your answer is complete; the arguments must match this tool's parameter schema exactly.` and the requested schema. This runtime-only definition is outside the generated shipped [tool package map](../../../docs/tool-catalog.md#tool-package-map). Success returns `Structured output recorded.`; a later call becomes ``Error: structured output already recorded: the run is complete, so `` is not executed``. **Token effect**: Fixed instruction and capability tokens are paid only by that child. Result text enters the child history, while the captured value alone becomes the parent result. +**KV Cache effect**: Prefix-stable inside the child while the structured-output instruction and schema are unchanged. Changing the schema or capability may invalidate the child's cache from that early segment; results append in child and parent histories. + #### Structured-output instruction ```markdown @@ -66,12 +70,16 @@ When you have your final answer, you MUST report it by calling the `structured_o **Token effect**: Zero tokens on a successful start; only the failed parent tool call retains this text. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Parent result, indirectly **What the model sees**: The driver extracts only the child's own last assistant output or captured structured value; seeded parent messages and intermediate child work do not become the result. **Token effect**: The parent receives one data-dependent result through the consumer; all other child tokens stay in the child session. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Runs expose no `sendMessage`/`resume`** — the optional runtime capabilities are absent on in-process runs. diff --git a/packages/subagent/subagent-spawn/README.md b/packages/subagent/subagent-spawn/README.md index 6f9982f15b..ca283eee2c 100644 --- a/packages/subagent/subagent-spawn/README.md +++ b/packages/subagent/subagent-spawn/README.md @@ -26,12 +26,16 @@ Spawn advertises `{ outputSchema: true, depthLimit: true, toolFilter: true, pers **Token effect**: The child pays for a new independent context and history; no parent-history tokens are duplicated. Persona changes this child's repeated prompt cost, while filtering changes its schema or generated SDK cost. +**KV Cache effect**: Independent of the parent request cache. Child history grows append-only, while persona, tool-filter, generated-SDK, provider, or model changes establish a different child prefix. + ### Parent tool result, indirectly **What the model sees**: Through `dsh-tool-subagent`, the parent receives only the child's final output or stop-reason error. **Token effect**: Parent input grows by one data-dependent result retained until compaction. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Runs expose no `sendMessage`/`resume`** — the optional runtime capabilities are absent on in-process runs. diff --git a/packages/subagent/subagent-subprocess/README.md b/packages/subagent/subagent-subprocess/README.md index f15defcf2a..cc536e6d0e 100644 --- a/packages/subagent/subagent-subprocess/README.md +++ b/packages/subagent/subagent-subprocess/README.md @@ -41,6 +41,8 @@ A per-run isolated config directory for an external CLI child (the target of `CL Indirectly, through process-based subagent backends, whose child composition is constrained by credential scrubbing and isolated config directories. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **The credential scrub is name-based** — only variables matching `KEY` / `SECRET` / `TOKEN` are removed; differently named secrets such as `PASSWORD` pass through unless the backend supplies a stricter environment. diff --git a/packages/subagent/subagent/README.md b/packages/subagent/subagent/README.md index 17edd0e3c7..76c66b84b1 100644 --- a/packages/subagent/subagent/README.md +++ b/packages/subagent/subagent/README.md @@ -66,6 +66,8 @@ The model-facing tool collects synchronously by default: it awaits the child res Indirectly, through `dsh-tool-subagent`, which renders provider-specific schemas and foreground or generic-background results while child working context remains child-only. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Runtime steering and continuation are seam-only capabilities** — `sendMessage` and `resume` have no model-facing consumer in the current tool. diff --git a/packages/subagent/tool-subagent/README.md b/packages/subagent/tool-subagent/README.md index 825e252923..a7f8775fa0 100644 --- a/packages/subagent/tool-subagent/README.md +++ b/packages/subagent/tool-subagent/README.md @@ -36,18 +36,24 @@ Foreground and background calls are exclusive. Children may share the parent's w **Token effect**: Fixed schema cost per parent request; each provider instance adds one schema. +**KV Cache effect**: Prefix-stable while provider instances, names, descriptions, and schemas are unchanged. Provider registration lifecycle may invalidate parent reuse from the first changed tool definition. + ### Foreground result **What the model sees**: The call retains the description and prompt. Success contains only the child's final text; other outcomes become `Error: `. Intermediate child steps stay out of the parent. **Token effect**: The prompt and result remain in parent history until compaction; child working context remains in the child. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Background task result **What the model sees**: Start returns exactly `started background subagent task `. The generic task surface provides later status, final output, cancellation responses, and notices. **Token effect**: The acknowledgement is retained; final output enters parent history only when collected or injected. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Background runs expose final output only** — intermediate child steps stay in the child session. diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md index 27e0012d19..dd22565eb5 100644 --- a/packages/support/acp-snapshot/README.md +++ b/packages/support/acp-snapshot/README.md @@ -46,6 +46,8 @@ Constraints: `suite.ts` imports vitest, so the package entry is importable only None, as this test-only harness records, normalizes, and compares ACP transcripts without changing the agent's assembled model request. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **Session harvest is JSONL-only** — `runScenario` collects persisted `.jsonl` logs, so an example composed over the SQLite persistence backend has no snapshot path. diff --git a/packages/support/agent-loop-testkit/README.md b/packages/support/agent-loop-testkit/README.md index 350a8643e1..24aefeb29a 100644 --- a/packages/support/agent-loop-testkit/README.md +++ b/packages/support/agent-loop-testkit/README.md @@ -22,6 +22,8 @@ Tests of injection failures, partial topology, service load order, or service te None, as this test-only composition helper neither drives nor modifies model requests. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **Only the mandatory prerequisite spine is shared** — adapters, optional plugins, `AgentLoop`, agents, and Context teardown remain caller-owned so scenario-specific ordering stays visible. diff --git a/packages/support/invariants/README.md b/packages/support/invariants/README.md index 661e073d13..d9bb74769b 100644 --- a/packages/support/invariants/README.md +++ b/packages/support/invariants/README.md @@ -56,6 +56,8 @@ A seeded or forked session arrives with events already in its log because constr None, as this observer only validates events and frozen requests and never rewrites prompts, schemas, messages, or streams. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **The request-reconstructability assertion covers loop-built requests only** — hand-built one-shots (e.g. compaction's summarize call) carry no live `sessionId` marker and are skipped. diff --git a/packages/support/llm-replay/README.md b/packages/support/llm-replay/README.md index b256ef9a59..2134e722a7 100644 --- a/packages/support/llm-replay/README.md +++ b/packages/support/llm-replay/README.md @@ -56,6 +56,8 @@ Named `name` / `inject` / `Config` / `apply`, with **no default export**: the co None, as this keyless test adapter sends no request to a provider model; it only replays recorded assistant chunks into the test loop. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **First-call-order script binding assumes sequential delegation** — a cut that runs sibling subagents concurrently (or a compaction summarize call landing mid-run) would bind live sessions to recorded scripts non-deterministically; a stronger keying is deferred until such a scenario exists (`XXX(concurrent-subagents)`). diff --git a/packages/support/loader-smoke/README.md b/packages/support/loader-smoke/README.md index 0b783efc0d..bdb25180ca 100644 --- a/packages/support/loader-smoke/README.md +++ b/packages/support/loader-smoke/README.md @@ -10,6 +10,8 @@ This is support-tier test infrastructure, not product API. None, as this test-only harness boots example processes and inspects their streams without changing an assembled model request. +**KV Cache effect**: None; this package neither assembles nor sends a provider request. + ## Known Limitations and Deferred Work - **Built mode requires a prior build** — the config must also resolve every named package upward through `examples/node_modules`. diff --git a/packages/tasks/tasks/README.md b/packages/tasks/tasks/README.md index b061038714..90946c2728 100644 --- a/packages/tasks/tasks/README.md +++ b/packages/tasks/tasks/README.md @@ -26,6 +26,8 @@ See the [task type catalog](../../../docs/core-data-structures/tasks.md) and [ru Indirectly, through producer plugins and [`dsh-tool-tasks`](../tool-tasks/README.md), which render task ids, output, status, cancellation, and completion notices. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Tasks are process-local** — durable or cross-restart execution needs a separate lifecycle. diff --git a/packages/tasks/tool-tasks/README.md b/packages/tasks/tool-tasks/README.md index 520a874df1..cd62770a93 100644 --- a/packages/tasks/tool-tasks/README.md +++ b/packages/tasks/tool-tasks/README.md @@ -31,6 +31,8 @@ A default above the cap fails at load. **Token effect**: Small fixed input cost per request while active. +**KV Cache effect**: Prefix-stable while the plugin scope and guidance text are unchanged. Activation or disposal may invalidate reuse from this prompt section. + #### Background-task guidance ```markdown @@ -43,12 +45,16 @@ Track every background task id you start. You are notified in-session when a tas **Token effect**: Fixed schema cost on each request where the tools are visible. +**KV Cache effect**: Prefix-stable while tool definitions and visibility are unchanged. Registration lifecycle or scoped restrictions may invalidate reuse from the first changed schema token. + ### Results and notices **What the model sees**: Reads return output or `(no new output)` followed by `[status: ]` and optional detail. An empty list returns `(no background tasks)`. Kill returns `requested cancellation of task ` or the existing terminal status. Unreported owned completion uses the notice above. **Token effect**: Results and notices remain in parent history until compaction. Stream reads do not repeat consumed output. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Completion notices do not wake idle agents** — callers needing an immediate result must use `task_output`. diff --git a/packages/timeout/timeout-policy/README.md b/packages/timeout/timeout-policy/README.md index 474185a4eb..37fd75ad01 100644 --- a/packages/timeout/timeout-policy/README.md +++ b/packages/timeout/timeout-policy/README.md @@ -41,6 +41,8 @@ Multiple `tools/execute` listeners compose by cordis registration order. Combine **Token effect**: Zero tokens on non-timeout calls. A timeout adds one small retained error result and can prevent a larger late provider result from entering context. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Cooperative, never a hard kill** — the deadline only notifies via `exec.signal`; a tool that ignores the signal does not stop on timeout (see § Cooperative, not a hard kill). diff --git a/packages/todo/tool-todo/README.md b/packages/todo/tool-todo/README.md index f6097abe96..9517f70e5e 100644 --- a/packages/todo/tool-todo/README.md +++ b/packages/todo/tool-todo/README.md @@ -32,12 +32,16 @@ A function/namespace plugin: it exports `name` / `inject` / `apply` and NO defau **Token effect**: Fixed schema cost on every request where the tool is visible. +**KV Cache effect**: Prefix-stable while the definition and visibility are unchanged. Plugin lifecycle or scoped restrictions may invalidate reuse from this schema. + ### Tool-call history and result **What the model sees**: Each assistant tool call retains the entire replacement list in its arguments. Success returns exactly `Updated todo list: pending, in progress, completed.` Stable failures are ``Error: invalid todo: `content` must be a non-empty string``, `Error: invalid todos: duplicate content ""`, `Error: invalid todos: at most one task may be in_progress, got `, and `Error: todo_write requires an owning agent session`. The full `todo/write` session event is UI and replay state, not a second model message. **Token effect**: Token growth scales with every full list the model submits, and those call arguments remain until compaction. The result itself is small and fixed-shape. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **Single-owner scope only** — the list belongs to the one calling agent session; subagent/shared/swarm scopes are a deliberate cut (see § Single owner), and a non-agent caller is rejected. diff --git a/packages/ui/acp/README.md b/packages/ui/acp/README.md index 4e19313302..11d3befbd6 100644 --- a/packages/ui/acp/README.md +++ b/packages/ui/acp/README.md @@ -98,30 +98,40 @@ The JSON-RPC frames go on stdout, so this plugin MUST run in an example that loa **Token effect**: Prompt tokens are data-dependent and remain in that session's history until compaction. Concurrent ACP sessions keep separate contexts. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Human answers and permission decisions **What the model sees**: When optional consumers are loaded, ACP form answers become the exact JSON shape documented by `dsh-tool-ask-user`. Failures become `Error: ACP user questions must come from an agent-owned request`, `Error: ACP user question has no matching session`, `Error: ACP elicitation request failed`, `Error: ask_user_question was cancelled by the user`, `Error: ask_user_question returned no answer`, or `Error: ask_user_question was aborted before the user answered`. Permission decisions control whether another tool yields success or denial. ACP tool cards, terminal output, diffs, and streamed session updates are UI-only. **Token effect**: Answer, error, and denial text enters context only through the owning tool result; presentation metadata adds zero model tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Permission preset switches **What the model sees**: `session/set_config_option` emits no model message itself. When `dsh-permission` is composed, the bridge writes the selected preset through that service; the resulting model-visible policy prompt and change notice belong to [`dsh-user-approval`](../user-approval/README.md), while sandbox-mode effects belong to [`dsh-tool-bash`](../../bash/tool-bash/README.md). The ACP `Permissions` select, its option descriptions, pending idle value, and refreshed config response remain client-only. **Token effect**: Zero direct tokens from the ACP option or the log-only `permission/preset` event. Downstream cost is limited to the owning plugins' policy prompt, conditional retained change notice, and any changed tool outcome. +**KV Cache effect**: The ACP option and log event cause no direct invalidation. The downstream policy-prompt change may invalidate reuse from that system section, while its change notice appends to history. + ### Model switches **What the model sees**: The ACP selector itself emits no message. The selected provider/model pair supplies the next step's `{{provider}}` / `{{model}}` prompt variables and request routing together; all other call-config fields continue through the `agent/request` waterfall unchanged. **Token effect**: The selector adds no direct tokens. A changed model may tokenize the same retained prompt/history differently, and any persona text that interpolates provider or model changes accordingly. +**KV Cache effect**: Switching provider or model selects a different cache domain. If the persona interpolates either value, the rendered system prompt also changes and prevents reuse from its first changed token. + ### Loaded sessions **What the model sees**: `session/load` resumes the persisted log, after which the loop sends its reconstructed history and request header. Replaying that log to the editor is not an extra model message. **Token effect**: Restored context has the persistence and session packages' normal retained cost; ACP replay to the client adds none. +**KV Cache effect**: Loading does not rewrite the stored log, but the next request is reconstructed under the current envelope and route. Reuse requires that reconstruction to match; ACP replay to the client has no cache effect. + ## Known Limitations and Deferred Work - **`additionalDirectories`** — rejected. A session operates in its single `cwd` (see Per-session cwd); widening the tool/filesystem scope to extra roots is a separate sandbox concern, not yet implemented. diff --git a/packages/ui/app-boot/README.md b/packages/ui/app-boot/README.md index 5fed7b5023..d0512d4047 100644 --- a/packages/ui/app-boot/README.md +++ b/packages/ui/app-boot/README.md @@ -20,6 +20,8 @@ This package carries no loader hooks and no dev-mode surface: the `dsh-scripts` Indirectly, through the plugin tree it loads, which determines the prompts, schemas, messages, and model adapter in the resulting application. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Bare package specifiers depend on Loader internals** — production bins need `node --expose-internals` or the Loader's optional native fallback; an in-process caller without either must use resolvable relative/file specifiers or tsx path mapping. diff --git a/packages/ui/jsonrpc/README.md b/packages/ui/jsonrpc/README.md index 6666867598..d3f610b3d3 100644 --- a/packages/ui/jsonrpc/README.md +++ b/packages/ui/jsonrpc/README.md @@ -30,6 +30,8 @@ The plugin answers `shutdown`, disposes SDK-owned agents and subscriptions to qu **Token effect**: Data-dependent user-message tokens enter retained session history and are resent on later turns until another package compacts them. The JSON-RPC frames, session notifications, and server bookkeeping add zero model-context tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **The wire has no per-session close or prompt-cancel method** — SDK-created agents remain live until process shutdown, and one accepted prompt runs to agent idle before that session accepts another. diff --git a/packages/ui/permission/README.md b/packages/ui/permission/README.md index bd37890dd1..35eda1c3ac 100644 --- a/packages/ui/permission/README.md +++ b/packages/ui/permission/README.md @@ -10,6 +10,8 @@ The service requires a confining `ctx.bash` executor and `ctx.approval`. A table Indirectly, through `dsh-user-approval` and `dsh-tool-bash`, which render the approval-policy prompt, switch notice, and sandboxed tool outcomes selected by this service's knob events; `permission/preset` itself is log-only. +**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes. + ## Known Limitations and Deferred Work - **Only two mechanism knobs are bundled** — presets select sandbox mode and approval policy; an agent/profile choice is not part of `PresetSpec` yet. diff --git a/packages/ui/stdio/README.md b/packages/ui/stdio/README.md index caaa779c6e..c606d20e30 100644 --- a/packages/ui/stdio/README.md +++ b/packages/ui/stdio/README.md @@ -29,12 +29,16 @@ The plugin seeds display labels from the live agent registry, then tracks `agent **Token effect**: Submitted text is retained under the agent loop's normal session-history and compaction rules. The welcome banner, `> ` prompt, rendered transcript, and `[tool call]` / `[tool result]` terminal lines add no tokens. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ### Terminal user-interaction answers **What the model sees**: When a consumer calls `ctx.userInteraction.ask()`, this provider renders the question in the terminal and returns selected option labels or `custom` text. Through `dsh-tool-ask-user`, closed stdin becomes `Error: ask_user_question cannot be answered because stdin is closed`; disposal or abort becomes `Error: ask_user_question was interrupted before the user answered`. **Token effect**: Waiting and terminal prompts add no tokens; the resolved answer or error is model-visible only through the calling tool or plugin's result. +**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries. + ## Known Limitations and Deferred Work - **One configured session receives stdin** — the session/event renderer can print output from any session, but input lines always drive the configured `sessionId` rather than routing by the visible label. diff --git a/packages/ui/tool-ask-user/README.md b/packages/ui/tool-ask-user/README.md index 880b56ecc5..c1d920bff3 100644 --- a/packages/ui/tool-ask-user/README.md +++ b/packages/ui/tool-ask-user/README.md @@ -27,12 +27,16 @@ This is the consumer package for the user-interaction seam. It does not render U **Token effect**: Fixed schema cost on every request where the tool is visible. +**KV Cache effect**: Prefix-stable while the definition and visibility are unchanged. Plugin lifecycle or scoped restrictions may invalidate reuse from this schema. + ### Tool-call history and result **What the model sees**: The model's full questions remain in the assistant tool-call arguments. After the human answers, the next step sees compact JSON in the exact shape `{"answers":[{"id":"","selected":["