diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 53db69125c..5ee653103e 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -871,7 +871,7 @@ export interface Config { } ``` -Source: [`packages/fs/tool-fs-search/src/index.ts:59`](../packages/fs/tool-fs-search/src/index.ts) +Source: [`packages/fs/tool-fs-search/src/index.ts:62`](../packages/fs/tool-fs-search/src/index.ts) ## `@deepseek-ai/dsh-tool-skill` diff --git a/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md b/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md index bc8a452c91..4644f3e07a 100644 --- a/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md +++ b/docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md @@ -10,13 +10,13 @@ Search output also has two distinct budgets. The tool needs enough raw `rg` outp ## Decision -`glob` and `grep` are model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, backed by the bash seam, not by new `ctx.fs` provider methods. The package registers model-facing filesystem discovery tools, but execution uses `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)` with fixed `rg` command templates assembled by the tool. The tool layer owns schemas, argument validation, shell quoting, result parsing, result formatting, retention, formatted-result spill handoff, and timeout declaration. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution across local, sandboxed, or remote bash implementations. +`glob` and `grep` are conditional model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, backed by the bash seam, not by new `ctx.fs` provider methods. At plugin load, the package checks `command -v rg >/dev/null 2>&1` through `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)`; if the command exits nonzero, the package logs a warning and registers neither tools nor prompt sections. A probe that cannot start, times out, aborts, is killed, or produces no exit code fails plugin load loudly because that is a broken bash executor rather than an absent optional binary. When registered, execution uses the same `ctx.bash.resolve(request)` followed by `ctx.bash.run(spec)` flow with fixed `rg` command templates assembled by the tool. The tool layer owns schemas, argument validation, shell quoting, result parsing, result formatting, retention, formatted-result spill handoff, and timeout declaration. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution across local, sandboxed, or remote bash implementations. The tools do not use `ctx.bash.start()` and do not create model-visible background tasks. They run as ordinary foreground tools from the agent loop's perspective: the tool call returns only after the `rg` command exits, times out, is aborted, or fails. `defineTool({ timeoutMs })` declares the cooperative tool-call budget, `@deepseek-ai/dsh-timeout-policy` enforces it through `exec.signal`, and the tool forwards that signal into the bash request before `resolve()` / `run()`. The bash backend's own timeout remains a second safety cap; whichever aborts first wins. The tools align `path` with Claude Code's search tools while binding resolution to the bash workdir, not to `ctx.fs`. The tool derives the bash request workdir from `exec.agent?.session.header.cwd`, mirroring `dsh-tool-bash` and `dsh-tool-fs`; when no session cwd exists, it omits `request.workdir` so the bash implementation applies its configured cwd or process cwd through `resolve()`. For `grep`, `path` is an optional ripgrep target and may be a file or directory; omitted means the resolved bash workdir. For `glob`, `path` is an optional directory search root; omitted means the resolved bash workdir. Relative `path` values resolve against that workdir. Returned paths are displayed relative to the resolved bash workdir when possible and are intended to be follow-up-readable only in co-located deployments where the bash workdir and filesystem `read` root are the same workspace. v1 documents that deployment requirement but does not perform runtime cross-service validation. Remote or virtual filesystem search is deferred until there is a shared workspace/root contract or a provider-specific search backend. -The package does not inject `fs`. It injects `tools`, `systemPrompt`, and `bash`; it deliberately reads `spillStore` with `ctx.get('spillStore')` instead of static inject because formatted-result spill is optional. Existing `@deepseek-ai/dsh-tool-fs` deployments that only want `read` / `write` / `edit` do not need to load bash. +The package does not inject `fs`. It injects `tools`, `systemPrompt`, and `bash`; it deliberately reads `spillStore` with `ctx.get('spillStore')` instead of static inject because formatted-result spill is optional. Existing `@deepseek-ai/dsh-tool-fs` deployments that only want `read` / `write` / `edit` do not need to load bash. Deployments that load search need `rg` available in the bash executor environment for the tools to enter the model-visible schema. ### Package shape @@ -79,9 +79,9 @@ The `path` field follows the same split as Claude Code: `grep.path` is a file-or Raw `rg` stdout is an internal transport detail. The tool requests `stdoutMaxBytes: rawOutputMaxBytes` through `ctx.bash.resolve()` and parses `stdout.text` only when the executor returns untruncated stdout within that cap. If stdout is larger than `rawOutputMaxBytes`, or the executor still returns `stdout.truncated`, the tool fails with a clear search error telling the model to narrow `pattern`, `path`, or `include`. The tool never exposes raw `rg` output or bash raw spill paths to the model. -Only stdout is a parse source. Stderr is diagnostic text for invalid patterns, missing `rg`, and search failures; if bash truncates stderr, the tool uses the retained stderr tail with a truncation note and does not read `stderr.spillPath`. +Only stdout is a parse source. Stderr is diagnostic text for invalid patterns, runtime `rg` disappearance after registration, and search failures; if bash truncates stderr, the tool uses the retained stderr tail with a truncation note and does not read `stderr.spillPath`. -If `ctx.bash.run()` reports `aborted` because the tool timeout or caller cancellation fired, the tool returns a structured failure rather than pretending there were no matches. If bash reports its own timeout first, the tool likewise fails with a clear timeout message. Nonzero ripgrep exit semantics are tool-owned: exit 0 is success with matches, exit 1 is success with no matches, invalid pattern / missing `rg` / inaccessible search workdir are failures. +If `ctx.bash.run()` reports `aborted` because the tool timeout or caller cancellation fired, the tool returns a structured failure rather than pretending there were no matches. If bash reports its own timeout first, the tool likewise fails with a clear timeout message. Nonzero ripgrep exit semantics are tool-owned: exit 0 is success with matches, exit 1 is success with no matches, invalid pattern / runtime `rg` disappearance / inaccessible search workdir are failures. Search failures use a package-owned `HarnessError` subclass with `SEARCH_*` codes, not `FsErrorCode`, because these tools are not `ctx.fs` provider operations. The v1 vocabulary is `SEARCH_INVALID_PATTERN`, `SEARCH_FAILED`, `SEARCH_RAW_OUTPUT_OVERFLOW`, and `SEARCH_ABORTED`. Model argument validation failures such as missing required fields, blank strings, or unsupported negated/list `include` values remain ordinary tool argument errors. @@ -116,7 +116,7 @@ Line 12: ... (Full grep result stored at: /.../session-abc123/9f8e7d-grep-results.txt. Use read with offset/limit, or grep this path to search within it.) ``` -If the complete logical result fits under the inline cap, no formatted spill artifact is created. If the complete logical result is too large but formatted spill is unavailable, the footer says that the result was capped and the complete result could not be saved. The `truncated` / omitted count is a budget fact, not an incomplete-search fact; timeout, invalid regex, missing `rg`, inaccessible workdirs, raw-output overflow, binary skips, and parse failures stay in tool-domain error or incomplete fields. +If the complete logical result fits under the inline cap, no formatted spill artifact is created. If the complete logical result is too large but formatted spill is unavailable, the footer says that the result was capped and the complete result could not be saved. The `truncated` / omitted count is a budget fact, not an incomplete-search fact; timeout, invalid regex, runtime `rg` disappearance, inaccessible workdirs, raw-output overflow, binary skips, and parse failures stay in tool-domain error or incomplete fields. ## Alternatives considered @@ -138,22 +138,24 @@ If the complete logical result fits under the inline cap, no formatted spill art **Expand the bash seam with a raw-output reader first.** Rejected: a portable `readRawOutput(ref, maxBytes)` API would add reference lifetime, permission, and backend storage semantics. A per-run `stdoutMaxBytes` request is the narrower seam: search either receives complete stdout within `rawOutputMaxBytes` or fails clearly. +**Always register and report missing `rg` only at execution time.** Rejected: a model-visible tool schema is a promise that the deployment can attempt that capability. If the bash executor cannot find ripgrep at load, the safer surface is no `glob` / `grep` tools or prompt guidance. Execution-time missing-`rg` classification remains as a defensive fallback for environments that change after registration. + ## Testing -- Tests prove an aborted `exec.signal` reaches the bash backend (same-reference spec assertion plus the `SEARCH_ABORTED` result), and cover command construction/quoting (malicious patterns, paths with spaces, leading-dash values, quotes, newlines, glob metacharacters — unit assertions plus a real `bash -c` round-trip for every hostile value), `grep.path` as file and directory targets, `glob.path` as a directory search root, invalid pattern handling, no matches, malformed `rg --json` output, matched-line preview truncation, raw-output overflow, timeout/abort, formatted spill success/failure, the package-owned `SEARCH_*` error codes, and the no-background-task invariant. +- Tests cover registration-time `rg` probing (probe success registers both tools and prompt sections, nonzero probe skips both tools and prompt sections with a warning, infrastructure probe failures reject plugin load), prove an aborted `exec.signal` reaches the bash backend (same-reference spec assertion plus the `SEARCH_ABORTED` result), and cover command construction/quoting (malicious patterns, paths with spaces, leading-dash values, quotes, newlines, glob metacharacters — unit assertions plus a real `bash -c` round-trip for every hostile value), `grep.path` as file and directory targets, `glob.path` as a directory search root, invalid pattern handling, no matches, malformed `rg --json` output, matched-line preview truncation, raw-output overflow, timeout/abort, formatted spill success/failure, the package-owned `SEARCH_*` error codes, and the no-background-task invariant. - The first-party tool-owned spill precedent is covered directly: spill backend present, spill backend absent, `saveText()` failure, and missing spill owner. - The package has real Loader-path coverage for the namespace plugin export shape (`name`, `inject`, `Config`, and `apply`, with no default export). -- A real-executor integration suite (`dsh-bash-local` + a real `rg`) verifies the world: hostile patterns stay inert, per-session cwd resolution, VCS-metadata exclusion, modification-time ordering, and real ripgrep stderr classification. It self-skips where `rg` is not on PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor suite alone carries the per-file 100% coverage gate. +- A real-executor integration suite (`dsh-bash-local` + a real `rg`) verifies the world: hostile patterns stay inert, per-session cwd resolution, VCS-metadata exclusion, modification-time ordering, and real ripgrep stderr classification. It self-skips where `rg` is not on the test process PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor suite carries registration and execution coverage for missing `rg`, plus the per-file 100% coverage gate. - Snapshot gap note for the transcript-visible spill notice: this landed with the gap note, not a snapshot. The snapshot tier replays the acp-agent tree, and adding the search plugin there changes the assembled system prompt — every golden would need re-recording with a real key, which the implementing environment did not hold. The spill notice's exact transcript text is pinned by unit tests (`formatGlobOutput`/`formatGrepOutput` and the through-the-registry spill tests); wiring the plugin into the acp-agent tree plus a `test:snapshot:record` pass is the follow-up for the next key-holding session. ## Consequences -- `glob` and `grep` are model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, not `ctx.fs` provider methods and not part of the existing `@deepseek-ai/dsh-tool-fs` root plugin. The package injects `tools`, `systemPrompt`, and `bash`; it does not inject `fs`, and `ctx.spillStore` stays optional via `ctx.get('spillStore')`. +- `glob` and `grep` are conditional model-facing tools in `@deepseek-ai/dsh-tool-fs-search`, not `ctx.fs` provider methods and not part of the existing `@deepseek-ai/dsh-tool-fs` root plugin. They register only when the bash executor can find `rg`; the package injects `tools`, `systemPrompt`, and `bash`, does not inject `fs`, and keeps `ctx.spillStore` optional via `ctx.get('spillStore')`. - The schemas are exactly `glob(pattern, path?)` and `grep(pattern, path?, include?)`; search caps and timeout are defaulted, validated Config fields (`globMaxResults`, `grepMaxMatches`, `grepMaxLineBytes`, `rawOutputMaxBytes`, `timeoutMs`). - The tools execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)`, forward `exec.signal`, never call `ctx.bash.start()`, and never expose a bash task id. The bash request workdir comes from `exec.agent?.session.header.cwd` when available; the resolved `spec.workdir` drives execution and relative-path display. - The tools request `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam, parse only untruncated stdout within that cap, and treat over-cap or still-truncated raw output as a clear search failure; raw `rg` output is never exposed to the model. - Oversized complete formatted results are saved through `ctx.spillStore.saveText()` when available while inline results stay bounded; spill failure, a missing backend, or a missing owner preserves the inline result and reports the unsaved remainder — never an `isError`. -- The package README, the generated config catalog, and exported JSDoc document the Config fields and `SEARCH_*` codes; the coding-agent example ships the tools (the acp-agent tree waits on the snapshot re-record above); the fs group README records the co-located bash/filesystem deployment requirement. +- The package README, the generated config catalog, and exported JSDoc document the Config fields and `SEARCH_*` codes; the coding-agent example ships the conditional tool plugin (the acp-agent tree waits on the snapshot re-record above); the fs group README records the `rg` availability and co-located bash/filesystem deployment requirements. ## Risks diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index 382eb04c4c..5b6e281b1a 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -20,7 +20,7 @@ This table connects model-visible tool names to the plugin package and service s | `@deepseek-ai/dsh-tool-bash` | `bash`, `bash_kill`, `bash_output` | `ctx.tools`, `ctx.bash` | `tool/call`, `tool/result`, `context/message via agent.inject() for background completion notices` | - | The bash/bash_output/bash_kill tools are model-facing consumers of the bash executor seam. | | `@deepseek-ai/dsh-tool-cordis` | `cordis_inspect`, `cordis_mount`, `cordis_unmount` | `ctx.tools` | `tool/call`, `tool/result`, `live plugin-tree mutations (mount/unmount)` | - | Ships in examples/cordis-agent only (a deliberate opt-in — mounted code gets the real ctx, see docs/rfc/implemented/feature/2026-07-08-self-referential-cordis-toolset.md). Plugins the model mounts may register ADDITIONAL model-visible tools at runtime; the request-header ToolsDelta logs those tool-set changes. | | `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after successful file operations`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The tool schemas above are identical with or without the policy plugin. | -| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | +| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. | | `@deepseek-ai/dsh-tool-skill` | `skill` | `ctx.tools`, `ctx.skills` | `tool/call`, `tool/result` | - | - | | `@deepseek-ai/dsh-tool-subagent` | `subagent` | `ctx.tools`, `ctx.subagents` | `tool/call`, `tool/result`, `child session events through the chosen provider` | `subagent`, `subagent_fork` | The registered tool name is the load-time `toolName` config (default `subagent`); the schema above is that default. The shipped example agents load this package once per subagent backend, so the model additionally sees `subagent_fork` (bound to the fork backend) with an identical schema — see `examples/coding-agent/cordis.yml` and `examples/acp-agent/cordis.yml`. | | `@deepseek-ai/dsh-tool-todo` | `todo_write` | `ctx.tools`, `owning Agent session` | `tool/call`, `todo/write`, `tool/result` | - | todo_write is session-owned state; UIs render the latest todo/write event as a checklist or ACP plan. | @@ -428,7 +428,7 @@ Search file contents with a ripgrep regular expression. Returns matching lines w Source: [`packages/fs/tool-fs-search/src/index.ts`](../packages/fs/tool-fs-search/src/index.ts) -glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. +glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. ## `@deepseek-ai/dsh-tool-skill` diff --git a/examples/coding-agent/cordis.yml b/examples/coding-agent/cordis.yml index b23d785439..698ae97403 100644 --- a/examples/coding-agent/cordis.yml +++ b/examples/coding-agent/cordis.yml @@ -133,9 +133,10 @@ - id: tool-fs name: '@deepseek-ai/dsh-tool-fs' -# Bash-backed discovery tools (glob/grep): fixed ripgrep commands through the -# local bash executor above — not ctx.fs. Capped results save the complete -# formatted list through the spill backend below (ctx.spillStore, optional). +# Bash-backed discovery tools (glob/grep): if the local bash executor above +# can find rg, register fixed ripgrep commands — not ctx.fs. Capped results +# save the complete formatted list through the spill backend below +# (ctx.spillStore, optional). - id: tool-fs-search name: '@deepseek-ai/dsh-tool-fs-search' diff --git a/packages/core/tools/tests/gen-tool-catalog.spec.ts b/packages/core/tools/tests/gen-tool-catalog.spec.ts index 43490b99dc..b77cbdfd8d 100644 --- a/packages/core/tools/tests/gen-tool-catalog.spec.ts +++ b/packages/core/tools/tests/gen-tool-catalog.spec.ts @@ -61,6 +61,19 @@ describe('gen-tool-catalog collectToolCatalog', () => { expect(bash?.source).toBe('packages/bash/tool-bash/src/index.ts') }) + it('harvests search tools without depending on the generator process PATH', async () => { + const oldPath = process.env.PATH + try { + process.env.PATH = '' + const catalog = await collectToolCatalog() + const search = catalog.find(entry => entry.pkg === '@deepseek-ai/dsh-tool-fs-search') + expect(search?.schemas.map(s => s.name).sort()).toEqual(['glob', 'grep']) + } finally { + if (oldPath === undefined) delete process.env.PATH + else process.env.PATH = oldPath + } + }) + it('records the shipped `subagent_fork` alias in a note (config-driven tool name)', async () => { // `tool-subagent`'s registered name is the load-time `toolName` config, so // the shipped agents surface this one package as both `subagent` and diff --git a/packages/fs/README.md b/packages/fs/README.md index 039cb39ae9..ba57eede25 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -8,9 +8,9 @@ The filesystem stack: a provider seam (text IO + atomic mutation with an optiona | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | | `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) | -| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools, backed by fixed ripgrep commands through the bash seam (`ctx.bash`), NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | +| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). ## No timeouts on file IO diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index 2f9df200de..81e0042e92 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -1,20 +1,20 @@ # @deepseek-ai/dsh-tool-fs-search -The **model-facing filesystem discovery tools** — `glob`, `grep` — backed by the **bash executor seam**, not by `ctx.fs` provider methods. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a bounded, workdir-relative result. The package injects `tools`, `systemPrompt`, and `bash` — deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. +The **model-facing filesystem discovery tools** — `glob`, `grep` — backed by the **bash executor seam**, not by `ctx.fs` provider methods. At load, the package probes `command -v rg` through `ctx.bash`; if the executor cannot find ripgrep on its `PATH`, it logs a warning and registers no tools or prompt sections. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a bounded, workdir-relative result. The package injects `tools`, `systemPrompt`, and `bash` — deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. ```ts ignore-check -// Default deployment: a bash executor, then the discovery tools. +// Default deployment: a bash executor whose PATH includes rg, then the discovery tools. await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local -await ctx.plugin(ToolFsSearch) // this package — registers glob/grep +await ctx.plugin(ToolFsSearch) // this package — conditionally registers glob/grep // Optional: a spill backend makes capped results fully recoverable. await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local ``` Why bash-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution (local, sandboxed, remote); this package owns schemas, argument validation, shell quoting, parsing, retention, formatted-result spill, and timeout declaration. The tools never call `ctx.bash.start()` and never expose a bash task id — the call returns only after `rg` exits, times out, is aborted, or fails. -## Deployment requirement: co-located bash + filesystem +## Deployment requirement: rg + co-located bash/filesystem -Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. +The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin load; otherwise `glob` and `grep` are absent from the model-visible tool schema. Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend. ## Config @@ -43,4 +43,4 @@ Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMax ## Errors -Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (missing `rg`, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. +Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors. diff --git a/packages/fs/tool-fs-search/src/index.ts b/packages/fs/tool-fs-search/src/index.ts index 8c33d5770a..5930890b7a 100644 --- a/packages/fs/tool-fs-search/src/index.ts +++ b/packages/fs/tool-fs-search/src/index.ts @@ -1,6 +1,7 @@ /** * The model-facing filesystem discovery tool suite (`glob`, `grep`) over the - * bash executor seam (`ctx.bash`). This single plugin registers both tools. + * bash executor seam (`ctx.bash`). This single plugin registers both tools + * only when the mounted bash executor can find `rg` on its `PATH`. * * ## Bash-backed, not a `ctx.fs` provider method * @@ -12,9 +13,11 @@ * parsing, retention, formatted-result spill, and timeout declaration; the * bash executor owns request defaulting/capping, subprocess execution, * process-group termination, environment scrubbing, raw output capture, and - * backend substitution. The package injects `tools`, `systemPrompt`, and - * `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read opportunistically - * with `ctx.get()` because formatted-result spill is optional. + * backend substitution. At load, the package probes `command -v rg` through the + * same bash seam; if ripgrep is absent, `glob` / `grep` and their prompt + * sections are not registered. The package injects `tools`, `systemPrompt`, + * and `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read + * opportunistically with `ctx.get()` because formatted-result spill is optional. * * Returned paths are displayed relative to the resolved bash workdir and are * follow-up-readable only in co-located deployments where the bash workdir and @@ -80,6 +83,9 @@ export const Config: z = z.object({ /** The shape after schemastery applied the defaults. */ type ResolvedConfig = Required +/** POSIX-shell builtin probe for the ripgrep binary in the bash executor environment. */ +const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + /** Every search cap counts items/bytes/milliseconds — a positive integer, or retention and timeout arithmetic misbehaves silently. */ function assertPositiveInteger(name: string, value: number): void { if (!Number.isInteger(value) || value < 1) { @@ -87,8 +93,38 @@ function assertPositiveInteger(name: string, value: number): void { } } -/** Register the `glob`/`grep` filesystem discovery tool suite. */ -export function apply(ctx: Context, config: Config): void { +/** + * Check whether the mounted bash executor can find `rg`. + * + * Nonzero exit means "not available" and disables this optional tool suite. + * Infrastructure failures stay loud: a deployment with a broken bash executor + * should not silently lose tools in a way that looks like a deliberate skip. + * + * @param ctx - plugin context whose `bash` service is the executor the tools will use. + * @returns true when `command -v rg` exits 0, false when it exits nonzero. + */ +async function ripgrepAvailable(ctx: Context): Promise { + const spec = ctx.bash.resolve({ command: RG_PROBE_COMMAND }) + let result + try { + result = await ctx.bash.run(spec) + } catch (error: unknown) { + throw new Error(`tool-fs-search: ripgrep availability probe could not start: ${String(error)}`, { cause: error }) + } + if (result.aborted || result.timedOut || result.signal !== null || result.exitCode === null) { + throw new Error('tool-fs-search: ripgrep availability probe did not complete') + } + return result.exitCode === 0 +} + +/** + * Register the `glob`/`grep` filesystem discovery tool suite when `rg` exists. + * + * @param ctx - plugin context; registrations are effects scoped to this plugin. + * @param config - resolved plugin configuration from schemastery. + * @returns when ripgrep is unavailable, resolves without registering any tools. + */ +export async function apply(ctx: Context, config: Config): Promise { // schemastery (Config) has already filled every defaulted field. const resolved = config as ResolvedConfig assertPositiveInteger('globMaxResults', resolved.globMaxResults) @@ -96,6 +132,10 @@ export function apply(ctx: Context, config: Config): void { assertPositiveInteger('grepMaxLineBytes', resolved.grepMaxLineBytes) assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes) assertPositiveInteger('timeoutMs', resolved.timeoutMs) + if (!await ripgrepAvailable(ctx)) { + ctx.logger.warn('tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered') + return + } applyGlobTool(ctx, { maxResults: resolved.globMaxResults, rawOutputMaxBytes: resolved.rawOutputMaxBytes, diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index 9131940de5..648eb19585 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -2,12 +2,12 @@ * Consumer-surface tests for the search tools over a FAKE bash executor and a * FAKE spill backend, exercised through `ctx.tools.execute()` so nothing * bypasses the tool registry. The fake executor makes every seam outcome - * scriptable — truncated stdout with/without a raw spill path, abort/timeout, - * signal kills, ripgrep exit codes — so these tests verify schemas, argument - * validation, shell-safe command construction, workdir derivation, signal - * forwarding, `SEARCH_*` error classification, retention, formatted-result - * spill handoff, and the no-background-task invariant. Real-`rg` behavior is - * pinned separately in integration.spec.ts. + * scriptable — registration-time `rg` probing, truncated stdout with/without a + * raw spill path, abort/timeout, signal kills, ripgrep exit codes — so these + * tests verify schemas, argument validation, shell-safe command construction, + * workdir derivation, signal forwarding, `SEARCH_*` error classification, + * retention, formatted-result spill handoff, and the no-background-task + * invariant. Real-`rg` behavior is pinned separately in integration.spec.ts. */ import { describe, expect, it } from 'vitest' @@ -31,6 +31,8 @@ import { toWorkdirRelative, } from '@deepseek-ai/dsh-tool-fs-search' +const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + /** A successful run result over the given stdout; overrides script the failure shapes. */ function runResult(stdout: string, overrides?: Partial): BashRunResult { return { @@ -52,13 +54,18 @@ function runResult(stdout: string, overrides?: Partial): BashRunR * create a background task. */ class FakeBash extends BashExecutor { + probeRequests: BashExecRequest[] = [] + probeSpecs: BashExecSpec[] = [] requests: BashExecRequest[] = [] specs: BashExecSpec[] = [] startCalls = 0 + probeResult: BashRunResult = runResult('') + probeError?: Error handler: (spec: BashExecSpec) => BashRunResult = () => runResult('') override resolve(request: BashExecRequest): BashExecSpec { - this.requests.push(request) + if (request.command === RG_PROBE_COMMAND) this.probeRequests.push(request) + else this.requests.push(request) return { command: request.command, workdir: request.workdir ?? '/work', @@ -69,9 +76,14 @@ class FakeBash extends BashExecutor { sandboxMode: request.sandboxMode, } } - override run(spec: BashExecSpec): Promise { + override async run(spec: BashExecSpec): Promise { + if (spec.command === RG_PROBE_COMMAND) { + this.probeSpecs.push(spec) + if (this.probeError) throw this.probeError + return this.probeResult + } this.specs.push(spec) - return Promise.resolve(this.handler(spec)) + return this.handler(spec) } override start(): BashTask { this.startCalls++ @@ -113,18 +125,36 @@ class FakeSpill extends SpillStore { interface SetupOptions { config?: ToolFsSearch.Config spill?: boolean + probeError?: Error + probeResult?: BashRunResult } async function setup(options: SetupOptions = {}) { const ctx = new Context() + const warnings: string[] = [] + ctx.logger.warn = ((message: unknown) => { warnings.push(String(message)) }) as typeof ctx.logger.warn await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(FakeBash) + const bash = ctx.bash as FakeBash + if (options.probeResult) bash.probeResult = options.probeResult + if (options.probeError) bash.probeError = options.probeError if (options.spill === true) await ctx.plugin(FakeSpill) const fiber = await ctx.plugin(ToolFsSearch, options.config) - const bash = ctx.bash as FakeBash const spill = options.spill === true ? ctx.get('spillStore') as FakeSpill : undefined - return { ctx, bash, spill, fiber } + return { ctx, bash, spill, fiber, warnings } +} + +/** Assert plugin setup rejects without letting Vitest pretty-print a live Context on failure. */ +async function expectSetupRejects(options: SetupOptions, message: RegExp): Promise { + let thrown: string | undefined + try { + const loaded = await setup(options) + await loaded.fiber.dispose() + } catch (error: unknown) { + thrown = error instanceof Error ? error.message : String(error) + } + expect(thrown).toMatch(message) } /** A stand-in agent whose session header carries the given cwd (and a stable id). */ @@ -152,13 +182,37 @@ function matchLine(path: string, lineNumber: number, lineText: string): string { describe('registration', () => { it('registers glob and grep with their prompt sections', async () => { - const { ctx } = await setup() + const { ctx, bash } = await setup() + expect(bash.probeRequests).toHaveLength(1) + expect(bash.probeRequests[0]?.command).toBe(RG_PROBE_COMMAND) + expect(bash.probeRequests[0]).not.toHaveProperty('workdir') expect(ctx.tools.schemas().map(s => s.name).sort()).toEqual(['glob', 'grep']) const prompt = renderPrompt(await ctx.systemPrompt.assemble()) expect(prompt).toContain('Use the glob tool') expect(prompt).toContain('Use the grep tool') }) + it('does not register glob or grep when the bash executor cannot find rg', async () => { + const { ctx, warnings } = await setup({ probeResult: runResult('', { exitCode: 1 }) }) + expect(ctx.tools.schemas()).toHaveLength(0) + const sections = (await ctx.systemPrompt.assemble()).sections.map(s => s.name) + expect(sections).not.toContain('tool:glob') + expect(sections).not.toContain('tool:grep') + expect(warnings).toEqual([ + 'tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered', + ]) + }) + + it('rejects plugin load when the rg availability probe cannot run', async () => { + await expectSetupRejects({ probeError: new Error('spawn bash ENOENT') }, /spawn bash ENOENT/) + }) + + it('rejects plugin load when the rg availability probe is aborted or killed', async () => { + await expectSetupRejects({ + probeResult: runResult('', { aborted: true, exitCode: null, signal: 'SIGTERM' }), + }, /tool-fs-search: ripgrep availability probe did not complete/) + }) + it('stays pending until ctx.bash exists (inject)', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index e317f2be24..91ac3adfd1 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -39,6 +39,8 @@ import { Context } from 'cordis' import type { ToolSchema } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools' +import { BashExecutor } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskId, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' import LocalBashExecutor from '@deepseek-ai/dsh-bash-local' import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import UserInteractionService from '@deepseek-ai/dsh-user-interaction' @@ -63,6 +65,65 @@ import * as ToolWorkflow from '@deepseek-ai/dsh-tool-workflow' const root = resolve(import.meta.dirname, '..') const OUT = 'docs/tool-catalog.md' +const CATALOG_RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1' + +/** + * Minimal bash service for harvesting `dsh-tool-fs-search` schemas. The search + * plugin now probes `rg` at registration time, but the generated catalog must + * remain independent of the host PATH and never execute a real search. + */ +class CatalogSearchBashExecutor extends BashExecutor { + override resolve(request: BashExecRequest): BashExecSpec { + return { + command: request.command, + workdir: request.workdir ?? root, + timeoutMs: request.timeoutMs ?? 60_000, + stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000, + signal: request.signal, + owner: request.owner, + sandboxMode: request.sandboxMode, + } + } + + override run(spec: BashExecSpec): Promise { + if (spec.command !== CATALOG_RG_PROBE_COMMAND) { + throw new Error(`gen-tool-catalog: unexpected search bash command during schema harvest: ${spec.command}`) + } + return Promise.resolve({ + exitCode: 0, + signal: null, + timedOut: false, + aborted: false, + timeoutMs: spec.timeoutMs, + stdout: { text: '', truncated: false }, + stderr: { text: '', truncated: false }, + }) + } + + override start(): BashTask { + throw new Error('gen-tool-catalog: search schema harvest must not start bash tasks') + } + + override get(): BashTask | undefined { + return undefined + } + + override ownerOf(): OwnerToken | undefined { + return undefined + } + + override list(): BashTask[] { + return [] + } + + override readOutput(id: BashTaskId): BashTaskRead { + throw new Error(`gen-tool-catalog: unknown bash task ${id}`) + } + + override kill(id: BashTaskId): boolean { + throw new Error(`gen-tool-catalog: unknown bash task ${id}`) + } +} /** * One tool-plugin package to boot. `mount` is a per-entry recipe (async): it @@ -192,14 +253,15 @@ const TOOL_PACKAGES: ToolPackage[] = [ writes: ['tool/call', 'tool/result'], async mount(ctx) { // The tools inject `bash` (search executes fixed `rg` commands through - // the executor seam, not ctx.fs); boot the local executor to satisfy it. - // `ctx.spillStore` is optional (read via ctx.get) and does not affect the - // schemas, so no spill backend is mounted. - await ctx.plugin(LocalBashExecutor) + // the executor seam, not ctx.fs). Use a catalog-only executor so the + // registration-time `rg` probe stays deterministic and the generator + // never depends on the host PATH. `ctx.spillStore` is optional (read via + // ctx.get) and does not affect the schemas, so no spill backend is mounted. + await ctx.plugin(CatalogSearchBashExecutor) await ctx.plugin(ToolFsSearch) }, note: - 'glob and grep are bash-backed discovery tools: they run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', + 'glob and grep are conditional bash-backed discovery tools: they register only when ctx.bash can find `rg`, then run fixed ripgrep commands through ctx.bash as ordinary foreground calls (never background tasks). Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments.', }, { pkg: '@deepseek-ai/dsh-tool-skill',