feat(fs-search): spawn the packaged ripgrep binary through the subprocess seam
glob/grep now run the @vscode/ripgrep binary via ctx.subprocess with a plain argv vector: no system rg install, no shell layer, unconditional registration. The load-time command -v rg probe and the bash-seam coupling are removed; timeouts ride the cooperative exec.signal plus the seam's terminate escalation. The fs-glob-sampling ACP snapshot executes the real packaged binary against an mtime-pinned fixture. Adds the packaged-ripgrep-search Agent Note, updates the roster-note facts and both shipped-composition e2es, and regenerates the doc catalogs and third-party notices (surfacing pre-existing manifest drift plus the new @vscode/ripgrep row; the notices generator also learns pnpm 11's truncated virtual-store names).
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/tool-fs-search/README.md
|
||||
README.md: b12ffda9869c7d6bef5ea5b54594781ecf555ff4
|
||||
README.zh.md: 7dd6cdf9a209f2fe357b4ffe48d20d574266ce60
|
||||
README.md: 0152be017ae15fc83a3d5cb7df927f25d04d2d53
|
||||
README.zh.md: 69ce49f3ad1621021dc1d0938cdc07a900d1cda8
|
||||
|
||||
@@ -2,21 +2,21 @@
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **bash executor seam**, not by `ctx.fs` provider methods. At load, the package probes `command -v rg` through `ctx.bash`; if the executor cannot find ripgrep on its `PATH`, it logs a warning and registers no tools or prompt sections. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `bash`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional.
|
||||
The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **packaged ripgrep binary** (`@vscode/ripgrep`), not by `ctx.fs` provider methods and not by a system `rg` install. Registration is unconditional: the binary ships inside the npm dependency, so there is no load-time availability probe. Each call spawns the binary through the `ctx.subprocess` seam with a fixed argv vector (model-controlled values are plain argv elements — no shell layer exists, so no quoting applies), parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `subprocess`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional.
|
||||
|
||||
```ts ignore-check
|
||||
// A deployment chooses how over-cap glob pages are selected.
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local
|
||||
await ctx.plugin(LocalSubprocessService) // @deepseek-ai/dsh-subprocess-local
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false })
|
||||
// Optional: a spill backend makes capped results fully recoverable.
|
||||
await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local
|
||||
```
|
||||
|
||||
Why bash-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution (local, sandboxed, remote); this package owns schemas, argument validation, shell quoting, parsing, retention, formatted-result spill, and timeout declaration. The tools never call `ctx.bash.start()` and never expose a bash task id — the call returns only after `rg` exits, times out, is aborted, or fails.
|
||||
Why spawn-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The subprocess seam owns spawn execution, process-tree termination, environment scrubbing, and bounded output capture; this package owns schemas, argument validation, argv construction, parsing, retention, formatted-result spill, and timeout declaration. The tools never expose a background task — the call returns only after `rg` exits, is terminated by the cooperative timeout, is aborted, or fails.
|
||||
|
||||
## Deployment requirement: rg + co-located bash/filesystem
|
||||
## Deployment requirement: no host rg, co-located workdir/filesystem
|
||||
|
||||
The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin load; otherwise `glob` and `grep` are absent from the model-visible tool schema. Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend.
|
||||
The binary ships with the package on every supported platform (macOS/Linux/Windows, x64/arm64), so no host `rg` install is required and the tools register on every deployment. Returned paths are displayed relative to the resolved workdir (the calling agent's session cwd when present, else `process.cwd()`) and are follow-up-readable with `read` only when that workdir and the filesystem root are the same workspace. v1 documents that co-location requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend.
|
||||
|
||||
## Config
|
||||
|
||||
@@ -29,24 +29,24 @@ The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin
|
||||
| `grepMaxMatches` | `250` | Max flat matches one `grep` call retains inline (matches Claude Code's `GrepTool` `head_limit`); later matches go to the formatted spill artifact. |
|
||||
| `grepMaxLineBytes` | `2000` | Byte cap per matched-line preview; the cut preserves UTF-8 boundaries and is marked `(line truncated)`. |
|
||||
| `rawOutputMaxBytes` | `20000000` | Max complete raw `rg` stdout a search will parse (matches Claude Code's ripgrep raw buffer); larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. |
|
||||
| `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the bash backend's own timeout stays a second safety cap. |
|
||||
| `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the subprocess seam's terminate escalation is the hard kill. |
|
||||
|
||||
## Tools
|
||||
|
||||
| Tool | Arguments | Behavior |
|
||||
|---|---|---|
|
||||
| `glob` | `pattern`, `path?` | `rg --files --glob <pattern> --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved bash workdir. Returns one FILE path per line; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. Complete results stay modification-time ordered; over-cap presentation follows `sampleOverCapGlobResults`. |
|
||||
| `glob` | `pattern`, `path?` | `rg --files --glob <pattern> --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved workdir. Returns one FILE path per line; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. Complete results stay modification-time ordered; over-cap presentation follows `sampleOverCapGlobResults`. |
|
||||
| `grep` | `pattern`, `path?`, `include?` | Line-oriented `rg --json` parse (no colon-splitting ambiguity). `pattern` is a ripgrep regex; `path` is an optional **file or directory** target; `include` is ONE positive glob filter — a comma-separated list or a negated (`!…`) value is rejected up front (brace alternation like `*.{ts,tsx}` is fine). Returns matches grouped by file as `Line N: <preview>`. |
|
||||
|
||||
Routine budgets stay out of the model-facing schema (no `head_limit`/`offset`/`case_insensitive`/output modes): a model that needs surrounding context reads the matched file with `read`; one that needs later results follows the returned spill locator's retrieval hint.
|
||||
|
||||
## Two budgets, two artifacts
|
||||
|
||||
Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam and parses only complete retained stdout; if the executor still returns `stdout.truncated`, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`.
|
||||
Raw `rg` stdout is an internal transport detail. Each search requests a collect-mode stdout budget of `rawOutputMaxBytes` from the subprocess seam and parses only complete retained stdout; if the seam still reports a lossy read, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`.
|
||||
|
||||
## Errors
|
||||
|
||||
Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors.
|
||||
Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (a failed `rg` launch, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still lossy after the requested stdout capture budget), and `SEARCH_ABORTED` (cooperative tool timeout or caller cancellation). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -54,7 +54,7 @@ Search failures carry the package-owned `SearchError` (a `HarnessError` subclass
|
||||
|
||||
#### What the model sees
|
||||
|
||||
After the load-time `rg` probe succeeds, every request in this plugin's registration scope contains the independently registered glob and grep guidance below. Agent-scoped tool restrictions can hide either schema without removing its prompt section.
|
||||
Every request in this plugin's registration scope contains the independently registered glob and grep guidance below. Agent-scoped tool restrictions can hide either schema without removing its prompt section.
|
||||
|
||||
##### Glob guidance with `sampleOverCapGlobResults: true`
|
||||
|
||||
@@ -86,7 +86,7 @@ Prefix-stable while the plugin scope, sampling choice, and guidance text are unc
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The glob description states the configured over-cap ordering. The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) use `sampleOverCapGlobResults: true`; schemas are visible only after the load-time `rg` probe succeeds.
|
||||
The glob description states the configured over-cap ordering. The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) use `sampleOverCapGlobResults: true`; the tools are registered unconditionally.
|
||||
|
||||
#### Token effect
|
||||
|
||||
@@ -126,7 +126,7 @@ Append-only; newly visible content follows the reusable request prefix and does
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the bash workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation.
|
||||
- **Ripgrep is a deployment dependency** — a missing `rg` executable makes the package register no tools or guidance; an incompatible executable or one that disappears after registration fails calls with `SEARCH_FAILED`. Remote or virtual filesystems need a co-located executor or another search consumer.
|
||||
- **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation.
|
||||
- **The packaged binary is fixed at dependency version** — `@vscode/ripgrep` covers the platforms it ships (macOS/Linux/Windows, x64/arm64); an unsupported platform or a corrupted install fails calls with `SEARCH_FAILED`. Remote or virtual filesystems need a co-located workspace or another search consumer.
|
||||
- **The schemas expose one bounded page** — offset pagination, case-mode switches, alternate output modes, and provider-backed discovery remain outside this package; capped complete output requires a spill backend.
|
||||
- **Sampling, when enabled, groups by first path segment beneath the search root only** — an over-cap `glob` page balances across those top-level entries, so a result concentrated deeper (one busy directory inside an otherwise even tree) is still shown unevenly below that level; recursive balancing is deferred.
|
||||
|
||||
@@ -2,21 +2,21 @@
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
**面向模型的文件系统发现工具**(`glob`、`grep`)由 **bash 执行器 seam** 支持,而不是由 `ctx.fs` 提供方方法支持。加载时,本包(package)探测 `command -v rg`,探测通过 `ctx.bash` 进行;如果执行器无法在其 `PATH` 上找到 ripgrep,就记录警告,并且不注册工具或提示词段。每次调用都会组装固定的 ripgrep 命令(所有模型控制的值都经过同一个包私有 shell 引用辅助函数),通过 `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` 作为普通前台工具调用运行,解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `bash`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。
|
||||
**面向模型的文件系统发现工具**(`glob`、`grep`)由 **打包的 ripgrep 二进制**(`@vscode/ripgrep`)支持,而不是由 `ctx.fs` 提供方方法或系统 `rg` 安装支持。注册是无条件的:二进制随 npm 依赖一起交付,因此没有加载期可用性探针。每次调用都通过 `ctx.subprocess` seam 以固定 argv 向量 spawn 该二进制(模型控制的值是普通 argv 元素——不存在 shell 层,因此无需引号),解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `subprocess`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。
|
||||
|
||||
```ts ignore-check
|
||||
// A deployment chooses how over-cap glob pages are selected.
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local
|
||||
await ctx.plugin(LocalSubprocessService) // @deepseek-ai/dsh-subprocess-local
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false })
|
||||
// Optional: a spill backend makes capped results fully recoverable.
|
||||
await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local
|
||||
```
|
||||
|
||||
采用 bash 支持的原因:本地工作区发现天然是由进程支持的 `rg` 工作流;如果把搜索放到 `ctx.fs` 上,就会迫使每个文件系统后端扩展搜索 API。bash 执行器负责请求默认值/上限、子进程执行、进程组终止、环境清理、原始输出捕获和后端替换(本地、沙箱化、远程);本包负责 schema、参数校验、shell 引用、解析、保留、格式化结果 spill 和超时声明。工具绝不调用 `ctx.bash.start()`,也不公开 bash task id;只有在 `rg` 退出、超时、中止或失败后,调用才会返回。
|
||||
采用 spawn 支持的原因:本地工作区发现天然是由进程支持的 `rg` 工作流;如果把搜索放到 `ctx.fs` 上,就会迫使每个文件系统后端扩展搜索 API。subprocess seam 负责 spawn 执行、进程树终止、环境清理和有界输出捕获;本包负责 schema、参数校验、argv 构造、解析、保留、格式化结果 spill 和超时声明。工具绝不暴露后台任务——只有在 `rg` 退出、被协作式超时终止、被中止或失败后,调用才会返回。
|
||||
|
||||
## 部署要求:rg 与共置的 bash/文件系统
|
||||
## 部署要求:无需宿主 rg,但工作目录与文件系统需共置
|
||||
|
||||
已挂载的 bash 执行器必须能在插件加载时解析 `rg`,其来源是执行器的 `PATH`;否则面向模型的工具 schema 中不会出现 `glob` 和 `grep`。返回路径会相对于解析后的 bash 工作目录显示(调用方 agent(智能体)有会话 cwd 时使用该 cwd,否则使用执行器配置的默认值);只有 bash 工作目录与文件系统根目录是同一工作区时,才能用 `read` 继续读取。v1 只记录这项共置要求,不执行运行时跨服务校验;远程或虚拟文件系统搜索需等待共享工作区契约或特定提供方的搜索后端。
|
||||
二进制随包交付,覆盖所有受支持平台(macOS/Linux/Windows,x64/arm64),因此无需宿主 `rg` 安装,工具在每个部署上都注册。返回路径会相对于解析后的工作目录显示(调用方 agent(智能体)有会话 cwd 时使用该 cwd,否则使用 `process.cwd()`);只有该工作目录与文件系统根目录是同一工作区时,才能用 `read` 继续读取。v1 只记录这项共置要求,不执行运行时跨服务校验;远程或虚拟文件系统搜索需等待共享工作区契约或特定提供方的搜索后端。
|
||||
|
||||
## 配置
|
||||
|
||||
@@ -29,24 +29,24 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
|
||||
| `grepMaxMatches` | `250` | 一次 `grep` 调用内联保留的最大平铺匹配数(与 Claude Code 的 `GrepTool` `head_limit` 相同);后续匹配写入格式化 spill 产物。 |
|
||||
| `grepMaxLineBytes` | `2000` | 每条匹配行预览的字节上限;截断会保留 UTF-8 边界,并标记为 `(line truncated)`。 |
|
||||
| `rawOutputMaxBytes` | `20000000` | 搜索将解析的完整原始 `rg` stdout 上限(与 Claude Code 的 ripgrep 原始 buffer 相同);更大的原始输出以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败。 |
|
||||
| `timeoutMs` | `30000` | 附加到两个工具定义上的协作式工具调用预算,由 `@deepseek-ai/dsh-timeout-policy` 通过 `exec.signal` 强制执行;bash 后端自身的超时仍作为第二道安全上限。 |
|
||||
| `timeoutMs` | `30000` | 附加到两个工具定义上的协作式工具调用预算,由 `@deepseek-ai/dsh-timeout-policy` 通过 `exec.signal` 强制执行;subprocess seam 的终止升级提供硬终止。 |
|
||||
|
||||
## 工具
|
||||
|
||||
| 工具 | 参数 | 行为 |
|
||||
|---|---|---|
|
||||
| `glob` | `pattern`、`path?` | 运行 `rg --files --glob <pattern> --sort=modified --no-ignore --hidden`,并排除 VCS 元数据(`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`)。`path` 是可选的**目录**搜索根;省略时使用解析后的 bash 工作目录。每行返回一个**文件**路径;`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义:不含 `/` 时匹配任意深度的基名,因此 `*` 匹配整棵树。完整结果保持按修改时间排序;超过上限时的呈现方式遵循 `sampleOverCapGlobResults`。 |
|
||||
| `glob` | `pattern`、`path?` | 运行 `rg --files --glob <pattern> --sort=modified --no-ignore --hidden`,并排除 VCS 元数据(`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`)。`path` 是可选的**目录**搜索根;省略时使用解析后的工作目录。每行返回一个**文件**路径;`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义:不含 `/` 时匹配任意深度的基名,因此 `*` 匹配整棵树。完整结果保持按修改时间排序;超过上限时的呈现方式遵循 `sampleOverCapGlobResults`。 |
|
||||
| `grep` | `pattern`、`path?`、`include?` | 按行解析 `rg --json`,避免按冒号拆分的歧义。`pattern` 是 ripgrep 正则表达式;`path` 是可选的**文件或目录**目标;`include` 是一个正向 glob 过滤器,前置拒绝逗号分隔列表或否定值(`!…`),但允许 `*.{ts,tsx}` 等花括号交替。返回按文件分组、形如 `Line N: <preview>` 的匹配。 |
|
||||
|
||||
常规预算不进入面向模型的 schema(没有 `head_limit`/`offset`/`case_insensitive`/输出模式):模型需要周边上下文时,用 `read` 读取匹配文件;需要后续结果时,遵循返回的 spill locator 检索提示。
|
||||
|
||||
## 两类预算、两类产物
|
||||
|
||||
原始 `rg` stdout 是内部传输细节。每次搜索从 bash seam 请求 `stdoutMaxBytes: rawOutputMaxBytes`,且只解析完整保留的 stdout;如果执行器仍返回 `stdout.truncated`,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。
|
||||
原始 `rg` stdout 是内部传输细节。每次搜索从 subprocess seam 请求 `rawOutputMaxBytes` 的 collect 模式 stdout 预算,且只解析完整保留的 stdout;如果 seam 仍报告 lossy 读取,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。
|
||||
|
||||
## 错误
|
||||
|
||||
搜索失败携带本包拥有的 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(注册后 `rg` 在运行时消失、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍被截断)和 `SEARCH_ABORTED`(工具超时、调用方取消或 bash 执行器自身超时)。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。
|
||||
搜索失败携带本包拥有的 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(`rg` 启动失败、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍 lossy)和 `SEARCH_ABORTED`(协作式工具超时或调用方取消)。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。
|
||||
|
||||
## 模型体验
|
||||
|
||||
@@ -54,7 +54,7 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
加载时 `rg` 探测成功后,该插件注册作用域内的每个请求都包含下方独立注册的 glob 与 grep 指导。agent 作用域的工具限制可以隐藏任一 schema,而不移除其提示词段。
|
||||
该插件注册作用域内的每个请求都包含下方独立注册的 glob 与 grep 指导。agent 作用域的工具限制可以隐藏任一 schema,而不移除其提示词段。
|
||||
|
||||
##### 启用 `sampleOverCapGlobResults: true` 时的 Glob 指导
|
||||
|
||||
@@ -76,57 +76,57 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read
|
||||
|
||||
#### Token 影响
|
||||
|
||||
工具注册期间,每个请求支付固定指导成本;必填的采样选项决定采用哪个 glob 变体。
|
||||
工具注册期间每个请求有固定的指导成本;必填的采样选择决定采用哪一个 glob 变体。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
只要插件作用域、采样选项和指导文本不变,前缀就保持稳定。启用、dispose(资源释放)或更改该选项,可能从该提示词段开始使复用失效。
|
||||
插件作用域、采样选择与指导文本不变时前缀稳定。激活、销毁或改变选择可能使该提示词段的复用失效。
|
||||
|
||||
### 工具 schema
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
glob 描述会说明配置所指定的超限结果排序方式。已生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) 使用 `sampleOverCapGlobResults: true`;只有加载时 `rg` 探测成功后,这些 schema 才可见。
|
||||
glob 描述声明了配置的超过上限排序方式。生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) 使用 `sampleOverCapGlobResults: true`;工具无条件注册。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
工具可见的每个请求都支付固定 schema 成本。
|
||||
工具可见时每个请求有固定的 schema 成本。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
只要工具可见性和定义不变,前缀就保持稳定。注册生命周期或作用域限制可能从首个变化的 schema token 开始使复用失效。
|
||||
工具可见性与定义不变时前缀稳定。注册生命周期或作用域限制可能从第一个改变的 schema token 起使复用失效。
|
||||
|
||||
### 结果与 spill 通知
|
||||
### 结果与 spill 提示
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
`glob` 每行返回一个路径;`grep` 在每个路径下对 `Line <line>: <preview>` 匹配分组。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果末尾会附加省略数量、spill locator 和后端检索提示,或说明完整结果无法保存。`sampleOverCapGlobResults: true` 时,超过上限的 `glob` 页面会在实际搜索根正下方的条目之间按轮转方式取路径,footer 会说明采样依据和触达的顶层条目数;若无法触达全部条目,footer 会要求模型缩小 `path`。设为 `false` 时,页面保留按修改时间排序的前部,并沿用通常用于达到上限结果的 footer。未超过上限的结果原样不动;扁平的采样结果也沿用普通 footer,因为其样本等同于按修改时间排序的前部。spill 产物始终保存按修改时间排序的完整列表。
|
||||
`glob` 每行返回一个路径;`grep` 在每个路径下分组展示 `Line <line>: <preview>` 匹配。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果以省略计数结尾,并附 spill locator 与后端检索提示;否则说明完整结果无法保存。启用 `sampleOverCapGlobResults: true` 时,超过上限的 `glob` 页面按实际搜索根正下方的条目轮转取路径,页脚说明采样依据及其覆盖的顶层条目数;无法覆盖全部条目时,页脚提示模型收窄 `path`。`false` 时页面是按修改时间排序的前部,并保留普通的上限结果页脚。未超过上限的结果原样呈现;扁平采样的结果也保留普通页脚,因为其采样等于按修改时间排序的前部。spill 产物始终持有按修改时间排序的完整列表。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
内联路径和匹配受 `globMaxResults`、`grepMaxMatches` 与 `grepMaxLineBytes` 限制;调用和保留结果会留在历史中,直到上下文压缩(compaction)。
|
||||
内联路径与匹配受 `globMaxResults`、`grepMaxMatches` 与 `grepMaxLineBytes` 约束;调用与保留结果在压缩前留在历史中。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV-cache 条目失效。
|
||||
只追加;新可见内容跟在可复用请求前缀之后,不会使既有 KV-cache 条目失效。
|
||||
|
||||
### 工具错误
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
失败会规范化为 `Error: <message>`,并向调用方提供结构化的 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据。
|
||||
失败被规范化为 `Error: <message>`,并携带结构化 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据供调用方使用。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
只有失败调用会添加这些保留 token。
|
||||
只有失败的调用会增加这些保留 token。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV-cache 条目失效。
|
||||
只追加;新可见内容跟在可复用请求前缀之后,不会使既有 KV-cache 条目失效。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
## 已知局限与延期工作
|
||||
|
||||
- **搜索和文件访问没有共享工作区证明**:只有 bash 工作目录和文件系统根目录表示同一工作区时,返回路径才能继续读取;本包不执行运行时跨服务校验。
|
||||
- **Ripgrep 是部署依赖**:缺失 `rg` 可执行文件时,本包不注册工具或指导;可执行文件不兼容或注册后消失时,调用以 `SEARCH_FAILED` 失败。远程或虚拟文件系统需要共置执行器或其他搜索消费方。
|
||||
- **schema 只公开一个有界页面**:offset 分页、大小写模式开关、其他输出模式和提供方支持的发现均不在本包内;达到上限的完整输出需要 spill 后端。
|
||||
- **启用采样时,只按搜索根下的路径首段分组**:超过上限的 `glob` 页面在这些顶层条目之间做均衡,因此集中在更深层的结果(一棵总体均匀的树里某个特别庞大的子目录)在该层级以下仍然分布不均;递归均衡已延期。
|
||||
- **搜索与文件访问没有共享工作区证明**——只有当工作目录与文件系统根目录指向同一工作区时,返回路径才保证可继续读取;本包不执行运行时跨服务校验。
|
||||
- **打包二进制固定在依赖版本上**——`@vscode/ripgrep` 覆盖其随附的平台(macOS/Linux/Windows,x64/arm64);不支持的平台或损坏的安装会以 `SEARCH_FAILED` 使调用失败。远程或虚拟文件系统需要共置的工作区或另一个搜索消费方。
|
||||
- **schema 只暴露一个有界页面**——偏移分页、大小写开关、替代输出模式与提供方支撑的发现仍不在本包范围内;达到上限的完整输出需要 spill 后端。
|
||||
- **启用采样时仅按搜索根正下方的第一段路径分组**——超过上限的 `glob` 页面在这些顶层条目之间平衡,因此集中在更深处的结果(一棵均匀树里某个繁忙目录)在该层级之下仍会呈现不均;递归平衡被延期。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-tool-fs-search",
|
||||
"description": "Model-facing filesystem discovery tools (glob, grep) backed by the DeepSeek Harness bash seam (ctx.bash)",
|
||||
"description": "Model-facing filesystem discovery tools (glob, grep) backed by the packaged ripgrep binary (@vscode/ripgrep)",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
@@ -27,23 +27,23 @@
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"@vscode/ripgrep": "^1.18.0",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-bash": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-retention": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-spill": "^0.0.1",
|
||||
"@deepseek-ai/dsh-subprocess": "^0.0.1",
|
||||
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-bash": "workspace:^",
|
||||
"@deepseek-ai/dsh-bash-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-subprocess": "workspace:^",
|
||||
"@deepseek-ai/dsh-subprocess-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
/**
|
||||
* The model-facing `glob` tool: discover files whose paths match a glob
|
||||
* pattern, sorted by modification time. Execution goes through the bash seam
|
||||
* (`ctx.bash`) with a fixed `rg --files` command — this module owns the
|
||||
* model-facing schema, argument validation, shell-safe command construction,
|
||||
* result parsing, inline sampling, and formatting; process concerns (defaulting,
|
||||
* scrubbing, kill, backend substitution) stay behind `ctx.bash`.
|
||||
* pattern, sorted by modification time. Execution spawns the packaged
|
||||
* ripgrep binary (`@vscode/ripgrep`) directly through the subprocess seam
|
||||
* with a plain argv vector — this module owns the model-facing schema,
|
||||
* argument validation, argv construction, result parsing, inline sampling,
|
||||
* and formatting; process concerns (spawn execution, tree termination,
|
||||
* environment scrubbing, output capture) stay behind `ctx.subprocess`.
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/glob
|
||||
*/
|
||||
|
||||
@@ -13,11 +14,9 @@ import { sep } from 'node:path'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { GenericCallView, SearchResultView, ToolResult } from '@deepseek-ai/dsh-tools'
|
||||
import type { SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import type {} from '@deepseek-ai/dsh-bash'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
import { runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts'
|
||||
import { globSearchMeta, searchViewFromMeta } from './presentation.ts'
|
||||
import { singleQuote } from './shell-quote.ts'
|
||||
import { acceptedSurfaceValue } from './surface.ts'
|
||||
|
||||
/**
|
||||
@@ -73,32 +72,35 @@ export function parseGlobArgs(args: { pattern: string; path?: string }): GlobInp
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the fixed `rg --files` command for one `glob` call. Every
|
||||
* Build the fixed `rg --files` argv for one `glob` call. Every
|
||||
* model-controlled value ({@link GlobInput.pattern}, {@link GlobInput.path})
|
||||
* passes through {@link singleQuote}; the search root rides behind `--` so a
|
||||
* leading-dash path can never be parsed as a flag. `--sort=modified` orders by
|
||||
* modification time, `--no-ignore --hidden` searches ignored and hidden files,
|
||||
* and {@link GLOB_VCS_EXCLUDES} keeps VCS metadata out.
|
||||
* is a plain argv element — no shell layer exists, so no quoting applies; the
|
||||
* search root rides behind `--` so a leading-dash path can never be parsed as
|
||||
* a flag. `--sort=modified` orders by modification time, `--no-ignore
|
||||
* --hidden` searches ignored and hidden files, and
|
||||
* {@link GLOB_VCS_EXCLUDES} keeps VCS metadata out.
|
||||
*
|
||||
* @param input - the validated arguments.
|
||||
* @returns the complete, shell-safe command string.
|
||||
* @returns the complete ripgrep argument vector (excluding the binary itself).
|
||||
*/
|
||||
export function buildGlobCommand(input: GlobInput): string {
|
||||
export function buildGlobCommand(input: GlobInput): string[] {
|
||||
const parts = [
|
||||
'rg --files',
|
||||
`--glob=${singleQuote(input.pattern)}`,
|
||||
'--sort=modified --no-ignore --hidden',
|
||||
'--files',
|
||||
`--glob=${input.pattern}`,
|
||||
'--sort=modified',
|
||||
'--no-ignore',
|
||||
'--hidden',
|
||||
// Two negated globs per VCS name: the bare form prunes the directory
|
||||
// during traversal; the /** form still excludes the contents when the
|
||||
// search root is AT or INSIDE the directory (where the bare form,
|
||||
// matched against root-prefixed paths, never fires).
|
||||
...GLOB_VCS_EXCLUDES.flatMap(name => [
|
||||
`--glob=${singleQuote(`!**/${name}`)}`,
|
||||
`--glob=${singleQuote(`!**/${name}/**`)}`,
|
||||
`--glob=!**/${name}`,
|
||||
`--glob=!**/${name}/**`,
|
||||
]),
|
||||
]
|
||||
if (input.path !== undefined) parts.push('--', singleQuote(input.path))
|
||||
return parts.join(' ')
|
||||
if (input.path !== undefined) parts.push('--', input.path)
|
||||
return parts
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -285,7 +287,7 @@ export function presentGlobResult(_args: { pattern: string; path?: string }, res
|
||||
* Register the `glob` tool and its system-prompt guidance.
|
||||
*
|
||||
* @param ctx - the plugin context; registrations are effects scoped to it, and
|
||||
* execution uses its `bash` service.
|
||||
* execution uses its `subprocess` service.
|
||||
* @param caps - the deployment's resolved glob caps (plugin config after defaulting).
|
||||
*/
|
||||
export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void {
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
/**
|
||||
* The model-facing `grep` tool: search file contents with a ripgrep regular
|
||||
* expression. Execution goes through the bash seam (`ctx.bash`) with a fixed
|
||||
* line-oriented `rg --json` command so file path, line number, and line text
|
||||
* parse without colon-splitting ambiguity — this module owns the model-facing
|
||||
* schema, argument validation, shell-safe command construction, `--json`
|
||||
* record parsing, per-line preview retention, match retention, grouping, and
|
||||
* formatting; process concerns stay behind `ctx.bash`.
|
||||
* expression. Execution spawns the packaged ripgrep binary
|
||||
* (`@vscode/ripgrep`) directly through the subprocess seam with a plain argv
|
||||
* vector using a fixed line-oriented `rg --json` command so file path, line
|
||||
* number, and line text parse without colon-splitting ambiguity — this module
|
||||
* owns the model-facing schema, argument validation, argv construction,
|
||||
* `--json` record parsing, per-line preview retention, match retention,
|
||||
* grouping, and formatting; process concerns stay behind `ctx.subprocess`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/grep
|
||||
*/
|
||||
@@ -15,12 +16,10 @@ import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { GenericCallView, SearchResultView, ToolResult } from '@deepseek-ai/dsh-tools'
|
||||
import type { RetainedItems } from '@deepseek-ai/dsh-retention'
|
||||
import type { SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import type {} from '@deepseek-ai/dsh-bash'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
import type { GrepMatch } from './search-core.ts'
|
||||
import { SearchError, previewLine, retainGrepMatches, runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts'
|
||||
import { grepSearchMeta, searchViewFromMeta } from './presentation.ts'
|
||||
import { singleQuote } from './shell-quote.ts'
|
||||
import { acceptedSurfaceValue } from './surface.ts'
|
||||
|
||||
/**
|
||||
@@ -96,20 +95,21 @@ export function parseGrepArgs(args: { pattern: string; path?: string; include?:
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the fixed line-oriented `rg --json` command for one `grep` call. Every
|
||||
* Build the fixed line-oriented `rg --json` argv for one `grep` call. Every
|
||||
* model-controlled value ({@link GrepInput.pattern}, {@link GrepInput.path},
|
||||
* {@link GrepInput.include}) passes through {@link singleQuote}; the pattern
|
||||
* and include ride in `--flag=value` form and the target behind `--`, so a
|
||||
* leading-dash value can never be parsed as a flag.
|
||||
* {@link GrepInput.include}) is a plain argv element — no shell layer exists,
|
||||
* so no quoting applies; the pattern and include ride in `--flag=value` form
|
||||
* and the target behind `--`, so a leading-dash value can never be parsed as
|
||||
* a flag.
|
||||
*
|
||||
* @param input - the validated arguments.
|
||||
* @returns the complete, shell-safe command string.
|
||||
* @returns the complete ripgrep argument vector (excluding the binary itself).
|
||||
*/
|
||||
export function buildGrepCommand(input: GrepInput): string {
|
||||
const parts = ['rg --json', `--regexp=${singleQuote(input.pattern)}`]
|
||||
if (input.include !== undefined) parts.push(`--glob=${singleQuote(input.include)}`)
|
||||
if (input.path !== undefined) parts.push('--', singleQuote(input.path))
|
||||
return parts.join(' ')
|
||||
export function buildGrepCommand(input: GrepInput): string[] {
|
||||
const parts = ['--json', `--regexp=${input.pattern}`]
|
||||
if (input.include !== undefined) parts.push(`--glob=${input.include}`)
|
||||
if (input.path !== undefined) parts.push('--', input.path)
|
||||
return parts
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,28 +1,27 @@
|
||||
/**
|
||||
* The model-facing filesystem discovery tool suite (`glob`, `grep`) over the
|
||||
* bash executor seam (`ctx.bash`). This single plugin registers both tools
|
||||
* only when the mounted bash executor can find `rg` on its `PATH`.
|
||||
* packaged ripgrep binary (`@vscode/ripgrep`). This single plugin registers
|
||||
* both tools; the binary ships inside the npm dependency, so no system `rg`
|
||||
* install and no shell layer is involved.
|
||||
*
|
||||
* ## Bash-backed, not a `ctx.fs` provider method
|
||||
* ## Spawn-backed, not a `ctx.fs` provider method
|
||||
*
|
||||
* Local workspace discovery is a process-backed `rg` workflow, so these tools
|
||||
* execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` with fixed
|
||||
* ripgrep command templates — never `ctx.bash.start()`, never a model-visible
|
||||
* background task. The tool layer owns schemas, argument validation, shell
|
||||
* quoting ({@link module:@deepseek-ai/dsh-tool-fs-search/shell-quote}), result
|
||||
* parsing, retention, formatted-result spill, and timeout declaration; the
|
||||
* bash executor owns request defaulting/capping, subprocess execution,
|
||||
* process-group termination, environment scrubbing, raw output capture, and
|
||||
* backend substitution. At load, the package probes `command -v rg` through the
|
||||
* same bash seam; if ripgrep is absent, `glob` / `grep` and their prompt
|
||||
* sections are not registered. The package injects `tools`, `systemPrompt`,
|
||||
* and `bash` — deliberately NOT `fs`, and `ctx.spillStore` is read
|
||||
* execute through `ctx.subprocess.spawn()` with fixed ripgrep argv templates —
|
||||
* never `ctx.bash`, never `ctx.bash.start()`, never a model-visible background
|
||||
* task. The tool layer owns schemas, argument validation, argv construction
|
||||
* ({@link module:@deepseek-ai/dsh-tool-fs-search/glob} /
|
||||
* {@link module:@deepseek-ai/dsh-tool-fs-search/grep}), result parsing,
|
||||
* retention, formatted-result spill, and timeout declaration; the subprocess
|
||||
* seam owns spawn execution, process-tree termination, environment scrubbing,
|
||||
* and raw output capture. The package injects `tools`, `systemPrompt`, and
|
||||
* `subprocess` — deliberately NOT `fs`, and `ctx.spillStore` is read
|
||||
* opportunistically with `ctx.get()` because formatted-result spill is optional.
|
||||
*
|
||||
* Returned paths are displayed relative to the resolved bash workdir and are
|
||||
* follow-up-readable only in co-located deployments where the bash workdir and
|
||||
* the filesystem `read` root are the same workspace — a documented v1
|
||||
* deployment requirement, not runtime-validated.
|
||||
* Returned paths are displayed relative to the resolved workdir and are
|
||||
* follow-up-readable only in co-located deployments where the workdir and the
|
||||
* filesystem `read` root are the same workspace — a documented v1 deployment
|
||||
* requirement, not runtime-validated.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search
|
||||
*/
|
||||
@@ -65,7 +64,7 @@ export { singleQuote } from './shell-quote.ts'
|
||||
export const name = 'tool-fs-search'
|
||||
|
||||
/** Services required by the search tool suite (`spillStore` is optional, read via `ctx.get()`). */
|
||||
export const inject = ['tools', 'systemPrompt', 'bash']
|
||||
export const inject = ['tools', 'systemPrompt', 'subprocess']
|
||||
|
||||
/** Plugin config; over-cap glob sampling is an explicit deployment choice and the remaining fields have defaults. */
|
||||
export interface Config {
|
||||
@@ -98,9 +97,6 @@ export const Config: z<Config> = z.object({
|
||||
/** The shape after schemastery applied the defaults. */
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/** POSIX-shell builtin probe for the ripgrep binary in the bash executor environment. */
|
||||
const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1'
|
||||
|
||||
/** Every search cap counts items/bytes/milliseconds — a positive integer, or retention and timeout arithmetic misbehaves silently. */
|
||||
function assertPositiveInteger(name: string, value: number): void {
|
||||
if (!Number.isInteger(value) || value < 1) {
|
||||
@@ -109,36 +105,14 @@ function assertPositiveInteger(name: string, value: number): void {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether the mounted bash executor can find `rg`.
|
||||
*
|
||||
* Nonzero exit means "not available" and disables this optional tool suite.
|
||||
* Infrastructure failures stay loud: a deployment with a broken bash executor
|
||||
* should not silently lose tools in a way that looks like a deliberate skip.
|
||||
*
|
||||
* @param ctx - plugin context whose `bash` service is the executor the tools will use.
|
||||
* @returns true when `command -v rg` exits 0, false when it exits nonzero.
|
||||
*/
|
||||
async function ripgrepAvailable(ctx: Context): Promise<boolean> {
|
||||
const spec = ctx.bash.resolve({ command: RG_PROBE_COMMAND })
|
||||
let result
|
||||
try {
|
||||
result = await ctx.bash.run(spec)
|
||||
} catch (error: unknown) {
|
||||
throw new Error(`tool-fs-search: ripgrep availability probe could not start: ${String(error)}`, { cause: error })
|
||||
}
|
||||
if (result.aborted || result.timedOut || result.signal !== null || result.exitCode === null) {
|
||||
throw new Error('tool-fs-search: ripgrep availability probe did not complete')
|
||||
}
|
||||
return result.exitCode === 0
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `glob`/`grep` filesystem discovery tool suite when `rg` exists.
|
||||
* Register the `glob`/`grep` filesystem discovery tool suite. The packaged
|
||||
* ripgrep binary is always available (an npm dependency), so registration is
|
||||
* unconditional.
|
||||
*
|
||||
* @param ctx - plugin context; registrations are effects scoped to this plugin.
|
||||
* @param config - resolved plugin configuration from schemastery.
|
||||
* @returns when ripgrep is unavailable, resolves without registering any tools.
|
||||
*/
|
||||
// oxlint-disable-next-line typescript/require-await -- async keeps a load-time config rejection a rejection, not a synchronous throw
|
||||
export async function apply(ctx: Context, config: Config): Promise<void> {
|
||||
// schemastery (Config) has already filled every defaulted field.
|
||||
const resolved = config as ResolvedConfig
|
||||
@@ -148,10 +122,6 @@ export async function apply(ctx: Context, config: Config): Promise<void> {
|
||||
assertPositiveInteger('searchMetaMaxBytes', resolved.searchMetaMaxBytes)
|
||||
assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes)
|
||||
assertPositiveInteger('timeoutMs', resolved.timeoutMs)
|
||||
if (!await ripgrepAvailable(ctx)) {
|
||||
ctx.logger.warn('tool-fs-search: ripgrep (rg) not found on the bash executor PATH; glob/grep tools not registered')
|
||||
return
|
||||
}
|
||||
applyGlobTool(ctx, {
|
||||
sampleOverCapGlobResults: resolved.sampleOverCapGlobResults,
|
||||
maxResults: resolved.globMaxResults,
|
||||
|
||||
12
packages/fs/tool-fs-search/src/ripgrep.d.ts
vendored
Normal file
12
packages/fs/tool-fs-search/src/ripgrep.d.ts
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
/**
|
||||
* Minimal type surface for the `@vscode/ripgrep` package: an ESM module that
|
||||
* resolves the platform ripgrep binary (`@vscode/ripgrep-<platform>-<arch>`
|
||||
* optional dependency) and exports its absolute path as the named export
|
||||
* `rgPath` (no bundled type declarations).
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/ripgrep-types
|
||||
*/
|
||||
|
||||
declare module '@vscode/ripgrep' {
|
||||
/** Absolute path to the packaged ripgrep executable for the current platform. */
|
||||
export const rgPath: string
|
||||
}
|
||||
@@ -1,16 +1,19 @@
|
||||
/**
|
||||
* Shared execution plumbing for the `glob` / `grep` search tools: the
|
||||
* package-owned `SEARCH_*` error vocabulary, one bash-seam run helper that
|
||||
* turns a fixed `rg` command into complete raw stdout, the best-effort
|
||||
* formatted-result spill handoff, and workdir-relative path display.
|
||||
* package-owned `SEARCH_*` error vocabulary, one spawn helper that runs the
|
||||
* PACKAGED ripgrep binary (`@vscode/ripgrep`) with a plain argv vector and
|
||||
* returns complete raw stdout, the best-effort formatted-result spill handoff,
|
||||
* and workdir-relative path display.
|
||||
*
|
||||
* Both tools execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)`
|
||||
* as ordinary foreground tool calls — never `ctx.bash.start()`, never a
|
||||
* model-visible background task. Raw `rg` stdout is an internal transport
|
||||
* detail: the tools request a per-run stdout capture budget from the bash seam,
|
||||
* parse only complete in-memory stdout within `rawOutputMaxBytes`, and never
|
||||
* read executor spill files. The model-facing recovery artifact is the
|
||||
* formatted result saved through `ctx.spillStore.saveText()`
|
||||
* Both tools execute as ordinary foreground spawns through `ctx.subprocess` —
|
||||
* never `ctx.bash`, never `ctx.bash.start()`, never a model-visible background
|
||||
* task. The ripgrep binary ships inside the npm package, so no system `rg`
|
||||
* install is required, and no shell layer exists between the argv vector and
|
||||
* ripgrep, so no shell quoting is involved. Raw `rg` stdout is an internal
|
||||
* transport detail: the tools request a per-run stdout capture budget from the
|
||||
* subprocess seam, parse only complete in-memory stdout within
|
||||
* `rawOutputMaxBytes`, and never read spill files. The model-facing recovery
|
||||
* artifact is the formatted result saved through `ctx.spillStore.saveText()`
|
||||
* ({@link trySaveFormattedResult}).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/search-core
|
||||
@@ -18,10 +21,11 @@
|
||||
|
||||
import { isAbsolute, relative, sep } from 'node:path'
|
||||
import type { Context } from 'cordis'
|
||||
import { rgPath } from '@vscode/ripgrep'
|
||||
import { HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import { ItemRetainer, TextRetainer } from '@deepseek-ai/dsh-retention'
|
||||
import type { RetainedItems } from '@deepseek-ai/dsh-retention'
|
||||
import type { BashRunResult, CollectedOutput } from '@deepseek-ai/dsh-bash'
|
||||
import type { SubprocessCollect, SubprocessOutcome, SubprocessOutputRead, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess'
|
||||
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import type { ToolExecution } from '@deepseek-ai/dsh-tools'
|
||||
|
||||
@@ -38,6 +42,18 @@ export const RAW_OUTPUT_MAX_BYTES = 20_000_000
|
||||
*/
|
||||
export const SEARCH_TIMEOUT_MS = 30_000
|
||||
|
||||
/**
|
||||
* Default cap in bytes on the retained stderr tail of one search run — a
|
||||
* diagnostic excerpt only (the tool never reads `stderr.spillPath`).
|
||||
*/
|
||||
const SEARCH_STDERR_MAX_BYTES = 64 * 1024
|
||||
|
||||
/** Default whole-stream spill cap for search output (the subprocess seam requires an explicit budget). */
|
||||
const SEARCH_SPILL_MAX_BYTES = 64 * 1024 * 1024
|
||||
|
||||
/** Default terminate grace period for a search process (ms). */
|
||||
const SEARCH_GRACE_MS = 3_000
|
||||
|
||||
/**
|
||||
* Default cap in bytes on one search's serialized `presentationMeta` (the
|
||||
* `searchMetaMaxBytes` config). The inline match/path caps already bound the item
|
||||
@@ -52,14 +68,14 @@ export const SEARCH_META_MAX_BYTES = 65_536
|
||||
|
||||
/**
|
||||
* Stable, machine-routable codes for search failures. Package-owned (not
|
||||
* `FsErrorCode`) because these tools are bash-backed discovery, not `ctx.fs`
|
||||
* `FsErrorCode`) because these tools are spawn-backed discovery, not `ctx.fs`
|
||||
* provider operations: `SEARCH_INVALID_PATTERN` — ripgrep rejected the regex or
|
||||
* glob; `SEARCH_FAILED` — the search could not run or its output could not be
|
||||
* parsed (missing `rg`, inaccessible target, signal kill, malformed `--json`);
|
||||
* `SEARCH_RAW_OUTPUT_OVERFLOW` — raw `rg` output exceeded `rawOutputMaxBytes`
|
||||
* or stayed truncated after that requested stdout budget; `SEARCH_ABORTED` — the tool
|
||||
* timeout, caller cancellation, or the bash executor's own timeout cut the
|
||||
* search short.
|
||||
* parsed (a failed `rg` launch, inaccessible target, signal kill, malformed
|
||||
* `--json`); `SEARCH_RAW_OUTPUT_OVERFLOW` — raw `rg` output exceeded
|
||||
* `rawOutputMaxBytes` or stayed truncated after that requested stdout budget;
|
||||
* `SEARCH_ABORTED` — the cooperative tool timeout or caller cancellation cut
|
||||
* the search short.
|
||||
*/
|
||||
export type SearchErrorCode =
|
||||
| 'SEARCH_INVALID_PATTERN'
|
||||
@@ -84,7 +100,7 @@ export class SearchError extends HarnessError {
|
||||
|
||||
/** The completed acquisition of one `rg` run: complete stdout plus the resolved workdir. */
|
||||
export interface RipgrepRun {
|
||||
/** Complete raw stdout retained by the bash executor within the requested cap. */
|
||||
/** Complete raw stdout retained by the subprocess seam within the requested cap. */
|
||||
stdout: string
|
||||
/** True when ripgrep exited 1: a successful search with zero results. */
|
||||
noMatches: boolean
|
||||
@@ -94,73 +110,71 @@ export interface RipgrepRun {
|
||||
|
||||
/**
|
||||
* The retained stderr tail as a diagnostic excerpt, with a truncation note when
|
||||
* the executor dropped bytes (the tool never reads `stderr.spillPath`).
|
||||
* the subprocess seam dropped bytes (the tool never reads `stderr.spillPath`).
|
||||
*/
|
||||
function stderrExcerpt(stderr: CollectedOutput): string {
|
||||
const text = stderr.text.trim()
|
||||
function stderrExcerpt(stderrText: string, truncated: boolean): string {
|
||||
const text = stderrText.trim()
|
||||
if (text.length === 0) return ''
|
||||
return stderr.truncated ? `${text} [stderr truncated]` : text
|
||||
return truncated ? `${text} [stderr truncated]` : text
|
||||
}
|
||||
|
||||
/** Classify a nonzero-exit `rg` run into the search error vocabulary (invalid pattern vs missing `rg` vs everything else). */
|
||||
function classifyRunFailure(toolName: string, result: BashRunResult): SearchError {
|
||||
const stderr = stderrExcerpt(result.stderr)
|
||||
function classifyRunFailure(toolName: string, exitCode: number, stderrText: string, stderrTruncated: boolean): SearchError {
|
||||
const stderr = stderrExcerpt(stderrText, stderrTruncated)
|
||||
if (/regex parse error|error parsing glob/i.test(stderr)) {
|
||||
return new SearchError(`${toolName} pattern rejected by ripgrep: ${stderr}`, 'SEARCH_INVALID_PATTERN')
|
||||
}
|
||||
if (result.exitCode === 127 || /command not found/i.test(stderr)) {
|
||||
return new SearchError(`${toolName} requires ripgrep (rg) on the bash executor's PATH${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
if (exitCode === 127 || /command not found/i.test(stderr)) {
|
||||
return new SearchError(`${toolName} requires ripgrep (rg) to launch${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
}
|
||||
return new SearchError(`${toolName} search failed (exit ${result.exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
return new SearchError(`${toolName} search failed (exit ${exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire the COMPLETE raw stdout of a finished run, enforcing
|
||||
* `rawOutputMaxBytes` on the in-memory transport. A truncated result means the
|
||||
* bash backend could not retain complete stdout within the requested budget, so
|
||||
* the tool fails clearly instead of parsing a silently-partial stream.
|
||||
* subprocess seam could not retain complete stdout within the requested
|
||||
* budget, so the tool fails clearly instead of parsing a silently-partial
|
||||
* stream.
|
||||
*/
|
||||
function completeStdout(toolName: string, result: BashRunResult, rawOutputMaxBytes: number): string {
|
||||
function completeStdout(toolName: string, stdout: SubprocessOutputRead, rawOutputMaxBytes: number): string {
|
||||
const narrow = 'narrow pattern, path, or include and retry'
|
||||
if (!result.stdout.truncated) {
|
||||
const inlineBytes = Buffer.byteLength(result.stdout.text, 'utf8')
|
||||
if (!stdout.lossy) {
|
||||
const inlineBytes = Buffer.byteLength(stdout.text, 'utf8')
|
||||
if (inlineBytes > rawOutputMaxBytes) {
|
||||
throw new SearchError(
|
||||
`${toolName} produced ${inlineBytes} bytes of raw output, over the ${rawOutputMaxBytes}-byte cap; ${narrow}`,
|
||||
'SEARCH_RAW_OUTPUT_OVERFLOW',
|
||||
)
|
||||
}
|
||||
return result.stdout.text
|
||||
return stdout.text
|
||||
}
|
||||
throw new SearchError(
|
||||
`${toolName} produced more raw output than the bash executor retained within the ${rawOutputMaxBytes}-byte cap; ${narrow}`,
|
||||
`${toolName} produced more raw output than the subprocess seam retained within the ${rawOutputMaxBytes}-byte cap; ${narrow}`,
|
||||
'SEARCH_RAW_OUTPUT_OVERFLOW',
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one fixed `rg` command through the bash seam and return its complete raw
|
||||
* stdout. The bash request workdir is the calling agent's session cwd
|
||||
* (`exec.agent.session.header.cwd`) when available — mirroring `dsh-tool-bash` /
|
||||
* `dsh-tool-fs` — else omitted so the implementation's `resolve()` applies its
|
||||
* configured default. `exec.signal` is forwarded so the cooperative tool
|
||||
* timeout (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation kill the
|
||||
* command; the bash backend's own timeout stays a second safety cap.
|
||||
* Run the packaged ripgrep binary with a plain argv vector and return its
|
||||
* complete raw stdout. The working directory is the calling agent's session
|
||||
* cwd (`exec.agent.session.header.cwd`) when available, else
|
||||
* `process.cwd()`. `exec.signal` is forwarded so the cooperative tool timeout
|
||||
* (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation terminate the
|
||||
* process tree.
|
||||
*
|
||||
* Exit semantics are tool-owned: exit 0 is success with results, exit 1 is
|
||||
* success with zero results (`noMatches`), anything else throws a
|
||||
* {@link SearchError} (abort/timeout → `SEARCH_ABORTED`, invalid pattern →
|
||||
* `SEARCH_INVALID_PATTERN`, the rest → `SEARCH_FAILED` /
|
||||
* `SEARCH_RAW_OUTPUT_OVERFLOW`). A `run()` REJECTION — the seam's
|
||||
* infrastructure failures (pre-aborted signal, unusable workdir, missing
|
||||
* shell) — is translated into the same taxonomy: a pre-aborted signal becomes
|
||||
* `SEARCH_ABORTED`, everything else `SEARCH_FAILED`, with the original as
|
||||
* `cause`.
|
||||
* `SEARCH_RAW_OUTPUT_OVERFLOW`). A spawn REJECTION — the seam's
|
||||
* infrastructure failures — is translated into `SEARCH_FAILED` with the
|
||||
* original as `cause`; a pre-aborted signal becomes `SEARCH_ABORTED`.
|
||||
*
|
||||
* @param ctx - the plugin context; execution uses its `bash` service.
|
||||
* @param ctx - the plugin context; execution uses its `subprocess` service.
|
||||
* @param exec - the tool-execution context; supplies the session cwd and the abort signal.
|
||||
* @param toolName - `glob` or `grep`, used in error messages.
|
||||
* @param command - the fully-quoted `rg` command string (every model value already through `singleQuote`).
|
||||
* @param argv - the ripgrep arguments (every model value an unquoted argv element; no shell layer exists).
|
||||
* @param rawOutputMaxBytes - cap on the complete raw stdout the tool will parse.
|
||||
* @returns the complete stdout, the zero-result flag, and the resolved workdir.
|
||||
*/
|
||||
@@ -168,54 +182,64 @@ export async function runRipgrep(
|
||||
ctx: Context,
|
||||
exec: ToolExecution,
|
||||
toolName: string,
|
||||
command: string,
|
||||
argv: readonly string[],
|
||||
rawOutputMaxBytes: number,
|
||||
): Promise<RipgrepRun> {
|
||||
const cwd = exec.agent?.session.header.cwd
|
||||
const spec = ctx.bash.resolve({
|
||||
command,
|
||||
stdoutMaxBytes: rawOutputMaxBytes,
|
||||
...cwd !== undefined ? { workdir: cwd } : {},
|
||||
signal: exec.signal,
|
||||
})
|
||||
let result: BashRunResult
|
||||
try {
|
||||
result = await ctx.bash.run(spec)
|
||||
} catch (error: unknown) {
|
||||
// The seam contract: run() REJECTS only for infrastructure failures — a
|
||||
// pre-aborted signal, an unusable workdir, a missing shell. Translate them
|
||||
// so these failures stay machine-routable under the SEARCH_* taxonomy.
|
||||
if (spec.signal?.aborted === true) {
|
||||
throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED', { cause: error })
|
||||
}
|
||||
throw new SearchError(`${toolName} could not start its search command (unusable working directory or missing shell)`, 'SEARCH_FAILED', { cause: error })
|
||||
}
|
||||
if (result.aborted) {
|
||||
if (exec.signal.aborted) {
|
||||
throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED')
|
||||
}
|
||||
if (result.timedOut) {
|
||||
throw new SearchError(`${toolName} timed out after ${result.timeoutMs}ms in the bash executor; narrow pattern, path, or include and retry`, 'SEARCH_ABORTED')
|
||||
const cwd = exec.agent?.session.header.cwd
|
||||
const workdir = cwd ?? process.cwd()
|
||||
const collect = (maxBytes: number): SubprocessCollect =>
|
||||
({ maxBytes, spill: { maxBytes: SEARCH_SPILL_MAX_BYTES } })
|
||||
const handle = ctx.subprocess.spawn({
|
||||
argv: [rgPath, ...argv],
|
||||
cwd: workdir,
|
||||
stdio: {
|
||||
stdin: 'ignore',
|
||||
stdout: collect(rawOutputMaxBytes),
|
||||
stderr: collect(SEARCH_STDERR_MAX_BYTES),
|
||||
},
|
||||
graceMs: SEARCH_GRACE_MS,
|
||||
signal: exec.signal,
|
||||
} satisfies SubprocessSpawnSpec)
|
||||
let outcome: SubprocessOutcome
|
||||
try {
|
||||
outcome = await handle.done
|
||||
} catch (error: unknown) {
|
||||
throw new SearchError(`${toolName} could not start its search command (ripgrep launch failed)`, 'SEARCH_FAILED', { cause: error })
|
||||
}
|
||||
if (result.signal !== null || result.exitCode === null) {
|
||||
throw new SearchError(`${toolName} search command was killed by signal ${result.signal ?? '(unknown)'}`, 'SEARCH_FAILED')
|
||||
const stdout = handle.collected.stdout?.readFrom(0)
|
||||
const stderr = handle.collected.stderr?.readFrom(0)
|
||||
if (stdout === undefined || stderr === undefined) {
|
||||
throw new SearchError(`${toolName} search command produced no collected output streams`, 'SEARCH_FAILED')
|
||||
}
|
||||
if (result.exitCode !== 0 && result.exitCode !== 1) {
|
||||
throw classifyRunFailure(toolName, result)
|
||||
// The signal can abort while the spawn is awaited; the static narrowing that
|
||||
// proves this re-check "always false" cannot see AbortSignal state changes.
|
||||
// oxlint-disable-next-line typescript/no-unnecessary-condition
|
||||
if (exec.signal.aborted) {
|
||||
throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED')
|
||||
}
|
||||
const stdout = completeStdout(toolName, result, rawOutputMaxBytes)
|
||||
return { stdout, noMatches: result.exitCode === 1, workdir: spec.workdir }
|
||||
if (outcome.signal !== null || outcome.exitCode === null) {
|
||||
throw new SearchError(`${toolName} search command was killed by signal ${outcome.signal ?? '(unknown)'}`, 'SEARCH_FAILED')
|
||||
}
|
||||
if (outcome.exitCode !== 0 && outcome.exitCode !== 1) {
|
||||
throw classifyRunFailure(toolName, outcome.exitCode, stderr.text, stderr.lossy)
|
||||
}
|
||||
const text = completeStdout(toolName, stdout, rawOutputMaxBytes)
|
||||
return { stdout: text, noMatches: outcome.exitCode === 1, workdir }
|
||||
}
|
||||
|
||||
/**
|
||||
* Map an `rg` output path to its display form: absolute paths inside the
|
||||
* resolved bash workdir become workdir-relative; everything else (relative
|
||||
* output, paths outside the workdir) passes through unchanged. Display-only —
|
||||
* returned paths are follow-up-readable in co-located bash/filesystem
|
||||
* resolved workdir become workdir-relative; everything else (relative output,
|
||||
* paths outside the workdir) passes through unchanged. Display-only —
|
||||
* returned paths are follow-up-readable in co-located workdir/filesystem
|
||||
* deployments where both resolve the same workspace (the documented v1
|
||||
* deployment requirement).
|
||||
*
|
||||
* @param path - one path as ripgrep printed it.
|
||||
* @param workdir - the resolved bash workdir the command ran in.
|
||||
* @param workdir - the resolved workdir the command ran in.
|
||||
* @returns the workdir-relative display path when possible, else `path` unchanged.
|
||||
*/
|
||||
export function toWorkdirRelative(path: string, workdir: string): string {
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
/**
|
||||
* The one shell-quoting helper both search tools MUST route every
|
||||
* model-controlled value through before it enters an `rg` command string. The
|
||||
* bash seam (`ctx.bash`) accepts a command STRING, not an argv vector, so this
|
||||
* is the safety boundary that stops a `pattern`, `path`, or `include` from
|
||||
* breaking out of its argument and injecting shell syntax.
|
||||
*
|
||||
* Command builders in `glob.ts` / `grep.ts` must never hand-roll quoting or
|
||||
* concatenate an unquoted model value — they call {@link singleQuote}.
|
||||
* POSIX single-quoting helper retained for compatibility with older
|
||||
* deployments and tests. The current `glob`/`grep` command builders spawn the
|
||||
* packaged ripgrep binary with a plain argv vector — no shell layer exists —
|
||||
* so no quoting is involved; this module is kept because its export is part
|
||||
* of the package surface.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/shell-quote
|
||||
*/
|
||||
|
||||
@@ -1,15 +1,16 @@
|
||||
/**
|
||||
* Integration tests: the REAL local bash executor (`dsh-bash-local`) plus a
|
||||
* REAL ripgrep binary, exercised through `ctx.tools.execute()`. These verify
|
||||
* the WORLD — actual files on disk are discovered and grepped, hostile
|
||||
* patterns stay inert in a real shell, and real `rg` stderr classifies into
|
||||
* the `SEARCH_*` vocabulary. The whole suite self-skips when `rg` is not on
|
||||
* PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor
|
||||
* suite (tools.spec.ts) carries the coverage gate.
|
||||
* Integration tests: the REAL local subprocess service plus the PACKAGED
|
||||
* ripgrep binary (`@vscode/ripgrep`), exercised through `ctx.tools.execute()`.
|
||||
* These verify the WORLD — actual files on disk are discovered and grepped,
|
||||
* hostile patterns stay inert (they are plain argv elements; there is no
|
||||
* shell layer to escape), and real `rg` stderr classifies into the
|
||||
* `SEARCH_*` vocabulary. The binary ships inside the npm dependency, so the
|
||||
* suite runs on every platform without a system `rg` install; the
|
||||
* fake-service suite (tools.spec.ts) carries the coverage gate.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { mkdir, mkdtemp, rm, utimes, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -17,14 +18,11 @@ import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH } from '@deepseek-ai/dsh-tools'
|
||||
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
|
||||
import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local'
|
||||
import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
const testToolSignal = new AbortController().signal
|
||||
|
||||
const hasRg = spawnSync('rg', ['--version'], { encoding: 'utf8' }).status === 0
|
||||
|
||||
let dir: string
|
||||
let ctx: Context
|
||||
|
||||
@@ -43,7 +41,10 @@ function text(result: { content: { type: string; text?: string }[] }): string {
|
||||
return result.content.filter(b => b.type === 'text').map(b => b.text).join('')
|
||||
}
|
||||
|
||||
describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () => {
|
||||
/** The fixture workspace as a session cwd, so relative paths resolve inside `dir`. */
|
||||
const agent = () => ({ session: { header: { id: 'session-int', cwd: dir } } })
|
||||
|
||||
describe('search tools over the real subprocess service + the packaged rg', () => {
|
||||
beforeEach(async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-int-'))
|
||||
await mkdir(join(dir, 'src'), { recursive: true })
|
||||
@@ -54,7 +55,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
await writeFile(join(dir, 'notes.md'), 'alpha appears here too\n')
|
||||
await writeFile(join(dir, '.hidden.ts'), 'export const hidden = 3\n')
|
||||
await writeFile(join(dir, '.git', 'config.ts'), 'never listed\n')
|
||||
await writeFile(join(dir, 'spaced dir', "wei'rd \"name\".ts"), 'const inside = true\n')
|
||||
await writeFile(join(dir, 'spaced dir', "wei'rd name.ts"), 'const inside = true\n')
|
||||
// Deterministic --sort=modified order: alpha oldest, beta newest.
|
||||
await utimes(join(dir, 'src', 'alpha.ts'), new Date(2000, 0, 1), new Date(2000, 0, 1))
|
||||
await utimes(join(dir, 'src', 'beta.ts'), new Date(2020, 0, 1), new Date(2020, 0, 1))
|
||||
@@ -63,7 +64,6 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(LocalSubprocessService)
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: dir, timeoutMs: 20_000 })
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: true })
|
||||
})
|
||||
|
||||
@@ -73,33 +73,33 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
|
||||
describe('glob', () => {
|
||||
it('discovers files by pattern, sorted by modification time, hidden included, .git excluded', async () => {
|
||||
const result = await call('glob', { pattern: '**/*.ts' })
|
||||
const result = await call('glob', { pattern: '**/*.ts' }, agent())
|
||||
expect(result.isError).toBe(false)
|
||||
const paths = text(result).split('\n')
|
||||
expect(paths.indexOf('src/alpha.ts')).toBeLessThan(paths.indexOf('src/beta.ts'))
|
||||
expect(paths.indexOf(join('src', 'alpha.ts'))).toBeLessThan(paths.indexOf(join('src', 'beta.ts')))
|
||||
expect(paths).toContain('.hidden.ts')
|
||||
expect(paths).toContain("spaced dir/wei'rd \"name\".ts")
|
||||
expect(paths).not.toContain('.git/config.ts')
|
||||
expect(paths).toContain(join('spaced dir', "wei'rd name.ts"))
|
||||
expect(paths).not.toContain(join('.git', 'config.ts'))
|
||||
expect(paths).not.toContain('notes.md')
|
||||
})
|
||||
|
||||
it('scopes to a directory search root (path arg)', async () => {
|
||||
const result = await call('glob', { pattern: '*.ts', path: 'src' })
|
||||
expect(text(result).split('\n').sort()).toEqual(['src/alpha.ts', 'src/beta.ts'])
|
||||
const result = await call('glob', { pattern: '*.ts', path: 'src' }, agent())
|
||||
expect(text(result).split('\n').sort()).toEqual([join('src', 'alpha.ts'), join('src', 'beta.ts')])
|
||||
})
|
||||
|
||||
it('reports zero discoveries as No files found', async () => {
|
||||
expect(text(await call('glob', { pattern: '*.nomatch' }))).toBe('No files found')
|
||||
expect(text(await call('glob', { pattern: '*.nomatch' }, agent()))).toBe('No files found')
|
||||
})
|
||||
|
||||
it('excludes VCS internals even when the search root IS the VCS directory', async () => {
|
||||
// The prune glob alone never matches root-prefixed paths when rg is
|
||||
// rooted at .git; the paired contents glob keeps the exclusion airtight.
|
||||
expect(text(await call('glob', { pattern: '*', path: '.git' }))).toBe('No files found')
|
||||
expect(text(await call('glob', { pattern: '*', path: '.git' }, agent()))).toBe('No files found')
|
||||
})
|
||||
|
||||
it('classifies an invalid glob as SEARCH_INVALID_PATTERN', async () => {
|
||||
const result = await call('glob', { pattern: '[' })
|
||||
const result = await call('glob', { pattern: '[' }, agent())
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_INVALID_PATTERN' } })
|
||||
})
|
||||
@@ -107,37 +107,42 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
|
||||
describe('grep', () => {
|
||||
it('greps a directory tree with grouped, line-numbered output', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha' })
|
||||
const result = await call('grep', { pattern: 'alpha' }, agent())
|
||||
expect(result.isError).toBe(false)
|
||||
const output = text(result)
|
||||
expect(output).toContain('Found 3 matches')
|
||||
expect(output).toContain('src/alpha.ts\nLine 1: export const alpha = 1\nLine 2: // TODO: refit alpha')
|
||||
expect(output).toContain(`${join('src', 'alpha.ts')}\nLine 1: export const alpha = 1\nLine 2: // TODO: refit alpha`)
|
||||
expect(output).toContain('notes.md\nLine 1: alpha appears here too')
|
||||
})
|
||||
|
||||
it('greps a single FILE target', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha', path: 'notes.md' })
|
||||
const result = await call('grep', { pattern: 'alpha', path: 'notes.md' }, agent())
|
||||
expect(text(result)).toBe('Found 1 match\n\nnotes.md\nLine 1: alpha appears here too')
|
||||
})
|
||||
|
||||
it('greps a directory target with an include filter', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha', path: '.', include: '*.ts' })
|
||||
const result = await call('grep', { pattern: 'alpha', path: '.', include: '*.ts' }, agent())
|
||||
const output = text(result)
|
||||
expect(output).toContain('alpha.ts')
|
||||
expect(output).not.toContain('notes.md')
|
||||
})
|
||||
|
||||
it('a hostile pattern stays inert (no command substitution, the world untouched)', async () => {
|
||||
it('a hostile pattern stays inert (a plain argv element, the world untouched)', async () => {
|
||||
// There is no shell layer between the argv vector and rg, so the pattern
|
||||
// is a literal regex — but the world-untouched guarantee is the shipped
|
||||
// contract, and a future shell-wrapping change must not reintroduce it.
|
||||
// The canary name carries no path so the regex stays valid on every
|
||||
// platform (a Windows path's backslashes would be regex escapes).
|
||||
const canary = join(dir, 'pwned')
|
||||
const result = await call('grep', { pattern: `$(touch ${canary})` })
|
||||
const result = await call('grep', { pattern: '$(touch pwned)' }, agent())
|
||||
expect(result.isError).toBe(false) // exit 1: found nothing, executed nothing
|
||||
expect(text(result)).toBe('No matches found')
|
||||
expect(spawnSync('test', ['-e', canary]).status).not.toBe(0)
|
||||
expect(existsSync(canary)).toBe(false)
|
||||
})
|
||||
|
||||
it('a leading-dash pattern is a pattern, not a flag', async () => {
|
||||
await writeFile(join(dir, 'dashes.txt'), 'value --flag value\n')
|
||||
const result = await call('grep', { pattern: '--flag', path: 'dashes.txt' })
|
||||
const result = await call('grep', { pattern: '--flag', path: 'dashes.txt' }, agent())
|
||||
expect(text(result)).toBe('Found 1 match\n\ndashes.txt\nLine 1: value --flag value')
|
||||
})
|
||||
|
||||
@@ -155,7 +160,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
})
|
||||
|
||||
describe('per-session cwd', () => {
|
||||
it('resolves the search in the SESSION workspace, not the executor config cwd', async () => {
|
||||
it('resolves the search in the SESSION workspace, not the process cwd', async () => {
|
||||
const sessionDir = await mkdtemp(join(tmpdir(), 'dsh-search-session-'))
|
||||
try {
|
||||
await writeFile(join(sessionDir, 'only-here.ts'), 'const sessionFile = true\n')
|
||||
@@ -170,7 +175,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
})
|
||||
})
|
||||
|
||||
describe('pre-dispatch cancellation and bash-start failures', () => {
|
||||
describe('pre-dispatch cancellation and spawn failures', () => {
|
||||
it('a pre-aborted registry call is ABORTED_BEFORE_DISPATCH', async () => {
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
|
||||
@@ -3,14 +3,15 @@
|
||||
* a NAMESPACE plugin with `inject` — so a stray `export default apply` would
|
||||
* make the cordis Loader's `unwrapExports` (`exports.default ?? exports`)
|
||||
* collapse the module to the bare `apply` function, DROPPING `inject`. The
|
||||
* plugin would then read `ctx.bash` without having injected it and throw
|
||||
* plugin would then read `ctx.subprocess` without having injected it and throw
|
||||
* `cannot get property … without inject` the moment it loads (postmortem 0001).
|
||||
*
|
||||
* A hand-built `ctx.plugin({ apply, inject })` mount CANNOT catch that — it
|
||||
* bypasses `unwrapExports`. So this test unwraps the module through the REAL
|
||||
* `Loader.prototype.unwrapExports` and mounts the result over a bash executor,
|
||||
* exercising the exact path the Loader uses. Prove the guard bites: add
|
||||
* `export default apply` to `src/index.ts`, watch this go red, revert.
|
||||
* `Loader.prototype.unwrapExports` and mounts the result over the real local
|
||||
* subprocess service, exercising the exact path the Loader uses. Prove the
|
||||
* guard bites: add `export default apply` to `src/index.ts`, watch this go
|
||||
* red, revert.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
@@ -18,48 +19,9 @@ import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import { BashExecutor } from '@deepseek-ai/dsh-bash'
|
||||
import type { BashExecRequest, BashExecSpec, BashProcess, BashRunResult } from '@deepseek-ai/dsh-bash'
|
||||
import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local'
|
||||
import * as toolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
const RG_PROBE_COMMAND = 'command -v rg >/dev/null 2>&1'
|
||||
|
||||
/**
|
||||
* Deterministic bash service for this Loader guard: the test wants to exercise
|
||||
* the real unwrap/inject path, not depend on whether the host image has rg.
|
||||
*/
|
||||
class ProbeSuccessBashExecutor extends BashExecutor {
|
||||
override resolve(request: BashExecRequest): BashExecSpec {
|
||||
return {
|
||||
command: request.command,
|
||||
workdir: request.workdir ?? '/work',
|
||||
timeoutMs: request.timeoutMs ?? 60_000,
|
||||
stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000,
|
||||
signal: request.signal,
|
||||
sandboxPolicy: request.sandboxPolicy,
|
||||
}
|
||||
}
|
||||
|
||||
override run(spec: BashExecSpec): Promise<BashRunResult> {
|
||||
if (spec.command !== RG_PROBE_COMMAND) {
|
||||
throw new Error(`unexpected command in load-path guard: ${spec.command}`)
|
||||
}
|
||||
return Promise.resolve({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
timeoutMs: spec.timeoutMs,
|
||||
stdout: { text: '', truncated: false },
|
||||
stderr: { text: '', truncated: false },
|
||||
})
|
||||
}
|
||||
|
||||
override start(): BashProcess {
|
||||
throw new Error('load-path guard must not start background processes')
|
||||
}
|
||||
}
|
||||
|
||||
describe('dsh-tool-fs-search real-load-path guard', () => {
|
||||
it('has no default export and keeps name/inject/Config through unwrapExports', () => {
|
||||
expect('default' in toolFsSearch).toBe(false)
|
||||
@@ -68,16 +30,16 @@ describe('dsh-tool-fs-search real-load-path guard', () => {
|
||||
const unwrapped = loader.unwrapExports(toolFsSearch) as Record<string, unknown>
|
||||
expect(unwrapped).toBe(toolFsSearch)
|
||||
expect(unwrapped.name).toBe('tool-fs-search')
|
||||
expect(unwrapped.inject).toEqual(['tools', 'systemPrompt', 'bash'])
|
||||
expect(unwrapped.inject).toEqual(['tools', 'systemPrompt', 'subprocess'])
|
||||
expect(typeof unwrapped.Config).toBe('function')
|
||||
expect(typeof unwrapped.apply).toBe('function')
|
||||
})
|
||||
|
||||
it('boots over ctx.bash through the unwrapped module without an inject error', async () => {
|
||||
it('boots over ctx.subprocess through the unwrapped module without an inject error', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(ProbeSuccessBashExecutor)
|
||||
await ctx.plugin(LocalSubprocessService)
|
||||
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolFsSearch) as Parameters<Context['plugin']>[0]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user