diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml index df6948209a..3647587fa5 100644 --- a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md -2026-08-01-packaged-ripgrep-search.md: 0470623163ff7686bdd77ca715bac68c9a466d2b -2026-08-01-packaged-ripgrep-search.zh.md: fe53643aff139615d49edd2e3b27d14428cd2029 +2026-08-01-packaged-ripgrep-search.md: 849cc0804a2081492297649ac8f13236e2ac60fe +2026-08-01-packaged-ripgrep-search.zh.md: 4381c3a8bca8e6bb9ab375ad352fec5a5ddb9f99 diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md index 0470623163..849cc0804a 100644 --- a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.md @@ -12,7 +12,7 @@ The `glob`/`grep` tools ran through the bash executor seam, which made a system ## Decision -`@deepseek-ai/dsh-tool-fs-search` now runs the PACKAGED ripgrep binary (`@vscode/ripgrep`, an npm dependency whose optional platform packages ship the binary) through the `ctx.subprocess` seam: `runRipgrep()` spawns `rgPath` with a plain argv vector, collect-mode stdout/stderr, `graceMs`, and `exec.signal` forwarded. There is no shell layer, so the shell-quoting boundary is gone from execution; `singleQuote` stays exported as a compatibility surface with its tests. Registration is unconditional — the load-time `command -v rg` probe and the conditional registration decision are deleted, and with them the "rg not found" warning. The package injects `tools`, `systemPrompt`, and `subprocess`. +`@deepseek-ai/dsh-tool-fs-search` now runs the PACKAGED ripgrep binary (`@vscode/ripgrep`, an npm dependency whose optional platform packages ship the binary) through the `ctx.subprocess` seam: `runRipgrep()` spawns `rgPath` with a plain argv vector prefixed by `--no-config`, collect-mode stdout/stderr, `graceMs`, and `exec.signal` forwarded. There is no shell layer, so the shell-quoting boundary is gone from execution; the `singleQuote` helper and its shell-spawning tests are deleted with it. The raw streams request the seam's diagnostic-tail collect shape (no spill files — the tool never reads a raw spill path; a lossy stdout read fails as `SEARCH_RAW_OUTPUT_OVERFLOW`). The terminate grace and the stderr tail budget are validated `Config` fields (`graceMs` default 3000, `stderrMaxBytes` default 64 KiB), no longer inherited from bash-local's config. Registration is unconditional — the load-time `command -v rg` probe and the conditional registration decision are deleted, and with them the "rg not found" warning. The package injects `tools`, `systemPrompt`, and `subprocess`. Exit semantics stay tool-owned: exit 0 is success with results, exit 1 is a successful empty search, anything else classifies into the existing `SEARCH_*` vocabulary (invalid pattern, launch failure, signal kill, raw-output overflow). Timeout is the cooperative tool-call budget attached to the tool definitions: `@deepseek-ai/dsh-timeout-policy` aborts `exec.signal`, the subprocess seam's terminate escalation provides the hard kill, and the tool reports `SEARCH_ABORTED`. The working directory is the session header cwd when present, else `process.cwd()` — there is no executor config to default through anymore, so the tool owns the fallback. @@ -30,6 +30,8 @@ The `fs-glob-sampling` ACP snapshot scenario now executes the real packaged bina - The discovery tools work on every platform the packaged binary covers (darwin/linux/win32, x64/arm64) with no host install; the shipped TUI/Web rosters gain `glob`/`grep` as fixed members ([even-out-shipped-tool-rosters](../feature/2026-07-31-even-out-shipped-tool-rosters.md)). - The shell-string attack surface is gone: hostile patterns are inert argv elements, pinned by the integration suite, which now runs on Windows too (it previously self-skipped without a system `rg`). +- The spawn is unconfined (a plain `ctx.subprocess` call), so `--no-config` is prepended: a host `RIPGREP_CONFIG_PATH` (or an `rg.conf` beside the binary) can otherwise inject a `--pre` preprocessor that executes an arbitrary command for every matched file. With `--no-config`, no config file — and therefore no preprocessor — can reach the search. +- The raw-output overflow path changed shape: the old bash-backed route inherited bash-local's always-on spill and could leave an unread multi-megabyte temp file; the subprocess seam now collects without spill, and overflow is a pure error (`SEARCH_RAW_OUTPUT_OVERFLOW`, "narrow pattern, path, or include and retry") with zero content returned. - Load-time failure modes changed: a broken subprocess seam now fails the first search call (`SEARCH_FAILED`) instead of failing plugin load through the probe; a missing binary is a launch failure with the packaged path, not a PATH problem. - The integration suite's fixture dropped a filename Windows cannot represent (`"` in a name), keeping the suite replayable on every platform. - Regenerating `THIRD_PARTY_NOTICES.md` surfaced a latent generator bug the new dependency made visible: Node's `fs.globSync` returns OS-native separators, so on Windows the `/`-suffixed dev-area prefixes in the notices tiering never matched and dev-only packages (test tooling, support leaves) were mis-tiered as runtime. The generator now normalizes manifest paths at ingestion, and the notices are platform-independent. diff --git a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md index fe53643aff..4381c3a8bc 100644 --- a/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-01-packaged-ripgrep-search.zh.md @@ -12,7 +12,7 @@ Status: implemented ## 决策 -`@deepseek-ai/dsh-tool-fs-search` 现在运行 PACKAGED(打包的)ripgrep 二进制(`@vscode/ripgrep`,一个 npm 依赖,其可选平台包随附二进制),经由 `ctx.subprocess` seam:`runRipgrep()` 以纯 argv 向量 spawn `rgPath`,配以 collect 模式 stdout/stderr、`graceMs` 与转发的 `exec.signal`。不再有 shell 层,执行路径上的 shell 引号边界随之消失;`singleQuote` 作为兼容导出与其测试保留。注册变为无条件——加载期 `command -v rg` 探针与条件注册决策被删除,连同那条 "rg not found" 警告。本包注入 `tools`、`systemPrompt` 与 `subprocess`。 +`@deepseek-ai/dsh-tool-fs-search` 现在运行 PACKAGED(打包的)ripgrep 二进制(`@vscode/ripgrep`,一个 npm 依赖,其可选平台包随附二进制),经由 `ctx.subprocess` seam:`runRipgrep()` 以纯 argv 向量 spawn `rgPath`,向量前缀 `--no-config`,配以 collect 模式 stdout/stderr、`graceMs` 与转发的 `exec.signal`。不再有 shell 层,执行路径上的 shell 引号边界随之消失;`singleQuote` 工具与其 shell spawn 测试一并删除。原始流使用 seam 的诊断尾部 collect 形态(无 spill 文件——工具从不读取原始 spill 路径;lossy stdout 读取以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败)。终止宽限与 stderr 尾部预算成为经校验的 `Config` 字段(`graceMs` 默认 3000,`stderrMaxBytes` 默认 64 KiB),不再继承自 bash-local 的配置。注册变为无条件——加载期 `command -v rg` 探针与条件注册决策被删除,连同那条 "rg not found" 警告。本包注入 `tools`、`systemPrompt` 与 `subprocess`。 退出语义仍由工具拥有:退出码 0 为有结果的成功,1 为成功的空搜索,其余归入既有 `SEARCH_*` 词汇(无效模式、启动失败、信号杀死、原始输出溢出)。超时是挂在工具定义上的协作式工具调用预算:`@deepseek-ai/dsh-timeout-policy` 中止 `exec.signal`,subprocess seam 的终止升级提供硬终止,工具报告 `SEARCH_ABORTED`。工作目录为会话 header cwd(存在时),否则为 `process.cwd()`——不再有执行器配置可供默认化,因此回退由工具自己拥有。 @@ -30,6 +30,8 @@ Status: implemented - 发现工具在打包二进制覆盖的每个平台(darwin/linux/win32,x64/arm64)上开箱即用,无需宿主安装;交付的 TUI/Web 工具清单把 `glob`/`grep` 变为固定成员(见 [拉平交付的工具清单](../feature/2026-07-31-even-out-shipped-tool-rosters.md))。 - shell 字符串攻击面消失:恶意模式只是惰性 argv 元素,由集成套件钉住;该套件现在也在 Windows 上运行(此前没有系统 `rg` 时它自行跳过)。 +- spawn 不受沙箱约束(普通的 `ctx.subprocess` 调用),因此前缀 `--no-config`:宿主的 `RIPGREP_CONFIG_PATH`(或二进制旁的 `rg.conf`)否则可注入 `--pre` 预处理器,对每个匹配文件执行任意命令。加上 `--no-config` 后,任何配置文件——因而任何预处理器——都无法触及搜索。 +- 原始输出溢出路径的形态改变:旧的 bash 承载路径继承了 bash-local 常开的 spill,可能留下没人读的多 MB 临时文件;subprocess seam 现在无 spill 收集,溢出是纯粹的错误(`SEARCH_RAW_OUTPUT_OVERFLOW`,"narrow pattern, path, or include and retry"),不返回任何内容。 - 加载期失败模式改变:subprocess seam 损坏现在让首次搜索调用失败(`SEARCH_FAILED`),而非通过探针使插件加载失败;二进制缺失是带打包路径的启动失败,而不是 PATH 问题。 - 集成套件的 fixture 去掉了 Windows 无法表示的文件名(名称含 `"`),保证套件在每个平台都能重放。 - 重新生成 `THIRD_PARTY_NOTICES.md` 暴露了一个由新依赖带出的潜在生成器 bug:Node 的 `fs.globSync` 返回操作系统原生分隔符,因此在 Windows 上 notices 分层中带 `/` 后缀的 dev 区前缀永远匹配不上,dev-only 包(测试工具、support 叶子)被错分为 runtime。生成器现在在入口处归一化清单路径,notices 与平台无关。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 20f835bb7f..1151f0ff0d 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1733,12 +1733,16 @@ export interface Config { searchMetaMaxBytes?: number /** Max complete raw `rg` stdout bytes a search will parse; larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. */ rawOutputMaxBytes?: number + /** Terminate-escalation grace period (ms) for one search process, handed to the subprocess seam. */ + graceMs?: number + /** Max bytes retained for one search's stderr diagnostic tail (never surfaced to the model). */ + stderrMaxBytes?: number /** Cooperative tool-call timeout budget (ms) on both tools, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`. */ timeoutMs?: number } ``` -Source: [`packages/fs/tool-fs-search/src/index.ts:70`](../packages/fs/tool-fs-search/src/index.ts) +Source: [`packages/fs/tool-fs-search/src/index.ts:71`](../packages/fs/tool-fs-search/src/index.ts) ## `@deepseek-ai/dsh-tool-goal` diff --git a/packages/fs/tool-fs-search/README.i18n.yaml b/packages/fs/tool-fs-search/README.i18n.yaml index 5ed7bfb8bb..a8e2222998 100644 --- a/packages/fs/tool-fs-search/README.i18n.yaml +++ b/packages/fs/tool-fs-search/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/tool-fs-search/README.md -README.md: 0152be017ae15fc83a3d5cb7df927f25d04d2d53 -README.zh.md: 69ce49f3ad1621021dc1d0938cdc07a900d1cda8 +README.md: 78ffa069e56da5fc987913acf761eb5c6ae15b1a +README.zh.md: 42b123d5c47d8f48bc21b6f9bed4905372ca8625 diff --git a/packages/fs/tool-fs-search/README.md b/packages/fs/tool-fs-search/README.md index 0152be017a..78ffa069e5 100644 --- a/packages/fs/tool-fs-search/README.md +++ b/packages/fs/tool-fs-search/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **packaged ripgrep binary** (`@vscode/ripgrep`), not by `ctx.fs` provider methods and not by a system `rg` install. Registration is unconditional: the binary ships inside the npm dependency, so there is no load-time availability probe. Each call spawns the binary through the `ctx.subprocess` seam with a fixed argv vector (model-controlled values are plain argv elements — no shell layer exists, so no quoting applies), parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `subprocess`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. +The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **packaged ripgrep binary** (`@vscode/ripgrep`), not by `ctx.fs` provider methods and not by a system `rg` install. Registration is unconditional: the binary ships inside the npm dependency, so there is no load-time availability probe. Each call spawns the binary through the `ctx.subprocess` seam with a fixed argv vector (`--no-config` prepended so a host `RIPGREP_CONFIG_PATH` cannot inject a `--pre` preprocessor into the unconfined spawn; model-controlled values are plain argv elements — no shell layer exists, so no quoting applies), parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `subprocess`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional. ```ts ignore-check // A deployment chooses how over-cap glob pages are selected. @@ -30,6 +30,8 @@ The binary ships with the package on every supported platform (macOS/Linux/Windo | `grepMaxLineBytes` | `2000` | Byte cap per matched-line preview; the cut preserves UTF-8 boundaries and is marked `(line truncated)`. | | `rawOutputMaxBytes` | `20000000` | Max complete raw `rg` stdout a search will parse (matches Claude Code's ripgrep raw buffer); larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. | | `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the subprocess seam's terminate escalation is the hard kill. | +| `graceMs` | `3000` | Terminate-escalation grace period the subprocess seam grants past `timeoutMs` before the search fails as `SEARCH_ABORTED`. | +| `stderrMaxBytes` | `65536` | Diagnostic-tail budget for `rg` stderr, captured through the subprocess seam's collect disposition; a lossy read keeps only the tail (marked `[stderr truncated]`). | ## Tools @@ -42,7 +44,7 @@ Routine budgets stay out of the model-facing schema (no `head_limit`/`offset`/`c ## Two budgets, two artifacts -Raw `rg` stdout is an internal transport detail. Each search requests a collect-mode stdout budget of `rawOutputMaxBytes` from the subprocess seam and parses only complete retained stdout; if the seam still reports a lossy read, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`. +Raw `rg` stdout and stderr are internal transport details. Each search requests collect-mode budgets from the subprocess seam — complete stdout within `rawOutputMaxBytes` and a `stderrMaxBytes` diagnostic tail — with no spill files on either stream (the tool never reads a raw spill path). If the seam still reports a lossy stdout read, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query; a lossy stderr read only marks the diagnostic excerpt `[stderr truncated]`. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`. ## Errors diff --git a/packages/fs/tool-fs-search/README.zh.md b/packages/fs/tool-fs-search/README.zh.md index 69ce49f3ad..42b123d5c4 100644 --- a/packages/fs/tool-fs-search/README.zh.md +++ b/packages/fs/tool-fs-search/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -**面向模型的文件系统发现工具**(`glob`、`grep`)由 **打包的 ripgrep 二进制**(`@vscode/ripgrep`)支持,而不是由 `ctx.fs` 提供方方法或系统 `rg` 安装支持。注册是无条件的:二进制随 npm 依赖一起交付,因此没有加载期可用性探针。每次调用都通过 `ctx.subprocess` seam 以固定 argv 向量 spawn 该二进制(模型控制的值是普通 argv 元素——不存在 shell 层,因此无需引号),解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `subprocess`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。 +**面向模型的文件系统发现工具**(`glob`、`grep`)由 **打包的 ripgrep 二进制**(`@vscode/ripgrep`)支持,而不是由 `ctx.fs` 提供方方法或系统 `rg` 安装支持。注册是无条件的:二进制随 npm 依赖一起交付,因此没有加载期可用性探针。每次调用都通过 `ctx.subprocess` seam 以固定 argv 向量 spawn 该二进制(前缀 `--no-config`,使宿主的 `RIPGREP_CONFIG_PATH` 无法向不受约束的 spawn 注入 `--pre` 预处理器;模型控制的值是普通 argv 元素——不存在 shell 层,因此无需引号),解析原始 `rg` 输出,并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `subprocess`,有意**不**注入 `fs`;格式化结果 spill 为可选功能,因此机会性读取 `ctx.spillStore`,调用方式为 `ctx.get()`。 ```ts ignore-check // A deployment chooses how over-cap glob pages are selected. @@ -30,6 +30,8 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh- | `grepMaxLineBytes` | `2000` | 每条匹配行预览的字节上限;截断会保留 UTF-8 边界,并标记为 `(line truncated)`。 | | `rawOutputMaxBytes` | `20000000` | 搜索将解析的完整原始 `rg` stdout 上限(与 Claude Code 的 ripgrep 原始 buffer 相同);更大的原始输出以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败。 | | `timeoutMs` | `30000` | 附加到两个工具定义上的协作式工具调用预算,由 `@deepseek-ai/dsh-timeout-policy` 通过 `exec.signal` 强制执行;subprocess seam 的终止升级提供硬终止。 | +| `graceMs` | `3000` | subprocess seam 在 `timeoutMs` 之外授予的终止升级宽限期;超过后搜索以 `SEARCH_ABORTED` 失败。 | +| `stderrMaxBytes` | `65536` | `rg` stderr 的诊断尾部预算,经 subprocess seam 的 collect 形态捕获;lossy 读取只保留尾部(标记 `[stderr truncated]`)。 | ## 工具 @@ -42,7 +44,7 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh- ## 两类预算、两类产物 -原始 `rg` stdout 是内部传输细节。每次搜索从 subprocess seam 请求 `rawOutputMaxBytes` 的 collect 模式 stdout 预算,且只解析完整保留的 stdout;如果 seam 仍报告 lossy 读取,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。 +原始 `rg` stdout 与 stderr 是内部传输细节。每次搜索从 subprocess seam 请求 collect 模式预算——`rawOutputMaxBytes` 内的完整 stdout 与 `stderrMaxBytes` 的诊断尾部——两条流都不产生 spill 文件(工具从不读取原始 spill 路径)。如果 seam 仍报告 lossy stdout 读取,搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败,并要求模型缩小查询;lossy stderr 读取只把诊断摘录标记为 `[stderr truncated]`。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径;启用采样时,借助 `root`,原生渲染器能以显式的相对或绝对搜索路径为根,按该根下的条目分组,而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`,并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时,后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览,并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill,因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面,并报告完整结果无法保存,绝不会成为 `isError`。 ## 错误 diff --git a/packages/fs/tool-fs-search/src/glob.ts b/packages/fs/tool-fs-search/src/glob.ts index 6e7d411a01..3eeea9c75c 100644 --- a/packages/fs/tool-fs-search/src/glob.ts +++ b/packages/fs/tool-fs-search/src/glob.ts @@ -47,6 +47,10 @@ export interface GlobToolCaps { maxMetaBytes: number /** Cap on the complete raw `rg` stdout the tool will parse. */ rawOutputMaxBytes: number + /** Terminate-escalation grace period (ms) for the search process. */ + graceMs: number + /** Cap on the retained stderr diagnostic tail. */ + stderrMaxBytes: number /** Cooperative tool-call budget (ms) attached as `ToolDefinition.timeoutMs`. */ timeoutMs: number } @@ -337,7 +341,7 @@ export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void { }, async execute(args, exec) { const input = parseGlobArgs(args) - const run = await runRipgrep(ctx, exec, 'glob', buildGlobCommand(input), caps.rawOutputMaxBytes) + const run = await runRipgrep(ctx, exec, 'glob', buildGlobCommand(input), caps.rawOutputMaxBytes, caps.graceMs, caps.stderrMaxBytes) const root = input.path === undefined ? '.' : toWorkdirRelative(input.path, run.workdir) if (run.noMatches) return { root, paths: [] } diff --git a/packages/fs/tool-fs-search/src/grep.ts b/packages/fs/tool-fs-search/src/grep.ts index 03b49f01e8..49548499ac 100644 --- a/packages/fs/tool-fs-search/src/grep.ts +++ b/packages/fs/tool-fs-search/src/grep.ts @@ -45,6 +45,10 @@ export interface GrepToolCaps { maxMetaBytes: number /** Cap on the complete raw `rg` stdout the tool will parse. */ rawOutputMaxBytes: number + /** Terminate-escalation grace period (ms) for the search process. */ + graceMs: number + /** Cap on the retained stderr diagnostic tail. */ + stderrMaxBytes: number /** Cooperative tool-call budget (ms) attached as `ToolDefinition.timeoutMs`. */ timeoutMs: number } @@ -265,7 +269,7 @@ export function presentGrepResult( * Register the `grep` tool and its system-prompt guidance. * * @param ctx - the plugin context; registrations are effects scoped to it, and - * execution uses its `bash` service. + * execution uses its `subprocess` service. * @param caps - the deployment's resolved grep caps (plugin config after defaulting). */ export function applyGrepTool(ctx: Context, caps: GrepToolCaps): void { @@ -315,7 +319,7 @@ export function applyGrepTool(ctx: Context, caps: GrepToolCaps): void { }, async execute(args, exec) { const input = parseGrepArgs(args) - const run = await runRipgrep(ctx, exec, 'grep', buildGrepCommand(input), caps.rawOutputMaxBytes) + const run = await runRipgrep(ctx, exec, 'grep', buildGrepCommand(input), caps.rawOutputMaxBytes, caps.graceMs, caps.stderrMaxBytes) if (run.noMatches) return { matches: [] } const all: GrepMatch[] = [] diff --git a/packages/fs/tool-fs-search/src/index.ts b/packages/fs/tool-fs-search/src/index.ts index e596ae3ca1..9a4042dde1 100644 --- a/packages/fs/tool-fs-search/src/index.ts +++ b/packages/fs/tool-fs-search/src/index.ts @@ -30,7 +30,7 @@ import type { Context } from 'cordis' import z from 'schemastery' import { GLOB_MAX_RESULTS, applyGlobTool } from './glob.ts' import { GREP_MAX_LINE_BYTES, GREP_MAX_MATCHES, applyGrepTool } from './grep.ts' -import { RAW_OUTPUT_MAX_BYTES, SEARCH_META_MAX_BYTES, SEARCH_TIMEOUT_MS } from './search-core.ts' +import { RAW_OUTPUT_MAX_BYTES, SEARCH_GRACE_MS, SEARCH_META_MAX_BYTES, SEARCH_STDERR_MAX_BYTES, SEARCH_TIMEOUT_MS } from './search-core.ts' export { GLOB_MAX_RESULTS, GLOB_VCS_EXCLUDES, applyGlobTool, buildGlobCommand, formatGlobOutput, parseGlobArgs, presentGlobCall, presentGlobResult, sampleAcrossTopLevel } from './glob.ts' export type { GlobInput, GlobSample, GlobToolCaps } from './glob.ts' @@ -49,7 +49,9 @@ export { export type { GrepInput, GrepToolCaps } from './grep.ts' export { RAW_OUTPUT_MAX_BYTES, + SEARCH_GRACE_MS, SEARCH_META_MAX_BYTES, + SEARCH_STDERR_MAX_BYTES, SEARCH_TIMEOUT_MS, SearchError, previewLine, @@ -58,7 +60,6 @@ export { trySaveFormattedResult, } from './search-core.ts' export type { GrepMatch, RipgrepRun, SearchErrorCode } from './search-core.ts' -export { singleQuote } from './shell-quote.ts' /** Cordis plugin name used by loader diagnostics. */ export const name = 'tool-fs-search' @@ -80,6 +81,10 @@ export interface Config { searchMetaMaxBytes?: number /** Max complete raw `rg` stdout bytes a search will parse; larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. */ rawOutputMaxBytes?: number + /** Terminate-escalation grace period (ms) for one search process, handed to the subprocess seam. */ + graceMs?: number + /** Max bytes retained for one search's stderr diagnostic tail (never surfaced to the model). */ + stderrMaxBytes?: number /** Cooperative tool-call timeout budget (ms) on both tools, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`. */ timeoutMs?: number } @@ -91,6 +96,8 @@ export const Config: z = z.object({ grepMaxLineBytes: z.number().default(GREP_MAX_LINE_BYTES), searchMetaMaxBytes: z.number().default(SEARCH_META_MAX_BYTES), rawOutputMaxBytes: z.number().default(RAW_OUTPUT_MAX_BYTES), + graceMs: z.number().default(SEARCH_GRACE_MS), + stderrMaxBytes: z.number().default(SEARCH_STDERR_MAX_BYTES), timeoutMs: z.number().default(SEARCH_TIMEOUT_MS), }) @@ -121,12 +128,16 @@ export async function apply(ctx: Context, config: Config): Promise { assertPositiveInteger('grepMaxLineBytes', resolved.grepMaxLineBytes) assertPositiveInteger('searchMetaMaxBytes', resolved.searchMetaMaxBytes) assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes) + assertPositiveInteger('graceMs', resolved.graceMs) + assertPositiveInteger('stderrMaxBytes', resolved.stderrMaxBytes) assertPositiveInteger('timeoutMs', resolved.timeoutMs) applyGlobTool(ctx, { sampleOverCapGlobResults: resolved.sampleOverCapGlobResults, maxResults: resolved.globMaxResults, maxMetaBytes: resolved.searchMetaMaxBytes, rawOutputMaxBytes: resolved.rawOutputMaxBytes, + graceMs: resolved.graceMs, + stderrMaxBytes: resolved.stderrMaxBytes, timeoutMs: resolved.timeoutMs, }) applyGrepTool(ctx, { @@ -134,6 +145,8 @@ export async function apply(ctx: Context, config: Config): Promise { maxLineBytes: resolved.grepMaxLineBytes, maxMetaBytes: resolved.searchMetaMaxBytes, rawOutputMaxBytes: resolved.rawOutputMaxBytes, + graceMs: resolved.graceMs, + stderrMaxBytes: resolved.stderrMaxBytes, timeoutMs: resolved.timeoutMs, }) } diff --git a/packages/fs/tool-fs-search/src/search-core.ts b/packages/fs/tool-fs-search/src/search-core.ts index c9f5f80b5b..9444b029cc 100644 --- a/packages/fs/tool-fs-search/src/search-core.ts +++ b/packages/fs/tool-fs-search/src/search-core.ts @@ -44,15 +44,13 @@ export const SEARCH_TIMEOUT_MS = 30_000 /** * Default cap in bytes on the retained stderr tail of one search run — a - * diagnostic excerpt only (the tool never reads `stderr.spillPath`). + * diagnostic excerpt only (the tool never reads a stderr spill path, and the + * collect disposition requests none). */ -const SEARCH_STDERR_MAX_BYTES = 64 * 1024 - -/** Default whole-stream spill cap for search output (the subprocess seam requires an explicit budget). */ -const SEARCH_SPILL_MAX_BYTES = 64 * 1024 * 1024 +export const SEARCH_STDERR_MAX_BYTES = 64 * 1024 /** Default terminate grace period for a search process (ms). */ -const SEARCH_GRACE_MS = 3_000 +export const SEARCH_GRACE_MS = 3_000 /** * Default cap in bytes on one search's serialized `presentationMeta` (the @@ -110,7 +108,7 @@ export interface RipgrepRun { /** * The retained stderr tail as a diagnostic excerpt, with a truncation note when - * the subprocess seam dropped bytes (the tool never reads `stderr.spillPath`). + * the subprocess seam dropped bytes. */ function stderrExcerpt(stderrText: string, truncated: boolean): string { const text = stderrText.trim() @@ -118,15 +116,16 @@ function stderrExcerpt(stderrText: string, truncated: boolean): string { return truncated ? `${text} [stderr truncated]` : text } -/** Classify a nonzero-exit `rg` run into the search error vocabulary (invalid pattern vs missing `rg` vs everything else). */ +/** + * Classify a nonzero-exit `rg` run into the search error vocabulary. There is + * no shell layer, so an exit 127 or shell "command not found" text cannot + * occur — a launch failure rejects at spawn (see {@link runRipgrep}). + */ function classifyRunFailure(toolName: string, exitCode: number, stderrText: string, stderrTruncated: boolean): SearchError { const stderr = stderrExcerpt(stderrText, stderrTruncated) if (/regex parse error|error parsing glob/i.test(stderr)) { return new SearchError(`${toolName} pattern rejected by ripgrep: ${stderr}`, 'SEARCH_INVALID_PATTERN') } - if (exitCode === 127 || /command not found/i.test(stderr)) { - return new SearchError(`${toolName} requires ripgrep (rg) to launch${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') - } return new SearchError(`${toolName} search failed (exit ${exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED') } @@ -163,6 +162,13 @@ function completeStdout(toolName: string, stdout: SubprocessOutputRead, rawOutpu * (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation terminate the * process tree. * + * The spawn is unconfined (a plain `ctx.subprocess` call), so `--no-config` + * is prepended: a host `RIPGREP_CONFIG_PATH` (or `rg.conf` next to the + * binary) can otherwise inject `--pre` and make ripgrep execute an arbitrary + * preprocessor for every matched file. The collect dispositions are the + * seam's diagnostic-tail shape (no spill files): the tools never read a raw + * spill path, and truncated stdout fails as `SEARCH_RAW_OUTPUT_OVERFLOW`. + * * Exit semantics are tool-owned: exit 0 is success with results, exit 1 is * success with zero results (`noMatches`), anything else throws a * {@link SearchError} (abort/timeout → `SEARCH_ABORTED`, invalid pattern → @@ -176,6 +182,8 @@ function completeStdout(toolName: string, stdout: SubprocessOutputRead, rawOutpu * @param toolName - `glob` or `grep`, used in error messages. * @param argv - the ripgrep arguments (every model value an unquoted argv element; no shell layer exists). * @param rawOutputMaxBytes - cap on the complete raw stdout the tool will parse. + * @param graceMs - the seam's terminate-escalation grace period. + * @param stderrMaxBytes - cap on the retained stderr diagnostic tail. * @returns the complete stdout, the zero-result flag, and the resolved workdir. */ export async function runRipgrep( @@ -184,6 +192,8 @@ export async function runRipgrep( toolName: string, argv: readonly string[], rawOutputMaxBytes: number, + graceMs: number, + stderrMaxBytes: number, ): Promise { if (exec.signal.aborted) { throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED') @@ -191,16 +201,16 @@ export async function runRipgrep( const cwd = exec.agent?.session.header.cwd const workdir = cwd ?? process.cwd() const collect = (maxBytes: number): SubprocessCollect => - ({ maxBytes, spill: { maxBytes: SEARCH_SPILL_MAX_BYTES } }) + ({ maxBytes }) const handle = ctx.subprocess.spawn({ - argv: [rgPath, ...argv], + argv: [rgPath, '--no-config', ...argv], cwd: workdir, stdio: { stdin: 'ignore', stdout: collect(rawOutputMaxBytes), - stderr: collect(SEARCH_STDERR_MAX_BYTES), + stderr: collect(stderrMaxBytes), }, - graceMs: SEARCH_GRACE_MS, + graceMs, signal: exec.signal, } satisfies SubprocessSpawnSpec) let outcome: SubprocessOutcome diff --git a/packages/fs/tool-fs-search/src/shell-quote.ts b/packages/fs/tool-fs-search/src/shell-quote.ts deleted file mode 100644 index ea67abf449..0000000000 --- a/packages/fs/tool-fs-search/src/shell-quote.ts +++ /dev/null @@ -1,24 +0,0 @@ -/** - * POSIX single-quoting helper retained for compatibility with older - * deployments and tests. The current `glob`/`grep` command builders spawn the - * packaged ripgrep binary with a plain argv vector — no shell layer exists — - * so no quoting is involved; this module is kept because its export is part - * of the package surface. - * - * @module @deepseek-ai/dsh-tool-fs-search/shell-quote - */ - -/** - * POSIX single-quote a string for safe use as ONE shell word. Wraps the value - * in single quotes and rewrites every embedded single quote as `'\''` (close - * quote, an escaped literal quote, reopen quote). Inside single quotes the shell - * treats every other byte literally — spaces, newlines, `$`, backticks, `;`, - * `|`, `&`, glob metacharacters, and a leading `-` are all inert — so the result - * is a single, injection-safe argument regardless of the input. - * - * @param value - the raw, possibly model-controlled string to quote. - * @returns the value wrapped as one safe single-quoted shell word. - */ -export function singleQuote(value: string): string { - return `'${value.replaceAll("'", "'\\''")}'` -} diff --git a/packages/fs/tool-fs-search/tests/shell-quote.spec.ts b/packages/fs/tool-fs-search/tests/shell-quote.spec.ts deleted file mode 100644 index 84c8506be1..0000000000 --- a/packages/fs/tool-fs-search/tests/shell-quote.spec.ts +++ /dev/null @@ -1,59 +0,0 @@ -/** - * Unit tests for the shell-quoting safety boundary, plus a REAL round-trip: - * every adversarial value, quoted, must survive `bash -c "printf '%s' "` - * byte-for-byte — proving the quoting is inert in an actual shell, not just - * against a mental model of one. - */ - -import { describe, expect, it } from 'vitest' -import { spawnSync } from 'node:child_process' -import { singleQuote } from '@deepseek-ai/dsh-tool-fs-search' - -/** Adversarial values a model could pass as pattern / path / include. */ -const HOSTILE: readonly string[] = [ - 'plain', - 'with spaces', - "it's got 'quotes'", - '"double quoted"', - '$(rm -rf /tmp/nope)', - '`touch /tmp/nope`', - '$HOME and ${PATH}', - 'semi;colon && chain || pipe | bg &', - 'newline\nin the middle', - '-leading-dash', - '--leading-double-dash', - '*?[a-z]{x,y}', - '!bang', - '\\backslash\\', - '~tilde', - '# not a comment', - '>redirect &1', -] - -describe('singleQuote', () => { - it('wraps a plain value in single quotes', () => { - expect(singleQuote('abc')).toBe("'abc'") - }) - - it("rewrites embedded single quotes as '\\''", () => { - expect(singleQuote("a'b")).toBe("'a'\\''b'") - expect(singleQuote("''")).toBe("''\\'''\\'''") - }) - - it.each(HOSTILE.map(value => [JSON.stringify(value), value] as const))( - 'round-trips %s through a real bash -c unchanged', - (_label, value) => { - const result = spawnSync('bash', ['-c', `printf '%s' ${singleQuote(value)}`], { encoding: 'utf8' }) - expect(result.status).toBe(0) - expect(result.stdout).toBe(value) - }, - ) - - it('a quoted command substitution does not execute (the world stays untouched)', () => { - const canary = `/tmp/dsh-quote-canary-${process.pid}` - const result = spawnSync('bash', ['-c', `printf '%s' ${singleQuote(`$(touch ${canary})`)}`], { encoding: 'utf8' }) - expect(result.stdout).toBe(`$(touch ${canary})`) - // The canary file must NOT exist — the substitution stayed literal. - expect(spawnSync('test', ['-e', canary]).status).not.toBe(0) - }) -}) diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index d786fa3181..5c6713a7f0 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -291,6 +291,8 @@ describe('config validation', () => { ['grepMaxMatches', { grepMaxMatches: -1 }], ['grepMaxLineBytes', { grepMaxLineBytes: 1.5 }], ['rawOutputMaxBytes', { rawOutputMaxBytes: 0 }], + ['graceMs', { graceMs: 0 }], + ['stderrMaxBytes', { stderrMaxBytes: -1 }], ['timeoutMs', { timeoutMs: -100 }], ] as const)('rejects a non-positive or fractional %s at load', async (name, config) => { const ctx = new Context() @@ -372,16 +374,30 @@ describe('workdir derivation and signal forwarding', () => { expect(subprocess.spawns[1]?.cwd).toBe(process.cwd()) }) - it('spawns the packaged ripgrep binary with the fixed argv and budgeted collect streams', async () => { - const { ctx, subprocess } = await setup({ config: { rawOutputMaxBytes: 1234 } }) + it('spawns the packaged ripgrep binary with --no-config, the fixed argv, and budgeted collect streams', async () => { + const { ctx, subprocess } = await setup({ + config: { rawOutputMaxBytes: 1234, graceMs: 5000, stderrMaxBytes: 4096 }, + }) subprocess.handler = () => runResult('', { exitCode: 1 }) await call(ctx, 'grep', { pattern: 'needle' }) const spec = subprocess.spawns[0] - expect(spec?.argv[0]).toBe(rgPath) - expect(spec?.argv).toEqual([rgPath, '--json', '--regexp=needle']) + // --no-config keeps a host RIPGREP_CONFIG_PATH from injecting a + // preprocessor into this unconfined spawn. + expect(spec?.argv).toEqual([rgPath, '--no-config', '--json', '--regexp=needle']) expect(spec?.stdio.stdin).toBe('ignore') - // stdout gets the tool's parse budget; stderr is a diagnostic excerpt. + // stdout gets the tool's parse budget; stderr is a diagnostic excerpt; + // both are the seam's diagnostic-tail shape (no spill files requested). expect((spec?.stdio.stdout as { maxBytes: number }).maxBytes).toBe(1234) + expect((spec?.stdio.stderr as { maxBytes: number }).maxBytes).toBe(4096) + expect(spec?.graceMs).toBe(5_000) + }) + + it('defaults the stderr tail budget and grace period when the config omits them', async () => { + const { ctx, subprocess } = await setup() + subprocess.handler = () => runResult('', { exitCode: 1 }) + await call(ctx, 'grep', { pattern: 'needle' }) + const spec = subprocess.spawns[0] + expect((spec?.stdio.stderr as { maxBytes: number }).maxBytes).toBe(64 * 1024) expect(spec?.graceMs).toBe(3_000) }) @@ -430,7 +446,7 @@ describe('workdir derivation and signal forwarding', () => { const controller = new AbortController() controller.abort() const exec = { signal: controller.signal, name: 'glob', callId: CallId('direct-pre-abort') } as unknown as ToolExecution - await expect(runRipgrep(ctx, exec, 'glob', ['--files'], 1_000_000)).rejects + await expect(runRipgrep(ctx, exec, 'glob', ['--files'], 1_000_000, 3_000, 64 * 1024)).rejects .toMatchObject({ name: 'SearchError', code: 'SEARCH_ABORTED' }) }) @@ -489,20 +505,6 @@ describe('exit semantics and failure classification', () => { expect(result.error).toMatchObject({ info: { code: 'SEARCH_INVALID_PATTERN' } }) }) - it('a failed ripgrep launch classifies as SEARCH_FAILED naming ripgrep', async () => { - const { ctx, subprocess } = await setup() - subprocess.handler = () => runResult('', { exitCode: 127, stderr: { text: 'sh: rg: command not found' } }) - const result = await call(ctx, 'glob', { pattern: '*' }) - expect(result.error).toMatchObject({ info: { code: 'SEARCH_FAILED' } }) - expect(text(result)).toContain('requires ripgrep (rg)') - // The same classification holds from either evidence alone: the 127 exit - // with silent stderr, or a shell's command-not-found text on another exit. - subprocess.handler = () => runResult('', { exitCode: 127 }) - expect(text(await call(ctx, 'glob', { pattern: '*' }))).toContain('requires ripgrep (rg)') - subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'sh: rg: command not found' } }) - expect(text(await call(ctx, 'grep', { pattern: 'x' }))).toContain('requires ripgrep (rg)') - }) - it('other nonzero exits are SEARCH_FAILED carrying the stderr excerpt', async () => { const { ctx, subprocess } = await setup() subprocess.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: missing.dir: IO error: no such file or directory' } }) @@ -682,7 +684,7 @@ describe('glob results', () => { subprocess.handler = () => runResult('sub/a.ts\n') const result = await call(ctx, 'glob', { pattern: '*.ts', path: 'sub' }) expect(result.isError).toBe(false) - expect(subprocess.spawns[0]?.argv).toEqual([rgPath, '--files', '--glob=*.ts', '--sort=modified', '--no-ignore', '--hidden', + expect(subprocess.spawns[0]?.argv).toEqual([rgPath, '--no-config', '--files', '--glob=*.ts', '--sort=modified', '--no-ignore', '--hidden', '--glob=!**/.git', '--glob=!**/.git/**', '--glob=!**/.svn', '--glob=!**/.svn/**', '--glob=!**/.hg', '--glob=!**/.hg/**',