fix(fs-search): make glob sampling an explicit choice
This commit is contained in:
Tianyi Cui
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/tool-fs-search/README.md
|
||||
README.md: 51b3fa5385330cdaba0b36dd71a6efe4fd3d0db5
|
||||
README.zh.md: de9e50c90724cbc40f6de5880e7000ae0552b52d
|
||||
README.md: b12ffda9869c7d6bef5ea5b54594781ecf555ff4
|
||||
README.zh.md: 7dd6cdf9a209f2fe357b4ffe48d20d574266ce60
|
||||
|
||||
@@ -5,9 +5,9 @@ English | [中文](README.zh.md)
|
||||
The **model-facing filesystem discovery tools**—`glob`, `grep`—are backed by the **bash executor seam**, not by `ctx.fs` provider methods. At load, the package probes `command -v rg` through `ctx.bash`; if the executor cannot find ripgrep on its `PATH`, it logs a warning and registers no tools or prompt sections. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a workdir-relative canonical value. The package injects `tools`, `systemPrompt`, and `bash`—deliberately **not** `fs`; `ctx.spillStore` is read opportunistically with `ctx.get()` because formatted-result spill is optional.
|
||||
|
||||
```ts ignore-check
|
||||
// Default deployment: a bash executor whose PATH includes rg, then the discovery tools.
|
||||
// A deployment chooses how over-cap glob pages are selected.
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local
|
||||
await ctx.plugin(ToolFsSearch) // this package — conditionally registers glob/grep
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false })
|
||||
// Optional: a spill backend makes capped results fully recoverable.
|
||||
await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local
|
||||
```
|
||||
@@ -20,11 +20,12 @@ The mounted bash executor must be able to resolve `rg` from its `PATH` at plugin
|
||||
|
||||
## Config
|
||||
|
||||
All keys are optional; the defaults are the shipped search caps.
|
||||
`sampleOverCapGlobResults` is required and has no fallback; deployments choose the over-cap ordering contract explicitly. The remaining keys are optional search caps with the defaults below.
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `globMaxResults` | `100` | Max paths one `glob` call shows inline (matches Claude Code's `GlobTool` limit). Within it the result is shown whole in modification-time order; beyond it the inline page is sampled across top-level entries and the complete sorted list goes to the formatted spill artifact. |
|
||||
| `sampleOverCapGlobResults` | none (required) | `true` samples an over-cap `glob` page across top-level entries; `false` keeps the modification-time-ordered head. When formatted spill succeeds, both modes preserve the complete sorted list in that artifact. |
|
||||
| `globMaxResults` | `100` | Max paths one `glob` call shows inline (matches Claude Code's `GlobTool` limit). A result within the cap remains complete and modification-time ordered. |
|
||||
| `grepMaxMatches` | `250` | Max flat matches one `grep` call retains inline (matches Claude Code's `GrepTool` `head_limit`); later matches go to the formatted spill artifact. |
|
||||
| `grepMaxLineBytes` | `2000` | Byte cap per matched-line preview; the cut preserves UTF-8 boundaries and is marked `(line truncated)`. |
|
||||
| `rawOutputMaxBytes` | `20000000` | Max complete raw `rg` stdout a search will parse (matches Claude Code's ripgrep raw buffer); larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. |
|
||||
@@ -34,14 +35,14 @@ All keys are optional; the defaults are the shipped search caps.
|
||||
|
||||
| Tool | Arguments | Behavior |
|
||||
|---|---|---|
|
||||
| `glob` | `pattern`, `path?` | `rg --files --glob <pattern> --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved bash workdir. Returns one FILE path per line, modification-time ordered; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. |
|
||||
| `glob` | `pattern`, `path?` | `rg --files --glob <pattern> --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved bash workdir. Returns one FILE path per line; `rg --files` never emits directory entries. The pattern keeps ripgrep semantics: without a `/` it matches the basename at any depth, so `*` matches the whole tree. Complete results stay modification-time ordered; over-cap presentation follows `sampleOverCapGlobResults`. |
|
||||
| `grep` | `pattern`, `path?`, `include?` | Line-oriented `rg --json` parse (no colon-splitting ambiguity). `pattern` is a ripgrep regex; `path` is an optional **file or directory** target; `include` is ONE positive glob filter — a comma-separated list or a negated (`!…`) value is rejected up front (brace alternation like `*.{ts,tsx}` is fine). Returns matches grouped by file as `Line N: <preview>`. |
|
||||
|
||||
Routine budgets stay out of the model-facing schema (no `head_limit`/`offset`/`case_insensitive`/output modes): a model that needs surrounding context reads the matched file with `read`; one that needs later results follows the returned spill locator's retrieval hint.
|
||||
|
||||
## Two budgets, two artifacts
|
||||
|
||||
Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam and parses only complete retained stdout; if the executor still returns `stdout.truncated`, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with a head page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`.
|
||||
Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMaxBytes: rawOutputMaxBytes` from the bash seam and parses only complete retained stdout; if the executor still returns `stdout.truncated`, the search fails with `SEARCH_RAW_OUTPUT_OVERFLOW` and tells the model to narrow the query. A successful `glob` keeps the displayed search root and every acquired path in `{ root, paths }`; when sampling is enabled, `root` lets the Native renderer group an explicit relative or absolute search path by entries beneath that root rather than by its workdir prefix. `grep` keeps every acquired `{ path, lineNumber, line }` in `{ matches }`. Inline item and per-line preview caps apply only in the Native renderer. For a direct surface call with more logical results than the inline cap, post-policy best-effort saves the complete formatted preview through `ctx.spillStore.saveText()` and replaces only presentation with the configured page plus locator. Nested Code dispatches skip that spill because their full canonical value does not enter model context. Missing/failed spill keeps the inline page and reports that the complete result could not be saved—never an `isError`.
|
||||
|
||||
## Errors
|
||||
|
||||
@@ -55,12 +56,18 @@ Search failures carry the package-owned `SearchError` (a `HarnessError` subclass
|
||||
|
||||
After the load-time `rg` probe succeeds, every request in this plugin's registration scope contains the independently registered glob and grep guidance below. Agent-scoped tool restrictions can hide either schema without removing its prompt section.
|
||||
|
||||
##### Glob guidance
|
||||
##### Glob guidance with `sampleOverCapGlobResults: true`
|
||||
|
||||
```markdown
|
||||
Use the glob tool — not shell find — to discover files by path pattern. A pattern with no "/" matches basenames at any depth, so "*" matches every file in the tree rather than its top level. Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, while a larger one is sampled across top-level entries, so it spans the tree instead of one subtree.
|
||||
```
|
||||
|
||||
##### Glob guidance with `sampleOverCapGlobResults: false`
|
||||
|
||||
```markdown
|
||||
Use the glob tool — not shell find — to discover files by path pattern. A pattern with no "/" matches basenames at any depth, so "*" matches every file in the tree rather than its top level. Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, while a larger one keeps the modification-time-ordered head.
|
||||
```
|
||||
|
||||
##### Grep guidance
|
||||
|
||||
```markdown
|
||||
@@ -69,17 +76,17 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read
|
||||
|
||||
#### Token effect
|
||||
|
||||
Fixed guidance cost per request while the tools are registered.
|
||||
Fixed guidance cost per request while the tools are registered; the required sampling choice selects one glob variant.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Prefix-stable while the plugin scope and guidance text are unchanged. Activation or disposal may invalidate reuse from this prompt section.
|
||||
Prefix-stable while the plugin scope, sampling choice, and guidance text are unchanged. Activation, disposal, or changing the choice may invalidate reuse from this prompt section.
|
||||
|
||||
### Tool schemas
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) after the load-time `rg` probe succeeds and while this surface is visible.
|
||||
The glob description states the configured over-cap ordering. The generated [`glob` and `grep` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) use `sampleOverCapGlobResults: true`; schemas are visible only after the load-time `rg` probe succeeds.
|
||||
|
||||
#### Token effect
|
||||
|
||||
@@ -93,7 +100,7 @@ Prefix-stable while tool visibility and definitions are unchanged. Registration
|
||||
|
||||
#### What the model sees
|
||||
|
||||
`glob` returns one path per line; `grep` groups `Line <line>: <preview>` matches beneath each path. Empty searches return `No files found` or `No matches found`. A capped result ends with its omission count plus the spill locator and backend retrieval hint, or says the complete result could not be saved. An over-cap `glob` result does not show the head of the sorted list: its inline page takes paths round-robin across entries immediately beneath the actual search root, so one old-timestamped subtree cannot own every slot, and the footer states the sampled basis and how many top-level entries it reached. When it cannot reach them all, the footer tells the model to narrow `path`. A result that fits inline is untouched, and a flat result — every path its own top-level entry — keeps the plain footer, because there the sample is the modification-time-ordered head. The spill artifact always holds the complete list in modification-time order.
|
||||
`glob` returns one path per line; `grep` groups `Line <line>: <preview>` matches beneath each path. Empty searches return `No files found` or `No matches found`. A capped result ends with its omission count plus the spill locator and backend retrieval hint, or says the complete result could not be saved. With `sampleOverCapGlobResults: true`, an over-cap `glob` page takes paths round-robin across entries immediately beneath the actual search root, and the footer states the sampled basis and how many top-level entries it reached; when it cannot reach them all, the footer tells the model to narrow `path`. With `false`, the page is the modification-time-ordered head and keeps the plain capped-result footer. A result that fits inline is untouched, and a flat sampled result also keeps the plain footer because its sample equals the modification-time head. The spill artifact always holds the complete list in modification-time order.
|
||||
|
||||
#### Token effect
|
||||
|
||||
@@ -122,4 +129,4 @@ Append-only; newly visible content follows the reusable request prefix and does
|
||||
- **Search and file access have no shared-workspace proof** — returned paths are follow-up-readable only when the bash workdir and filesystem root denote the same workspace; the package performs no runtime cross-service validation.
|
||||
- **Ripgrep is a deployment dependency** — a missing `rg` executable makes the package register no tools or guidance; an incompatible executable or one that disappears after registration fails calls with `SEARCH_FAILED`. Remote or virtual filesystems need a co-located executor or another search consumer.
|
||||
- **The schemas expose one bounded page** — offset pagination, case-mode switches, alternate output modes, and provider-backed discovery remain outside this package; capped complete output requires a spill backend.
|
||||
- **Sampling groups by first path segment beneath the search root only** — an over-cap `glob` page balances across those top-level entries, so a result concentrated deeper (one busy directory inside an otherwise even tree) is still shown unevenly below that level; recursive balancing is deferred.
|
||||
- **Sampling, when enabled, groups by first path segment beneath the search root only** — an over-cap `glob` page balances across those top-level entries, so a result concentrated deeper (one busy directory inside an otherwise even tree) is still shown unevenly below that level; recursive balancing is deferred.
|
||||
|
||||
@@ -5,9 +5,9 @@
|
||||
**面向模型的文件系统发现工具**（`glob`、`grep`）由 **bash 执行器 seam** 支持，而不是由 `ctx.fs` 提供方方法支持。加载时，本包（package）探测 `command -v rg`，探测通过 `ctx.bash` 进行；如果执行器无法在其 `PATH` 上找到 ripgrep，就记录警告，并且不注册工具或提示词段。每次调用都会组装固定的 ripgrep 命令（所有模型控制的值都经过同一个包私有 shell 引用辅助函数），通过 `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` 作为普通前台工具调用运行，解析原始 `rg` 输出，并返回相对于工作目录的规范值。本包注入 `tools`、`systemPrompt` 和 `bash`，有意**不**注入 `fs`；格式化结果 spill 为可选功能，因此机会性读取 `ctx.spillStore`，调用方式为 `ctx.get()`。
|
||||
|
||||
```ts ignore-check
|
||||
// Default deployment: a bash executor whose PATH includes rg, then the discovery tools.
|
||||
// A deployment chooses how over-cap glob pages are selected.
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local
|
||||
await ctx.plugin(ToolFsSearch) // this package — conditionally registers glob/grep
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: false })
|
||||
// Optional: a spill backend makes capped results fully recoverable.
|
||||
await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-spill-local
|
||||
```
|
||||
@@ -20,11 +20,12 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
|
||||
|
||||
## 配置
|
||||
|
||||
所有键均为可选；默认值是随产品交付的搜索上限。
|
||||
`sampleOverCapGlobResults` 是必填项且没有回退值；部署必须显式选择超过上限时的排序契约。其余键是可选的搜索上限，默认值如下。
|
||||
|
||||
| 键 | 默认值 | 含义 |
|
||||
|---|---|---|
|
||||
| `globMaxResults` | `100` | 一次 `glob` 调用内联展示的最大路径数（与 Claude Code 的 `GlobTool` 上限相同）。未超过时结果整体按修改时间展示；超过时内联页面改为跨顶层条目取样，完整的排序列表写入格式化 spill 产物。 |
|
||||
| `sampleOverCapGlobResults` | 无（必填） | `true` 会在顶层条目之间对超过上限的 `glob` 页面采样；`false` 保留按修改时间排序的前部。格式化 spill 成功时，两种模式都会在该产物中保留完整排序列表。 |
|
||||
| `globMaxResults` | `100` | 一次 `glob` 调用内联展示的最大路径数（与 Claude Code 的 `GlobTool` 上限相同）。未超过上限的结果保持完整，并按修改时间排序。 |
|
||||
| `grepMaxMatches` | `250` | 一次 `grep` 调用内联保留的最大平铺匹配数（与 Claude Code 的 `GrepTool` `head_limit` 相同）；后续匹配写入格式化 spill 产物。 |
|
||||
| `grepMaxLineBytes` | `2000` | 每条匹配行预览的字节上限；截断会保留 UTF-8 边界，并标记为 `(line truncated)`。 |
|
||||
| `rawOutputMaxBytes` | `20000000` | 搜索将解析的完整原始 `rg` stdout 上限（与 Claude Code 的 ripgrep 原始 buffer 相同）；更大的原始输出以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败。 |
|
||||
@@ -34,14 +35,14 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
|
||||
|
||||
| 工具 | 参数 | 行为 |
|
||||
|---|---|---|
|
||||
| `glob` | `pattern`、`path?` | 运行 `rg --files --glob <pattern> --sort=modified --no-ignore --hidden`，并排除 VCS 元数据（`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`）。`path` 是可选的**目录**搜索根；省略时使用解析后的 bash 工作目录。每行返回一个**文件**路径；`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义：不含 `/` 时匹配任意深度的基名，因此 `*` 匹配整棵树。未超过 `globMaxResults` 的结果按修改时间排序；超过时内联页面改为跨顶层条目取样（见下）。 |
|
||||
| `glob` | `pattern`、`path?` | 运行 `rg --files --glob <pattern> --sort=modified --no-ignore --hidden`，并排除 VCS 元数据（`.git`、`.svn`、`.hg`、`.bzr`、`.jj`、`.sl`）。`path` 是可选的**目录**搜索根；省略时使用解析后的 bash 工作目录。每行返回一个**文件**路径；`rg --files` 从不输出目录条目。pattern 保留 ripgrep 语义：不含 `/` 时匹配任意深度的基名，因此 `*` 匹配整棵树。完整结果保持按修改时间排序；超过上限时的呈现方式遵循 `sampleOverCapGlobResults`。 |
|
||||
| `grep` | `pattern`、`path?`、`include?` | 按行解析 `rg --json`，避免按冒号拆分的歧义。`pattern` 是 ripgrep 正则表达式；`path` 是可选的**文件或目录**目标；`include` 是一个正向 glob 过滤器，前置拒绝逗号分隔列表或否定值（`!…`），但允许 `*.{ts,tsx}` 等花括号交替。返回按文件分组、形如 `Line N: <preview>` 的匹配。 |
|
||||
|
||||
常规预算不进入面向模型的 schema（没有 `head_limit`/`offset`/`case_insensitive`/输出模式）：模型需要周边上下文时，用 `read` 读取匹配文件；需要后续结果时，遵循返回的 spill locator 检索提示。
|
||||
|
||||
## 两类预算、两类产物
|
||||
|
||||
原始 `rg` stdout 是内部传输细节。每次搜索从 bash seam 请求 `stdoutMaxBytes: rawOutputMaxBytes`，且只解析完整保留的 stdout；如果执行器仍返回 `stdout.truncated`，搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败，并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径；借助 `root`，原生渲染器能以显式的相对或绝对搜索路径为根，按该根下的条目分组，而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`，并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时，后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览，并只把呈现替换为头部页面加 locator。嵌套 Code 分派会跳过 spill，因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面，并报告完整结果无法保存，绝不会成为 `isError`。
|
||||
原始 `rg` stdout 是内部传输细节。每次搜索从 bash seam 请求 `stdoutMaxBytes: rawOutputMaxBytes`，且只解析完整保留的 stdout；如果执行器仍返回 `stdout.truncated`，搜索会以 `SEARCH_RAW_OUTPUT_OVERFLOW` 失败，并要求模型缩小查询。成功的 `glob` 在 `{ root, paths }` 中保留所显示的搜索根及所有已取得路径；启用采样时，借助 `root`，原生渲染器能以显式的相对或绝对搜索路径为根，按该根下的条目分组，而不是按其工作目录前缀分组。`grep` 保留所有已取得的 `{ path, lineNumber, line }`，并将其存入 `{ matches }`。内联条目和每行预览上限只应用于原生渲染器。直接接口调用的逻辑结果超过内联上限时，后置策略会尽力通过 `ctx.spillStore.saveText()` 保存完整格式化预览，并只把呈现替换为配置指定的页面与 locator。嵌套 Code 分派会跳过 spill，因为其完整规范值不会进入模型上下文。spill 缺失/失败时保留内联页面，并报告完整结果无法保存，绝不会成为 `isError`。
|
||||
|
||||
## 错误
|
||||
|
||||
@@ -55,12 +56,18 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
|
||||
|
||||
加载时 `rg` 探测成功后，该插件注册作用域内的每个请求都包含下方独立注册的 glob 与 grep 指导。agent 作用域的工具限制可以隐藏任一 schema，而不移除其提示词段。
|
||||
|
||||
##### Glob 指导
|
||||
##### 启用 `sampleOverCapGlobResults: true` 时的 Glob 指导
|
||||
|
||||
```markdown
|
||||
Use the glob tool — not shell find — to discover files by path pattern. A pattern with no "/" matches basenames at any depth, so "*" matches every file in the tree rather than its top level. Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, while a larger one is sampled across top-level entries, so it spans the tree instead of one subtree.
|
||||
```
|
||||
|
||||
##### 启用 `sampleOverCapGlobResults: false` 时的 Glob 指导
|
||||
|
||||
```markdown
|
||||
Use the glob tool — not shell find — to discover files by path pattern. A pattern with no "/" matches basenames at any depth, so "*" matches every file in the tree rather than its top level. Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, while a larger one keeps the modification-time-ordered head.
|
||||
```
|
||||
|
||||
##### Grep 指导
|
||||
|
||||
```markdown
|
||||
@@ -69,17 +76,17 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read
|
||||
|
||||
#### Token 影响
|
||||
|
||||
工具注册期间，每个请求支付固定指导成本。
|
||||
工具注册期间，每个请求支付固定指导成本；必填的采样选项决定采用哪个 glob 变体。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
只要插件作用域和指导文本不变，前缀就保持稳定。启用或 dispose（资源释放）可能从该提示词段开始使复用失效。
|
||||
只要插件作用域、采样选项和指导文本不变，前缀就保持稳定。启用、dispose（资源释放）或更改该选项，可能从该提示词段开始使复用失效。
|
||||
|
||||
### 工具 schema
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
当前接口可见时，公开已生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search)；前提是加载时 `rg` 探测成功。
|
||||
glob 描述会说明配置所指定的超限结果排序方式。已生成的 [`glob` 和 `grep` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs-search) 使用 `sampleOverCapGlobResults: true`；只有加载时 `rg` 探测成功后，这些 schema 才可见。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
@@ -93,7 +100,7 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
`glob` 每行返回一个路径；`grep` 在每个路径下对 `Line <line>: <preview>` 匹配分组。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果末尾会附加省略数量、spill locator 和后端检索提示，或说明完整结果无法保存。超过上限的 `glob` 结果不再展示排序列表的头部：其内联页面按轮转方式跨实际搜索根正下方的条目取样，因此单个时间戳较旧的子树无法占满所有位置；footer 会说明取样依据及其触达的顶层条目数。无法触达全部时，footer 会要求模型缩小 `path`。未超过上限的结果原样不动；扁平结果（每个路径各自构成一个顶层条目）保留朴素 footer，因为此时取样结果就是按修改时间排序的头部。spill 产物始终保存按修改时间排序的完整列表。
|
||||
`glob` 每行返回一个路径；`grep` 在每个路径下对 `Line <line>: <preview>` 匹配分组。空搜索返回 `No files found` 或 `No matches found`。达到上限的结果末尾会附加省略数量、spill locator 和后端检索提示，或说明完整结果无法保存。`sampleOverCapGlobResults: true` 时，超过上限的 `glob` 页面会在实际搜索根正下方的条目之间按轮转方式取路径，footer 会说明采样依据和触达的顶层条目数；若无法触达全部条目，footer 会要求模型缩小 `path`。设为 `false` 时，页面保留按修改时间排序的前部，并沿用通常用于达到上限结果的 footer。未超过上限的结果原样不动；扁平的采样结果也沿用普通 footer，因为其样本等同于按修改时间排序的前部。spill 产物始终保存按修改时间排序的完整列表。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
@@ -122,4 +129,4 @@ Use the grep tool — not shell grep or rg — to search file contents. Use read
|
||||
- **搜索和文件访问没有共享工作区证明**：只有 bash 工作目录和文件系统根目录表示同一工作区时，返回路径才能继续读取；本包不执行运行时跨服务校验。
|
||||
- **Ripgrep 是部署依赖**：缺失 `rg` 可执行文件时，本包不注册工具或指导；可执行文件不兼容或注册后消失时，调用以 `SEARCH_FAILED` 失败。远程或虚拟文件系统需要共置执行器或其他搜索消费方。
|
||||
- **schema 只公开一个有界页面**：offset 分页、大小写模式开关、其他输出模式和提供方支持的发现均不在本包内；达到上限的完整输出需要 spill 后端。
|
||||
- **取样只按搜索根下的路径首段分组**：超过上限的 `glob` 页面在这些顶层条目之间做均衡，因此集中在更深层的结果（一棵总体均匀的树里某个特别庞大的子目录）在该层级以下仍然分布不均；递归均衡已延期。
|
||||
- **启用采样时，只按搜索根下的路径首段分组**：超过上限的 `glob` 页面在这些顶层条目之间做均衡，因此集中在更深层的结果（一棵总体均匀的树里某个特别庞大的子目录）在该层级以下仍然分布不均；递归均衡已延期。
|
||||
|
||||
@@ -38,6 +38,8 @@ export const GLOB_VCS_EXCLUDES: readonly string[] = ['.git', '.svn', '.hg', '.bz
|
||||
|
||||
/** Resolved glob-tool caps — plugin config after defaulting (see `Config` in index.ts). */
|
||||
export interface GlobToolCaps {
|
||||
/** Whether over-cap pages are sampled across top-level entries instead of taking the modification-time head. */
|
||||
sampleOverCapGlobResults: boolean
|
||||
/** Max paths retained inline; later paths go to the formatted spill file. */
|
||||
maxResults: number
|
||||
/** Cap on the complete raw `rg` stdout the tool will parse. */
|
||||
@@ -183,24 +185,32 @@ export function sampleAcrossTopLevel(paths: readonly string[], maxItems: number,
|
||||
* @returns the model-facing text.
|
||||
*/
|
||||
export function formatGlobOutput(sample: GlobSample, seen: number, spillRef: SpillRef | undefined): string {
|
||||
const body = sample.items.join('\n')
|
||||
const recovery = spillRef !== undefined
|
||||
? `Full sorted result stored at: ${spillRef.locator}. ${spillRef.retrievalHint}`
|
||||
: 'The complete result could not be saved; narrow pattern or path to see more.'
|
||||
const basis = sample.total === seen
|
||||
? '.'
|
||||
: `, sampled across ${sample.shown} of the ${sample.total} top-level entries this pattern matched instead of taken in modification-time order.`
|
||||
+ (sample.shown < sample.total ? ' Narrow path to inspect a specific subtree.' : '')
|
||||
return `${body}\n\n(Showing ${sample.items.length} of ${seen} paths${basis} ${recovery})`
|
||||
return formatGlobPage(sample.items, seen, spillRef, basis)
|
||||
}
|
||||
|
||||
/** Format one bounded page and the recovery path for its complete sorted result. */
|
||||
function formatGlobPage(items: readonly string[], seen: number, spillRef: SpillRef | undefined, basis: string): string {
|
||||
const body = items.join('\n')
|
||||
const recovery = spillRef !== undefined
|
||||
? `Full sorted result stored at: ${spillRef.locator}. ${spillRef.retrievalHint}`
|
||||
: 'The complete result could not be saved; narrow pattern or path to see more.'
|
||||
return `${body}\n\n(Showing ${items.length} of ${seen} paths${basis} ${recovery})`
|
||||
}
|
||||
|
||||
/** Bound and format one canonical path list for the Native surface relative to its search root. */
|
||||
function renderGlobPaths(paths: string[], maxResults: number, root: string, spillRef?: SpillRef): string {
|
||||
function renderGlobPaths(paths: string[], caps: GlobToolCaps, root: string, spillRef?: SpillRef): string {
|
||||
if (paths.length === 0) return 'No files found'
|
||||
// A result that fits is shown whole, untouched: modification-time order is the
|
||||
// tool's contract, and over a complete result it is what answers age questions.
|
||||
if (paths.length <= maxResults) return paths.join('\n')
|
||||
return formatGlobOutput(sampleAcrossTopLevel(paths, maxResults, root), paths.length, spillRef)
|
||||
if (paths.length <= caps.maxResults) return paths.join('\n')
|
||||
if (!caps.sampleOverCapGlobResults) {
|
||||
return formatGlobPage(paths.slice(0, caps.maxResults), paths.length, spillRef, '.')
|
||||
}
|
||||
return formatGlobOutput(sampleAcrossTopLevel(paths, caps.maxResults, root), paths.length, spillRef)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -222,19 +232,24 @@ export function presentGlobCall(args: { pattern: string; path?: string }): Gener
|
||||
* @param caps - the deployment's resolved glob caps (plugin config after defaulting).
|
||||
*/
|
||||
export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void {
|
||||
const overCapGuidance = caps.sampleOverCapGlobResults
|
||||
? 'while a larger one is sampled across top-level entries, so it spans the tree instead of one subtree.'
|
||||
: 'while a larger one keeps the modification-time-ordered head.'
|
||||
ctx.systemPrompt.section({
|
||||
name: 'tool:glob',
|
||||
order: 103,
|
||||
text: 'Use the glob tool — not shell find — to discover files by path pattern. A pattern with no "/" matches basenames at any depth, so "*" matches every file in the tree rather than its top level. '
|
||||
+ 'Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, while a larger one is sampled across top-level entries, '
|
||||
+ 'so it spans the tree instead of one subtree.',
|
||||
+ `Results are files only, never directories, and include hidden and ignored files: a result that fits comes back in modification-time order, ${overCapGuidance}`,
|
||||
})
|
||||
|
||||
const overCapDescription = caps.sampleOverCapGlobResults
|
||||
? `a larger result instead returns ${caps.maxResults} paths sampled across top-level entries`
|
||||
: `a larger result returns the first ${caps.maxResults} paths in modification-time order`
|
||||
const tool = defineTool({
|
||||
name: 'glob',
|
||||
description: 'Find files whose paths match a glob pattern. Returns matching file paths — never directories — '
|
||||
+ 'including hidden and ignored files (VCS metadata directories are excluded). '
|
||||
+ `Up to ${caps.maxResults} paths come back in modification-time order; a larger result instead returns ${caps.maxResults} paths sampled across top-level entries, `
|
||||
+ `Up to ${caps.maxResults} paths come back in modification-time order; ${overCapDescription}, `
|
||||
+ 'says so, and reports where the complete sorted list was saved. This tool does not enumerate directory entries.',
|
||||
parameters: {
|
||||
pattern: {
|
||||
@@ -255,7 +270,7 @@ export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void {
|
||||
paths: { type: 'array', required: true, items: { type: 'string' } },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [{ type: 'text', text: renderGlobPaths(value.paths, caps.maxResults, value.root) }],
|
||||
render: (_args, value) => [{ type: 'text', text: renderGlobPaths(value.paths, caps, value.root) }],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const input = parseGlobArgs(args)
|
||||
@@ -284,7 +299,7 @@ export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void {
|
||||
const spillRef = await trySaveFormattedResult(ctx, exec, 'glob-results.txt', paths.join('\n'))
|
||||
return {
|
||||
kind: 'accept',
|
||||
content: [{ type: 'text', text: renderGlobPaths(paths, caps.maxResults, value.root, spillRef) }],
|
||||
content: [{ type: 'text', text: renderGlobPaths(paths, caps, value.root, spillRef) }],
|
||||
...decision.additionalContexts !== undefined ? { additionalContexts: decision.additionalContexts } : {},
|
||||
}
|
||||
})
|
||||
|
||||
@@ -58,8 +58,10 @@ export const name = 'tool-fs-search'
|
||||
/** Services required by the search tool suite (`spillStore` is optional, read via `ctx.get()`). */
|
||||
export const inject = ['tools', 'systemPrompt', 'bash']
|
||||
|
||||
/** Plugin config (all optional — `Config` supplies the defaults). */
|
||||
/** Plugin config; over-cap glob sampling is an explicit deployment choice and the remaining fields have defaults. */
|
||||
export interface Config {
|
||||
/** Whether an over-cap `glob` page is sampled across top-level entries instead of taking the modification-time head. */
|
||||
sampleOverCapGlobResults: boolean
|
||||
/** Max paths one `glob` call retains inline; later paths go to the formatted spill file. */
|
||||
globMaxResults?: number
|
||||
/** Max flat matches one `grep` call retains inline; later matches go to the formatted spill file. */
|
||||
@@ -73,6 +75,7 @@ export interface Config {
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
sampleOverCapGlobResults: z.boolean().required(),
|
||||
globMaxResults: z.number().default(GLOB_MAX_RESULTS),
|
||||
grepMaxMatches: z.number().default(GREP_MAX_MATCHES),
|
||||
grepMaxLineBytes: z.number().default(GREP_MAX_LINE_BYTES),
|
||||
@@ -137,6 +140,7 @@ export async function apply(ctx: Context, config: Config): Promise<void> {
|
||||
return
|
||||
}
|
||||
applyGlobTool(ctx, {
|
||||
sampleOverCapGlobResults: resolved.sampleOverCapGlobResults,
|
||||
maxResults: resolved.globMaxResults,
|
||||
rawOutputMaxBytes: resolved.rawOutputMaxBytes,
|
||||
timeoutMs: resolved.timeoutMs,
|
||||
|
||||
@@ -64,7 +64,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(LocalSubprocessService)
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: dir, timeoutMs: 20_000 })
|
||||
await ctx.plugin(ToolFsSearch)
|
||||
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: true })
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
|
||||
@@ -82,7 +82,7 @@ describe('dsh-tool-fs-search real-load-path guard', () => {
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolFsSearch) as Parameters<Context['plugin']>[0]
|
||||
// A collapsed export shape (dropped inject) would throw "without inject" here.
|
||||
const fiber = await ctx.plugin(unwrapped)
|
||||
const fiber = await ctx.plugin(unwrapped, { sampleOverCapGlobResults: true })
|
||||
expect(ctx.tools.schemas().map(s => s.name)).toEqual(expect.arrayContaining(['glob', 'grep']))
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
@@ -111,12 +111,14 @@ class FakeSpill extends SpillStore {
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
config?: ToolFsSearch.Config
|
||||
config?: Partial<ToolFsSearch.Config>
|
||||
spill?: boolean
|
||||
probeError?: Error
|
||||
probeResult?: BashRunResult
|
||||
}
|
||||
|
||||
const DEFAULT_CONFIG = { sampleOverCapGlobResults: true } satisfies ToolFsSearch.Config
|
||||
|
||||
async function setup(options: SetupOptions = {}) {
|
||||
const ctx = new Context()
|
||||
const warnings: string[] = []
|
||||
@@ -128,7 +130,7 @@ async function setup(options: SetupOptions = {}) {
|
||||
if (options.probeResult) bash.probeResult = options.probeResult
|
||||
if (options.probeError) bash.probeError = options.probeError
|
||||
if (options.spill === true) await ctx.plugin(FakeSpill)
|
||||
const fiber = await ctx.plugin(ToolFsSearch, options.config)
|
||||
const fiber = await ctx.plugin(ToolFsSearch, { ...DEFAULT_CONFIG, ...options.config })
|
||||
const spill = options.spill === true ? ctx.get('spillStore') as FakeSpill : undefined
|
||||
return { ctx, bash, spill, fiber, warnings }
|
||||
}
|
||||
@@ -216,7 +218,7 @@ describe('registration', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(ToolFsSearch) // no bash executor
|
||||
await ctx.plugin(ToolFsSearch, DEFAULT_CONFIG) // no bash executor
|
||||
expect(ctx.tools.schemas()).toHaveLength(0)
|
||||
})
|
||||
|
||||
@@ -241,9 +243,27 @@ describe('registration', () => {
|
||||
expect(ctx.tools.get('glob')?.timeoutMs).toBe(30_000)
|
||||
expect(ctx.tools.get('grep')?.timeoutMs).toBe(30_000)
|
||||
})
|
||||
|
||||
it('describes the modification-time head when over-cap sampling is disabled', async () => {
|
||||
const { ctx } = await setup({ config: { sampleOverCapGlobResults: false } })
|
||||
const prompt = renderPrompt(await ctx.systemPrompt.assemble())
|
||||
expect(prompt).toContain('a larger one keeps the modification-time-ordered head')
|
||||
expect(prompt).not.toContain('sampled across top-level entries')
|
||||
const glob = ctx.tools.schemas().find(schema => schema.name === 'glob')
|
||||
expect(glob?.description).toContain('a larger result returns the first 100 paths in modification-time order')
|
||||
expect(glob?.description).not.toContain('sampled across top-level entries')
|
||||
})
|
||||
})
|
||||
|
||||
describe('config validation', () => {
|
||||
it('requires an explicit over-cap glob sampling choice', () => {
|
||||
expect(() => new ToolFsSearch.Config()).toThrow(/sampleOverCapGlobResults/)
|
||||
expect(new ToolFsSearch.Config({ sampleOverCapGlobResults: false })).toMatchObject({
|
||||
sampleOverCapGlobResults: false,
|
||||
globMaxResults: 100,
|
||||
})
|
||||
})
|
||||
|
||||
it.each([
|
||||
['globMaxResults', { globMaxResults: 0 }],
|
||||
['grepMaxMatches', { grepMaxMatches: -1 }],
|
||||
@@ -255,7 +275,7 @@ describe('config validation', () => {
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(FakeBash)
|
||||
await expect(ctx.plugin(ToolFsSearch, config)).rejects.toThrow(new RegExp(`tool-fs-search: ${name} must be a positive integer`))
|
||||
await expect(ctx.plugin(ToolFsSearch, { ...DEFAULT_CONFIG, ...config })).rejects.toThrow(new RegExp(`tool-fs-search: ${name} must be a positive integer`))
|
||||
})
|
||||
})
|
||||
|
||||
@@ -625,6 +645,16 @@ describe('glob results', () => {
|
||||
+ 'The complete result could not be saved; narrow pattern or path to see more.)')
|
||||
})
|
||||
|
||||
it('keeps the modification-time head when over-cap sampling is disabled', async () => {
|
||||
const { ctx, bash } = await setup({
|
||||
config: { globMaxResults: 3, sampleOverCapGlobResults: false },
|
||||
})
|
||||
bash.handler = () => runResult(['vendor/a.ts', 'vendor/b.ts', 'vendor/c.ts', 'src/d.ts', 'guide/e.md'].join('\n'))
|
||||
expect(text(await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') })))
|
||||
.toBe('vendor/a.ts\nvendor/b.ts\nvendor/c.ts\n\n'
|
||||
+ '(Showing 3 of 5 paths. The complete result could not be saved; narrow pattern or path to see more.)')
|
||||
})
|
||||
|
||||
it('samples relative to the explicit search root instead of its workdir prefix', async () => {
|
||||
const { ctx, bash } = await setup({ config: { globMaxResults: 3 } })
|
||||
bash.handler = () => runResult([
|
||||
|
||||
Reference in New Issue
Block a user