feat: bash-backed glob/grep discovery tools (dsh-tool-fs-search)
Implements docs/rfc/implemented/feature/2026-07-09-bash-backed-grep-glob- discovery.md: model-facing glob/grep in a new @deepseek-ai/dsh-tool-fs-search package, executing fixed ripgrep templates through ctx.bash.resolve/run — not ctx.fs provider methods — so filesystem backends stay free of a search contract and sandboxed/remote executors substitute cleanly. The tools never call ctx.bash.start(); the tool layer owns quoting (one singleQuote safety boundary), rg --json parsing, ItemRetainer/TextRetainer retention, and the first tool-owned ctx.spillFiles.saveText() handoff (item-level retention the generic post-execute spill policy cannot recover). RFC amendments on the way to implemented/: a shared src/search-core.ts (the SEARCH_* vocabulary + bash-run/raw-spill/spill plumbing was byte-identical across both tools — the missed-extraction smell), and a snapshot-gap note: wiring the acp-agent tree changes the assembled prompt, so goldens need a keyed re-record; the spill notice text is pinned by unit tests instead and only the coding-agent example ships the tools for now.
This commit is contained in:
Dudu-0223
@@ -12,7 +12,7 @@ Packages are grouped by role at `packages/<group>/<pkg>/`. The group directory i
|
||||
| [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface |
|
||||
| [`bash/`](bash/README.md) | Bash capability family: the executor seam, a local impl, and the model-facing tool | Product — stable surface |
|
||||
| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the abstract runtime seam for model-written programs + a worker-thread backend | Product — stable surface |
|
||||
| [`fs/`](fs/README.md) | Filesystem capability family: the abstract seam, a local impl, and the model-facing file tools | Product — stable surface |
|
||||
| [`fs/`](fs/README.md) | Filesystem capability family: the abstract seam, a local impl, the model-facing file tools, and the bash-backed discovery tools | Product — stable surface |
|
||||
| [`compact/`](compact/README.md) | Compaction capability family: the abstract seam + a basic backend (tool deferred) | Product — stable surface |
|
||||
| [`subagent/`](subagent/README.md) | Subagent capability family: the provider-registry seam and the model-facing delegation tool | Product — stable surface |
|
||||
| [`web/`](web/README.md) | Web capability family: the abstract seam, search/fetch provider impls, and the model-facing web tools | Product — stable surface |
|
||||
|
||||
@@ -35,7 +35,7 @@ describe('gen-tool-catalog collectToolCatalog', () => {
|
||||
it('boots every shipped tool package and harvests its model-facing schemas', async () => {
|
||||
const catalog = await collectToolCatalog()
|
||||
const names = catalog.flatMap(entry => entry.schemas.map(s => s.name)).sort()
|
||||
expect(names).toEqual(['bash', 'bash_kill', 'bash_output', 'edit', 'read', 'subagent', 'todo_write', 'web_fetch', 'web_search', 'write'])
|
||||
expect(names).toEqual(['bash', 'bash_kill', 'bash_output', 'edit', 'glob', 'grep', 'read', 'subagent', 'todo_write', 'web_fetch', 'web_search', 'write'])
|
||||
// Every tool carries a JSON-Schema `parameters` object (what the model sees).
|
||||
for (const entry of catalog) {
|
||||
for (const schema of entry.schemas) {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# fs/ - filesystem capability family
|
||||
|
||||
The filesystem stack: a provider seam (text IO + atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), and the model-facing file tools + executor. All **product** packages.
|
||||
The filesystem stack: a provider seam (text IO + atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), the model-facing file tools + executor, and the bash-backed discovery tools. All **product** packages.
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
@@ -8,9 +8,10 @@ The filesystem stack: a provider seam (text IO + atomic mutation with an optiona
|
||||
| `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) |
|
||||
| `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) |
|
||||
| `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) |
|
||||
| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools, backed by fixed ripgrep commands through the bash seam (`ctx.bash`), NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) |
|
||||
|
||||
The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it.
|
||||
The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents).
|
||||
|
||||
## No timeouts on file IO
|
||||
|
||||
`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web, which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md). A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries.
|
||||
`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web (which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)) and the bash-backed `glob`/`grep` (whose declared `timeoutMs` is enforced by `@deepseek-ai/dsh-timeout-policy`): those are process-backed, where a deadline can really kill the work. A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries.
|
||||
|
||||
46
packages/fs/tool-fs-search/README.md
Normal file
46
packages/fs/tool-fs-search/README.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# @deepseek-ai/dsh-tool-fs-search
|
||||
|
||||
The **model-facing filesystem discovery tools** — `glob`, `grep` — backed by the **bash executor seam**, not by `ctx.fs` provider methods. Each call assembles a fixed ripgrep command (every model-controlled value through one package-private shell-quoting helper), runs it via `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` as an ordinary foreground tool call, parses the raw `rg` output, and returns a bounded, workdir-relative result. The package injects `tools`, `systemPrompt`, and `bash` — deliberately **not** `fs`; `ctx.spillFiles` is read opportunistically with `ctx.get()` because formatted-result spill is optional.
|
||||
|
||||
```ts ignore-check
|
||||
// Default deployment: a bash executor, then the discovery tools.
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: process.cwd() }) // @deepseek-ai/dsh-bash-local
|
||||
await ctx.plugin(ToolFsSearch) // this package — registers glob/grep
|
||||
// Optional: a spill backend makes capped results fully recoverable.
|
||||
await ctx.plugin(LocalSpillFiles) // @deepseek-ai/dsh-spill-local
|
||||
```
|
||||
|
||||
Why bash-backed: local workspace discovery is naturally a process-backed `rg` workflow, and putting search on `ctx.fs` would force every filesystem backend to grow a search API. The bash executor owns request defaulting/capping, subprocess execution, process-group termination, environment scrubbing, raw output capture, and backend substitution (local, sandboxed, remote); this package owns schemas, argument validation, shell quoting, parsing, retention, formatted-result spill, and timeout declaration. The tools never call `ctx.bash.start()` and never expose a bash task id — the call returns only after `rg` exits, times out, is aborted, or fails.
|
||||
|
||||
## Deployment requirement: co-located bash + filesystem
|
||||
|
||||
Returned paths are displayed relative to the resolved bash workdir (the calling agent's session cwd when present, else the executor's configured default) and are follow-up-readable with `read` only when the bash workdir and the filesystem root are the same workspace. v1 documents that requirement and performs no runtime cross-service validation; remote or virtual filesystem search waits for a shared workspace contract or a provider-specific search backend.
|
||||
|
||||
## Config
|
||||
|
||||
All keys are optional; the defaults are the shipped search caps.
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `globMaxResults` | `100` | Max paths one `glob` call retains inline (matches Claude Code's `GlobTool` limit); later paths go to the formatted spill file. |
|
||||
| `grepMaxMatches` | `250` | Max flat matches one `grep` call retains inline (matches Claude Code's `GrepTool` `head_limit`); later matches go to the formatted spill file. |
|
||||
| `grepMaxLineBytes` | `2000` | Byte cap per matched-line preview; the cut preserves UTF-8 boundaries and is marked `(line truncated)`. |
|
||||
| `rawOutputMaxBytes` | `20000000` | Max complete raw `rg` stdout a search will parse (matches Claude Code's ripgrep raw buffer); larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. |
|
||||
| `timeoutMs` | `30000` | Cooperative tool-call budget attached to both tool definitions, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`; the bash backend's own timeout stays a second safety cap. |
|
||||
|
||||
## Tools
|
||||
|
||||
| Tool | Arguments | Behavior |
|
||||
|---|---|---|
|
||||
| `glob` | `pattern`, `path?` | `rg --files --glob <pattern> --sort=modified --no-ignore --hidden` plus VCS metadata excludes (`.git`, `.svn`, `.hg`, `.bzr`, `.jj`, `.sl`). `path` is an optional **directory** search root; omitted means the resolved bash workdir. Returns one path per line, modification-time ordered. |
|
||||
| `grep` | `pattern`, `path?`, `include?` | Line-oriented `rg --json` parse (no colon-splitting ambiguity). `pattern` is a ripgrep regex; `path` is an optional **file or directory** target; `include` is ONE positive glob filter — a comma-separated list or a negated (`!…`) value is rejected up front (brace alternation like `*.{ts,tsx}` is fine). Returns matches grouped by file as `Line N: <preview>`. |
|
||||
|
||||
Routine budgets stay out of the model-facing schema (no `head_limit`/`offset`/`case_insensitive`/output modes): a model that needs surrounding context reads the matched file with `read`; one that needs later results reads the formatted spill file with `read offset/limit`.
|
||||
|
||||
## Two budgets, two artifacts
|
||||
|
||||
Raw `rg` stdout is an internal transport detail. When the executor truncates it, the tool recovers the complete stream from the executor's **raw bash spill file** — read locally, capped at `rawOutputMaxBytes`, never shown to the model. The model-facing recovery artifact is different: when a search yields more logical results than the inline cap, the tool saves the COMPLETE formatted result through `ctx.spillFiles.saveText()` (suggested names `glob-results.txt` / `grep-results.txt`, owner = the calling session, source = the tool execution identity) and appends a footer naming the saved path. This is the first tool-owned spill call in the codebase — deliberate, because retention here is item-level: the generic `@deepseek-ai/dsh-spill-policy` only sees the final text on `tools/post-execute`, by which point a capped search has already omitted later paths/matches. A missing spill backend, a call with no session owner, or a `saveText()` failure keeps the inline page and reports that the complete result could not be saved — never an `isError`.
|
||||
|
||||
## Errors
|
||||
|
||||
Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (missing `rg`, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or truncated with no recovery file), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors.
|
||||
49
packages/fs/tool-fs-search/package.json
Normal file
49
packages/fs/tool-fs-search/package.json
Normal file
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-tool-fs-search",
|
||||
"description": "Model-facing filesystem discovery tools (glob, grep) backed by the DeepSeek Harness bash seam (ctx.bash)",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-bash": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-retention": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-spill": "^0.0.1",
|
||||
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-bash": "workspace:^",
|
||||
"@deepseek-ai/dsh-bash-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-retention": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-spill": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
168
packages/fs/tool-fs-search/src/glob.ts
Normal file
168
packages/fs/tool-fs-search/src/glob.ts
Normal file
@@ -0,0 +1,168 @@
|
||||
/**
|
||||
* The model-facing `glob` tool: discover files whose paths match a glob
|
||||
* pattern, sorted by modification time. Execution goes through the bash seam
|
||||
* (`ctx.bash`) with a fixed `rg --files` command — this module owns the
|
||||
* model-facing schema, argument validation, shell-safe command construction,
|
||||
* result parsing, retention, and formatting; process concerns (defaulting,
|
||||
* scrubbing, kill, backend substitution) stay behind `ctx.bash`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/glob
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { GenericCallView } from '@deepseek-ai/dsh-tools'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import { ItemRetainer } from '@deepseek-ai/dsh-retention'
|
||||
import type { RetainedItems } from '@deepseek-ai/dsh-retention'
|
||||
import type {} from '@deepseek-ai/dsh-bash'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
import { runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts'
|
||||
import { singleQuote } from './shell-quote.ts'
|
||||
|
||||
/**
|
||||
* Default cap on paths retained inline by one `glob` call (the `globMaxResults`
|
||||
* config), matching Claude Code's default `GlobTool` result limit.
|
||||
*/
|
||||
export const GLOB_MAX_RESULTS = 100
|
||||
|
||||
/**
|
||||
* Directory names ripgrep must never descend into for a discovery listing: VCS
|
||||
* metadata stores. `--no-ignore --hidden` would otherwise surface them in every
|
||||
* broad search. Each is excluded with a negated any-depth `--glob` (see
|
||||
* {@link buildGlobCommand}), which matches — and prunes — the directory
|
||||
* wherever it appears.
|
||||
*/
|
||||
export const GLOB_VCS_EXCLUDES: readonly string[] = ['.git', '.svn', '.hg', '.bzr', '.jj', '.sl']
|
||||
|
||||
/** Resolved glob-tool caps — plugin config after defaulting (see `Config` in index.ts). */
|
||||
export interface GlobToolCaps {
|
||||
/** Max paths retained inline; later paths go to the formatted spill file. */
|
||||
maxResults: number
|
||||
/** Cap on the complete raw `rg` stdout the tool will parse. */
|
||||
rawOutputMaxBytes: number
|
||||
/** Cooperative tool-call budget (ms) attached as `ToolDefinition.timeoutMs`. */
|
||||
timeoutMs: number
|
||||
}
|
||||
|
||||
/** Validated `glob` arguments. */
|
||||
export interface GlobInput {
|
||||
pattern: string
|
||||
path?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate value constraints the schema DSL can't express: a non-blank
|
||||
* `pattern`, and a non-blank `path` when given. Throws a plain `Error` (an
|
||||
* ordinary tool argument error) otherwise.
|
||||
*
|
||||
* @param args - the schema-validated `glob` arguments.
|
||||
* @returns the accepted input, unchanged.
|
||||
*/
|
||||
export function parseGlobArgs(args: { pattern: string; path?: string }): GlobInput {
|
||||
if (args.pattern.trim().length === 0) throw new Error('pattern must be a non-empty string')
|
||||
if (args.path !== undefined && args.path.trim().length === 0) throw new Error('path must be a non-empty string when given')
|
||||
return { pattern: args.pattern, ...args.path !== undefined ? { path: args.path } : {} }
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the fixed `rg --files` command for one `glob` call. Every
|
||||
* model-controlled value ({@link GlobInput.pattern}, {@link GlobInput.path})
|
||||
* passes through {@link singleQuote}; the search root rides behind `--` so a
|
||||
* leading-dash path can never be parsed as a flag. `--sort=modified` orders by
|
||||
* modification time, `--no-ignore --hidden` searches ignored and hidden files,
|
||||
* and {@link GLOB_VCS_EXCLUDES} keeps VCS metadata out.
|
||||
*
|
||||
* @param input - the validated arguments.
|
||||
* @returns the complete, shell-safe command string.
|
||||
*/
|
||||
export function buildGlobCommand(input: GlobInput): string {
|
||||
const parts = [
|
||||
'rg --files',
|
||||
`--glob=${singleQuote(input.pattern)}`,
|
||||
'--sort=modified --no-ignore --hidden',
|
||||
...GLOB_VCS_EXCLUDES.map(name => `--glob=${singleQuote(`!**/${name}`)}`),
|
||||
]
|
||||
if (input.path !== undefined) parts.push('--', singleQuote(input.path))
|
||||
return parts.join(' ')
|
||||
}
|
||||
|
||||
/**
|
||||
* Format the model-facing `glob` result: the retained paths, then — when the
|
||||
* result was capped — a footer carrying either the formatted-spill recovery
|
||||
* path or the could-not-save explanation. The omitted count is a budget fact:
|
||||
* the search itself completed.
|
||||
*
|
||||
* @param retained - the retention outcome over every discovered path.
|
||||
* @param spillPath - the saved complete-result path, or `undefined` when unsaved.
|
||||
* @returns the model-facing text.
|
||||
*/
|
||||
export function formatGlobOutput(retained: RetainedItems<string>, spillPath: string | undefined): string {
|
||||
const body = retained.items.join('\n')
|
||||
if (!retained.truncated) return body
|
||||
const recovery = spillPath !== undefined
|
||||
? `Full sorted result saved to: ${spillPath}. Use read with offset/limit to inspect it.`
|
||||
: 'The complete result could not be saved; narrow pattern or path to see more.'
|
||||
return `${body}\n\n(Showing ${retained.kept} of ${retained.seen} paths. ${recovery})`
|
||||
}
|
||||
|
||||
/**
|
||||
* Pending-call presentation: a search card titled by the pattern (and root).
|
||||
*
|
||||
* @param args - the raw tool arguments; `pattern` and `path` feed the title.
|
||||
* @returns the generic card view (`kind: 'search'`) shown while the call runs.
|
||||
*/
|
||||
export function presentGlobCall(args: { pattern: string; path?: string }): GenericCallView {
|
||||
const where = args.path !== undefined ? ` in ${args.path}` : ''
|
||||
return { card: 'generic', title: `Glob ${args.pattern}${where}`, kind: 'search', rawInput: args.pattern }
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `glob` tool and its system-prompt guidance.
|
||||
*
|
||||
* @param ctx - the plugin context; registrations are effects scoped to it, and
|
||||
* execution uses its `bash` service.
|
||||
* @param caps - the deployment's resolved glob caps (plugin config after defaulting).
|
||||
*/
|
||||
export function applyGlobTool(ctx: Context, caps: GlobToolCaps): void {
|
||||
ctx.systemPrompt.section({
|
||||
name: 'tool:glob',
|
||||
order: 103,
|
||||
text: 'Use the glob tool — not shell find or ls — to discover files by path pattern. Results are sorted by modification time and include hidden and ignored files.',
|
||||
})
|
||||
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'glob',
|
||||
description: 'Find files whose paths match a glob pattern. Returns matching paths sorted by modification time, '
|
||||
+ 'including hidden and ignored files (VCS metadata directories are excluded). '
|
||||
+ `Returns the first ${caps.maxResults} paths inline; a capped result reports where the complete list was saved.`,
|
||||
parameters: {
|
||||
pattern: { type: 'string', required: true, description: 'Glob pattern to match file paths against (e.g. "**/*.ts", "src/**/*.test.js").' },
|
||||
path: { type: 'string', description: 'Directory to search in. Defaults to the session workspace; a relative path resolves against it.' },
|
||||
},
|
||||
timeoutMs: caps.timeoutMs,
|
||||
async execute(args, exec): Promise<ContentBlock[]> {
|
||||
const input = parseGlobArgs(args)
|
||||
const run = await runRipgrep(ctx, exec, 'glob', buildGlobCommand(input), caps.rawOutputMaxBytes)
|
||||
if (run.noMatches) return [{ type: 'text', text: 'No files found' }]
|
||||
|
||||
const retainer = new ItemRetainer<string>({ kind: 'head', maxItems: caps.maxResults })
|
||||
const all: string[] = []
|
||||
for (const line of run.stdout.split('\n')) {
|
||||
if (line.length === 0) continue
|
||||
const displayPath = toWorkdirRelative(line, run.workdir)
|
||||
all.push(displayPath)
|
||||
retainer.push(displayPath)
|
||||
}
|
||||
const retained = retainer.finish()
|
||||
|
||||
// The complete sorted list is the recovery artifact; save it only when
|
||||
// the inline page omitted paths (an uncapped result needs no spill file).
|
||||
const spillPath = retained.truncated
|
||||
? await trySaveFormattedResult(ctx, exec, 'glob-results.txt', all.join('\n'))
|
||||
: undefined
|
||||
return [{ type: 'text', text: formatGlobOutput(retained, spillPath) }]
|
||||
},
|
||||
presentCall: presentGlobCall,
|
||||
}))
|
||||
}
|
||||
314
packages/fs/tool-fs-search/src/grep.ts
Normal file
314
packages/fs/tool-fs-search/src/grep.ts
Normal file
@@ -0,0 +1,314 @@
|
||||
/**
|
||||
* The model-facing `grep` tool: search file contents with a ripgrep regular
|
||||
* expression. Execution goes through the bash seam (`ctx.bash`) with a fixed
|
||||
* line-oriented `rg --json` command so file path, line number, and line text
|
||||
* parse without colon-splitting ambiguity — this module owns the model-facing
|
||||
* schema, argument validation, shell-safe command construction, `--json`
|
||||
* record parsing, per-line preview retention, match retention, grouping, and
|
||||
* formatting; process concerns stay behind `ctx.bash`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/grep
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { GenericCallView } from '@deepseek-ai/dsh-tools'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import { ItemRetainer, TextRetainer } from '@deepseek-ai/dsh-retention'
|
||||
import type { RetainedItems } from '@deepseek-ai/dsh-retention'
|
||||
import type {} from '@deepseek-ai/dsh-bash'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
import { SearchError, runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts'
|
||||
import { singleQuote } from './shell-quote.ts'
|
||||
|
||||
/**
|
||||
* Default cap on flat matches retained inline by one `grep` call (the
|
||||
* `grepMaxMatches` config), matching Claude Code's default `GrepTool`
|
||||
* `head_limit`.
|
||||
*/
|
||||
export const GREP_MAX_MATCHES = 250
|
||||
|
||||
/**
|
||||
* Default cap in bytes on one matched-line preview (the `grepMaxLineBytes`
|
||||
* config); the cut preserves UTF-8 boundaries.
|
||||
*/
|
||||
export const GREP_MAX_LINE_BYTES = 2000
|
||||
|
||||
/** Resolved grep-tool caps — plugin config after defaulting (see `Config` in index.ts). */
|
||||
export interface GrepToolCaps {
|
||||
/** Max flat matches retained inline; later matches go to the formatted spill file. */
|
||||
maxMatches: number
|
||||
/** Max bytes retained per matched-line preview. */
|
||||
maxLineBytes: number
|
||||
/** Cap on the complete raw `rg` stdout the tool will parse. */
|
||||
rawOutputMaxBytes: number
|
||||
/** Cooperative tool-call budget (ms) attached as `ToolDefinition.timeoutMs`. */
|
||||
timeoutMs: number
|
||||
}
|
||||
|
||||
/** Validated `grep` arguments. */
|
||||
export interface GrepInput {
|
||||
pattern: string
|
||||
path?: string
|
||||
include?: string
|
||||
}
|
||||
|
||||
/** One parsed match: the file, the 1-based line number, and the (possibly previewed) line text. */
|
||||
export interface GrepMatch {
|
||||
path: string
|
||||
lineNumber: number
|
||||
line: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject an `include` that is not ONE positive glob filter: blank strings,
|
||||
* negated patterns (`!…`), and comma-separated lists. A comma inside a brace
|
||||
* group is fine — `*.{ts,tsx}` is one glob with alternation, not a list.
|
||||
*/
|
||||
function validateInclude(include: string): void {
|
||||
if (include.trim().length === 0) throw new Error('include must be a non-empty glob when given')
|
||||
if (include.startsWith('!')) throw new Error('include must be a positive glob filter; negated patterns ("!…") are not supported')
|
||||
let braceDepth = 0
|
||||
for (const char of include) {
|
||||
if (char === '{') braceDepth++
|
||||
else if (char === '}') braceDepth = Math.max(0, braceDepth - 1)
|
||||
else if (char === ',' && braceDepth === 0) {
|
||||
throw new Error('include must be one glob, not a comma-separated list (use {a,b} alternation instead)')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate value constraints the schema DSL can't express: a non-EMPTY
|
||||
* `pattern` (whitespace is a legitimate regex), a non-blank `path` when given,
|
||||
* and a single positive `include` glob ({@link GrepInput}). Throws a plain
|
||||
* `Error` (an ordinary tool argument error) otherwise.
|
||||
*
|
||||
* @param args - the schema-validated `grep` arguments.
|
||||
* @returns the accepted input, unchanged.
|
||||
*/
|
||||
export function parseGrepArgs(args: { pattern: string; path?: string; include?: string }): GrepInput {
|
||||
if (args.pattern.length === 0) throw new Error('pattern must be a non-empty string')
|
||||
if (args.path !== undefined && args.path.trim().length === 0) throw new Error('path must be a non-empty string when given')
|
||||
if (args.include !== undefined) validateInclude(args.include)
|
||||
return {
|
||||
pattern: args.pattern,
|
||||
...args.path !== undefined ? { path: args.path } : {},
|
||||
...args.include !== undefined ? { include: args.include } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the fixed line-oriented `rg --json` command for one `grep` call. Every
|
||||
* model-controlled value ({@link GrepInput.pattern}, {@link GrepInput.path},
|
||||
* {@link GrepInput.include}) passes through {@link singleQuote}; the pattern
|
||||
* and include ride in `--flag=value` form and the target behind `--`, so a
|
||||
* leading-dash value can never be parsed as a flag.
|
||||
*
|
||||
* @param input - the validated arguments.
|
||||
* @returns the complete, shell-safe command string.
|
||||
*/
|
||||
export function buildGrepCommand(input: GrepInput): string {
|
||||
const parts = ['rg --json', `--regexp=${singleQuote(input.pattern)}`]
|
||||
if (input.include !== undefined) parts.push(`--glob=${singleQuote(input.include)}`)
|
||||
if (input.path !== undefined) parts.push('--', singleQuote(input.path))
|
||||
return parts.join(' ')
|
||||
}
|
||||
|
||||
/**
|
||||
* The uniform malformed-output failure: raw `rg --json` is an internal
|
||||
* transport, so a shape surprise is a search failure, not a partial result.
|
||||
*/
|
||||
function malformedRecord(detail: string, cause?: unknown): SearchError {
|
||||
return new SearchError(`grep received malformed ripgrep --json output (${detail})`, 'SEARCH_FAILED', cause !== undefined ? { cause } : undefined)
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse one `rg --json` NDJSON line into a match, `undefined` for the
|
||||
* non-match record types (`begin`/`end`/`context`/`summary`). A line that is
|
||||
* not JSON, or a `match` record missing its path / line number / line content,
|
||||
* throws {@link SearchError} `SEARCH_FAILED`. A match whose line is not valid
|
||||
* UTF-8 (ripgrep sends base64 `bytes` instead of `text`) yields a placeholder
|
||||
* preview rather than failing the whole search.
|
||||
*/
|
||||
function parseRecord(line: string): GrepMatch | undefined {
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(line)
|
||||
} catch (error: unknown) {
|
||||
throw malformedRecord('a line is not JSON', error)
|
||||
}
|
||||
if (typeof parsed !== 'object' || parsed === null) throw malformedRecord('a record is not an object')
|
||||
const record = parsed as { type?: unknown; data?: unknown }
|
||||
// Non-match record types (begin/end/context/summary — and any future type)
|
||||
// are transport framing, not results: skipped, not malformed.
|
||||
if (record.type !== 'match') return undefined
|
||||
if (typeof record.data !== 'object' || record.data === null) throw malformedRecord('a match record has no data')
|
||||
const data = record.data as { path?: unknown; line_number?: unknown; lines?: unknown }
|
||||
const pathText = typeof data.path === 'object' && data.path !== null ? (data.path as { text?: unknown }).text : undefined
|
||||
if (typeof pathText !== 'string') throw malformedRecord('a match record has no path text')
|
||||
if (typeof data.line_number !== 'number') throw malformedRecord('a match record has no line number')
|
||||
if (typeof data.lines !== 'object' || data.lines === null) throw malformedRecord('a match record has no line content')
|
||||
const lines = data.lines as { text?: unknown; bytes?: unknown }
|
||||
if (typeof lines.text === 'string') {
|
||||
return { path: pathText, lineNumber: data.line_number, line: lines.text.replace(/\r?\n$/, '') }
|
||||
}
|
||||
if (typeof lines.bytes === 'string') {
|
||||
return { path: pathText, lineNumber: data.line_number, line: '(line is not valid UTF-8)' }
|
||||
}
|
||||
throw malformedRecord('a match record has neither line text nor bytes')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse complete `rg --json` stdout into flat matches, in output order (ripgrep
|
||||
* emits one file's matches contiguously). Only `match` records are consumed.
|
||||
*
|
||||
* @param stdout - the complete raw `rg --json` stdout.
|
||||
* @returns the flat matches; empty for output with no match records.
|
||||
*/
|
||||
export function parseGrepMatches(stdout: string): GrepMatch[] {
|
||||
const matches: GrepMatch[] = []
|
||||
for (const line of stdout.split('\n')) {
|
||||
if (line.length === 0) continue
|
||||
const match = parseRecord(line)
|
||||
if (match !== undefined) matches.push(match)
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
/**
|
||||
* Bound one matched-line preview to `maxBytes` (UTF-8 boundary preserved) and
|
||||
* mark the cut. The cap is a per-line budget fact; the complete line stays in
|
||||
* the searched file for `read`.
|
||||
*
|
||||
* @param line - the matched line text (trailing newline already stripped).
|
||||
* @param maxBytes - the preview budget in bytes.
|
||||
* @returns the preview, suffixed with ` (line truncated)` when bytes were cut.
|
||||
*/
|
||||
export function previewLine(line: string, maxBytes: number): string {
|
||||
const retainer = new TextRetainer({ kind: 'head', maxBytes })
|
||||
retainer.push(line)
|
||||
const kept = retainer.finish()
|
||||
return kept.truncated ? `${kept.text} (line truncated)` : kept.text
|
||||
}
|
||||
|
||||
/** `match` / `matches` for a count. */
|
||||
function matchNoun(count: number): string {
|
||||
return count === 1 ? 'match' : 'matches'
|
||||
}
|
||||
|
||||
/**
|
||||
* Group flat matches by file (first-seen order) into the model-facing body:
|
||||
* each file's display path, then one `Line N: <text>` row per match.
|
||||
*
|
||||
* @param matches - the flat matches to render.
|
||||
* @returns the grouped body text.
|
||||
*/
|
||||
export function formatGrepMatches(matches: GrepMatch[]): string {
|
||||
const byFile = new Map<string, GrepMatch[]>()
|
||||
for (const match of matches) {
|
||||
const group = byFile.get(match.path)
|
||||
if (group !== undefined) group.push(match)
|
||||
else byFile.set(match.path, [match])
|
||||
}
|
||||
const sections: string[] = []
|
||||
for (const [path, group] of byFile) {
|
||||
sections.push(`${path}\n${group.map(m => `Line ${m.lineNumber}: ${m.line}`).join('\n')}`)
|
||||
}
|
||||
return sections.join('\n\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Format the model-facing `grep` result: a found-count header, the retained
|
||||
* matches grouped by file, then — when the result was capped — a footer
|
||||
* carrying either the formatted-spill recovery path or the could-not-save
|
||||
* explanation. The omitted count is a budget fact: the search itself completed.
|
||||
*
|
||||
* @param retained - the retention outcome over every parsed match.
|
||||
* @param spillPath - the saved complete-result path, or `undefined` when unsaved.
|
||||
* @returns the model-facing text.
|
||||
*/
|
||||
export function formatGrepOutput(retained: RetainedItems<GrepMatch>, spillPath: string | undefined): string {
|
||||
const header = retained.truncated
|
||||
? `Found ${retained.kept} of ${retained.seen} matches`
|
||||
: `Found ${retained.seen} ${matchNoun(retained.seen)}`
|
||||
const body = formatGrepMatches(retained.items)
|
||||
if (!retained.truncated) return `${header}\n\n${body}`
|
||||
const recovery = spillPath !== undefined
|
||||
? `Full grep result saved to: ${spillPath}. Use read with offset/limit to inspect it.`
|
||||
: 'The complete result could not be saved; narrow pattern, path, or include to see more.'
|
||||
return `${header}\n\n${body}\n\n(${recovery})`
|
||||
}
|
||||
|
||||
/**
|
||||
* Pending-call presentation: a search card titled by the pattern (and target /
|
||||
* include filter).
|
||||
*
|
||||
* @param args - the raw tool arguments; `pattern`, `path`, and `include` feed the title.
|
||||
* @returns the generic card view (`kind: 'search'`) shown while the call runs.
|
||||
*/
|
||||
export function presentGrepCall(args: { pattern: string; path?: string; include?: string }): GenericCallView {
|
||||
const where = args.path !== undefined ? ` in ${args.path}` : ''
|
||||
const filter = args.include !== undefined ? ` (${args.include})` : ''
|
||||
return { card: 'generic', title: `Grep ${args.pattern}${where}${filter}`, kind: 'search', rawInput: args.pattern }
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `grep` tool and its system-prompt guidance.
|
||||
*
|
||||
* @param ctx - the plugin context; registrations are effects scoped to it, and
|
||||
* execution uses its `bash` service.
|
||||
* @param caps - the deployment's resolved grep caps (plugin config after defaulting).
|
||||
*/
|
||||
export function applyGrepTool(ctx: Context, caps: GrepToolCaps): void {
|
||||
ctx.systemPrompt.section({
|
||||
name: 'tool:grep',
|
||||
order: 104,
|
||||
text: 'Use the grep tool — not shell grep or rg — to search file contents. Use read on a matched file when you need surrounding context.',
|
||||
})
|
||||
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'grep',
|
||||
description: 'Search file contents with a ripgrep regular expression. Returns matching lines with line numbers, grouped by file. '
|
||||
+ `Returns the first ${caps.maxMatches} matches inline; a capped result reports where the complete match list was saved. `
|
||||
+ 'Use read on a matched file for surrounding context.',
|
||||
parameters: {
|
||||
pattern: { type: 'string', required: true, description: 'Regular expression to search for (ripgrep syntax).' },
|
||||
path: { type: 'string', description: 'File or directory to search. Defaults to the session workspace; a relative path resolves against it.' },
|
||||
include: { type: 'string', description: 'One glob filter for which files to search (e.g. "*.ts", "*.{js,jsx}"). Not a list; negation is not supported.' },
|
||||
},
|
||||
timeoutMs: caps.timeoutMs,
|
||||
async execute(args, exec): Promise<ContentBlock[]> {
|
||||
const input = parseGrepArgs(args)
|
||||
const run = await runRipgrep(ctx, exec, 'grep', buildGrepCommand(input), caps.rawOutputMaxBytes)
|
||||
if (run.noMatches) return [{ type: 'text', text: 'No matches found' }]
|
||||
|
||||
const retainer = new ItemRetainer<GrepMatch>({ kind: 'head', maxItems: caps.maxMatches })
|
||||
const all: GrepMatch[] = []
|
||||
for (const raw of parseGrepMatches(run.stdout)) {
|
||||
const match: GrepMatch = {
|
||||
path: toWorkdirRelative(raw.path, run.workdir),
|
||||
lineNumber: raw.lineNumber,
|
||||
line: previewLine(raw.line, caps.maxLineBytes),
|
||||
}
|
||||
all.push(match)
|
||||
retainer.push(match)
|
||||
}
|
||||
const retained = retainer.finish()
|
||||
|
||||
// The spill file stores the FULL formatted match list (same grouped,
|
||||
// per-line-previewed shape the model saw), so read offset/limit pages the
|
||||
// same logical result; save only when the inline page omitted matches.
|
||||
const spillPath = retained.truncated
|
||||
? await trySaveFormattedResult(
|
||||
ctx,
|
||||
exec,
|
||||
'grep-results.txt',
|
||||
`Found ${all.length} ${matchNoun(all.length)}\n\n${formatGrepMatches(all)}`,
|
||||
)
|
||||
: undefined
|
||||
return [{ type: 'text', text: formatGrepOutput(retained, spillPath) }]
|
||||
},
|
||||
presentCall: presentGrepCall,
|
||||
}))
|
||||
}
|
||||
110
packages/fs/tool-fs-search/src/index.ts
Normal file
110
packages/fs/tool-fs-search/src/index.ts
Normal file
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* The model-facing filesystem discovery tool suite (`glob`, `grep`) over the
|
||||
* bash executor seam (`ctx.bash`). This single plugin registers both tools.
|
||||
*
|
||||
* ## Bash-backed, not a `ctx.fs` provider method
|
||||
*
|
||||
* Local workspace discovery is a process-backed `rg` workflow, so these tools
|
||||
* execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)` with fixed
|
||||
* ripgrep command templates — never `ctx.bash.start()`, never a model-visible
|
||||
* background task. The tool layer owns schemas, argument validation, shell
|
||||
* quoting ({@link module:@deepseek-ai/dsh-tool-fs-search/shell-quote}), result
|
||||
* parsing, retention, formatted-result spill, and timeout declaration; the
|
||||
* bash executor owns request defaulting/capping, subprocess execution,
|
||||
* process-group termination, environment scrubbing, raw output capture, and
|
||||
* backend substitution. The package injects `tools`, `systemPrompt`, and
|
||||
* `bash` — deliberately NOT `fs`, and `ctx.spillFiles` is read opportunistically
|
||||
* with `ctx.get()` because formatted-result spill is optional.
|
||||
*
|
||||
* Returned paths are displayed relative to the resolved bash workdir and are
|
||||
* follow-up-readable only in co-located deployments where the bash workdir and
|
||||
* the filesystem `read` root are the same workspace — a documented v1
|
||||
* deployment requirement, not runtime-validated.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { GLOB_MAX_RESULTS, applyGlobTool } from './glob.ts'
|
||||
import { GREP_MAX_LINE_BYTES, GREP_MAX_MATCHES, applyGrepTool } from './grep.ts'
|
||||
import { RAW_OUTPUT_MAX_BYTES, SEARCH_TIMEOUT_MS } from './search-core.ts'
|
||||
|
||||
export { GLOB_MAX_RESULTS, GLOB_VCS_EXCLUDES, applyGlobTool, buildGlobCommand, formatGlobOutput, parseGlobArgs, presentGlobCall } from './glob.ts'
|
||||
export type { GlobInput, GlobToolCaps } from './glob.ts'
|
||||
export {
|
||||
GREP_MAX_LINE_BYTES,
|
||||
GREP_MAX_MATCHES,
|
||||
applyGrepTool,
|
||||
buildGrepCommand,
|
||||
formatGrepMatches,
|
||||
formatGrepOutput,
|
||||
parseGrepArgs,
|
||||
parseGrepMatches,
|
||||
presentGrepCall,
|
||||
previewLine,
|
||||
} from './grep.ts'
|
||||
export type { GrepInput, GrepMatch, GrepToolCaps } from './grep.ts'
|
||||
export { RAW_OUTPUT_MAX_BYTES, SEARCH_TIMEOUT_MS, SearchError, runRipgrep, toWorkdirRelative, trySaveFormattedResult } from './search-core.ts'
|
||||
export type { RipgrepRun, SearchErrorCode } from './search-core.ts'
|
||||
export { singleQuote } from './shell-quote.ts'
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'tool-fs-search'
|
||||
|
||||
/** Services required by the search tool suite (`spillFiles` is optional, read via `ctx.get()`). */
|
||||
export const inject = ['tools', 'systemPrompt', 'bash']
|
||||
|
||||
/** Plugin config (all optional — `Config` supplies the defaults). */
|
||||
export interface Config {
|
||||
/** Max paths one `glob` call retains inline; later paths go to the formatted spill file. */
|
||||
globMaxResults?: number
|
||||
/** Max flat matches one `grep` call retains inline; later matches go to the formatted spill file. */
|
||||
grepMaxMatches?: number
|
||||
/** Max bytes retained for one matched-line preview (the cut preserves UTF-8 boundaries). */
|
||||
grepMaxLineBytes?: number
|
||||
/** Max complete raw `rg` stdout bytes a search will parse; larger raw output fails with `SEARCH_RAW_OUTPUT_OVERFLOW`. */
|
||||
rawOutputMaxBytes?: number
|
||||
/** Cooperative tool-call timeout budget (ms) on both tools, enforced by `@deepseek-ai/dsh-timeout-policy` through `exec.signal`. */
|
||||
timeoutMs?: number
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
globMaxResults: z.number().default(GLOB_MAX_RESULTS),
|
||||
grepMaxMatches: z.number().default(GREP_MAX_MATCHES),
|
||||
grepMaxLineBytes: z.number().default(GREP_MAX_LINE_BYTES),
|
||||
rawOutputMaxBytes: z.number().default(RAW_OUTPUT_MAX_BYTES),
|
||||
timeoutMs: z.number().default(SEARCH_TIMEOUT_MS),
|
||||
})
|
||||
|
||||
/** The shape after schemastery applied the defaults. */
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/** Every search cap counts items/bytes/milliseconds — a positive integer, or retention and timeout arithmetic misbehaves silently. */
|
||||
function assertPositiveInteger(name: string, value: number): void {
|
||||
if (!Number.isInteger(value) || value < 1) {
|
||||
throw new Error(`tool-fs-search: ${name} must be a positive integer`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Register the `glob`/`grep` filesystem discovery tool suite. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
// schemastery (Config) has already filled every defaulted field.
|
||||
const resolved = config as ResolvedConfig
|
||||
assertPositiveInteger('globMaxResults', resolved.globMaxResults)
|
||||
assertPositiveInteger('grepMaxMatches', resolved.grepMaxMatches)
|
||||
assertPositiveInteger('grepMaxLineBytes', resolved.grepMaxLineBytes)
|
||||
assertPositiveInteger('rawOutputMaxBytes', resolved.rawOutputMaxBytes)
|
||||
assertPositiveInteger('timeoutMs', resolved.timeoutMs)
|
||||
applyGlobTool(ctx, {
|
||||
maxResults: resolved.globMaxResults,
|
||||
rawOutputMaxBytes: resolved.rawOutputMaxBytes,
|
||||
timeoutMs: resolved.timeoutMs,
|
||||
})
|
||||
applyGrepTool(ctx, {
|
||||
maxMatches: resolved.grepMaxMatches,
|
||||
maxLineBytes: resolved.grepMaxLineBytes,
|
||||
rawOutputMaxBytes: resolved.rawOutputMaxBytes,
|
||||
timeoutMs: resolved.timeoutMs,
|
||||
})
|
||||
}
|
||||
257
packages/fs/tool-fs-search/src/search-core.ts
Normal file
257
packages/fs/tool-fs-search/src/search-core.ts
Normal file
@@ -0,0 +1,257 @@
|
||||
/**
|
||||
* Shared execution plumbing for the `glob` / `grep` search tools: the
|
||||
* package-owned `SEARCH_*` error vocabulary, one bash-seam run helper that
|
||||
* turns a fixed `rg` command into complete raw stdout, the best-effort
|
||||
* formatted-result spill handoff, and workdir-relative path display.
|
||||
*
|
||||
* Both tools execute through `ctx.bash.resolve(request)` → `ctx.bash.run(spec)`
|
||||
* as ordinary foreground tool calls — never `ctx.bash.start()`, never a
|
||||
* model-visible background task. Raw `rg` stdout is an internal transport
|
||||
* detail: when the executor truncates it, the ONLY recovery source is the
|
||||
* executor's local raw spill file, read here up to `rawOutputMaxBytes` and
|
||||
* never exposed to the model. The model-facing recovery artifact is the
|
||||
* formatted result saved through `ctx.spillFiles.saveText()`
|
||||
* ({@link trySaveFormattedResult}) — a different artifact from the bash raw
|
||||
* spill file.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/search-core
|
||||
*/
|
||||
|
||||
import { readFile, stat } from 'node:fs/promises'
|
||||
import { isAbsolute, relative, sep } from 'node:path'
|
||||
import type { Context } from 'cordis'
|
||||
import { HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import type { BashRunResult, CollectedOutput } from '@deepseek-ai/dsh-bash'
|
||||
import type { SaveTextSpill } from '@deepseek-ai/dsh-spill'
|
||||
import type { ToolExecution } from '@deepseek-ai/dsh-tools'
|
||||
|
||||
/**
|
||||
* Default cap on the complete raw `rg` stdout the tools will parse (the
|
||||
* `rawOutputMaxBytes` config), matching Claude Code's ripgrep raw buffer.
|
||||
*/
|
||||
export const RAW_OUTPUT_MAX_BYTES = 20_000_000
|
||||
|
||||
/**
|
||||
* Default cooperative tool-call timeout budget in milliseconds (the `timeoutMs`
|
||||
* config), attached to both tool definitions for
|
||||
* `@deepseek-ai/dsh-timeout-policy` to enforce through `exec.signal`.
|
||||
*/
|
||||
export const SEARCH_TIMEOUT_MS = 30_000
|
||||
|
||||
/**
|
||||
* Stable, machine-routable codes for search failures. Package-owned (not
|
||||
* `FsErrorCode`) because these tools are bash-backed discovery, not `ctx.fs`
|
||||
* provider operations: `SEARCH_INVALID_PATTERN` — ripgrep rejected the regex or
|
||||
* glob; `SEARCH_FAILED` — the search could not run or its output could not be
|
||||
* parsed (missing `rg`, inaccessible target, signal kill, malformed `--json`);
|
||||
* `SEARCH_RAW_OUTPUT_OVERFLOW` — raw `rg` output exceeded `rawOutputMaxBytes`
|
||||
* (or was truncated with no recovery file); `SEARCH_ABORTED` — the tool
|
||||
* timeout, caller cancellation, or the bash executor's own timeout cut the
|
||||
* search short.
|
||||
*/
|
||||
export type SearchErrorCode =
|
||||
| 'SEARCH_INVALID_PATTERN'
|
||||
| 'SEARCH_FAILED'
|
||||
| 'SEARCH_RAW_OUTPUT_OVERFLOW'
|
||||
| 'SEARCH_ABORTED'
|
||||
|
||||
/**
|
||||
* Typed search failure. Extends {@link HarnessError} so it carries a stable
|
||||
* {@link SearchErrorCode} and chains `cause`; the tool registry surfaces
|
||||
* `{ name, code }` on `isError` results so retry/permission/UI layers can
|
||||
* branch without parsing messages.
|
||||
*/
|
||||
export class SearchError extends HarnessError {
|
||||
override readonly code: SearchErrorCode
|
||||
|
||||
constructor(message: string, code: SearchErrorCode, options?: ErrorOptions) {
|
||||
super(message, code, options)
|
||||
this.code = code
|
||||
}
|
||||
}
|
||||
|
||||
/** The completed acquisition of one `rg` run: complete stdout plus the resolved workdir. */
|
||||
export interface RipgrepRun {
|
||||
/** Complete raw stdout — inline executor text, or the raw spill file's content. */
|
||||
stdout: string
|
||||
/** True when ripgrep exited 1: a successful search with zero results. */
|
||||
noMatches: boolean
|
||||
/** The resolved working directory the command ran in (the display-relativization base). */
|
||||
workdir: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The retained stderr tail as a diagnostic excerpt, with a truncation note when
|
||||
* the executor dropped bytes (the tool never reads `stderr.spillPath`).
|
||||
*/
|
||||
function stderrExcerpt(stderr: CollectedOutput): string {
|
||||
const text = stderr.text.trim()
|
||||
if (text.length === 0) return ''
|
||||
return stderr.truncated ? `${text} [stderr truncated]` : text
|
||||
}
|
||||
|
||||
/** Classify a nonzero-exit `rg` run into the search error vocabulary (invalid pattern vs missing `rg` vs everything else). */
|
||||
function classifyRunFailure(toolName: string, result: BashRunResult): SearchError {
|
||||
const stderr = stderrExcerpt(result.stderr)
|
||||
if (/regex parse error|error parsing glob/i.test(stderr)) {
|
||||
return new SearchError(`${toolName} pattern rejected by ripgrep: ${stderr}`, 'SEARCH_INVALID_PATTERN')
|
||||
}
|
||||
if (result.exitCode === 127 || /command not found/i.test(stderr)) {
|
||||
return new SearchError(`${toolName} requires ripgrep (rg) on the bash executor's PATH${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
}
|
||||
return new SearchError(`${toolName} search failed (exit ${result.exitCode})${stderr.length > 0 ? `: ${stderr}` : ''}`, 'SEARCH_FAILED')
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire the COMPLETE raw stdout of a finished run. Untruncated stdout is used
|
||||
* as-is; truncated stdout is recovered from the executor's local raw spill file
|
||||
* only when the complete file fits within `rawOutputMaxBytes`. A missing spill
|
||||
* path or an over-cap file is a clear failure telling the model to narrow the
|
||||
* search — never a silently-partial parse.
|
||||
*/
|
||||
async function completeStdout(toolName: string, result: BashRunResult, rawOutputMaxBytes: number): Promise<string> {
|
||||
if (!result.stdout.truncated) return result.stdout.text
|
||||
const narrow = 'narrow pattern, path, or include and retry'
|
||||
const spillPath = result.stdout.spillPath
|
||||
if (spillPath === undefined) {
|
||||
throw new SearchError(
|
||||
`${toolName} produced more raw output than the bash executor retained and no raw spill file is available; ${narrow}`,
|
||||
'SEARCH_RAW_OUTPUT_OVERFLOW',
|
||||
)
|
||||
}
|
||||
try {
|
||||
const { size } = await stat(spillPath)
|
||||
if (size > rawOutputMaxBytes) {
|
||||
throw new SearchError(
|
||||
`${toolName} produced ${size} bytes of raw output, over the ${rawOutputMaxBytes}-byte cap; ${narrow}`,
|
||||
'SEARCH_RAW_OUTPUT_OVERFLOW',
|
||||
)
|
||||
}
|
||||
return await readFile(spillPath, 'utf8')
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof SearchError) throw error
|
||||
throw new SearchError(`${toolName} could not read the executor's raw output spill file`, 'SEARCH_FAILED', { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one fixed `rg` command through the bash seam and return its complete raw
|
||||
* stdout. The bash request workdir is the calling agent's session cwd
|
||||
* (`exec.agent.session.header.cwd`) when available — mirroring `dsh-tool-bash` /
|
||||
* `dsh-tool-fs` — else omitted so the implementation's `resolve()` applies its
|
||||
* configured default. `exec.signal` is forwarded so the cooperative tool
|
||||
* timeout (`@deepseek-ai/dsh-timeout-policy`) and caller cancellation kill the
|
||||
* command; the bash backend's own timeout stays a second safety cap.
|
||||
*
|
||||
* Exit semantics are tool-owned: exit 0 is success with results, exit 1 is
|
||||
* success with zero results (`noMatches`), anything else throws a
|
||||
* {@link SearchError} (abort/timeout → `SEARCH_ABORTED`, invalid pattern →
|
||||
* `SEARCH_INVALID_PATTERN`, the rest → `SEARCH_FAILED` /
|
||||
* `SEARCH_RAW_OUTPUT_OVERFLOW`).
|
||||
*
|
||||
* @param ctx - the plugin context; execution uses its `bash` service.
|
||||
* @param exec - the tool-execution context; supplies the session cwd and the abort signal.
|
||||
* @param toolName - `glob` or `grep`, used in error messages.
|
||||
* @param command - the fully-quoted `rg` command string (every model value already through `singleQuote`).
|
||||
* @param rawOutputMaxBytes - cap on the complete raw stdout the tool will parse.
|
||||
* @returns the complete stdout, the zero-result flag, and the resolved workdir.
|
||||
*/
|
||||
export async function runRipgrep(
|
||||
ctx: Context,
|
||||
exec: ToolExecution,
|
||||
toolName: string,
|
||||
command: string,
|
||||
rawOutputMaxBytes: number,
|
||||
): Promise<RipgrepRun> {
|
||||
const cwd = exec.agent?.session.header.cwd
|
||||
const spec = ctx.bash.resolve({
|
||||
command,
|
||||
...cwd !== undefined ? { workdir: cwd } : {},
|
||||
...exec.signal ? { signal: exec.signal } : {},
|
||||
})
|
||||
const result = await ctx.bash.run(spec)
|
||||
if (result.aborted) {
|
||||
throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED')
|
||||
}
|
||||
if (result.timedOut) {
|
||||
throw new SearchError(`${toolName} timed out after ${result.timeoutMs}ms in the bash executor; narrow pattern, path, or include and retry`, 'SEARCH_ABORTED')
|
||||
}
|
||||
if (result.signal !== null || result.exitCode === null) {
|
||||
throw new SearchError(`${toolName} search command was killed by signal ${result.signal ?? '(unknown)'}`, 'SEARCH_FAILED')
|
||||
}
|
||||
if (result.exitCode !== 0 && result.exitCode !== 1) {
|
||||
throw classifyRunFailure(toolName, result)
|
||||
}
|
||||
const stdout = await completeStdout(toolName, result, rawOutputMaxBytes)
|
||||
return { stdout, noMatches: result.exitCode === 1, workdir: spec.workdir }
|
||||
}
|
||||
|
||||
/**
|
||||
* Map an `rg` output path to its display form: absolute paths inside the
|
||||
* resolved bash workdir become workdir-relative; everything else (relative
|
||||
* output, paths outside the workdir) passes through unchanged. Display-only —
|
||||
* returned paths are follow-up-readable in co-located bash/filesystem
|
||||
* deployments where both resolve the same workspace (the documented v1
|
||||
* deployment requirement).
|
||||
*
|
||||
* @param path - one path as ripgrep printed it.
|
||||
* @param workdir - the resolved bash workdir the command ran in.
|
||||
* @returns the workdir-relative display path when possible, else `path` unchanged.
|
||||
*/
|
||||
export function toWorkdirRelative(path: string, workdir: string): string {
|
||||
if (!isAbsolute(path)) return path
|
||||
const rel = relative(workdir, path)
|
||||
if (rel.length === 0) return '.'
|
||||
if (rel === '..' || rel.startsWith(`..${sep}`)) return path
|
||||
return rel
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort save of one COMPLETE formatted search result through
|
||||
* `ctx.spillFiles.saveText()` — the model-facing recovery path for a capped
|
||||
* result. `spillFiles` is read with `ctx.get()` (not static inject) because
|
||||
* formatted-result spill is optional; the spill owner is the calling agent's
|
||||
* session header id and the source is the tool execution identity. A missing
|
||||
* backend, a call with no session owner, or a `saveText()` rejection logs a
|
||||
* warning and returns `undefined` — the caller keeps the inline result and
|
||||
* reports that the complete result could not be saved; search success never
|
||||
* turns into `isError` because spill storage is unavailable.
|
||||
*
|
||||
* @param ctx - the plugin context; `spillFiles` is looked up opportunistically.
|
||||
* @param exec - the tool-execution context; supplies the owning session, tool name, and call id.
|
||||
* @param suggestedName - the backend-sanitized filename hint (e.g. `grep-results.txt`).
|
||||
* @param content - the complete formatted result to persist.
|
||||
* @returns the saved spill path, or `undefined` when the result could not be saved.
|
||||
*/
|
||||
export async function trySaveFormattedResult(
|
||||
ctx: Context,
|
||||
exec: ToolExecution,
|
||||
suggestedName: string,
|
||||
content: string,
|
||||
): Promise<string | undefined> {
|
||||
const sessionId = exec.agent?.session.header.id
|
||||
if (sessionId === undefined) {
|
||||
ctx.logger.warn(`tool-fs-search: no session owner for ${exec.name} result; complete result not saved`)
|
||||
return undefined
|
||||
}
|
||||
const spillFiles = ctx.get('spillFiles')
|
||||
if (!spillFiles) {
|
||||
ctx.logger.warn(`tool-fs-search: no ctx.spillFiles backend loaded; complete ${exec.name} result not saved`)
|
||||
return undefined
|
||||
}
|
||||
const save: SaveTextSpill = {
|
||||
owner: { sessionId },
|
||||
source: { toolName: exec.name, callId: exec.callId, label: 'result' },
|
||||
suggestedName,
|
||||
content,
|
||||
}
|
||||
try {
|
||||
const { path } = await spillFiles.saveText(save)
|
||||
return path
|
||||
} catch (error: unknown) {
|
||||
// Best-effort: a storage failure must never fail the search or hide the
|
||||
// inline result — the footer reports the unsaved remainder instead.
|
||||
ctx.logger.warn(`tool-fs-search: saveText failed for ${exec.name}: ${String(error)}; complete result not saved`)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
27
packages/fs/tool-fs-search/src/shell-quote.ts
Normal file
27
packages/fs/tool-fs-search/src/shell-quote.ts
Normal file
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* The one shell-quoting helper both search tools MUST route every
|
||||
* model-controlled value through before it enters an `rg` command string. The
|
||||
* bash seam (`ctx.bash`) accepts a command STRING, not an argv vector, so this
|
||||
* is the safety boundary that stops a `pattern`, `path`, or `include` from
|
||||
* breaking out of its argument and injecting shell syntax.
|
||||
*
|
||||
* Command builders in `glob.ts` / `grep.ts` must never hand-roll quoting or
|
||||
* concatenate an unquoted model value — they call {@link singleQuote}.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-fs-search/shell-quote
|
||||
*/
|
||||
|
||||
/**
|
||||
* POSIX single-quote a string for safe use as ONE shell word. Wraps the value
|
||||
* in single quotes and rewrites every embedded single quote as `'\''` (close
|
||||
* quote, an escaped literal quote, reopen quote). Inside single quotes the shell
|
||||
* treats every other byte literally — spaces, newlines, `$`, backticks, `;`,
|
||||
* `|`, `&`, glob metacharacters, and a leading `-` are all inert — so the result
|
||||
* is a single, injection-safe argument regardless of the input.
|
||||
*
|
||||
* @param value - the raw, possibly model-controlled string to quote.
|
||||
* @returns the value wrapped as one safe single-quoted shell word.
|
||||
*/
|
||||
export function singleQuote(value: string): string {
|
||||
return `'${value.replaceAll("'", "'\\''")}'`
|
||||
}
|
||||
161
packages/fs/tool-fs-search/tests/integration.spec.ts
Normal file
161
packages/fs/tool-fs-search/tests/integration.spec.ts
Normal file
@@ -0,0 +1,161 @@
|
||||
/**
|
||||
* Integration tests: the REAL local bash executor (`dsh-bash-local`) plus a
|
||||
* REAL ripgrep binary, exercised through `ctx.tools.execute()`. These verify
|
||||
* the WORLD — actual files on disk are discovered and grepped, hostile
|
||||
* patterns stay inert in a real shell, and real `rg` stderr classifies into
|
||||
* the `SEARCH_*` vocabulary. The whole suite self-skips when `rg` is not on
|
||||
* PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor
|
||||
* suite (tools.spec.ts) carries the coverage gate.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { mkdir, mkdtemp, rm, utimes, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
|
||||
import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
const hasRg = spawnSync('rg', ['--version'], { encoding: 'utf8' }).status === 0
|
||||
|
||||
let dir: string
|
||||
let ctx: Context
|
||||
|
||||
let callCounter = 0
|
||||
function call(name: string, args: unknown, agentObj?: object) {
|
||||
return ctx.tools.execute({
|
||||
callId: CallId(`it-${++callCounter}`),
|
||||
name,
|
||||
arguments: args,
|
||||
...agentObj ? { agent: agentObj as never } : {},
|
||||
})
|
||||
}
|
||||
|
||||
function text(result: { content: { type: string; text?: string }[] }): string {
|
||||
return result.content.filter(b => b.type === 'text').map(b => b.text).join('')
|
||||
}
|
||||
|
||||
describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', () => {
|
||||
beforeEach(async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-int-'))
|
||||
await mkdir(join(dir, 'src'), { recursive: true })
|
||||
await mkdir(join(dir, '.git'), { recursive: true })
|
||||
await mkdir(join(dir, 'spaced dir'), { recursive: true })
|
||||
await writeFile(join(dir, 'src', 'alpha.ts'), 'export const alpha = 1\n// TODO: refit alpha\n')
|
||||
await writeFile(join(dir, 'src', 'beta.ts'), 'export const beta = 2\n')
|
||||
await writeFile(join(dir, 'notes.md'), 'alpha appears here too\n')
|
||||
await writeFile(join(dir, '.hidden.ts'), 'export const hidden = 3\n')
|
||||
await writeFile(join(dir, '.git', 'config.ts'), 'never listed\n')
|
||||
await writeFile(join(dir, 'spaced dir', "wei'rd \"name\".ts"), 'const inside = true\n')
|
||||
// Deterministic --sort=modified order: alpha oldest, beta newest.
|
||||
await utimes(join(dir, 'src', 'alpha.ts'), new Date(2000, 0, 1), new Date(2000, 0, 1))
|
||||
await utimes(join(dir, 'src', 'beta.ts'), new Date(2020, 0, 1), new Date(2020, 0, 1))
|
||||
|
||||
ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(LocalBashExecutor, { cwd: dir, timeoutMs: 20_000 })
|
||||
await ctx.plugin(ToolFsSearch)
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(dir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
describe('glob', () => {
|
||||
it('discovers files by pattern, sorted by modification time, hidden included, .git excluded', async () => {
|
||||
const result = await call('glob', { pattern: '**/*.ts' })
|
||||
expect(result.isError).toBe(false)
|
||||
const paths = text(result).split('\n')
|
||||
expect(paths.indexOf('src/alpha.ts')).toBeLessThan(paths.indexOf('src/beta.ts'))
|
||||
expect(paths).toContain('.hidden.ts')
|
||||
expect(paths).toContain("spaced dir/wei'rd \"name\".ts")
|
||||
expect(paths).not.toContain('.git/config.ts')
|
||||
expect(paths).not.toContain('notes.md')
|
||||
})
|
||||
|
||||
it('scopes to a directory search root (path arg)', async () => {
|
||||
const result = await call('glob', { pattern: '*.ts', path: 'src' })
|
||||
expect(text(result).split('\n').sort()).toEqual(['src/alpha.ts', 'src/beta.ts'])
|
||||
})
|
||||
|
||||
it('reports zero discoveries as No files found', async () => {
|
||||
expect(text(await call('glob', { pattern: '*.nomatch' }))).toBe('No files found')
|
||||
})
|
||||
|
||||
it('classifies an invalid glob as SEARCH_INVALID_PATTERN', async () => {
|
||||
const result = await call('glob', { pattern: '[' })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ name: 'SearchError', code: 'SEARCH_INVALID_PATTERN' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('grep', () => {
|
||||
it('greps a directory tree with grouped, line-numbered output', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha' })
|
||||
expect(result.isError).toBe(false)
|
||||
const output = text(result)
|
||||
expect(output).toContain('Found 3 matches')
|
||||
expect(output).toContain('src/alpha.ts\nLine 1: export const alpha = 1\nLine 2: // TODO: refit alpha')
|
||||
expect(output).toContain('notes.md\nLine 1: alpha appears here too')
|
||||
})
|
||||
|
||||
it('greps a single FILE target', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha', path: 'notes.md' })
|
||||
expect(text(result)).toBe('Found 1 match\n\nnotes.md\nLine 1: alpha appears here too')
|
||||
})
|
||||
|
||||
it('greps a directory target with an include filter', async () => {
|
||||
const result = await call('grep', { pattern: 'alpha', path: '.', include: '*.ts' })
|
||||
const output = text(result)
|
||||
expect(output).toContain('alpha.ts')
|
||||
expect(output).not.toContain('notes.md')
|
||||
})
|
||||
|
||||
it('a hostile pattern stays inert (no command substitution, the world untouched)', async () => {
|
||||
const canary = join(dir, 'pwned')
|
||||
const result = await call('grep', { pattern: `$(touch ${canary})` })
|
||||
expect(result.isError).toBe(false) // exit 1: found nothing, executed nothing
|
||||
expect(text(result)).toBe('No matches found')
|
||||
expect(spawnSync('test', ['-e', canary]).status).not.toBe(0)
|
||||
})
|
||||
|
||||
it('a leading-dash pattern is a pattern, not a flag', async () => {
|
||||
await writeFile(join(dir, 'dashes.txt'), 'value --flag value\n')
|
||||
const result = await call('grep', { pattern: '--flag', path: 'dashes.txt' })
|
||||
expect(text(result)).toBe('Found 1 match\n\ndashes.txt\nLine 1: value --flag value')
|
||||
})
|
||||
|
||||
it('classifies a real rg regex error as SEARCH_INVALID_PATTERN', async () => {
|
||||
const result = await call('grep', { pattern: '(unclosed' })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_INVALID_PATTERN' })
|
||||
})
|
||||
|
||||
it('classifies a missing target as SEARCH_FAILED', async () => {
|
||||
const result = await call('grep', { pattern: 'x', path: 'no-such-dir' })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('per-session cwd', () => {
|
||||
it('resolves the search in the SESSION workspace, not the executor config cwd', async () => {
|
||||
const sessionDir = await mkdtemp(join(tmpdir(), 'dsh-search-session-'))
|
||||
try {
|
||||
await writeFile(join(sessionDir, 'only-here.ts'), 'const sessionFile = true\n')
|
||||
const agentObj = { session: { header: { id: 'session-int', cwd: sessionDir } } }
|
||||
const globbed = await call('glob', { pattern: '*.ts' }, agentObj)
|
||||
expect(text(globbed)).toBe('only-here.ts')
|
||||
const grepped = await call('grep', { pattern: 'sessionFile' }, agentObj)
|
||||
expect(text(grepped)).toContain('only-here.ts\nLine 1: const sessionFile = true')
|
||||
} finally {
|
||||
await rm(sessionDir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
50
packages/fs/tool-fs-search/tests/load-path.spec.ts
Normal file
50
packages/fs/tool-fs-search/tests/load-path.spec.ts
Normal file
@@ -0,0 +1,50 @@
|
||||
/**
|
||||
* Real-load-path guard for @deepseek-ai/dsh-tool-fs-search. `tool-fs-search` is
|
||||
* a NAMESPACE plugin with `inject` — so a stray `export default apply` would
|
||||
* make the cordis Loader's `unwrapExports` (`exports.default ?? exports`)
|
||||
* collapse the module to the bare `apply` function, DROPPING `inject`. The
|
||||
* plugin would then read `ctx.bash` without having injected it and throw
|
||||
* `cannot get property … without inject` the moment it loads (postmortem 0001).
|
||||
*
|
||||
* A hand-built `ctx.plugin({ apply, inject })` mount CANNOT catch that — it
|
||||
* bypasses `unwrapExports`. So this test unwraps the module through the REAL
|
||||
* `Loader.prototype.unwrapExports` and mounts the result over a bash executor,
|
||||
* exercising the exact path the Loader uses. Prove the guard bites: add
|
||||
* `export default apply` to `src/index.ts`, watch this go red, revert.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
|
||||
import * as toolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
describe('dsh-tool-fs-search real-load-path guard', () => {
|
||||
it('has no default export and keeps name/inject/Config through unwrapExports', () => {
|
||||
expect('default' in toolFsSearch).toBe(false)
|
||||
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolFsSearch) as Record<string, unknown>
|
||||
expect(unwrapped).toBe(toolFsSearch)
|
||||
expect(unwrapped.name).toBe('tool-fs-search')
|
||||
expect(unwrapped.inject).toEqual(['tools', 'systemPrompt', 'bash'])
|
||||
expect(typeof unwrapped.Config).toBe('function')
|
||||
expect(typeof unwrapped.apply).toBe('function')
|
||||
})
|
||||
|
||||
it('boots over ctx.bash through the unwrapped module without an inject error', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(LocalBashExecutor, {})
|
||||
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolFsSearch) as Parameters<Context['plugin']>[0]
|
||||
// A collapsed export shape (dropped inject) would throw "without inject" here.
|
||||
const fiber = await ctx.plugin(unwrapped)
|
||||
expect(ctx.tools.schemas().map(s => s.name)).toEqual(expect.arrayContaining(['glob', 'grep']))
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
59
packages/fs/tool-fs-search/tests/shell-quote.spec.ts
Normal file
59
packages/fs/tool-fs-search/tests/shell-quote.spec.ts
Normal file
@@ -0,0 +1,59 @@
|
||||
/**
|
||||
* Unit tests for the shell-quoting safety boundary, plus a REAL round-trip:
|
||||
* every adversarial value, quoted, must survive `bash -c "printf '%s' <quoted>"`
|
||||
* byte-for-byte — proving the quoting is inert in an actual shell, not just
|
||||
* against a mental model of one.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { singleQuote } from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
/** Adversarial values a model could pass as pattern / path / include. */
|
||||
const HOSTILE: readonly string[] = [
|
||||
'plain',
|
||||
'with spaces',
|
||||
"it's got 'quotes'",
|
||||
'"double quoted"',
|
||||
'$(rm -rf /tmp/nope)',
|
||||
'`touch /tmp/nope`',
|
||||
'$HOME and ${PATH}',
|
||||
'semi;colon && chain || pipe | bg &',
|
||||
'newline\nin the middle',
|
||||
'-leading-dash',
|
||||
'--leading-double-dash',
|
||||
'*?[a-z]{x,y}',
|
||||
'!bang',
|
||||
'\\backslash\\',
|
||||
'~tilde',
|
||||
'# not a comment',
|
||||
'>redirect <input 2>&1',
|
||||
]
|
||||
|
||||
describe('singleQuote', () => {
|
||||
it('wraps a plain value in single quotes', () => {
|
||||
expect(singleQuote('abc')).toBe("'abc'")
|
||||
})
|
||||
|
||||
it("rewrites embedded single quotes as '\\''", () => {
|
||||
expect(singleQuote("a'b")).toBe("'a'\\''b'")
|
||||
expect(singleQuote("''")).toBe("''\\'''\\'''")
|
||||
})
|
||||
|
||||
it.each(HOSTILE.map(value => [JSON.stringify(value), value] as const))(
|
||||
'round-trips %s through a real bash -c unchanged',
|
||||
(_label, value) => {
|
||||
const result = spawnSync('bash', ['-c', `printf '%s' ${singleQuote(value)}`], { encoding: 'utf8' })
|
||||
expect(result.status).toBe(0)
|
||||
expect(result.stdout).toBe(value)
|
||||
},
|
||||
)
|
||||
|
||||
it('a quoted command substitution does not execute (the world stays untouched)', () => {
|
||||
const canary = `/tmp/dsh-quote-canary-${process.pid}`
|
||||
const result = spawnSync('bash', ['-c', `printf '%s' ${singleQuote(`$(touch ${canary})`)}`], { encoding: 'utf8' })
|
||||
expect(result.stdout).toBe(`$(touch ${canary})`)
|
||||
// The canary file must NOT exist — the substitution stayed literal.
|
||||
expect(spawnSync('test', ['-e', canary]).status).not.toBe(0)
|
||||
})
|
||||
})
|
||||
623
packages/fs/tool-fs-search/tests/tools.spec.ts
Normal file
623
packages/fs/tool-fs-search/tests/tools.spec.ts
Normal file
@@ -0,0 +1,623 @@
|
||||
/**
|
||||
* Consumer-surface tests for the search tools over a FAKE bash executor and a
|
||||
* FAKE spill backend, exercised through `ctx.tools.execute()` so nothing
|
||||
* bypasses the tool registry. The fake executor makes every seam outcome
|
||||
* scriptable — truncated stdout with/without a raw spill file, abort/timeout,
|
||||
* signal kills, ripgrep exit codes — so these tests verify schemas, argument
|
||||
* validation, shell-safe command construction, workdir derivation, signal
|
||||
* forwarding, `SEARCH_*` error classification, retention, formatted-result
|
||||
* spill handoff, and the no-background-task invariant. Real-`rg` behavior is
|
||||
* pinned separately in integration.spec.ts.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import { BashExecutor } from '@deepseek-ai/dsh-bash'
|
||||
import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskId, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash'
|
||||
import { SpillFiles, SpillPath } from '@deepseek-ai/dsh-spill'
|
||||
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
|
||||
import {
|
||||
buildGlobCommand,
|
||||
buildGrepCommand,
|
||||
formatGrepMatches,
|
||||
parseGrepMatches,
|
||||
presentGlobCall,
|
||||
presentGrepCall,
|
||||
previewLine,
|
||||
toWorkdirRelative,
|
||||
} from '@deepseek-ai/dsh-tool-fs-search'
|
||||
|
||||
/** A successful run result over the given stdout; overrides script the failure shapes. */
|
||||
function runResult(stdout: string, overrides?: Partial<BashRunResult>): BashRunResult {
|
||||
return {
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
timeoutMs: 60_000,
|
||||
stdout: { text: stdout, truncated: false },
|
||||
stderr: { text: '', truncated: false },
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A scriptable fake executor: `resolve()` mirrors the real request→spec
|
||||
* defaulting (workdir falls back to `/work`), `run()` returns whatever the
|
||||
* test armed via `handler`, and `start()` throws — the search tools must NEVER
|
||||
* create a background task.
|
||||
*/
|
||||
class FakeBash extends BashExecutor {
|
||||
requests: BashExecRequest[] = []
|
||||
specs: BashExecSpec[] = []
|
||||
startCalls = 0
|
||||
handler: (spec: BashExecSpec) => BashRunResult = () => runResult('')
|
||||
|
||||
override resolve(request: BashExecRequest): BashExecSpec {
|
||||
this.requests.push(request)
|
||||
return {
|
||||
command: request.command,
|
||||
workdir: request.workdir ?? '/work',
|
||||
timeoutMs: request.timeoutMs ?? 60_000,
|
||||
signal: request.signal,
|
||||
owner: request.owner,
|
||||
}
|
||||
}
|
||||
override run(spec: BashExecSpec): Promise<BashRunResult> {
|
||||
this.specs.push(spec)
|
||||
return Promise.resolve(this.handler(spec))
|
||||
}
|
||||
override start(): BashTask {
|
||||
this.startCalls++
|
||||
throw new Error('search tools must never start a background task')
|
||||
}
|
||||
override get(): BashTask | undefined {
|
||||
return undefined
|
||||
}
|
||||
override ownerOf(): OwnerToken | undefined {
|
||||
return undefined
|
||||
}
|
||||
override list(): BashTask[] {
|
||||
return []
|
||||
}
|
||||
override readOutput(id: BashTaskId): BashTaskRead {
|
||||
throw new Error(`unknown bash task ${id}`)
|
||||
}
|
||||
override kill(id: BashTaskId): boolean {
|
||||
throw new Error(`unknown bash task ${id}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** A recording spill backend; arm `failWith` to script a storage failure. */
|
||||
class FakeSpill extends SpillFiles {
|
||||
saves: SaveTextSpill[] = []
|
||||
failWith?: Error
|
||||
|
||||
override saveText(input: SaveTextSpill): Promise<SpillRef> {
|
||||
if (this.failWith) return Promise.reject(this.failWith)
|
||||
this.saves.push(input)
|
||||
return Promise.resolve({ path: SpillPath(`/spill/${input.suggestedName}`), bytes: Buffer.byteLength(input.content, 'utf8') })
|
||||
}
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
config?: ToolFsSearch.Config
|
||||
spill?: boolean
|
||||
}
|
||||
|
||||
async function setup(options: SetupOptions = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(FakeBash)
|
||||
if (options.spill === true) await ctx.plugin(FakeSpill)
|
||||
const fiber = await ctx.plugin(ToolFsSearch, options.config)
|
||||
const bash = ctx.bash as FakeBash
|
||||
const spill = options.spill === true ? ctx.get('spillFiles') as FakeSpill : undefined
|
||||
return { ctx, bash, spill, fiber }
|
||||
}
|
||||
|
||||
/** A stand-in agent whose session header carries the given cwd (and a stable id). */
|
||||
const agent = (cwd?: string) => ({ session: { header: { id: 'session-1', ...cwd !== undefined ? { cwd } : {} } } })
|
||||
|
||||
let callCounter = 0
|
||||
function call(ctx: Context, name: string, args: unknown, options: { agent?: object; signal?: AbortSignal } = {}) {
|
||||
return ctx.tools.execute({
|
||||
callId: CallId(`call-${++callCounter}`),
|
||||
name,
|
||||
arguments: args,
|
||||
...options.agent ? { agent: options.agent as never } : {},
|
||||
...options.signal ? { signal: options.signal } : {},
|
||||
})
|
||||
}
|
||||
|
||||
function text(result: { content: { type: string; text?: string }[] }): string {
|
||||
return result.content.filter(b => b.type === 'text').map(b => b.text).join('')
|
||||
}
|
||||
|
||||
/** One rg --json match record line. */
|
||||
function matchLine(path: string, lineNumber: number, lineText: string): string {
|
||||
return JSON.stringify({ type: 'match', data: { path: { text: path }, lines: { text: lineText }, line_number: lineNumber, absolute_offset: 0, submatches: [] } })
|
||||
}
|
||||
|
||||
describe('registration', () => {
|
||||
it('registers glob and grep with their prompt sections', async () => {
|
||||
const { ctx } = await setup()
|
||||
expect(ctx.tools.schemas().map(s => s.name).sort()).toEqual(['glob', 'grep'])
|
||||
const prompt = renderPrompt(await ctx.systemPrompt.assemble())
|
||||
expect(prompt).toContain('Use the glob tool')
|
||||
expect(prompt).toContain('Use the grep tool')
|
||||
})
|
||||
|
||||
it('stays pending until ctx.bash exists (inject)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(ToolFsSearch) // no bash executor
|
||||
expect(ctx.tools.schemas()).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('unregisters everything on fiber disposal (HMR safety)', async () => {
|
||||
const { ctx, fiber } = await setup()
|
||||
expect(ctx.tools.schemas()).toHaveLength(2)
|
||||
await fiber.dispose()
|
||||
expect(ctx.tools.schemas()).toHaveLength(0)
|
||||
const sections = (await ctx.systemPrompt.assemble()).sections.map(s => s.name)
|
||||
expect(sections).not.toContain('tool:glob')
|
||||
expect(sections).not.toContain('tool:grep')
|
||||
})
|
||||
|
||||
it('attaches the configured timeoutMs to both tool definitions', async () => {
|
||||
const { ctx } = await setup({ config: { timeoutMs: 5000 } })
|
||||
expect(ctx.tools.get('glob')?.timeoutMs).toBe(5000)
|
||||
expect(ctx.tools.get('grep')?.timeoutMs).toBe(5000)
|
||||
})
|
||||
|
||||
it('defaults the timeout budget to 30 seconds', async () => {
|
||||
const { ctx } = await setup()
|
||||
expect(ctx.tools.get('glob')?.timeoutMs).toBe(30_000)
|
||||
expect(ctx.tools.get('grep')?.timeoutMs).toBe(30_000)
|
||||
})
|
||||
})
|
||||
|
||||
describe('config validation', () => {
|
||||
it.each([
|
||||
['globMaxResults', { globMaxResults: 0 }],
|
||||
['grepMaxMatches', { grepMaxMatches: -1 }],
|
||||
['grepMaxLineBytes', { grepMaxLineBytes: 1.5 }],
|
||||
['rawOutputMaxBytes', { rawOutputMaxBytes: 0 }],
|
||||
['timeoutMs', { timeoutMs: -100 }],
|
||||
] as const)('rejects a non-positive or fractional %s at load', async (name, config) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(FakeBash)
|
||||
await expect(ctx.plugin(ToolFsSearch, config)).rejects.toThrow(new RegExp(`tool-fs-search: ${name} must be a positive integer`))
|
||||
})
|
||||
})
|
||||
|
||||
describe('command construction (shell-safe)', () => {
|
||||
it('glob: fixed rg --files template with quoted pattern and VCS excludes', () => {
|
||||
const command = buildGlobCommand({ pattern: '**/*.ts' })
|
||||
expect(command).toBe(
|
||||
"rg --files --glob='**/*.ts' --sort=modified --no-ignore --hidden "
|
||||
+ "--glob='!**/.git' --glob='!**/.svn' --glob='!**/.hg' --glob='!**/.bzr' --glob='!**/.jj' --glob='!**/.sl'",
|
||||
)
|
||||
})
|
||||
|
||||
it('glob: the search root rides behind -- and is quoted', () => {
|
||||
const command = buildGlobCommand({ pattern: '*.md', path: 'docs dir' })
|
||||
expect(command).toContain("-- 'docs dir'")
|
||||
})
|
||||
|
||||
it('grep: fixed rg --json template with the pattern in --regexp= form', () => {
|
||||
expect(buildGrepCommand({ pattern: 'foo.*bar' })).toBe("rg --json --regexp='foo.*bar'")
|
||||
})
|
||||
|
||||
it('grep: include and path are quoted, include in --glob= form, path behind --', () => {
|
||||
const command = buildGrepCommand({ pattern: 'x', path: '-leading-dash', include: '*.{ts,tsx}' })
|
||||
expect(command).toBe("rg --json --regexp='x' --glob='*.{ts,tsx}' -- '-leading-dash'")
|
||||
})
|
||||
|
||||
it.each([
|
||||
['a command-substitution pattern', '$(rm -rf /)', "'$(rm -rf /)'"],
|
||||
['a backtick pattern', '`touch pwned`', "'`touch pwned`'"],
|
||||
['a pattern with double quotes and spaces', 'say "hi there"', '\'say "hi there"\''],
|
||||
['a pattern with single quotes', "it's", '\'it\'\\\'\'s\''],
|
||||
['a pattern with newlines', 'a\nb', "'a\nb'"],
|
||||
['a leading-dash pattern', '--flag', "'--flag'"],
|
||||
['glob metacharacters', '*?[a-z]{x,y}', "'*?[a-z]{x,y}'"],
|
||||
])('quotes %s into one inert shell word', (_label, raw, quoted) => {
|
||||
expect(buildGrepCommand({ pattern: raw })).toBe(`rg --json --regexp=${quoted}`)
|
||||
})
|
||||
})
|
||||
|
||||
describe('workdir derivation and signal forwarding', () => {
|
||||
it('forwards the session cwd as the request workdir', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('a.ts\n')
|
||||
await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/sessions/s1') })
|
||||
expect(bash.requests[0]?.workdir).toBe('/sessions/s1')
|
||||
expect(bash.specs[0]?.workdir).toBe('/sessions/s1')
|
||||
})
|
||||
|
||||
it('omits the request workdir without a session cwd so resolve() defaults apply', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('a.ts\n')
|
||||
await call(ctx, 'glob', { pattern: '*' }, { agent: agent() })
|
||||
expect(bash.requests[0]).not.toHaveProperty('workdir')
|
||||
expect(bash.specs[0]?.workdir).toBe('/work')
|
||||
// A non-agent caller takes the same default path.
|
||||
await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(bash.requests[1]).not.toHaveProperty('workdir')
|
||||
})
|
||||
|
||||
it('forwards exec.signal into the bash spec (the abort reaches the backend)', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
bash.handler = spec => runResult('', { aborted: spec.signal?.aborted === true })
|
||||
const result = await call(ctx, 'grep', { pattern: 'x' }, { signal: controller.signal })
|
||||
expect(bash.specs[0]?.signal).toBe(controller.signal)
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ name: 'SearchError', code: 'SEARCH_ABORTED' })
|
||||
expect(text(result)).toContain('aborted')
|
||||
})
|
||||
|
||||
it('reports the bash executor timeout as SEARCH_ABORTED with the budget', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { timedOut: true, timeoutMs: 1234, exitCode: null, signal: 'SIGTERM' })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_ABORTED' })
|
||||
expect(text(result)).toContain('timed out after 1234ms')
|
||||
})
|
||||
})
|
||||
|
||||
describe('exit semantics and failure classification', () => {
|
||||
it('exit 1 is a successful empty search', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 1 })
|
||||
const glob = await call(ctx, 'glob', { pattern: '*.nope' })
|
||||
expect(glob.isError).toBe(false)
|
||||
expect(text(glob)).toBe('No files found')
|
||||
const grep = await call(ctx, 'grep', { pattern: 'nope' })
|
||||
expect(grep.isError).toBe(false)
|
||||
expect(text(grep)).toBe('No matches found')
|
||||
})
|
||||
|
||||
it('a regex parse error classifies as SEARCH_INVALID_PATTERN', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: regex parse error:\n (\nerror: unclosed group', truncated: false } })
|
||||
const result = await call(ctx, 'grep', { pattern: '(' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_INVALID_PATTERN' })
|
||||
expect(text(result)).toContain('regex parse error')
|
||||
})
|
||||
|
||||
it('a glob parse error classifies as SEARCH_INVALID_PATTERN', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: error parsing glob \'[\': unclosed character class', truncated: false } })
|
||||
const result = await call(ctx, 'glob', { pattern: '[' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_INVALID_PATTERN' })
|
||||
})
|
||||
|
||||
it('a missing rg binary classifies as SEARCH_FAILED naming ripgrep', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 127, stderr: { text: 'bash: line 1: rg: command not found', truncated: false } })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
expect(text(result)).toContain('requires ripgrep (rg)')
|
||||
// The same classification holds from either evidence alone: the 127 exit
|
||||
// with silent stderr, or a shell's command-not-found text on another exit.
|
||||
bash.handler = () => runResult('', { exitCode: 127 })
|
||||
expect(text(await call(ctx, 'glob', { pattern: '*' }))).toContain('requires ripgrep (rg)')
|
||||
bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'sh: rg: command not found', truncated: false } })
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x' }))).toContain('requires ripgrep (rg)')
|
||||
})
|
||||
|
||||
it('other nonzero exits are SEARCH_FAILED carrying the stderr excerpt', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'rg: missing.dir: IO error: no such file or directory', truncated: false } })
|
||||
const result = await call(ctx, 'grep', { pattern: 'x', path: 'missing.dir' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
expect(text(result)).toContain('IO error')
|
||||
})
|
||||
|
||||
it('a nonzero exit with EMPTY stderr still reports the exit code', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 3 })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
expect(text(result)).toContain('exit 3')
|
||||
})
|
||||
|
||||
it('truncated stderr gains a truncation note and stderr.spillPath is never read', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', {
|
||||
exitCode: 2,
|
||||
stderr: { text: 'tail of diagnostics', truncated: true, spillPath: '/does/not/exist-and-never-read' },
|
||||
})
|
||||
const result = await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(text(result)).toContain('tail of diagnostics [stderr truncated]')
|
||||
})
|
||||
|
||||
it('a signal kill (not timeout, not abort) is SEARCH_FAILED', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: null, signal: 'SIGKILL' })
|
||||
const result = await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
expect(text(result)).toContain('SIGKILL')
|
||||
})
|
||||
|
||||
it('a null exit with no signal (defensive) is SEARCH_FAILED', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: null, signal: null })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('raw output acquisition', () => {
|
||||
let dir: string
|
||||
afterEach(async () => {
|
||||
await rm(dir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
it('parses the complete raw spill file when stdout is truncated', async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-raw-'))
|
||||
const spillPath = join(dir, 'raw.txt')
|
||||
await writeFile(spillPath, 'one.ts\ntwo.ts\nthree.ts\n')
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { stdout: { text: 'one.ts\n', truncated: true, spillPath } })
|
||||
const result = await call(ctx, 'glob', { pattern: '*.ts' })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(text(result)).toBe('one.ts\ntwo.ts\nthree.ts')
|
||||
})
|
||||
|
||||
it('fails with SEARCH_RAW_OUTPUT_OVERFLOW when the raw spill file exceeds the cap', async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-raw-'))
|
||||
const spillPath = join(dir, 'raw.txt')
|
||||
await writeFile(spillPath, 'x'.repeat(64))
|
||||
const { ctx, bash } = await setup({ config: { rawOutputMaxBytes: 16 } })
|
||||
bash.handler = () => runResult('', { stdout: { text: 'x', truncated: true, spillPath } })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_RAW_OUTPUT_OVERFLOW' })
|
||||
expect(text(result)).toContain('narrow pattern, path, or include')
|
||||
})
|
||||
|
||||
it('fails with SEARCH_RAW_OUTPUT_OVERFLOW when truncated stdout has no spill path', async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-raw-'))
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { stdout: { text: 'partial', truncated: true } })
|
||||
const result = await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_RAW_OUTPUT_OVERFLOW' })
|
||||
})
|
||||
|
||||
it('fails with SEARCH_FAILED when the raw spill file cannot be read', async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'dsh-search-raw-'))
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { stdout: { text: 'partial', truncated: true, spillPath: join(dir, 'gone.txt') } })
|
||||
const result = await call(ctx, 'glob', { pattern: '*' })
|
||||
expect(result.error).toMatchObject({ code: 'SEARCH_FAILED' })
|
||||
expect(text(result)).toContain('raw output spill file')
|
||||
})
|
||||
})
|
||||
|
||||
describe('glob results', () => {
|
||||
it('lists workdir-relative paths (absolute output under the workdir is relativized)', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('/sessions/s1/src/a.ts\n/elsewhere/b.ts\nrel/c.ts\n')
|
||||
const result = await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/sessions/s1') })
|
||||
expect(text(result)).toBe('src/a.ts\n/elsewhere/b.ts\nrel/c.ts')
|
||||
})
|
||||
|
||||
it('validates arguments (blank pattern, blank path)', async () => {
|
||||
const { ctx } = await setup()
|
||||
expect(text(await call(ctx, 'glob', { pattern: ' ' }))).toContain('pattern must be a non-empty string')
|
||||
expect(text(await call(ctx, 'glob', { pattern: '*', path: ' ' }))).toContain('path must be a non-empty string')
|
||||
})
|
||||
|
||||
it('caps at globMaxResults and saves the FULL sorted list through spillFiles', async () => {
|
||||
const { ctx, bash, spill } = await setup({ config: { globMaxResults: 2 }, spill: true })
|
||||
bash.handler = () => runResult('a.ts\nb.ts\nc.ts\nd.ts\n')
|
||||
const result = await call(ctx, 'glob', { pattern: '*.ts' }, { agent: agent('/w') })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(text(result)).toBe('a.ts\nb.ts\n\n(Showing 2 of 4 paths. Full sorted result saved to: /spill/glob-results.txt. Use read with offset/limit to inspect it.)')
|
||||
expect(spill?.saves).toHaveLength(1)
|
||||
expect(spill?.saves[0]).toMatchObject({
|
||||
owner: { sessionId: 'session-1' },
|
||||
source: { toolName: 'glob', label: 'result' },
|
||||
suggestedName: 'glob-results.txt',
|
||||
content: 'a.ts\nb.ts\nc.ts\nd.ts',
|
||||
})
|
||||
expect(spill?.saves[0]?.source.callId).toBeDefined()
|
||||
})
|
||||
|
||||
it('does not create a spill file when the result fits inline', async () => {
|
||||
const { ctx, bash, spill } = await setup({ spill: true })
|
||||
bash.handler = () => runResult('a.ts\nb.ts\n')
|
||||
const result = await call(ctx, 'glob', { pattern: '*' }, { agent: agent('/w') })
|
||||
expect(text(result)).toBe('a.ts\nb.ts')
|
||||
expect(spill?.saves).toHaveLength(0)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['no spill backend loaded', { fail: false, spill: false, ownerless: false }],
|
||||
['saveText fails', { fail: true, spill: true, ownerless: false }],
|
||||
['no session owner', { fail: false, spill: true, ownerless: true }],
|
||||
])('keeps the inline page and reports the unsaved remainder when %s', async (_label, mode) => {
|
||||
const { ctx, bash, spill } = await setup({ config: { globMaxResults: 1 }, spill: mode.spill })
|
||||
if (mode.fail && spill) spill.failWith = new Error('disk full')
|
||||
bash.handler = () => runResult('a.ts\nb.ts\n')
|
||||
const result = await call(ctx, 'glob', { pattern: '*' }, mode.ownerless ? {} : { agent: agent('/w') })
|
||||
expect(result.isError).toBe(false) // spill unavailability never fails the search
|
||||
expect(text(result)).toBe('a.ts\n\n(Showing 1 of 2 paths. The complete result could not be saved; narrow pattern or path to see more.)')
|
||||
})
|
||||
})
|
||||
|
||||
describe('grep results', () => {
|
||||
it('groups matches by file with line numbers', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult([
|
||||
JSON.stringify({ type: 'begin', data: { path: { text: 'a.ts' } } }),
|
||||
matchLine('a.ts', 3, 'const x = 1\n'),
|
||||
matchLine('a.ts', 9, 'const y = 2\n'),
|
||||
JSON.stringify({ type: 'end', data: { path: { text: 'a.ts' } } }),
|
||||
matchLine('b.ts', 1, 'const z = 3'),
|
||||
JSON.stringify({ type: 'summary', data: {} }),
|
||||
'',
|
||||
].join('\n'))
|
||||
const result = await call(ctx, 'grep', { pattern: 'const' })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(text(result)).toBe('Found 3 matches\n\na.ts\nLine 3: const x = 1\nLine 9: const y = 2\n\nb.ts\nLine 1: const z = 3')
|
||||
})
|
||||
|
||||
it('reports a single match in the singular', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult(`${matchLine('a.ts', 1, 'hit')}\n`)
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'hit' }))).toBe('Found 1 match\n\na.ts\nLine 1: hit')
|
||||
})
|
||||
|
||||
it('relativizes absolute match paths against the resolved workdir', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult(`${matchLine('/sessions/s1/deep/a.ts', 2, 'hit')}\n`)
|
||||
const result = await call(ctx, 'grep', { pattern: 'hit', path: '/sessions/s1' }, { agent: agent('/sessions/s1') })
|
||||
expect(text(result)).toContain('deep/a.ts\nLine 2: hit')
|
||||
})
|
||||
|
||||
it('previews a long matched line at grepMaxLineBytes preserving UTF-8', async () => {
|
||||
const { ctx, bash } = await setup({ config: { grepMaxLineBytes: 7 } })
|
||||
// 'héllo wörld' cut at 7 bytes lands mid-'é'? h(1)é(2)l(1)l(1)o(1)=6, space=7 → clean cut at 7.
|
||||
// Use a multibyte straddle instead: 'aé' repeated — cut at 7 bytes: a(1)é(2)a(1)é(2)=6 +a(1)=7 → next é straddles: trimmed.
|
||||
bash.handler = () => runResult(`${matchLine('a.txt', 1, 'aéaéaéaé')}\n`)
|
||||
const result = await call(ctx, 'grep', { pattern: 'a' })
|
||||
expect(text(result)).toContain('Line 1: aéaéa (line truncated)')
|
||||
})
|
||||
|
||||
it('renders a non-UTF-8 line (rg bytes form) as a placeholder instead of failing', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
const record = JSON.stringify({ type: 'match', data: { path: { text: 'bin.dat' }, lines: { bytes: 'AAECww==' }, line_number: 4 } })
|
||||
bash.handler = () => runResult(`${record}\n`)
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x' }))).toContain('Line 4: (line is not valid UTF-8)')
|
||||
})
|
||||
|
||||
it('strips a CRLF terminator from the matched line text', () => {
|
||||
const matches = parseGrepMatches(`${matchLine('a.txt', 1, 'windows line\r\n')}\n`)
|
||||
expect(matches[0]?.line).toBe('windows line')
|
||||
})
|
||||
|
||||
it('caps at grepMaxMatches and spills the full formatted match list', async () => {
|
||||
const { ctx, bash, spill } = await setup({ config: { grepMaxMatches: 2 }, spill: true })
|
||||
bash.handler = () => runResult([
|
||||
matchLine('a.ts', 1, 'one'),
|
||||
matchLine('a.ts', 2, 'two'),
|
||||
matchLine('b.ts', 3, 'three'),
|
||||
'',
|
||||
].join('\n'))
|
||||
const result = await call(ctx, 'grep', { pattern: 'e' }, { agent: agent('/w') })
|
||||
expect(text(result)).toBe('Found 2 of 3 matches\n\na.ts\nLine 1: one\nLine 2: two\n\n(Full grep result saved to: /spill/grep-results.txt. Use read with offset/limit to inspect it.)')
|
||||
expect(spill?.saves[0]).toMatchObject({
|
||||
source: { toolName: 'grep', label: 'result' },
|
||||
suggestedName: 'grep-results.txt',
|
||||
content: 'Found 3 matches\n\na.ts\nLine 1: one\nLine 2: two\n\nb.ts\nLine 3: three',
|
||||
})
|
||||
})
|
||||
|
||||
it('reports the unsaved remainder when capped with no spill backend', async () => {
|
||||
const { ctx, bash } = await setup({ config: { grepMaxMatches: 1 } })
|
||||
bash.handler = () => runResult(`${matchLine('a.ts', 1, 'one')}\n${matchLine('a.ts', 2, 'two')}\n`)
|
||||
const result = await call(ctx, 'grep', { pattern: 'o' }, { agent: agent('/w') })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(text(result)).toBe('Found 1 of 2 matches\n\na.ts\nLine 1: one\n\n(The complete result could not be saved; narrow pattern, path, or include to see more.)')
|
||||
})
|
||||
|
||||
it('validates arguments (empty pattern, blank path, bad include)', async () => {
|
||||
const { ctx } = await setup()
|
||||
expect(text(await call(ctx, 'grep', { pattern: '' }))).toContain('pattern must be a non-empty string')
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x', path: ' ' }))).toContain('path must be a non-empty string')
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x', include: ' ' }))).toContain('include must be a non-empty glob')
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x', include: '!*.ts' }))).toContain('negated patterns')
|
||||
expect(text(await call(ctx, 'grep', { pattern: 'x', include: '*.ts,*.js' }))).toContain('comma-separated list')
|
||||
})
|
||||
|
||||
it('accepts a whitespace-only pattern (a legitimate regex) and brace alternation in include', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('', { exitCode: 1 })
|
||||
const result = await call(ctx, 'grep', { pattern: ' ', include: '*.{ts,tsx}' })
|
||||
expect(result.isError).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('rg --json transport failures (SEARCH_FAILED)', () => {
|
||||
it.each([
|
||||
['a non-JSON line', 'not json at all'],
|
||||
['a non-object record', '42'],
|
||||
['a match record with no data', JSON.stringify({ type: 'match' })],
|
||||
['a match record with no path text', JSON.stringify({ type: 'match', data: { path: {}, lines: { text: 'x' }, line_number: 1 } })],
|
||||
['a match record with a non-object path', JSON.stringify({ type: 'match', data: { path: 'a.ts', lines: { text: 'x' }, line_number: 1 } })],
|
||||
['a match record with no line number', JSON.stringify({ type: 'match', data: { path: { text: 'a.ts' }, lines: { text: 'x' } } })],
|
||||
['a match record with no line content', JSON.stringify({ type: 'match', data: { path: { text: 'a.ts' }, line_number: 1 } })],
|
||||
['a match record with neither text nor bytes', JSON.stringify({ type: 'match', data: { path: { text: 'a.ts' }, lines: {}, line_number: 1 } })],
|
||||
])('%s fails the search', async (_label, line) => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult(`${line}\n`)
|
||||
const result = await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toMatchObject({ name: 'SearchError', code: 'SEARCH_FAILED' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('the no-background-task invariant', () => {
|
||||
it('never calls ctx.bash.start() across successful and failed searches', async () => {
|
||||
const { ctx, bash } = await setup()
|
||||
bash.handler = () => runResult('a.ts\n')
|
||||
await call(ctx, 'glob', { pattern: '*' })
|
||||
bash.handler = () => runResult('', { exitCode: 2, stderr: { text: 'boom', truncated: false } })
|
||||
await call(ctx, 'grep', { pattern: 'x' })
|
||||
expect(bash.startCalls).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('presentation', () => {
|
||||
it('glob titles carry the pattern and optional root', () => {
|
||||
expect(presentGlobCall({ pattern: '**/*.ts' })).toMatchObject({ card: 'generic', title: 'Glob **/*.ts', kind: 'search' })
|
||||
expect(presentGlobCall({ pattern: '*.md', path: 'docs' }).title).toBe('Glob *.md in docs')
|
||||
})
|
||||
|
||||
it('grep titles carry the pattern, target, and include filter', () => {
|
||||
expect(presentGrepCall({ pattern: 'todo' })).toMatchObject({ card: 'generic', title: 'Grep todo', kind: 'search' })
|
||||
expect(presentGrepCall({ pattern: 'todo', path: 'src', include: '*.ts' }).title).toBe('Grep todo in src (*.ts)')
|
||||
})
|
||||
})
|
||||
|
||||
describe('helpers', () => {
|
||||
it('toWorkdirRelative maps inside-workdir absolutes and passes everything else through', () => {
|
||||
expect(toWorkdirRelative('/w/a/b.ts', '/w')).toBe('a/b.ts')
|
||||
expect(toWorkdirRelative('/w', '/w')).toBe('.')
|
||||
expect(toWorkdirRelative('/other/b.ts', '/w')).toBe('/other/b.ts')
|
||||
expect(toWorkdirRelative('/w-sibling/b.ts', '/w')).toBe('/w-sibling/b.ts')
|
||||
expect(toWorkdirRelative('rel/b.ts', '/w')).toBe('rel/b.ts')
|
||||
// Normalization makes this land OUTSIDE the workdir → original path kept.
|
||||
expect(toWorkdirRelative('/w/../up.ts', '/w')).toBe('/w/../up.ts')
|
||||
})
|
||||
|
||||
it('previewLine keeps a within-budget line untouched', () => {
|
||||
expect(previewLine('short', 100)).toBe('short')
|
||||
})
|
||||
|
||||
it('formatGrepMatches groups by first-seen file order', () => {
|
||||
const grouped = formatGrepMatches([
|
||||
{ path: 'b.ts', lineNumber: 2, line: 'x' },
|
||||
{ path: 'a.ts', lineNumber: 1, line: 'y' },
|
||||
{ path: 'b.ts', lineNumber: 5, line: 'z' },
|
||||
])
|
||||
expect(grouped).toBe('b.ts\nLine 2: x\nLine 5: z\n\na.ts\nLine 1: y')
|
||||
})
|
||||
})
|
||||
20
packages/fs/tool-fs-search/tsconfig.json
Normal file
20
packages/fs/tool-fs-search/tsconfig.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [
|
||||
{ "path": "../../../vendor/cosmokit" },
|
||||
{ "path": "../../../vendor/cordis" },
|
||||
{ "path": "../../../vendor/schemastery" },
|
||||
{ "path": "../../util/retention" },
|
||||
{ "path": "../../llm/llm" },
|
||||
{ "path": "../../core/session" },
|
||||
{ "path": "../../core/tools" },
|
||||
{ "path": "../../core/system-prompt" },
|
||||
{ "path": "../../bash/bash" },
|
||||
{ "path": "../../spill/spill" }
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user