feat(fs): add a minimal read_image tool over the attachment and fs seams

The model reads a PNG/JPEG/WebP/GIF file, the bytes commit through the
durable attachment lifecycle, and the tool result carries the real
ImageBlock so the image enters context from the next request onward.
FileSystem gains a bounded readBytes primitive (local + E2B providers);
registration is conditional on the attachment store, and a strict
execution gate refuses routes that do not declare image input, so a
text route's durable history stays free of image blocks. llm-replay
models may declare inputModalities, letting keyless ACP snapshots pin
both the sha256-referenced success and the verbatim refusal.

Supersedes the withdrawn route-scoped design of PR #598; the decision
record is .agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md.
This commit is contained in:
creatixchu
2026-08-10 15:09:07 +08:00
parent 3764ce62a5
commit 1861a3fc7c
65 changed files with 1973 additions and 67 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/tool-fs/README.md
README.md: 27b53aca50f470fe9ead4da27d87328264440ff7
README.zh.md: 5bcf9c471d933702c46d50c56c9539cb9eede3ca
README.md: e295ad63902cbae245229fa86b780aeb59de808d
README.zh.md: be87bb18a1c49654d07977b5c8e2577188517077

View File

@@ -2,17 +2,20 @@
English | [中文](README.zh.md)
The **model-facing filesystem tools** — `read`, `write`, `edit` — and their **executor**. This is the consumer layer of the filesystem stack: it owns tool names, JSON schemas, argument validation, prompt sections, **read windowing**, and result formatting. It reads/writes/edits through the `ctx.fs` provider contract ([`@deepseek-ai/dsh-fs`](../fs)) **directly**. The freshness/observation policy is contributed by a separate plugin ([`@deepseek-ai/dsh-fs-policy`](../fs-policy)) through the `fs/*` event gate; the tool is not method-coupled to it. Under a confining provider, the shared sandbox-policy service is required for per-session execution and the tool exposes escalation for filesystem mutations.
The **model-facing filesystem tools** — `read`, `read_image`, `write`, `edit` — and their **executor**. This is the consumer layer of the filesystem stack: it owns tool names, JSON schemas, argument validation, prompt sections, **read windowing**, and result formatting. It reads/writes/edits through the `ctx.fs` provider contract ([`@deepseek-ai/dsh-fs`](../fs)) **directly**. The freshness/observation policy is contributed by a separate plugin ([`@deepseek-ai/dsh-fs-policy`](../fs-policy)) through the `fs/*` event gate; the tool is not method-coupled to it. Under a confining provider, the shared sandbox-policy service is required for per-session execution and the tool exposes escalation for filesystem mutations.
```ts ignore-check
// Default deployment: a ctx.fs provider, the policy plugin, then the tools.
await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // @deepseek-ai/dsh-fs-local
await ctx.plugin(FsPolicy) // @deepseek-ai/dsh-fs-policy (policy gate)
await ctx.plugin(ToolFs) // this package — registers read/write/edit
await ctx.plugin(LocalAttachmentStore, { dshHome }) // optional — enables durable read_image results
await ctx.plugin(ToolFs) // this package — read/write/edit, plus read_image with attachments
```
`@deepseek-ai/dsh-fs-policy` is **optional**: omit it and the tools run against the bare provider (unconditional write/overwrite/edit, no observed-state). A deployment that loads these tools is expected to also load it, so the behavior is read-before-write/edit.
`read_image` registers only while a durable `ctx.attachments` service is mounted — without one the deployment cannot commit image bytes, so the tool never appears. Execution additionally requires the exact routed model to declare `image` input (resolved through `ctx.llm.resolveModelInfo` from the session's latest request header, falling back to agent options); an unknown or text-only route gets a refusal result before any filesystem I/O, so a text route's durable history stays free of image blocks.
## Config
All keys are optional; the defaults are the shipped read caps.
@@ -29,18 +32,20 @@ All keys are optional; the defaults are the shipped read caps.
| Tool | Arguments | Behavior |
|---|---|---|
| `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer. `offset` is 1-based; `limit` defaults to and caps at the configured `readLimit` (2000). |
| `read_image` | `file_path` | Reads a PNG/JPEG/WebP/GIF file through the bounded byte seam, persists it through `ctx.attachments.saveImage`, and returns an image block beside a small metadata envelope. It succeeds only when the exact routed model declares image input. |
| `write` | `file_path`, `content` | Create or fully replace a file. With the policy plugin: overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. Without it: unconditional. |
| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. With the policy plugin: requires a prior `read` (any window) and the file unchanged since. Without it: unconditional. |
Field names are snake_case to match Claude Code and existing harness tool schemas.
Canonical successes are `read` → `{ path, offset, lines: [{ number, text }], totalLines }`, `write` → `{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit` → `{ path, before, after }`. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`, from these canonical values; the canonical values themselves are execution-local and are not added to `tool/result`, only the derived presentation metadata is persisted.
Canonical successes are `read` → `{ path, offset, lines: [{ number, text }], totalLines }`, `read_image` → `{ path, image: { attachmentId, mediaType, bytes, width, height, name? } }`, `write` → `{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit` → `{ path, before, after }`. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`, from these canonical values; the canonical values themselves are execution-local and are not added to `tool/result`, only the derived presentation metadata is persisted.
## The tool is the executor; policy is an event gate
The tools do **not** inject a policy service or inspect any cache. Each tool resolves the path via `ctx.fs.resolve(path, { cwd, signal })` — passing the calling agent's session cwd (`exec.agent.session.header.cwd`) so a relative path resolves against the session's workspace, matching `dsh-tool-bash`, and forwarding tool cancellation through resolution (see [the per-session cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)) — then:
- **read** — one `ctx.fs.stat` (type + size routing + version), then `readText`/`streamText`, then builds the line window, then emits `fs/observed` with a plain `ctx.emit`. (1 stat.)
- **read_image** — validates the argument, extension, attachment availability, deployment media types, and the image-capable route before any I/O; then one `ctx.fs.stat`, a bounded `ctx.fs.readBytes` capped at `imageLimits.maxImageBytes`, `attachments.saveImage` (content-addressed, so the image block references a durably committed object by the time `tool/result` is appended), and finally `fs/observed`. (1 stat.)
- **write** — `ctx.waterfall('fs/write-intent', target, exec, () => undefined)` for the optional guard, then `ctx.fs.writeText(target, content, intent)`, then `fs/observed`. (0 stat.)
- **edit** — `ctx.waterfall('fs/edit-intent', target, exec, () => undefined)` for the optional guard, then `ctx.fs.editText(target, edit, intent)`, then `fs/observed`. (0 stat.)
@@ -50,11 +55,11 @@ When `ctx.fs.sandboxMode` reports confinement, write/edit advertise `sandbox_per
## `fs/observed` is fire-and-forget
`fs/observed` fires AFTER the read/write/edit already succeeded, via a plain `ctx.emit`. A listener is contractually a synchronous, side-effect-only recorder (`@deepseek-ai/dsh-fs-policy`'s is a `WeakMap.set`); the tool does not guard the emit, so a listener that throws would surface as the tool's `isError` result — async or fallible observation does not belong on this event.
`fs/observed` fires AFTER the read/read_image/write/edit already succeeded, via a plain `ctx.emit`. A listener is contractually a synchronous, side-effect-only recorder (`@deepseek-ai/dsh-fs-policy`'s is a `WeakMap.set`); the tool does not guard the emit, so a listener that throws would surface as the tool's `isError` result — async or fallible observation does not belong on this event.
`read` opts into concurrent scheduling because its only mutation is the synchronous version recorder. Recorder races fail closed when a later `write` or `edit` re-checks the version under its target lock; both mutation tools remain exclusive. See the [parallel tool-call Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-parallel-tool-call-execution.md).
The package root exports only the Cordis plugin contract (`name`, `inject`, `Config`, and `apply`). Read rendering (line windowing + output formatting) lives in `src/read-render.ts` (Cordis-free, independently unit-tested); `src/read.ts`/`write.ts`/`edit.ts` are the tool executors and `src/index.ts` composes them.
The package root exports only the Cordis plugin contract (`name`, `inject`, `Config`, and `apply`). Read rendering (line windowing + output formatting) lives in `src/read-render.ts` (Cordis-free, independently unit-tested); `src/read.ts`/`read-image.ts`/`write.ts`/`edit.ts` are the tool executors and `src/index.ts` composes them.
## Model Experience
@@ -94,7 +99,7 @@ Prefix-stable while the plugin scope and guidance text are unchanged. Tool restr
#### What the model sees
The model sees the generated [`read`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. Scoped tool restrictions can remove any definition for one agent.
The model sees the generated [`read`, `read_image`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. `read_image` appears only while a durable attachment store is mounted; the schema itself is route-independent, and the strict gate refuses at execution. Scoped tool restrictions can remove any definition for one agent.
#### Token effect
@@ -118,6 +123,20 @@ Read output is capped by `readLimit`, `readMaxLineLength`, and `readMaxBytes`; t
Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Image read result
#### What the model sees
A successful `read_image` returns `<path><displayPath></path>`, `<type>image</type>`, and a `<content>` envelope naming the media type, dimensions, and byte size, followed by the image itself as a native image block. The session log stores only the durable `sha256:` attachment reference; the routed provider re-reads and digest-verifies the bytes on each request.
#### Token effect
The image is billed on every later request until compaction. Each call is independently bounded by the attachment store's `maxImageBytes`/`maxImagePixels`; repeated successful calls accumulate history, and content addressing deduplicates only the stored bytes, not the per-request token cost.
#### KV Cache effect
Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Write and edit results
#### What the model sees
@@ -136,7 +155,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, and `offset <offset> is out of range for "<path>" (<total> lines)`; provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, `offset <offset> is out of range for "<path>" (<total> lines)`, `cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual PNG/JPEG/WebP/GIF format`; provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
#### Token effect
@@ -149,5 +168,8 @@ Append-only; newly visible content follows the reusable request prefix and does
## Known Limitations and Deferred Work
- **No model-facing directory listing ships** — `ctx.fs.listDir` serves provider code such as skill discovery, while the sibling [`dsh-tool-fs-search`](../tool-fs-search/) package supplies ripgrep-backed `glob` and `grep` rather than extending the filesystem seam.
- **`read` handles UTF-8 text files only** — binary-safe reads and PDF/image/multimodal content are deferred; a directory target is `FS_NOT_REGULAR_FILE`.
- **`read` handles UTF-8 text files only** — images use the separate extension-routed `read_image` tool; PDF, audio, and video remain deferred. A directory target is `FS_NOT_REGULAR_FILE`.
- **The route gate races a concurrent model switch** — `read_image` checks the latest routed model at execution; a switch committed between that check and the next request can leave an image block on a route that rejects image content. The Web host already refuses switching an image-bearing session to a text-only model; other front doors own their equivalent guard.
- **Extension-declared media type** — the extension selects the declared type and the attachment store's magic-byte validation stays authoritative; a correctly formatted image under a wrong extension is refused with the rename remedy rather than sniffed.
- **No inline image preview on the tool-result card** — UI surfaces render the image result generically (the durable reference, not pixels); inline rendering is deferred to the UI packages.
- **No timeout surface** — `read`/`write`/`edit` take no timeout argument and declare no `timeout-policy` budget; cancellation rides `exec.signal` only ([provider rationale](../README.md#no-timeouts-on-file-io)).

View File

@@ -2,17 +2,20 @@
[English](README.md) | 中文
**面向模型的文件系统工具**(`read`、`write`、`edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取/写入/编辑。新鲜度/观察策略由独立插件([`@deepseek-ai/dsh-fs-policy`](../fs-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
**面向模型的文件系统工具**(`read`、`read_image`、`write`、`edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取/写入/编辑。新鲜度/观察策略由独立插件([`@deepseek-ai/dsh-fs-policy`](../fs-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
```ts ignore-check
// Default deployment: a ctx.fs provider, the policy plugin, then the tools.
await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // @deepseek-ai/dsh-fs-local
await ctx.plugin(FsPolicy) // @deepseek-ai/dsh-fs-policy (policy gate)
await ctx.plugin(ToolFs) // this package — registers read/write/edit
await ctx.plugin(LocalAttachmentStore, { dshHome }) // optional — enables durable read_image results
await ctx.plugin(ToolFs) // this package — read/write/edit, plus read_image with attachments
```
`@deepseek-ai/dsh-fs-policy` 是**可选的**:省略时,工具直接使用裸提供方(无条件写入/覆盖/编辑,无已观察状态)。加载这些工具的部署也应加载该插件,从而提供写入/编辑前读取行为。
`read_image` 只在持久 `ctx.attachments` 服务已挂载时注册:没有它,部署无法持久提交图像字节,工具就不会出现。执行时还要求确切路由的模型声明 `image` 输入(通过 `ctx.llm.resolveModelInfo` 从会话最新请求 header 解析,缺失时回退到 agent 选项);未知或纯文本路由在任何文件系统 I/O 之前就得到拒绝结果,因此文本路由的持久历史不会出现图像块。
## 配置
所有键均为可选;默认值是随产品交付的读取上限。
@@ -29,18 +32,20 @@ await ctx.plugin(ToolFs) // this package — re
| 工具 | 参数 | 行为 |
|---|---|---|
| `read` | `file_path`、`offset?`、`limit?` | 带行号的 UTF-8 内容和分页 footer。`offset` 从 1 开始;`limit` 默认为配置的 `readLimit`(2000),上限也为该值。 |
| `read_image` | `file_path` | 通过有界字节 seam 读取 PNG/JPEG/WebP/GIF 文件,经 `ctx.attachments.saveImage` 持久保存,并在小型元数据信封旁返回图像块。只有确切路由的模型声明图像输入时才会成功。 |
| `write` | `file_path`、`content` | 创建文件或完整替换文件。有策略插件时:覆盖现有文件要求先在未变版本上执行 `read`;创建新文件不需要。没有插件时:无条件执行。 |
| `edit` | `file_path`、非空 `old_string`、`new_string`、`replace_all?` | 字面量替换;除非 `replace_all` 为 true,否则要求唯一匹配。有策略插件时:要求先执行 `read`(任何窗口),且文件此后未变。没有插件时:无条件执行。 |
字段名使用 snake_case,与 Claude Code 和现有 harness 工具 schema 一致。
规范成功值分别为:`read` → `{ path, offset, lines: [{ number, text }], totalLines }`,`write` → `{ path, operation: 'create' | 'update', before: string | null, after }`,`edit` → `{ path, before, after }`。原生渲染器会保留下方带行号的读取结果和变更确认。`write`/`edit` 从这些规范值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`;规范值本身仅限于本次执行,不会添加到 `tool/result`,只有派生出的呈现元数据会被持久化。
规范成功值分别为:`read` → `{ path, offset, lines: [{ number, text }], totalLines }`,`read_image` → `{ path, image: { attachmentId, mediaType, bytes, width, height, name? } }`,`write` → `{ path, operation: 'create' | 'update', before: string | null, after }`,`edit` → `{ path, before, after }`。原生渲染器会保留下方带行号的读取结果和变更确认。`write`/`edit` 从这些规范值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`;规范值本身仅限于本次执行,不会添加到 `tool/result`,只有派生出的呈现元数据会被持久化。
## 工具就是执行器;策略是事件门禁
工具**不**注入策略服务,也不检查任何缓存。每个工具通过 `ctx.fs.resolve(path, { cwd, signal })` 解析路径;它会传入调用 agent(智能体)的会话 cwd(`exec.agent.session.header.cwd`),使相对路径以会话工作区为基准解析并与 `dsh-tool-bash` 一致,同时把工具取消转发到解析过程(见[每会话 cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md))。随后执行:
- **read**:一次 `ctx.fs.stat`(用于类型、大小路由和版本),随后调用 `readText`/`streamText`,构建行窗口,再发出 `fs/observed`,使用普通 `ctx.emit`。(1 次 stat。)
- **read_image**:在任何 I/O 之前校验参数、扩展名、附件可用性、部署接受的媒体类型和图像路由;随后一次 `ctx.fs.stat`、以 `imageLimits.maxImageBytes` 为上限的有界 `ctx.fs.readBytes`、`attachments.saveImage`(内容寻址,因此在 `tool/result` 事件追加时图像块引用的对象已持久提交),最后发出 `fs/observed`。(1 次 stat。)
- **write**:调用 `ctx.waterfall('fs/write-intent', target, exec, () => undefined)` 取得可选防护,然后调用 `ctx.fs.writeText(target, content, intent)`,再发出 `fs/observed`。(0 次 stat。)
- **edit**:调用 `ctx.waterfall('fs/edit-intent', target, exec, () => undefined)` 取得可选防护,然后调用 `ctx.fs.editText(target, edit, intent)`,再发出 `fs/observed`。(0 次 stat。)
@@ -50,11 +55,11 @@ await ctx.plugin(ToolFs) // this package — re
## `fs/observed` 发后即忘
`fs/observed` 在读取/写入/编辑已经成功之后,通过普通 `ctx.emit` 发出。监听器的约定是同步且只有副作用的记录器(`@deepseek-ai/dsh-fs-policy` 使用 `WeakMap.set`);工具不保护这次发出,因此监听器抛出会作为工具的 `isError` 结果出现。异步或可能失败的观察不属于该事件。
`fs/observed` 在 read/read_image/write/edit 已经成功之后,通过普通 `ctx.emit` 发出。监听器的约定是同步且只有副作用的记录器(`@deepseek-ai/dsh-fs-policy` 使用 `WeakMap.set`);工具不保护这次发出,因此监听器抛出会作为工具的 `isError` 结果出现。异步或可能失败的观察不属于该事件。
`read` 允许并发调度,因为其唯一变更是同步版本记录器。稍后的 `write` 或 `edit` 会在目标锁内重新检查版本,因此记录器竞态会以拒绝方式关闭;两个变更工具仍保持互斥。见[并行工具调用 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-parallel-tool-call-execution.md)。
包根目录只导出 Cordis 插件约定(`name`、`inject`、`Config` 和 `apply`)。读取渲染(行窗口与输出格式化)位于 `src/read-render.ts`(不依赖 Cordis,单独进行单元测试);`src/read.ts`/`write.ts`/`edit.ts` 是工具执行器,`src/index.ts` 负责组合。
包根目录只导出 Cordis 插件约定(`name`、`inject`、`Config` 和 `apply`)。读取渲染(行窗口与输出格式化)位于 `src/read-render.ts`(不依赖 Cordis,单独进行单元测试);`src/read.ts`/`read-image.ts`/`write.ts`/`edit.ts` 是工具执行器,`src/index.ts` 负责组合。
## 模型体验
@@ -94,7 +99,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
模型会看到已生成的 [`read`、`write` 和 `edit` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs),参数使用 snake_case。作用域工具限制可以为某个 agent 移除任一定义。
模型会看到已生成的 [`read`、`read_image`、`write` 和 `edit` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs),参数使用 snake_case。`read_image` 只在持久附件存储已挂载时出现;schema 本身与路由无关,严格门禁在执行时拒绝。作用域工具限制可以为某个 agent 移除任一定义。
#### Token 影响
@@ -118,6 +123,20 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV Cache 条目失效。
### 图像读取结果
#### 模型看到什么
成功的 `read_image` 返回 `<path><displayPath></path>`、`<type>image</type>` 和写明媒体类型、尺寸与字节数的 `<content>` 信封,随后是作为原生图像块的图像本身。会话日志只存储持久的 `sha256:` 附件引用;路由到的提供方在每次请求时重新读取并校验字节摘要。
#### Token 影响
图像在之后每次请求中都会计费,直到压缩。每次调用都独立受附件存储的 `maxImageBytes`/`maxImagePixels` 约束;重复成功调用会在历史中累积,内容寻址只去重存储的字节,不去重每次请求的 token 成本。
#### KV 缓存影响
只追加;新可见内容跟在可复用请求前缀之后,不会使既有 KV 缓存条目失效。
### 写入与编辑结果
#### 模型看到的内容
@@ -136,7 +155,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string`、`limit must be less than or equal to <max>`、`old_string must be a non-empty string`、`old_string and new_string must differ`、`cannot read "<path>": not found`、`cannot read "<path>": not a regular file` 和 `offset <offset> is out of range for "<path>" (<total> lines)`;提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `— re-read the file, then retry`,`FS_NOT_OBSERVED` 追加 `— read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`,而不会重复陈旧恢复指令;write 则使用带防护的创建。
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string`、`limit must be less than or equal to <max>`、`old_string must be a non-empty string`、`old_string and new_string must differ`、`cannot read "<path>": not found`、`cannot read "<path>": not a regular file`、`offset <offset> is out of range for "<path>" (<total> lines)`、`cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`、`cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`,以及类型不匹配的修复消息 `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual PNG/JPEG/WebP/GIF format`;提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `— re-read the file, then retry`,`FS_NOT_OBSERVED` 追加 `— read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`,而不会重复陈旧恢复指令;write 则使用带防护的创建。
#### Token 影响
@@ -149,5 +168,8 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
## 已知限制与暂缓事项
- **未交付面向模型的目录列表工具**:`ctx.fs.listDir` 服务于 skill(技能)发现等提供方代码,同级 [`dsh-tool-fs-search`](../tool-fs-search/) 包则提供基于 ripgrep 的 `glob` 与 `grep`,而不是扩展文件系统 seam。
- **`read` 只处理 UTF-8 文本文件**:二进制安全读取和 PDF/图像/多模态内容均延期处理;目录目标为 `FS_NOT_REGULAR_FILE`。
- **`read` 只处理 UTF-8 文本文件**:图像使用独立的、按扩展名路由的 `read_image` 工具;PDF、音频和视频仍延期处理。目录目标为 `FS_NOT_REGULAR_FILE`。
- **路由门禁与并发模型切换存在竞态**:`read_image` 在执行时检查最新路由的模型;在该检查与下一次请求之间提交的切换,可能让图像块落在拒绝图像内容的路由上。Web 宿主已拒绝把含图像的会话切到纯文本模型;其他前端拥有各自的等价防护。
- **媒体类型按扩展名声明**:扩展名选择声明类型,附件存储的魔数校验保持权威;扩展名错误但格式正确的图像会得到改名修复提示,而不是被嗅探接受。
- **工具结果卡片没有内嵌图像预览**:UI 表面以通用形式渲染图像结果(持久引用而非像素);内嵌渲染延后到 UI 包处理。
- **没有超时接口**:`read`/`write`/`edit` 不接受超时参数,也不声明 `timeout-policy` 预算;取消只通过 `exec.signal` 传递(见[提供方理由](../README.md#no-timeouts-on-file-io))。

View File

@@ -29,6 +29,7 @@
"schemastery": "^3.18.0"
},
"peerDependencies": {
"@deepseek-ai/dsh-attachment": "^0.0.1",
"@deepseek-ai/dsh-fs": "^0.0.1",
"@deepseek-ai/dsh-invariants": "^0.0.1",
"@deepseek-ai/dsh-llm": "^0.0.1",
@@ -44,6 +45,7 @@
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-agent-loop": "workspace:^",
"@deepseek-ai/dsh-agent-loop-testkit": "workspace:^",
"@deepseek-ai/dsh-attachment": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-fs-local": "workspace:^",
"@deepseek-ai/dsh-fs-policy": "workspace:^",

View File

@@ -11,6 +11,7 @@ import type {} from '@deepseek-ai/dsh-user-approval'
import { applyReadTool, READ_LIMIT, STREAM_MIN_SIZE } from './read.ts'
import { applyWriteTool } from './write.ts'
import { applyEditTool } from './edit.ts'
import { applyReadImageTool } from './read-image.ts'
import { READ_MAX_BYTES, READ_MAX_LINE_LENGTH } from './read-render.ts'
import { FsSandboxSurface } from './sandbox.ts'
@@ -63,6 +64,12 @@ export function apply(ctx: Context, config: Config): void {
maxBytes: resolved.readMaxBytes,
streamMinSize: resolved.readStreamMinSize,
})
// read_image is composition-conditional: without a mounted attachment store
// the deployment cannot durably commit image bytes, so the tool never
// registers; the execute body keeps a defensive re-check for direct callers.
ctx.inject(['attachments'], (imageCtx) => {
applyReadImageTool(imageCtx)
})
// One escalation surface shared by both mutating tools: advertisement gating,
// per-call policy resolution, and denial-marker mapping, all keyed off whether
// the mounted ctx.fs confines (ctx.fs.sandboxMode).

View File

@@ -0,0 +1,231 @@
/**
* The model-facing `read_image` tool: reads a PNG/JPEG/WebP/GIF file, durably
* commits its bytes through the attachment service (the same lifecycle as a
* user-uploaded image), and returns an image block so the image enters model
* context from the next request onward.
*
* The route gate is deliberately stricter than the host upload preflight: a
* tool result enters durable session history, so emitting an image on a route
* that cannot carry it would break that route's continuation. Unknown
* capability therefore refuses instead of relying on the adapter guard.
* @module @deepseek-ai/dsh-tool-fs/src/read-image
*/
import { basename, extname } from 'node:path'
import type { Context } from 'cordis'
import { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentRef, ImageMediaType } from '@deepseek-ai/dsh-attachment'
import { createUserMessage } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { defineTool } from '@deepseek-ai/dsh-tools'
import type { GenericCallView, ToolExecution } from '@deepseek-ai/dsh-tools'
import { FsError } from '@deepseek-ai/dsh-fs'
import type {} from '@deepseek-ai/dsh-fs'
import { sessionResolveOptions } from './session-cwd.ts'
/** Extensions `read_image` accepts; magic-byte validation at the attachment service stays authoritative. */
const IMAGE_EXTENSIONS: Readonly<Record<string, ImageMediaType>> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.webp': 'image/webp',
'.gif': 'image/gif',
}
/** The canonical outcome declared by the `read_image` output schema. */
export interface ImageReadValue {
path: string
image: {
attachmentId: string
mediaType: ImageMediaType
bytes: number
width: number
height: number
name?: string
}
}
/**
* Map a model-supplied path to its declared image media type by extension.
* @param filePath - the raw `file_path` argument (not yet resolved).
* @returns the declared media type, or undefined when the path does not claim an image.
*/
export function imageMediaTypeForPath(filePath: string): ImageMediaType | undefined {
return IMAGE_EXTENSIONS[extname(filePath).toLowerCase()]
}
/**
* Enforce the strict image-capability gate for the calling route. Resolves the
* session's latest routed provider/model (request header config, then agent
* options) and requires the exact resolved route to declare `image` input explicitly.
* @param ctx - the plugin context used to resolve the optional `llm` service.
* @param exec - the tool-execution context supplying the calling agent.
* @param displayPath - the path rendered in refusal messages.
*/
export async function assertImageCapableRoute(ctx: Context, exec: ToolExecution, displayPath: string): Promise<void> {
const routed = exec.agent?.session.requestHeader()?.config
const provider = routed?.provider ?? exec.agent?.options.provider
const model = routed?.model ?? exec.agent?.options.model
const llm = ctx.get('llm')
if (provider === undefined || model === undefined || llm === undefined) {
throw new Error(`cannot read "${displayPath}" as an image: the current model route could not be resolved`)
}
const active = await llm.resolveModelInfo(provider, model, exec.signal)
if (active.inputModalities === undefined || !active.inputModalities.includes('image')) {
throw new Error(`cannot read "${displayPath}" as an image: model "${model}" does not declare image input; switch to an image-capable model to read images`)
}
}
/**
* Re-brand a canonical image outcome into the durable attachment reference an
* `ImageBlock` carries.
* @param image - the canonical image metadata from the output schema.
* @returns the branded attachment reference.
*/
export function imageRefFromValue(image: ImageReadValue['image']): ImageAttachmentRef {
return {
attachmentId: AttachmentId(image.attachmentId),
mediaType: image.mediaType,
bytes: image.bytes,
width: image.width,
height: image.height,
...image.name === undefined ? {} : { name: image.name },
}
}
/**
* Format an image read as the model-facing envelope beside its image block.
* @param displayPath - the backend-resolved path rendered in the envelope's `<path>` element.
* @param image - the canonical image metadata to summarize.
* @returns the model-facing envelope; the image itself rides the adjacent image block.
*/
export function formatImageReadOutput(displayPath: string, image: ImageReadValue['image']): string {
return `<path>${displayPath}</path>
<type>image</type>
<content>
${image.mediaType} image, ${image.width}x${image.height} px, ${image.bytes} bytes
</content>`
}
/**
* Project one canonical image read into its model-facing envelope and image.
* @param value - the canonical image-read outcome.
* @returns the two content blocks used by native and nested dispatches.
*/
function imageReadContent(value: ImageReadValue): ContentBlock[] {
return [
{ type: 'text', text: formatImageReadOutput(value.path, value.image) },
{ type: 'image', attachment: imageRefFromValue(value.image) },
]
}
/**
* Register the `read_image` tool. Execution gates on the optional
* `attachments`/`llm` services and the calling route's declared image input;
* registration itself is unconditional so denial happens at the operation
* boundary rather than through schema omission.
* @param ctx - the plugin context; registrations are effects scoped to it, and
* execution uses its `fs` service plus the optional `attachments`/`llm` services.
*/
export function applyReadImageTool(ctx: Context): void {
ctx.tools.register(defineTool({
name: 'read_image',
description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.',
parameters: {
file_path: { type: 'string', required: true, description: 'Path to the image file, resolved by the filesystem backend.' },
},
output: {
schema: {
type: 'object',
additionalProperties: false,
properties: {
path: { type: 'string', required: true },
image: {
type: 'object',
additionalProperties: false,
required: true,
properties: {
attachmentId: { type: 'string', required: true },
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
bytes: { type: 'integer', required: true },
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
name: { type: 'string' },
},
},
},
},
render: (_args, value) => imageReadContent(value),
},
// Content-addressed attachment writes are idempotent, so concurrent reads
// of the same file cannot conflict.
isConcurrencySafe: () => true,
async execute(args, exec) {
if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string')
// Every gate runs before any filesystem I/O so a refusal never leaks
// partial reads or attachment writes.
const mediaType = imageMediaTypeForPath(args.file_path)
if (mediaType === undefined) {
throw new Error(`cannot read "${args.file_path}": read_image only accepts PNG/JPEG/WebP/GIF paths`)
}
const attachments = ctx.get('attachments')
if (attachments === undefined) {
throw new Error(`cannot read "${args.file_path}" as an image: no attachment service is mounted`)
}
if (!attachments.imageLimits.mediaTypes.includes(mediaType)) {
throw new Error(`cannot read "${args.file_path}": ${mediaType} images are not accepted by this deployment`)
}
await assertImageCapableRoute(ctx, exec, args.file_path)
const target = await ctx.fs.resolve(args.file_path, sessionResolveOptions(exec, args.file_path))
const info = await ctx.fs.stat(target, exec.signal)
if (!info) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND')
if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
const data = await ctx.fs.readBytes(target, exec.signal, attachments.imageLimits.maxImageBytes)
// Persist before returning: the image block must reference a durably
// committed object by the time the tool/result event is appended.
let ref: ImageAttachmentRef
try {
ref = await attachments.saveImage({ data, mediaType, name: basename(target.displayPath) })
} catch (error: unknown) {
if (!(error instanceof AttachmentError) || error.code !== 'IMAGE_TYPE_MISMATCH') throw error
const extension = extname(target.displayPath).toLowerCase()
throw new Error(
`cannot read "${target.displayPath}": the ${extension} extension declares ${mediaType}, but the bytes use a different image format; rename the file to match its actual PNG/JPEG/WebP/GIF format`,
{ cause: error },
)
}
ctx.emit('fs/observed', target, { kind: 'present', version: info.version }, exec)
const value: ImageReadValue = {
path: target.displayPath,
image: {
attachmentId: ref.attachmentId,
mediaType: ref.mediaType,
bytes: ref.bytes,
width: ref.width,
height: ref.height,
...ref.name === undefined ? {} : { name: ref.name },
},
}
if (exec.parent !== undefined) {
exec.deferContext(createUserMessage({
content: imageReadContent(value),
source: { kind: 'plugin', plugin: 'tool-fs' },
}))
}
return value
},
// Pure display: a generic card in the read family with a follow-along
// location on the image file.
presentCall(args): GenericCallView {
return {
card: 'generic',
title: `Read image ${args.file_path}`,
kind: 'read',
locations: [{ path: args.file_path }],
}
},
}))
}

View File

@@ -0,0 +1,462 @@
/**
* The `read_image` tool over the REAL local filesystem and attachment store:
* extension routing, the strict image-modality gate (every refusal arm),
* durable commit + image-block rendering, attachment admission failures, and
* the regression that `read` keeps its text-only contract.
*/
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { Context } from 'cordis'
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
import { CallId, LlmAdapter, LlmService } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { RUN_CODE_NAME } from '@deepseek-ai/dsh-tools'
import type { Config as ToolConfig } from '@deepseek-ai/dsh-tools'
import LocalFileSystem from '@deepseek-ai/dsh-fs-local'
import * as FsPolicy from '@deepseek-ai/dsh-fs-policy'
import LocalAttachmentStore from '@deepseek-ai/dsh-attachment-local'
import { AttachmentId, AttachmentStore } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentLimits, ImageAttachmentRef, SaveImageAttachment, StoredImageAttachment } from '@deepseek-ai/dsh-attachment'
import * as ToolFs from '@deepseek-ai/dsh-tool-fs'
import {
applyReadImageTool,
formatImageReadOutput,
imageMediaTypeForPath,
imageRefFromValue,
} from '../src/read-image.ts'
/** 1x1 red PNG (valid signature, IHDR, IDAT). */
const PNG_1X1 = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC', 'base64')
/** 3x3 red PNG used to trip a tiny configured pixel limit. */
const PNG_3X3 = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAIAAADZSiLoAAAAEElEQVR4nGP4z8AAQQxYWACPjgj4kWPEuQAAAABJRU5ErkJggg==', 'base64')
const testToolSignal = new AbortController().signal
/** Exact-route fake adapter; `stream` is unreachable in these tests. */
class CatalogAdapter extends LlmAdapter {
constructor(
private readonly models: LlmModelInfo[],
private readonly resolvedModels: LlmModelInfo[] = models,
) {
super()
}
override listModels(_provider: string): Promise<readonly LlmModelInfo[]> {
return Promise.resolve(this.models)
}
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
const resolved = this.resolvedModels.find(candidate => candidate.id === model)
return Promise.resolve({
provider,
id: model,
name: resolved?.name ?? model,
...resolved?.inputModalities === undefined ? {} : { inputModalities: [...resolved.inputModalities] },
})
}
override stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
throw new Error('read_image tests never stream')
}
}
/** In-process Code Mode seam fake that invokes the real registry bindings. */
class FakeRuntime extends CodeRuntime {
readonly language = 'typescript'
readonly isolation = 'fake'
behavior: (request: CodeRunRequest) => Promise<CodeRunResult> = () => Promise.resolve({ logs: [] })
run(request: CodeRunRequest): Promise<CodeRunResult> {
return this.behavior(request)
}
}
let dir: string
let home: string
beforeEach(async () => {
dir = await mkdtemp(join(tmpdir(), 'dsh-read-image-'))
home = await mkdtemp(join(tmpdir(), 'dsh-read-image-home-'))
})
afterEach(async () => {
await rm(dir, { recursive: true, force: true })
await rm(home, { recursive: true, force: true })
})
interface SetupOptions {
models?: LlmModelInfo[]
resolvedModels?: LlmModelInfo[]
attachments?: boolean
llm?: boolean
storeConfig?: { maxImageBytes?: number; maxImagePixels?: number }
toolMode?: ToolConfig['mode']
}
async function setup(options: SetupOptions = {}) {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry, { mode: options.toolMode ?? 'native' })
if (options.toolMode === 'code' || options.toolMode === 'both') {
await ctx.plugin(FakeRuntime)
}
await ctx.plugin(LocalFileSystem, { cwd: dir })
await ctx.plugin(FsPolicy)
if (options.attachments !== false) {
await ctx.plugin(LocalAttachmentStore, { dshHome: home, ...options.storeConfig })
}
if (options.llm !== false) {
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['visual'], new CatalogAdapter(options.models ?? [
{ provider: 'visual', id: 'vision-model', name: 'Vision', inputModalities: ['text', 'image'] },
{ provider: 'visual', id: 'text-model', name: 'Text', inputModalities: ['text'] },
{ provider: 'visual', id: 'legacy-model', name: 'Legacy' },
], options.resolvedModels))
}
await ctx.plugin(ToolFs)
return ctx
}
/** A fake calling agent pinned to one routed provider/model. */
function agentOn(model: string | undefined, provider = 'visual'): object {
return {
options: {},
session: {
header: { cwd: dir },
requestHeader: () => (model === undefined ? undefined : { config: { provider, model } }),
append: () => undefined,
},
}
}
let callCounter = 0
function call(ctx: Context, name: string, args: unknown, agent?: object) {
return ctx.tools.execute({
signal: testToolSignal,
callId: CallId(`img-call-${++callCounter}`),
name,
arguments: args,
...agent ? { agent: agent as never } : {},
})
}
function readImage(ctx: Context, args: unknown, agent?: object) {
return call(ctx, 'read_image', args, agent)
}
function text(result: { content: { type: string; text?: string }[] }): string {
return result.content.filter(b => b.type === 'text').map(b => b.text).join('')
}
describe('imageMediaTypeForPath', () => {
it('maps the four extensions case-insensitively and rejects everything else', () => {
expect(imageMediaTypeForPath('a.png')).toBe('image/png')
expect(imageMediaTypeForPath('a.JPG')).toBe('image/jpeg')
expect(imageMediaTypeForPath('b.jpeg')).toBe('image/jpeg')
expect(imageMediaTypeForPath('c.webp')).toBe('image/webp')
expect(imageMediaTypeForPath('d.Gif')).toBe('image/gif')
expect(imageMediaTypeForPath('note.txt')).toBeUndefined()
expect(imageMediaTypeForPath('png')).toBeUndefined()
})
})
describe('imageRefFromValue', () => {
it('re-brands with and without the optional display name', () => {
const base = { attachmentId: 'sha256:00', mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1 }
expect(imageRefFromValue(base)).toEqual(base)
expect(imageRefFromValue({ ...base, name: 'a.png' })).toEqual({ ...base, name: 'a.png' })
})
})
describe('read_image happy path', () => {
it('commits the bytes durably and renders the envelope beside an image block', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup()
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(false)
expect(result.content).toHaveLength(2)
const image = result.content[1] as { type: string; attachment: ImageAttachmentRef }
expect(image.type).toBe('image')
expect(image.attachment.mediaType).toBe('image/png')
expect(image.attachment.width).toBe(1)
expect(image.attachment.height).toBe(1)
expect(image.attachment.bytes).toBe(PNG_1X1.length)
expect(image.attachment.name).toBe('red.png')
expect(image.attachment.attachmentId).toMatch(/^sha256:[0-9a-f]{64}$/)
expect(text(result)).toBe(formatImageReadOutput(join(dir, 'red.png'), {
attachmentId: image.attachment.attachmentId,
mediaType: 'image/png',
bytes: PNG_1X1.length,
width: 1,
height: 1,
}))
// The committed object must read back verbatim through the store.
const attachments = ctx.get('attachments')
if (attachments === undefined) throw new Error('expected the attachment service')
const stored = await attachments.readImage(image.attachment)
expect(Buffer.from(stored.data)).toEqual(PNG_1X1)
})
it('emits fs/observed for the read image', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup()
const observed: string[] = []
ctx.on('fs/observed', target => void observed.push(target.displayPath))
await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(observed).toEqual([join(dir, 'red.png')])
})
it('falls back to agent options when no request header exists yet', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup()
const agent = {
options: { provider: 'visual', model: 'vision-model' },
session: { header: { cwd: dir }, requestHeader: () => undefined },
}
const result = await readImage(ctx, { file_path: 'red.png' }, agent)
expect(result.isError).toBe(false)
})
it('forwards a nested Code Mode image through the outer run_code context', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ toolMode: 'code' })
const runtime = ctx.codeRuntime as FakeRuntime
runtime.behavior = async (request) => {
const value = await request.bindings[0]!.functions.read_image!({ file_path: 'red.png' })
return { logs: [], value }
}
const result = await call(ctx, RUN_CODE_NAME, {
code: 'return await tools.read_image({ file_path: "red.png" })',
description: 'Read the image through Code Mode',
}, agentOn('vision-model'))
expect(result.isError).toBe(false)
expect(result.content.every(block => block.type === 'text')).toBe(true)
expect(result.additionalContexts).toHaveLength(1)
const forwarded = result.additionalContexts?.[0]?.content
expect(forwarded).toHaveLength(2)
expect(forwarded?.[0]?.type).toBe('text')
expect(forwarded?.[0]?.type === 'text' ? forwarded[0].text : '').toContain('<type>image</type>')
expect(forwarded?.[1]).toMatchObject({
type: 'image',
attachment: { mediaType: 'image/png', width: 1, height: 1 },
})
})
})
describe('strict image-modality gate', () => {
it('accepts an exact visual route even when the advisory model catalog omits it', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({
models: [],
resolvedModels: [
{ provider: 'visual', id: 'hidden-vision', name: 'Hidden Vision', inputModalities: ['text', 'image'] },
],
})
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('hidden-vision'))
expect(result.isError).toBe(false)
})
it.each([
['a text-only model', 'text-model'],
['a model without declared modalities', 'legacy-model'],
['a model absent from the catalog', 'unknown-model'],
])('refuses on %s', async (_label, model) => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup()
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn(model))
expect(result.isError).toBe(true)
expect(text(result)).toContain('does not declare image input')
})
it('refuses when the route cannot be resolved (no agent, or no header and no options)', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup()
const noAgent = await readImage(ctx, { file_path: 'red.png' })
expect(noAgent.isError).toBe(true)
expect(text(noAgent)).toContain('route could not be resolved')
const noRoute = await readImage(ctx, { file_path: 'red.png' }, agentOn(undefined))
expect(noRoute.isError).toBe(true)
expect(text(noRoute)).toContain('route could not be resolved')
})
it('refuses when no llm service is mounted', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ llm: false })
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('route could not be resolved')
})
})
describe('argument and service preconditions', () => {
it('rejects an empty path and a non-image extension', async () => {
const ctx = await setup()
const empty = await readImage(ctx, { file_path: ' ' }, agentOn('vision-model'))
expect(empty.isError).toBe(true)
expect(text(empty)).toContain('non-empty')
const nonImage = await readImage(ctx, { file_path: 'notes.txt' }, agentOn('vision-model'))
expect(nonImage.isError).toBe(true)
expect(text(nonImage)).toContain('only accepts PNG/JPEG/WebP/GIF paths')
})
it('refuses when no attachment service is mounted', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ attachments: false })
expect(ctx.tools.get('read_image')).toBeUndefined()
expect(ctx.tools.schemas().map(schema => schema.name)).not.toContain('read_image')
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('unknown tool "read_image"')
})
it('defensively refuses execution without an attachment service', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ attachments: false })
applyReadImageTool(ctx)
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('no attachment service is mounted')
})
it('refuses a media type the deployment does not accept', async () => {
/** Store whose deployment accepts JPEG only. */
class JpegOnlyStore extends AttachmentStore {
readonly imageLimits: ImageAttachmentLimits = Object.freeze({
maxImageBytes: 1024,
maxImagesPerMessage: 1,
maxMessageImageBytes: 1024,
maxImagePixels: 100,
mediaTypes: Object.freeze(['image/jpeg'] as const),
})
validateImage(_input: SaveImageAttachment): Promise<void> {
throw new Error('unreachable: admission refuses before validation')
}
saveImage(_input: SaveImageAttachment): Promise<ImageAttachmentRef> {
throw new Error('unreachable: admission refuses before save')
}
readImage(_ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
throw new Error('unreachable in this test')
}
}
const ctx = await setup({ attachments: false })
await ctx.plugin(JpegOnlyStore)
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('image/png images are not accepted by this deployment')
})
})
describe('image admission failures', () => {
it('explains how to repair a declared/actual media-type mismatch', async () => {
await writeFile(join(dir, 'wrong.jpg'), PNG_1X1)
const ctx = await setup()
const result = await readImage(ctx, { file_path: 'wrong.jpg' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('the .jpg extension declares image/jpeg')
expect(text(result)).toContain('rename the file to match its actual PNG/JPEG/WebP/GIF format')
})
it('fails with FS_TOO_LARGE before reading a file past maxImageBytes', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ storeConfig: { maxImageBytes: PNG_1X1.length - 1 } })
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('exceeds')
})
it('surfaces the pixel limit from the attachment admission', async () => {
await writeFile(join(dir, 'big.png'), PNG_3X3)
const ctx = await setup({ storeConfig: { maxImagePixels: 4 } })
const result = await readImage(ctx, { file_path: 'big.png' }, agentOn('vision-model'))
expect(result.isError).toBe(true)
})
it('reports a missing image file and a directory target through the fs vocabulary', async () => {
await mkdir(join(dir, 'folder.png'))
const ctx = await setup()
const missing = await readImage(ctx, { file_path: 'absent.png' }, agentOn('vision-model'))
expect(missing.isError).toBe(true)
expect(text(missing)).toContain('not found')
const directory = await readImage(ctx, { file_path: 'folder.png' }, agentOn('vision-model'))
expect(directory.isError).toBe(true)
expect(text(directory)).toContain('not a regular file')
})
it('omits the display name when the store returns a reference without one', async () => {
/** Store echoing a fixed nameless reference; deployments may strip names entirely. */
class NamelessStore extends AttachmentStore {
readonly imageLimits: ImageAttachmentLimits = Object.freeze({
maxImageBytes: 1024,
maxImagesPerMessage: 1,
maxMessageImageBytes: 1024,
maxImagePixels: 100,
mediaTypes: Object.freeze(['image/png'] as const),
})
validateImage(_input: SaveImageAttachment): Promise<void> {
return Promise.resolve()
}
async saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef> {
return { attachmentId: AttachmentId('sha256:feed'), mediaType: input.mediaType, bytes: input.data.length, width: 1, height: 1 }
}
readImage(_ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
throw new Error('unreachable in this test')
}
}
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ attachments: false })
await ctx.plugin(NamelessStore)
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(false)
const image = result.content[1] as { attachment: ImageAttachmentRef }
expect(image.attachment.name).toBeUndefined()
})
})
describe('registration surface', () => {
it('declares read_image parallel-safe and presents a read-family card', async () => {
const ctx = await setup()
expect(ctx.tools.executionMode({
signal: testToolSignal, callId: CallId('img-parallel'), name: 'read_image', arguments: { file_path: 'a.png' },
})).toEqual({ kind: 'parallel' })
expect(ctx.tools.get('read_image')?.presentCall?.({ file_path: 'shot.png' })).toEqual({
card: 'generic',
title: 'Read image shot.png',
kind: 'read',
locations: [{ path: 'shot.png' }],
})
})
})
describe('read keeps its text-only contract', () => {
it('still refuses a PNG as a binary file and line-numbers text', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
await writeFile(join(dir, 'note.txt'), 'hello\nworld')
const ctx = await setup()
const png = await call(ctx, 'read', { file_path: 'red.png' }, agentOn('vision-model'))
expect(png.isError).toBe(true)
expect(text(png)).toContain('binary file')
const txt = await call(ctx, 'read', { file_path: 'note.txt' }, agentOn('text-model'))
expect(txt.isError).toBe(false)
expect(text(txt)).toContain('1: hello')
expect(text(txt)).toContain('<type>file</type>')
})
})

View File

@@ -71,6 +71,13 @@ class FakeFs extends FileSystem {
const content = this.files.get(target.targetKey) ?? ''
return (async function* () { yield content })()
}
override async readBytes(target: FsTarget, _signal: AbortSignal | undefined, maxBytes: number): Promise<Uint8Array> {
const bytes = new TextEncoder().encode(this.files.get(target.targetKey) ?? '')
if (bytes.length > maxBytes) {
throw new FsError(`too large: ${target.displayPath}`, 'FS_TOO_LARGE')
}
return bytes
}
override async listDir(_target: FsTarget): Promise<FsDirEntry[]> {
return []
}

View File

@@ -41,6 +41,9 @@
},
{
"path": "../../interaction/user-approval"
},
{
"path": "../../attachment/attachment"
}
]
}