feat(tool-web): replace the regex HTML-to-markdown converter with turndown

Implements the turndown Agent Note from the NIH dependency audit (full
variant, not the minimal entities-only fallback): dsh-tool-web's fetch
rendering now converts HTML through turndown + @joplin/turndown-plugin-gfm
(atx headings, fenced code, dash bullets, GFM tables/strikethrough) over
the real domino DOM, with script/style/noscript removed wholesale. The
hand-rolled ~86-line regex converter html.ts and its entity tables are
deleted; renderBody wraps the conversion in try/catch falling back to
the raw HTML body, because turndown's recursive DOM walk overflows with
a RangeError on pathological nesting (measured: 4k levels on the main
thread, 8k in a worker) where the regex version could never throw.

Closure weight, measured: tool-web IS in the single-exe runtime closure,
and the exe asset globs would pack ~7.9 MB of the three new packages —
but ~6 MB of that is domino's test corpus, with runtime lib/ at ~550 KB
against a ~174 MB artifact (<0.5% either way), so the swap wins.

Per testing policy the previously-missing keyless web_fetch snapshot
ships in the same change: the acp-agent `web-fetch` scenario boots a new
web.cordis.yml overlay (web seam + real dsh-web-fetch-local provider +
tool-web fetch-only + a loopback HTTP fixture server on a fixed port
serving deterministic HTML with entities, a GFM table, and nesting), so
recording and keyless replay both drive the real HTTP fetch and real
conversion end to end; the scenario pins the new `web` header class.

The Agent Note moves proposed -> implemented and is rewritten per the
lifecycle contract (Decision/Consequences/Testing, closure verdict and
alternatives recorded); tool-web and acp-agent READMEs updated in both
languages and pairs re-recorded.
This commit is contained in:
Tianyi Cui
2026-07-27 00:22:13 +08:00
parent 05adf5da4a
commit 45a5175e44
29 changed files with 962 additions and 210 deletions

View File

@@ -11,7 +11,7 @@
| 工具 | 参数 | 行为 |
|---|---|---|
| `web_search` | `query`(string) | 发现。返回可选答案与源 URL。`max_results` **不** 面向模型:工具设置上限(`searchMaxResults` 配置,默认 8)并传给 seam。 |
| `web_fetch` | `url`(string) | 获取特定 URL。HTML 主体渲染为近似 markdown 的文本;文本主体原样通过。非 2xx 状态会报告,而非报错。工具调用超时是部署策略(`dsh-timeout-policy`),不是模型参数。 |
| `web_fetch` | `url`(string) | 获取特定 URL。HTML 主体渲染为 markdown(turndown,带 GFM 表格/删除线);文本主体原样通过。非 2xx 状态会报告,而非报错。工具调用超时是部署策略(`dsh-timeout-policy`),不是模型参数。 |
两个工具都选择并发调度,因为提供方读取会返回内容,不会修改父 agent 状态。
@@ -126,6 +126,6 @@ Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for ex
## 已知限制与暂缓事项
- **`htmlToMarkdown` 是最小正则转换器,不是 HTML parser**:它会移除 script/style/noscript,保留标题/项目符号/链接,并解码约十余个命名 entity;表格、图片与嵌套格式会丢失。
- **HTML→markdown 转换在病态输入上回退为原始 HTML**:[turndown](https://github.com/mixmark-io/turndown)(带 GFM 表格/删除线)通过真实 DOM 转换抓取到的 HTML,但其递归遍历在极深嵌套(数千层)上会栈溢出;此类主体不经转换原样通过,而非报错([决策记录](../../../.agents/notes/implemented/simplification/2026-07-26-turndown-for-tool-web-html-markdown.md))。
- **面向模型的表层有意保持最小,提升项暂缓**:`max_results` 保持为配置上限(不是模型参数),`web_fetch` 只接受 `url`(没有 `format`/`prompt`/LLM 摘要模式);两项都列为 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-24-web-capability-seam.md) 中的后续步骤。
- **没有 web 专用权限策略**:两个工具都不会请求 `ctx.approval` 就直接执行;需要确认的部署必须添加 `tools/pre-execute` 策略,该包不定义持久 URL/domain 授权。