docs: rebalance prose cleanup and add trimming skill

This commit is contained in:
Tianyi Cui
2026-07-13 23:27:00 +08:00
parent fcdc318dda
commit 148046b9c8
392 changed files with 2801 additions and 1754 deletions

View File

@@ -1,8 +1,8 @@
/**
* The model-facing `web_fetch` tool: retrieve the content of a specific URL. Execution goes
* through `ctx.web` — this module owns the model-facing schema, argument validation, and
* PRESENTATION (HTML→markdown, truncation formatting), while the fetch provider owns safe
* retrieval (transport, redirects, caps).
* The model-facing `web_fetch` tool. This module owns its schema, validation, and presentation;
* `ctx.web` owns retrieval. Timeout is deployment policy, not a model argument: config becomes
* `ToolDefinition.timeoutMs`, timeout policy enforces it, and this tool forwards the resulting
* signal. A provider timeout remains a backstop for direct seam callers.
*/
import type { Context } from 'cordis'

View File

@@ -1,5 +1,8 @@
/**
* Minimal, dependency-free HTML→markdown-ish text conversion for `web_fetch` presentation.
* Minimal dependency-free HTML-to-readable-text conversion for `web_fetch`, not a full parser. It
* removes non-content elements and tags, decodes common entities, collapses whitespace, and keeps
* basic headings, lists, and links. A richer converter can replace it without changing the seam or
* tool schema.
* @module @deepseek-ai/dsh-tool-web/html
*/

View File

@@ -1,7 +1,8 @@
/**
* The model-facing web tool suite (`web_search`, `web_fetch`) over the `ctx.web` seam. This
* root plugin registers the tools the product has ENABLED, composing the per-tool registration
* helpers (`applyWebSearchTool`, `applyWebFetchTool`).
* Model-facing `web_search` and `web_fetch` tools over `ctx.web`. This package owns schemas,
* validation, prompt guidance, limits, and presentation, never concrete providers. Enablement
* controls tool registration; an enabled tool remains visible when its provider is unavailable
* and fails with a structured error at execution time.
* @module @deepseek-ai/dsh-tool-web
*/

View File

@@ -2,7 +2,8 @@
* Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search provider
* (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool (`dsh-tool-web`) + the
* tool-call timeout policy (`dsh-timeout-policy`), exercised through `ctx.tools.execute()` —
* nothing bypasses the tool registry.
* nothing bypasses the tool registry. Fetch verifies world effects against loopback HTTP; search
* uses the real Exa provider with only its network boundary stubbed.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
@@ -157,7 +158,8 @@ describe('tool-call timeout returns TOOL_TIMEOUT (deadline wins over a slow fetc
it('the provider backstop still protects a DIRECT ctx.web.fetch() call (no tool-call policy in that path)', async () => {
// A direct seam caller does not go through tools/execute, so the tool-call policy never
// applies; the provider's own timeout is the only budget.
// applies; the provider's own timeout is the only budget. A short request hint must therefore
// produce provider-owned `WEB_FETCH_TIMEOUT`, never `TOOL_TIMEOUT`.
const err = await tctx.web.fetch({ url: slowBase, timeoutMs: 50 }).then(
() => undefined,
(e: unknown) => e as { code?: string },

View File

@@ -1,4 +1,9 @@
/** Real Loader-path coverage for the namespace plugin's export shape. */
/**
* Real Loader-path guard for an injected namespace plugin. A default export would make
* `unwrapExports` collapse the namespace and drop `inject`, causing access to `ctx.web` to fail.
* Hand-built mounting bypasses that path, so this test unwraps through the real Loader first; see
* postmortem 0001.
*/
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
@@ -28,6 +33,7 @@ describe('dsh-tool-web real-load-path guard', () => {
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(toolWeb) as Parameters<Context['plugin']>[0]
// Mounting the collapsed shape would throw for missing injection here.
const fiber = await ctx.plugin(unwrapped)
expect(ctx.tools.schemas().map(s => s.name)).toEqual(expect.arrayContaining(['web_search', 'web_fetch']))
await fiber.dispose()

View File

@@ -8,7 +8,9 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, a resource-backstop timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource.
The provider's `timeoutMs`/`maxTimeoutMs` is a **resource backstop** for direct `ctx.web.fetch()` callers and misconfigured deployments — it is NOT the model-facing tool-call budget. The tool-call budget for `web_fetch` is deployment policy owned by [`@deepseek-ai/dsh-timeout-policy`](../../timeout/timeout-policy/README.md), which arms a per-call deadline on `exec.signal`. A shipped web-tool deployment sets the provider backstop **above** the `tool-timeout` budget, so the tool-call policy normally wins for model calls (returning `TOOL_TIMEOUT`); when the outer deadline signal reaches this provider first, it classifies as `WEB_ABORTED` and the outer wrapper replaces the result with `TOOL_TIMEOUT`. The provider's own `WEB_FETCH_TIMEOUT` only fires for a direct seam caller whose own budget elapsed.
The provider's `timeoutMs`/`maxTimeoutMs` is a resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments, not the model-facing tool-call budget. [`dsh-timeout-policy`](../../timeout/timeout-policy/README.md) owns the `web_fetch` tool-call budget by arming `exec.signal`.
A shipping web-tool deployment sets the provider backstop above the tool budget, so model calls normally return `TOOL_TIMEOUT`. If the outer deadline reaches the provider first, the provider reports `WEB_ABORTED` and the outer policy replaces it with `TOOL_TIMEOUT`. `WEB_FETCH_TIMEOUT` therefore identifies a direct seam caller whose provider budget elapsed.
## Transport hygiene

View File

@@ -1,7 +1,10 @@
/**
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public HTTP(S) URL with
* platform-native `fetch` at the repo's Node floor and returns a status code plus bounded
* decoded content.
* Safe HTTP(S) retrieval for `ctx.web`: validates URLs, follows only same-origin redirects,
* enforces time and size limits, classifies and decodes text, and leaves presentation to
* `@deepseek-ai/dsh-tool-web`. Requests carry no browser cookies or ambient credentials.
*
* Private-network and SSRF protection is not implemented; do not enable this provider where
* it can reach sensitive internal targets.
* @module @deepseek-ai/dsh-web-fetch-local/provider
*/
@@ -46,8 +49,8 @@ export class LocalFetchProvider implements WebFetchProvider {
if (exec?.signal?.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED')
const timeoutMs = clampTimeout(request.timeoutMs, this.limits.timeoutMs, this.limits.maxTimeoutMs)
// One deadline signal fuses the caller's abort with our own timeout, so the network request
// and the streaming read both stop on either.
// One signal stops both the request and body read. The deadline's TimeoutReason later
// distinguishes this provider's timeout from caller or outer-deadline cancellation.
using d = deadline(exec?.signal, timeoutMs, 'WEB_FETCH_TIMEOUT')
return await this.followAndRead(request.url, d.signal)
}

View File

@@ -33,4 +33,8 @@ It reuses `$DEEPSEEK_API_KEY` (no new secret) but **not** `$DEEPSEEK_BASE_URL`:
## Mapping
DeepSeek returns no provider-generated answer surface this provider trusts as `content`, so `content` is omitted. `sources[]` is built from the `web_search_result` items inside `web_search_tool_result` blocks: `url``url`, `title``title`, `publishedAt``page_age`. The per-source `snippet` lives separately in a `text` block's `citations[]` (a `cited_text` keyed by `url`), so the provider joins the two — a result with no citation excerpt simply has no `snippet`. Results are deduped by `url` (a `maxUses > 1` request can surface the same URL across searches). DeepSeek's `web_search` has no result-count knob (only `maxUses`), so `maxResults` is enforced by the seam (truncating `sources[]` and setting `truncated`). Provider failures surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`.
DeepSeek returns no provider-generated answer surface this provider trusts as `content`, so `content` is omitted. `sources[]` comes from `web_search_result` items inside `web_search_tool_result` blocks: `url``url`, `title``title`, and `publishedAt``page_age`. Snippets live separately as URL-keyed `cited_text` entries in a text block's `citations[]`; the provider joins them, leaving `snippet` absent when no excerpt exists.
Results are deduplicated by URL because one request may surface the same page across searches. DeepSeek exposes `maxUses`, not a result-count knob, so the seam enforces `maxResults` by truncating `sources[]` and setting `truncated`.
Provider failures become `WEB_PROVIDER_ERROR`; caller cancellation becomes `WEB_ABORTED`.

View File

@@ -1,6 +1,7 @@
/**
* `@deepseek-ai/dsh-web-search-deepseek`: registers a DeepSeek-backed `WebSearchProvider` with
* `ctx.web`.
* Register a DeepSeek-backed provider in `ctx.web`. It calls the Anthropic-compatible Messages API
* with native `web_search_20250305`. The provider reuses `DEEPSEEK_API_KEY` but not
* `DEEPSEEK_BASE_URL`, because search and chat-completions use different bases.
* @module @deepseek-ai/dsh-web-search-deepseek
*/

View File

@@ -1,6 +1,8 @@
/**
* `DeepSeekSearchProvider`: a `WebSearchProvider` backed by DeepSeek's Anthropic-compatible
* Messages API with the native `web_search_20250305` server tool enabled.
* DeepSeek search through an Anthropic-compatible Messages model call with the native
* `web_search_20250305` server tool. Each search costs a model turn, but returns structured
* result blocks; absence of those blocks is an error rather than a prose-scraping fallback.
* The wire format and native `fetch` client are provider-private and do not use `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-deepseek/provider
*/
@@ -94,6 +96,7 @@ export function citationSnippets(blocks: readonly ContentBlock[]): Map<string, s
* @param query - the original request query, echoed on the result.
* @param response - the parsed Messages response body.
* @returns the normalized result with deduped, snippet-joined sources.
* @throws {@link WebError} when native search produced no result block.
*/
export function mapAnthropicResponse(query: string, response: AnthropicResponse): WebSearchResult {
const blocks = response.content ?? []

View File

@@ -1,6 +1,7 @@
/**
* Wire types for DeepSeek's Anthropic-compatible Messages API (`POST {baseURL}/messages`) with
* the native `web_search_20250305` server tool enabled.
* Provider-private wire types for DeepSeek's Anthropic-compatible Messages API. Citeable
* result items and citation excerpts arrive in separate blocks; the provider joins them by
* URL. These types do not create a dependency on `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-deepseek/types
*/

View File

@@ -300,7 +300,8 @@ describe('web-search-deepseek plugin registration', () => {
})
it('survives the real Loader unwrapExports path keeping name/inject/Config', () => {
// A default export would make Loader discard the required web injection metadata.
// A default export would make `unwrapExports` collapse the namespace and drop `inject: ['web']`.
// Drive the real Loader path because hand-built namespace mounting cannot expose that failure.
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(deepseekPlugin) as Record<string, unknown>
expect(unwrapped).toBe(deepseekPlugin)

View File

@@ -1,6 +1,8 @@
/**
* `ExaSearchProvider`: a `WebSearchProvider` backed by the Exa search API (`POST /search` with
* highlight contents).
* highlight contents). It maps the first non-blank highlight to `snippet`, maps
* `publishedDate` to `publishedAt`, drops entries without a snippet, and omits `content`
* because Exa returns no generated answer.
* @module @deepseek-ai/dsh-web-search-exa/provider
*/

View File

@@ -1,6 +1,8 @@
/**
* `PerplexitySearchProvider`: a `WebSearchProvider` backed by the Perplexity search API (an
* OpenAI-compatible `POST /chat/completions`).
* Perplexity search over its OpenAI-compatible chat-completions endpoint. The generated answer
* becomes `content`; sources prefer structured `search_results[]` and fall back to URL-only
* `citations[]`. The wire format and native `fetch` client are provider-private and do not use
* `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-perplexity/provider
*/

View File

@@ -1,6 +1,7 @@
/**
* Wire types for the Perplexity search API (`POST https://api.perplexity.ai/chat/completions`,
* an OpenAI-compatible chat shape).
* an OpenAI-compatible chat shape). Results prefer structured `search_results` and fall back to
* URL-only `citations`; the provider-private wire shape does not depend on `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-perplexity/types
*/

View File

@@ -1,9 +1,8 @@
/**
* The web access seam (`ctx.web`): a provider registry plus a provider-selecting execution
* surface for two capabilities — search and fetch. Provider packages register concrete
* backends with `registerSearchProvider` / `registerFetchProvider`; the model-facing consumer
* (`@deepseek-ai/dsh-tool-web`) executes through `search()` / `fetch()` and routes on the
* structured {@link WebError} codes selection throws.
* The web access seam (`ctx.web`): registries and provider-selecting execution for search and
* fetch. Duplicate ids are rejected. At execution time, a configured provider must exist and
* be usable; without one, exactly one usable provider is required, so selection never depends
* on registration order.
* @module @deepseek-ai/dsh-web
*/

View File

@@ -1,7 +1,7 @@
/**
* Vocabulary for the web capability seam (`ctx.web`): the search/fetch request/result shapes
* providers produce and consumers format, the provider status discriminant selection reads,
* the execution-control context, and the typed error taxonomy.
* Vocabulary for the web capability seam (`ctx.web`). Search and fetch deliberately share one
* seam so provider selection, cancellation, errors, and product configuration have one owner,
* while retaining separate request and result shapes.
* @module @deepseek-ai/dsh-web/types
*/
@@ -150,9 +150,11 @@ export interface WebFetchProvider {
}
/**
* Typed web error. Extends {@link HarnessError} so it carries a stable, machine-routable
* `code` (a `string`, like every other seam's error) and chains `cause`.
* `ToolRegistry.execute()` converts a thrown `WebError` into an error tool result whose
* structured metadata exposes the code, so callers (hooks, tests, UI) route on it.
* Typed web error with a machine-routable, open-string `code` and chained `cause`.
* Consumers must tolerate provider-specific codes. Shared codes cover unavailable,
* missing, unusable, ambiguous, or duplicate providers, cancellation, and provider failure;
* the local fetch provider additionally distinguishes invalid or blocked URLs, redirects,
* size and timeout limits, and unsupported content types. Tool execution exposes the code in
* structured error metadata.
*/
export class WebError extends HarnessError {}