docs: trim generated prose

This commit is contained in:
Tianyi Cui
2026-07-12 03:36:43 +08:00
parent 3dca90261c
commit 75838e10b5
323 changed files with 2857 additions and 11833 deletions

View File

@@ -1,15 +1,8 @@
/**
* The model-facing `web_fetch` tool: retrieve the content of a specific URL.
* Execution goes through `ctx.web` — this module owns the model-facing schema,
* argument validation, and PRESENTATION (HTML→markdown, truncation formatting),
* while the fetch provider owns safe retrieval (transport, redirects, caps).
*
* The model-facing schema exposes NO timeout knob: the tool-call budget is
* deployment policy DECLARED via this package's `fetchTimeoutMs` config (attached
* as `ToolDefinition.timeoutMs`) and ENFORCED by `@deepseek-ai/dsh-timeout-policy`
* (a `tools/execute` wrapper), matching the reference-agent `WebFetch` shape. This
* tool just forwards the (possibly deadline-derived) `exec.signal` to `ctx.web`;
* the provider keeps its own timeout only as a resource backstop for direct callers.
* The model-facing `web_fetch` tool: retrieve the content of a specific URL. Execution goes
* through `ctx.web` — this module owns the model-facing schema, argument validation, and
* PRESENTATION (HTML→markdown, truncation formatting), while the fetch provider owns safe
* retrieval (transport, redirects, caps).
*/
import type { Context } from 'cordis'

View File

@@ -1,11 +1,5 @@
/**
* Minimal, dependency-free HTML→markdown-ish text conversion for `web_fetch`
* presentation. This is intentionally NOT a full HTML parser: it strips
* script/style/noscript, drops tags, decodes the common named/numeric entities,
* and collapses whitespace into a readable plain-text approximation with a few
* markdown affordances (headings, list bullets, links). A heavier converter can
* replace this without touching the seam or the tool schema.
*
* Minimal, dependency-free HTML→markdown-ish text conversion for `web_fetch` presentation.
* @module @deepseek-ai/dsh-tool-web/html
*/

View File

@@ -1,19 +1,7 @@
/**
* The model-facing web tool suite (`web_search`, `web_fetch`) over the `ctx.web`
* seam. This root plugin registers the tools the product has ENABLED, composing
* the per-tool registration helpers (`applyWebSearchTool`, `applyWebFetchTool`).
*
* The package owns model-facing concerns only — tool names, JSON schemas,
* argument validation, prompt sections, result-cap constants, result formatting,
* HTML→markdown presentation. All web access goes through `ctx.web`; this
* package never imports a concrete provider package.
*
* Tool registration follows product/app ENABLEMENT, not backend availability: a
* tool stays visible even when its selected provider is missing/misconfigured,
* and execution fails with a structured `WebError` (resolved by the seam at call
* time). That keeps the model schema stable without making plugin load order,
* credential state, or HMR timing part of the model-facing contract.
*
* The model-facing web tool suite (`web_search`, `web_fetch`) over the `ctx.web` seam. This
* root plugin registers the tools the product has ENABLED, composing the per-tool registration
* helpers (`applyWebSearchTool`, `applyWebFetchTool`).
* @module @deepseek-ai/dsh-tool-web
*/

View File

@@ -1,11 +1,8 @@
/**
* Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search
* provider (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool
* (`dsh-tool-web`) + the tool-call timeout policy (`dsh-timeout-policy`),
* exercised through `ctx.tools.execute()` — nothing bypasses the tool registry.
* Fetch hits a real loopback HTTP server (verifying the WORLD); search runs the
* real Exa provider over a stubbed global `fetch` (the network is the one
* boundary we mock).
* Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search provider
* (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool (`dsh-tool-web`) + the
* tool-call timeout policy (`dsh-timeout-policy`), exercised through `ctx.tools.execute()` —
* nothing bypasses the tool registry.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
@@ -159,10 +156,8 @@ describe('tool-call timeout returns TOOL_TIMEOUT (deadline wins over a slow fetc
})
it('the provider backstop still protects a DIRECT ctx.web.fetch() call (no tool-call policy in that path)', async () => {
// A direct seam caller does not go through tools/execute, so the tool-call
// policy never applies; the provider's OWN timeout is the only budget. A
// short per-request hint proves the provider backstop is intact and classifies
// as WEB_FETCH_TIMEOUT (the provider-owned code), never TOOL_TIMEOUT.
// A direct seam caller does not go through tools/execute, so the tool-call policy never
// applies; the provider's own timeout is the only budget.
const err = await tctx.web.fetch({ url: slowBase, timeoutMs: 50 }).then(
() => undefined,
(e: unknown) => e as { code?: string },

View File

@@ -1,17 +1,4 @@
/**
* Real-load-path guard for @deepseek-ai/dsh-tool-web. `tool-web` is a NAMESPACE
* plugin with `inject` — so a stray `export default apply` would make the cordis
* Loader's `unwrapExports` (`exports.default ?? exports`) collapse the module to
* the bare `apply` function, DROPPING `inject`. The plugin would then read
* `ctx.web` without having injected it and throw `cannot get property … without
* inject` the moment it loads (postmortem 0001).
*
* A hand-built `ctx.plugin({ apply, inject })` mount CANNOT catch that — it
* bypasses `unwrapExports`. So this test unwraps the module through the REAL
* `Loader.prototype.unwrapExports` and mounts the result over `ctx.web`,
* exercising the exact path the Loader uses. Prove the guard bites: add
* `export default apply` to `src/index.ts`, watch this go red, revert.
*/
/** Real Loader-path coverage for the namespace plugin's export shape. */
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
@@ -41,7 +28,6 @@ describe('dsh-tool-web real-load-path guard', () => {
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(toolWeb) as Parameters<Context['plugin']>[0]
// A collapsed export shape (dropped inject) would throw "without inject" here.
const fiber = await ctx.plugin(unwrapped)
expect(ctx.tools.schemas().map(s => s.name)).toEqual(expect.arrayContaining(['web_search', 'web_fetch']))
await fiber.dispose()

View File

@@ -1,21 +1,7 @@
/**
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public
* HTTP(S) URL with platform-native `fetch` at the repo's Node floor and returns a status
* code plus bounded decoded content. It owns SAFE RESOURCE RETRIEVAL — URL
* validation, redirect policy, timeout, abort, byte caps, charset decoding,
* content-type classification, binary rejection — but NOT presentation
* (HTML→markdown lives in `@deepseek-ai/dsh-tool-web`).
*
* Redirects are followed manually (`redirect: 'manual'`) so the provider can
* enforce a same-origin-only policy: a cross-origin redirect is refused with
* `WEB_REDIRECT_BLOCKED`, requiring a fresh tool call (Claude Code's WebFetch
* uses the same model). It does NOT carry browser cookies, editor/git
* credentials, or implicit access to private services.
*
* SSRF / private-network protection is DEFERRED (see the package RFC); until it
* lands this provider is an SSRF primitive and must not be enabled where it can
* reach sensitive internal targets.
*
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public HTTP(S) URL with
* platform-native `fetch` at the repo's Node floor and returns a status code plus bounded
* decoded content.
* @module @deepseek-ai/dsh-web-fetch-local/provider
*/
@@ -60,11 +46,8 @@ export class LocalFetchProvider implements WebFetchProvider {
if (exec?.signal?.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED')
const timeoutMs = clampTimeout(request.timeoutMs, this.limits.timeoutMs, this.limits.maxTimeoutMs)
// One deadline signal fuses the caller's abort with our own timeout, so the
// network request and the streaming read both stop on either. The timeout
// abort carries a TimeoutReason we recover afterward to classify the cause
// (translateAbortOrNetwork), instead of hand-rolling a controller + timer +
// reason-recovery dance.
// One deadline signal fuses the caller's abort with our own timeout, so the network request
// and the streaming read both stop on either.
using d = deadline(exec?.signal, timeoutMs, 'WEB_FETCH_TIMEOUT')
return await this.followAndRead(request.url, d.signal)
}
@@ -78,11 +61,7 @@ export class LocalFetchProvider implements WebFetchProvider {
const response = await this.requestOnce(currentUrl, signal)
if (isRedirectStatus(response.status)) {
// The redirect budget is enforced BEFORE this hop's target is resolved
// or origin-checked, so `maxRedirects: N` follows at most N redirects
// exactly: the (N+1)th redirect is refused as "exceeded" regardless of
// where it points (a same-origin/cross-origin distinction on a hop we
// are not allowed to follow would be the wrong diagnosis).
// Enforce the redirect budget before resolving or validating the next hop.
if (redirectsFollowed >= this.limits.maxRedirects) {
await response.body?.cancel()
throw new WebError(`exceeded the maximum of ${this.limits.maxRedirects} redirects`, 'WEB_REDIRECT_BLOCKED')
@@ -95,10 +74,9 @@ export class LocalFetchProvider implements WebFetchProvider {
throw new WebError(`redirect response (HTTP ${response.status}) without a Location header`, 'WEB_PROVIDER_ERROR')
}
const target = resolveRedirect(location, currentUrl)
// Re-validate the target against the same transport hygiene a direct
// request gets: a redirect must not be a back door to a credentialed,
// non-http(s), or over-long URL that validateFetchUrl would reject. A
// rejection here must still cancel the body first (see below).
// Re-validate the target against the same transport hygiene a direct request gets: a
// redirect must not be a back door to a credentialed, non-http(s), or over-long URL
// that validateFetchUrl would reject.
let validatedTarget: URL
try {
validatedTarget = validateFetchUrl(target.toString(), this.limits.maxUrlLength)

View File

@@ -1,15 +1,6 @@
/**
* `@deepseek-ai/dsh-web-search-deepseek`: registers a DeepSeek-backed
* `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a
* default-export service): it registers INTO the seam's provider registry, like
* `@deepseek-ai/dsh-llm-deepseek` registers an adapter into `ctx.llm`.
*
* The provider talks to DeepSeek's Anthropic-compatible Messages API with the
* native `web_search_20250305` server tool. It reuses `$DEEPSEEK_API_KEY` (no
* new secret) but NOT `$DEEPSEEK_BASE_URL` — the search endpoint is the
* Anthropic-compatible base, distinct from the chat-completions base the LLM
* adapter uses.
*
* `@deepseek-ai/dsh-web-search-deepseek`: registers a DeepSeek-backed `WebSearchProvider` with
* `ctx.web`.
* @module @deepseek-ai/dsh-web-search-deepseek
*/

View File

@@ -1,22 +1,6 @@
/**
* `DeepSeekSearchProvider`: a `WebSearchProvider` backed by DeepSeek's
* Anthropic-compatible Messages API with the native `web_search_20250305` server
* tool enabled.
*
* Unlike a dedicated search endpoint (Exa's `POST /search`, Perplexity's
* `/chat/completions`), this issues a FULL Messages model call carrying a server
* tool, so a search costs a complete model turn in latency and tokens. In return
* DeepSeek runs the search server-side and returns STRUCTURED
* `web_search_tool_result` blocks — this provider parses those blocks and never
* scrapes URLs out of model prose. Strict mode: if the response carries no
* `web_search_tool_result` block (native search did not trigger), it throws
* `WEB_PROVIDER_ERROR` rather than degrading to prose-scraping.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
* The Anthropic wire shape is a provider-private detail and does NOT make this
* provider depend on `ctx.llm`.
*
* `DeepSeekSearchProvider`: a `WebSearchProvider` backed by DeepSeek's Anthropic-compatible
* Messages API with the native `web_search_20250305` server tool enabled.
* @module @deepseek-ai/dsh-web-search-deepseek/provider
*/
@@ -101,15 +85,11 @@ export function citationSnippets(blocks: readonly ContentBlock[]): Map<string, s
}
/**
* Map a DeepSeek Anthropic Messages response to a normalized search result.
* Walks `web_search_tool_result` blocks for citeable `web_search_result` items,
* joins each to its citation excerpt as `snippet`, and dedupes by `url` (a
* `max_uses > 1` request can surface the same URL across searches). The seam
* owns the final `maxResults` truncation, so `truncated` is always `false` here.
*
* Throws `WEB_PROVIDER_ERROR` (strict mode) when no `web_search_tool_result`
* block is present — native search did not trigger, and prose-scraping is not a
* fallback.
* Map a DeepSeek Anthropic Messages response to a normalized search result. Walks
* `web_search_tool_result` blocks for citeable `web_search_result` items, joins each to its
* citation excerpt as `snippet`, and dedupes by `url` (a `max_uses > 1` request can surface
* the same URL across searches). The seam owns the final `maxResults` truncation, so
* `truncated` is always `false` here.
*
* @param query - the original request query, echoed on the result.
* @param response - the parsed Messages response body.

View File

@@ -1,16 +1,6 @@
/**
* Wire types for DeepSeek's Anthropic-compatible Messages API
* (`POST {baseURL}/messages`) with the native `web_search_20250305` server tool
* enabled. Types only — no runtime code.
*
* DeepSeek returns structured content blocks: `web_search_tool_result` blocks
* carry the citeable `web_search_result` items (`url`/`title`/`page_age`), while
* the snippet/excerpt for a URL lives separately in a `text` block's
* `citations[]` (a `cited_text` keyed by `url`). The provider joins the two.
*
* The Anthropic wire shape is a provider-private detail; it does not make this
* provider depend on `ctx.llm`.
*
* Wire types for DeepSeek's Anthropic-compatible Messages API (`POST {baseURL}/messages`) with
* the native `web_search_20250305` server tool enabled.
* @module @deepseek-ai/dsh-web-search-deepseek/types
*/

View File

@@ -300,14 +300,7 @@ describe('web-search-deepseek plugin registration', () => {
})
it('survives the real Loader unwrapExports path keeping name/inject/Config', () => {
// A stray `export default apply` would make the cordis Loader's
// unwrapExports (`exports.default ?? exports`) collapse the module to the
// bare `apply` function, DROPPING `inject: ['web']` — the plugin would then
// read ctx.web without injecting it and throw "cannot get property … without
// inject" the moment it loads. A hand-built ctx.plugin(namespace) mount
// bypasses unwrapExports and cannot catch that, so drive the real path.
// Prove it bites: add `export default apply` to src/index.ts, watch this go
// red, revert.
// A default export would make Loader discard the required web injection metadata.
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(deepseekPlugin) as Record<string, unknown>
expect(unwrapped).toBe(deepseekPlugin)

View File

@@ -1,14 +1,6 @@
/**
* `ExaSearchProvider`: a `WebSearchProvider` backed by the Exa search API
* (`POST /search` with highlight contents). Maps Exa's flat `results[]` into the
* seam's normalized `WebSearchResult`. Exa returns no provider-generated answer,
* so `content` is omitted; each result maps to a `WebSearchSource` with `url`,
* `title`, the first highlight as `snippet`, and `publishedDate` as
* `publishedAt`.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
*
* `ExaSearchProvider`: a `WebSearchProvider` backed by the Exa search API (`POST /search` with
* highlight contents).
* @module @deepseek-ai/dsh-web-search-exa/provider
*/

View File

@@ -1,15 +1,6 @@
/**
* `PerplexitySearchProvider`: a `WebSearchProvider` backed by the Perplexity
* search API (an OpenAI-compatible `POST /chat/completions`). Maps the generated
* answer (`choices[0].message.content`) into `content`, and prefers the
* structured `search_results[]` for `sources[]`, falling back to the URL-only
* `citations[]` when `search_results` is absent.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter. The OpenAI-compatible request shape
* is a provider-private detail and does NOT make this provider depend on
* `ctx.llm`.
*
* `PerplexitySearchProvider`: a `WebSearchProvider` backed by the Perplexity search API (an
* OpenAI-compatible `POST /chat/completions`).
* @module @deepseek-ai/dsh-web-search-perplexity/provider
*/

View File

@@ -1,13 +1,6 @@
/**
* Wire types for the Perplexity search API
* (`POST https://api.perplexity.ai/chat/completions`, an OpenAI-compatible chat
* shape). Types only — no runtime code. Perplexity returns a generated answer in
* `choices[0].message.content` plus citation surfaces: a structured
* `search_results[]` (preferred) and a URL-only `citations[]` fallback.
*
* The OpenAI-compatible wire shape is a provider-private detail; it does not make
* this provider depend on `ctx.llm`.
*
* Wire types for the Perplexity search API (`POST https://api.perplexity.ai/chat/completions`,
* an OpenAI-compatible chat shape).
* @module @deepseek-ai/dsh-web-search-perplexity/types
*/

View File

@@ -1,17 +1,9 @@
/**
* The web access seam (`ctx.web`): a provider registry plus a provider-selecting
* execution surface for two capabilities — search and fetch. Provider packages
* register concrete backends with `registerSearchProvider` /
* `registerFetchProvider`; the model-facing consumer
* (`@deepseek-ai/dsh-tool-web`) executes through `search()` / `fetch()` and
* routes on the structured {@link WebError} codes selection throws.
*
* The registry half stays close to `LlmService`: a `Map<id, provider>` per
* capability kind, register methods that return disposers, duplicate ids that
* throw, and execution-time resolution that throws when the selected provider is
* absent or unusable — with selection rules that never depend on registration
* order.
*
* The web access seam (`ctx.web`): a provider registry plus a provider-selecting execution
* surface for two capabilities — search and fetch. Provider packages register concrete
* backends with `registerSearchProvider` / `registerFetchProvider`; the model-facing consumer
* (`@deepseek-ai/dsh-tool-web`) executes through `search()` / `fetch()` and routes on the
* structured {@link WebError} codes selection throws.
* @module @deepseek-ai/dsh-web
*/

View File

@@ -1,19 +1,7 @@
/**
* Vocabulary for the web capability seam (`ctx.web`): the search/fetch
* request/result shapes providers produce and consumers format, the provider
* status discriminant selection reads, the execution-control context, and the
* typed error taxonomy.
*
* These types are shared by every provider backend
* (`@deepseek-ai/dsh-web-search-exa`, `@deepseek-ai/dsh-web-search-perplexity`,
* `@deepseek-ai/dsh-web-fetch-local`, and future backends) and by the
* model-facing consumer (`@deepseek-ai/dsh-tool-web`). Search and fetch share no
* request schema and no business logic, but they are deliberately one seam:
* `ctx.web` is a single web-access middle layer with one provider-selection
* policy, one abort/error vocabulary, and one product-facing configuration
* point. The cost is the parallel `Search`/`Fetch` shapes below; that
* parallelism is intentional.
*
* Vocabulary for the web capability seam (`ctx.web`): the search/fetch request/result shapes
* providers produce and consumers format, the provider status discriminant selection reads,
* the execution-control context, and the typed error taxonomy.
* @module @deepseek-ai/dsh-web/types
*/
@@ -162,39 +150,9 @@ export interface WebFetchProvider {
}
/**
* Typed web error. Extends {@link HarnessError} so it carries a stable,
* machine-routable `code` (a `string`, like every other seam's error) and
* chains `cause`. `ToolRegistry.execute()` converts a thrown `WebError` into an
* error tool result whose structured metadata exposes the code, so callers
* (hooks, tests, UI) route on it.
*
* The `code` is an open `string`, NOT a closed union: a provider may raise its
* own codes without editing this package, and a consumer must tolerate an
* unknown code (a future provider will introduce ones this file never named).
* The codes split by who owns them — seam-neutral codes any provider may see,
* versus codes specific to a single implementation:
*
* Seam-neutral (raised by `WebService` selection and the shared contract):
* - `WEB_PROVIDER_UNAVAILABLE`: no provider configured and none usable.
* - `WEB_PROVIDER_CONFIGURED_MISSING`: a configured id is not registered.
* - `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`: a configured id is registered but its
* `status()` reports unavailable.
* - `WEB_PROVIDER_AMBIGUOUS`: no id configured and multiple usable providers
* exist (selection refuses to pick by registration order).
* - `WEB_DUPLICATE_PROVIDER`: a registration-time programming error — an id is
* already registered for that capability kind.
* - `WEB_ABORTED`: the operation was aborted via `WebExecContext.signal`.
* - `WEB_PROVIDER_ERROR`: catch-all for a provider's own failure surfaced
* through the seam, including network/transport failure (DNS, connection
* refused, TLS).
*
* Fetch-transport codes (owned by the `dsh-web-fetch-local` implementation; a
* different fetch backend need not raise these and may raise its own):
* - `WEB_INVALID_URL`: the fetch URL is malformed or not http(s).
* - `WEB_BLOCKED_URL`: the fetch URL is rejected by policy (credentials in URL).
* - `WEB_REDIRECT_BLOCKED`: a cross-origin redirect was refused.
* - `WEB_FETCH_TOO_LARGE`: the response exceeded the byte/character cap.
* - `WEB_FETCH_TIMEOUT`: the fetch exceeded its timeout.
* - `WEB_UNSUPPORTED_CONTENT_TYPE`: the response content type cannot be decoded.
* Typed web error. Extends {@link HarnessError} so it carries a stable, machine-routable
* `code` (a `string`, like every other seam's error) and chains `cause`.
* `ToolRegistry.execute()` converts a thrown `WebError` into an error tool result whose
* structured metadata exposes the code, so callers (hooks, tests, UI) route on it.
*/
export class WebError extends HarnessError {}