Merge origin/master into codex/truncated-design

This commit is contained in:
Dudu-0223
2026-07-17 18:21:54 +08:00
1057 changed files with 42961 additions and 18526 deletions

View File

@@ -32,4 +32,54 @@ Each tool is registered independently; a product that wants only one disables th
Tool registration follows product **enablement**, not backend availability. A tool stays visible even when its selected provider is missing, misconfigured, ambiguous, or temporarily unavailable; the seam resolves the provider at execution time and execution fails with a structured `WebError` (e.g. `WEB_PROVIDER_UNAVAILABLE`, `WEB_PROVIDER_AMBIGUOUS`), which `ToolRegistry.execute()` turns into an error tool result the model can read and hooks/UI can route on. This keeps the model schema stable without making plugin load order, credential state, or HMR timing part of the model-facing contract. To remove a web tool entirely, disable it here in config.
The tool never calls a provider's `status()` and never enumerates providers — its only execution path is `ctx.web.search()` / `ctx.web.fetch()`, and provider unavailability reaches it as the structured `WebError` codes selection throws at execution time. Provider selection stays entirely inside the seam, with one owner.
The tool never calls a provider's `available()` and never enumerates providers — its only execution path is `ctx.web.search()` / `ctx.web.fetch()`, and provider unavailability reaches it as the structured `WebError` codes selection throws at execution time. Provider selection stays entirely inside the seam, with one owner.
## Model Experience
### System prompt
**What the model sees**: Search and fetch contribute the web-search and web-fetch guidance below. A scoped tool restriction does not remove these independently registered sections.
**Token effect**: Fixed guidance cost per request for each config-enabled tool, even when a restriction hides its schema.
#### Web search guidance
```markdown
Use the web_search tool to discover current information on the web. It returns an optional answer plus a list of source URLs. Follow up with web_fetch when you need the full content of a specific result, and cite the relevant URLs as markdown links.
```
#### Web fetch guidance
```markdown
Use the web_fetch tool to retrieve the content of a specific HTTP(S) URL (for example a result from web_search). It returns the page content decoded to text. Cite the URL as a markdown link when you use its content.
```
### Tool schemas
**What the model sees**: The model sees the generated [`web_search` and `web_fetch` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-web). Result-count and timeout budgets are deployment settings, not model arguments.
**Token effect**: Fixed schema cost per request; config disablement removes both schema and guidance, while a scoped restriction removes only the schema.
### Search result
**What the model sees**: The optional provider-owned answer is followed by `Sources:` and data-dependent lines shaped exactly `- [<title-or-url>](<url>)`, optionally suffixed ` — <snippet> (<publishedAt>)`. With neither answer nor sources the result says `No results found.` A capped list adds `(Showing the first <count> sources. Refine the query for more.)`; every result ends `Cite the relevant URLs above as markdown links in your answer.`
**Token effect**: Data-dependent results are resent until compaction and sources are capped by `searchMaxResults`.
### Fetch result
**What the model sees**: A successful fetch is exactly `Fetched <finalUrl> (HTTP <statusCode>)`, a blank line, and the provider-owned decoded body. Truncation adds a blank line and `(Content truncated. Fetch a more specific URL or section for the full text.)`; failures become `Error: <message>`. Queries and URLs remain in call history.
**Token effect**: Provider caps bound body size; retained call arguments and results are resent until compaction, and timeout policy can replace a late result with a short error.
### Argument errors
**What the model sees**: Blank inputs become exactly `Error: query must be a non-empty string` or `Error: url must be a non-empty string`.
**Token effect**: Only the failing call adds these retained tokens.
## Known Limitations and Deferred Work
- **`htmlToMarkdown` is a minimal regex converter, not an HTML parser** — it strips script/style/noscript, keeps headings/bullets/links, and decodes about a dozen named entities; tables, images, and nested formatting are lost.
- **The model-facing surface is minimal by design, with promotions deferred** — `max_results` stays a config bound (not a model argument), and `web_fetch` takes only `url` (no `format`/`prompt`/LLM-summarization mode); both are named later steps in [the seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md).
- **No web-specific permission policy** — both tools execute without requesting `ctx.approval`; a deployment that needs confirmation must add a `tools/pre-execute` policy, and the package does not define persistent URL/domain grants.

View File

@@ -26,7 +26,7 @@
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
"@deepseek-ai/dsh-tools": "^0.0.1",
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
@@ -43,6 +43,6 @@
"@deepseek-ai/dsh-web": "workspace:^",
"@deepseek-ai/dsh-web-fetch-local": "workspace:^",
"@deepseek-ai/dsh-web-search-exa": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -1,15 +1,8 @@
/**
* The model-facing `web_fetch` tool: retrieve the content of a specific URL.
* Execution goes through `ctx.web` — this module owns the model-facing schema,
* argument validation, and PRESENTATION (HTML→markdown, truncation formatting),
* while the fetch provider owns safe retrieval (transport, redirects, caps).
*
* The model-facing schema exposes NO timeout knob: the tool-call budget is
* deployment policy DECLARED via this package's `fetchTimeoutMs` config (attached
* as `ToolDefinition.timeoutMs`) and ENFORCED by `@deepseek-ai/dsh-timeout-policy`
* (a `tools/execute` wrapper), matching the reference-agent `WebFetch` shape. This
* tool just forwards the (possibly deadline-derived) `exec.signal` to `ctx.web`;
* the provider keeps its own timeout only as a resource backstop for direct callers.
* The model-facing `web_fetch` tool. This module owns its schema, validation, and presentation;
* `ctx.web` owns retrieval. Timeout is deployment policy, not a model argument: config becomes
* `ToolDefinition.timeoutMs`, timeout policy enforces it, and this tool forwards the resulting
* signal. A provider timeout remains a backstop for direct seam callers.
*/
import type { Context } from 'cordis'
@@ -103,7 +96,7 @@ export function applyWebFetchTool(ctx: Context, timeoutMs: number): void {
const input = parseFetchArgs(args)
const result = await ctx.web.fetch(
{ url: input.url },
exec.signal ? { signal: exec.signal } : undefined,
exec.signal,
)
return [{ type: 'text', text: formatFetchOutput(result) }]
},

View File

@@ -1,11 +1,8 @@
/**
* Minimal, dependency-free HTML→markdown-ish text conversion for `web_fetch`
* presentation. This is intentionally NOT a full HTML parser: it strips
* script/style/noscript, drops tags, decodes the common named/numeric entities,
* and collapses whitespace into a readable plain-text approximation with a few
* markdown affordances (headings, list bullets, links). A heavier converter can
* replace this without touching the seam or the tool schema.
*
* Minimal dependency-free HTML-to-readable-text conversion for `web_fetch`, not a full parser. It
* removes non-content elements and tags, decodes common entities, collapses whitespace, and keeps
* basic headings, lists, and links. A richer converter can replace it without changing the seam or
* tool schema.
* @module @deepseek-ai/dsh-tool-web/html
*/

View File

@@ -1,19 +1,8 @@
/**
* The model-facing web tool suite (`web_search`, `web_fetch`) over the `ctx.web`
* seam. This root plugin registers the tools the product has ENABLED, composing
* the per-tool registration helpers (`applyWebSearchTool`, `applyWebFetchTool`).
*
* The package owns model-facing concerns only — tool names, JSON schemas,
* argument validation, prompt sections, result-cap constants, result formatting,
* HTML→markdown presentation. All web access goes through `ctx.web`; this
* package never imports a concrete provider package.
*
* Tool registration follows product/app ENABLEMENT, not backend availability: a
* tool stays visible even when its selected provider is missing/misconfigured,
* and execution fails with a structured `WebError` (resolved by the seam at call
* time). That keeps the model schema stable without making plugin load order,
* credential state, or HMR timing part of the model-facing contract.
*
* Model-facing `web_search` and `web_fetch` tools over `ctx.web`. This package owns schemas,
* validation, prompt guidance, limits, and presentation, never concrete providers. Enablement
* controls tool registration; an enabled tool remains visible when its provider is unavailable
* and fails with a structured error at execution time.
* @module @deepseek-ai/dsh-tool-web
*/

View File

@@ -113,7 +113,7 @@ export function applyWebSearchTool(ctx: Context, maxResults: number, timeoutMs:
const input = parseSearchArgs(args)
const result = await ctx.web.search(
{ query: input.query, maxResults },
exec.signal ? { signal: exec.signal } : undefined,
exec.signal,
)
return [{ type: 'text', text: formatSearchOutput(result) }]
},

View File

@@ -1,11 +1,9 @@
/**
* Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search
* provider (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool
* (`dsh-tool-web`) + the tool-call timeout policy (`dsh-timeout-policy`),
* exercised through `ctx.tools.execute()` — nothing bypasses the tool registry.
* Fetch hits a real loopback HTTP server (verifying the WORLD); search runs the
* real Exa provider over a stubbed global `fetch` (the network is the one
* boundary we mock).
* Integration: the real fetch backend (`dsh-web-fetch-local`) + a real search provider
* (`dsh-web-search-exa`) + the real seam (`dsh-web`) + the model tool (`dsh-tool-web`) + the
* tool-call timeout policy (`dsh-timeout-policy`), exercised through `ctx.tools.execute()` —
* nothing bypasses the tool registry. Fetch verifies world effects against loopback HTTP; search
* uses the real Exa provider with only its network boundary stubbed.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
@@ -136,7 +134,7 @@ describe('tool-call timeout returns TOOL_TIMEOUT (deadline wins over a slow fetc
await tctx.plugin(ToolRegistry)
await tctx.plugin(WebService, { fetchProvider: WebFetchLocal.LOCAL_FETCH_PROVIDER_ID })
// Provider backstop well ABOVE the tool-call budget, so the policy wins.
await tctx.plugin(WebFetchLocal, { timeoutMs: 30_000, maxTimeoutMs: 60_000 })
await tctx.plugin(WebFetchLocal, { timeoutMs: 30_000 })
await tctx.plugin(TimeoutPolicy)
// The tool-call budget is declared by tool-web config, enforced by the policy.
tfiber = await tctx.plugin(ToolWeb, { fetchTimeoutMs: 50 })
@@ -158,12 +156,18 @@ describe('tool-call timeout returns TOOL_TIMEOUT (deadline wins over a slow fetc
expect(text).toContain('timed out after 50ms')
})
it('the provider backstop still protects a DIRECT ctx.web.fetch() call (no tool-call policy in that path)', async () => {
// A direct seam caller does not go through tools/execute, so the tool-call
// policy never applies; the provider's OWN timeout is the only budget. A
// short per-request hint proves the provider backstop is intact and classifies
// as WEB_FETCH_TIMEOUT (the provider-owned code), never TOOL_TIMEOUT.
const err = await tctx.web.fetch({ url: slowBase, timeoutMs: 50 }).then(
it('the provider backstop still protects a direct provider call (no tool-call policy in that path)', async () => {
// A direct provider caller bypasses tools/execute, so a short configured backstop
// must produce provider-owned WEB_FETCH_TIMEOUT rather than TOOL_TIMEOUT.
const direct = new WebFetchLocal.LocalFetchProvider({
maxUrlLength: 2048,
maxResponseBytes: 5_000_000,
maxBodyChars: 100_000,
timeoutMs: 50,
maxRedirects: 5,
userAgent: 'integration-test',
})
const err = await direct.fetch({ url: slowBase }).then(
() => undefined,
(e: unknown) => e as { code?: string },
)

View File

@@ -1,16 +1,8 @@
/**
* Real-load-path guard for @deepseek-ai/dsh-tool-web. `tool-web` is a NAMESPACE
* plugin with `inject` — so a stray `export default apply` would make the cordis
* Loader's `unwrapExports` (`exports.default ?? exports`) collapse the module to
* the bare `apply` function, DROPPING `inject`. The plugin would then read
* `ctx.web` without having injected it and throw `cannot get property … without
* inject` the moment it loads (postmortem 0001).
*
* A hand-built `ctx.plugin({ apply, inject })` mount CANNOT catch that — it
* bypasses `unwrapExports`. So this test unwraps the module through the REAL
* `Loader.prototype.unwrapExports` and mounts the result over `ctx.web`,
* exercising the exact path the Loader uses. Prove the guard bites: add
* `export default apply` to `src/index.ts`, watch this go red, revert.
* Real Loader-path guard for an injected namespace plugin. A default export would make
* `unwrapExports` collapse the namespace and drop `inject`, causing access to `ctx.web` to fail.
* Hand-built mounting bypasses that path, so this test unwraps through the real Loader first; see
* postmortem 0001.
*/
import { describe, expect, it } from 'vitest'
@@ -41,7 +33,7 @@ describe('dsh-tool-web real-load-path guard', () => {
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(toolWeb) as Parameters<Context['plugin']>[0]
// A collapsed export shape (dropped inject) would throw "without inject" here.
// Mounting the collapsed shape would throw for missing injection here.
const fiber = await ctx.plugin(unwrapped)
expect(ctx.tools.schemas().map(s => s.name)).toEqual(expect.arrayContaining(['web_search', 'web_fetch']))
await fiber.dispose()

View File

@@ -4,7 +4,7 @@ import { CallId } from '@deepseek-ai/dsh-llm'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry from '@deepseek-ai/dsh-tools'
import WebService from '@deepseek-ai/dsh-web'
import type { WebSearchProvider, WebSearchResult, WebProviderStatus } from '@deepseek-ai/dsh-web'
import type { WebSearchProvider, WebSearchResult } from '@deepseek-ai/dsh-web'
import * as ToolWeb from '@deepseek-ai/dsh-tool-web'
import {
formatSearchOutput,
@@ -18,10 +18,10 @@ import {
WEB_SEARCH_MAX_RESULTS,
} from '@deepseek-ai/dsh-tool-web'
const available: WebProviderStatus = { available: true }
const available = true
function searchProvider(result: WebSearchResult, status: WebProviderStatus = available): WebSearchProvider {
return { id: 'stub-search', status: () => status, search: () => Promise.resolve(result) }
function searchProvider(result: WebSearchResult, isAvailable = available): WebSearchProvider {
return { id: 'stub-search', available: () => isAvailable, search: () => Promise.resolve(result) }
}
/** Mount the real registry, seam, and tool-web; return an executor helper. */
@@ -46,7 +46,7 @@ async function mountTools(opts: {
describe('search formatting', () => {
it('renders content, sources with titles/hostnames, snippets, and a citation reminder', () => {
const out = formatSearchOutput({
providerId: 'p', query: 'q', content: 'an answer', truncated: false,
content: 'an answer', truncated: false,
sources: [
{ url: 'https://a.test/x', title: 'A', snippet: 'about a', publishedAt: '2026-01-01' },
{ url: 'https://b.test/y' },
@@ -59,19 +59,19 @@ describe('search formatting', () => {
})
it('reports no results when there is neither content nor sources', () => {
expect(formatSearchOutput({ providerId: 'p', query: 'q', sources: [], truncated: false }))
expect(formatSearchOutput({ sources: [], truncated: false }))
.toContain('No results found.')
})
it('renders content alone when there are no sources', () => {
const out = formatSearchOutput({ providerId: 'p', query: 'q', content: 'just an answer', sources: [], truncated: false })
const out = formatSearchOutput({ content: 'just an answer', sources: [], truncated: false })
expect(out).toContain('just an answer')
expect(out).not.toContain('No results found.')
expect(out).not.toContain('Sources:')
})
it('notes truncation', () => {
const out = formatSearchOutput({ providerId: 'p', query: 'q', sources: [{ url: 'https://a.test' }], truncated: true })
const out = formatSearchOutput({ sources: [{ url: 'https://a.test' }], truncated: true })
expect(out).toContain('Showing the first 1 sources')
})
@@ -88,7 +88,7 @@ describe('search formatting', () => {
describe('fetch formatting', () => {
it('renders an html body to markdown text with a status header', () => {
const out = formatFetchOutput({
providerId: 'p', url: 'https://a.test', statusCode: 200, truncated: false,
url: 'https://a.test', statusCode: 200, truncated: false,
body: { kind: 'html', content: '<h1>Title</h1><p>Body text</p>' },
})
expect(out).toContain('Fetched https://a.test (HTTP 200)')
@@ -98,7 +98,7 @@ describe('fetch formatting', () => {
it('passes a text body through and notes truncation', () => {
const out = formatFetchOutput({
providerId: 'p', url: 'https://a.test', statusCode: 200, truncated: true,
url: 'https://a.test', statusCode: 200, truncated: true,
body: { kind: 'text', content: 'plain' },
})
expect(out).toContain('plain')
@@ -155,7 +155,7 @@ describe('htmlToMarkdown', () => {
})
it('falls back to the raw URL as a source label when the URL is unparseable', () => {
const out = formatSearchOutput({ providerId: 'p', query: 'q', truncated: false, sources: [{ url: 'not a url' }] })
const out = formatSearchOutput({ truncated: false, sources: [{ url: 'not a url' }] })
expect(out).toContain('[not a url](not a url)')
})
})
@@ -209,7 +209,7 @@ describe('tool-web registration', () => {
describe('tool-web execution through the real registry', () => {
it('executes web_search and formats the result', async () => {
const result: WebSearchResult = {
providerId: 'stub-search', query: 'q', content: 'answer', truncated: false,
content: 'answer', truncated: false,
sources: [{ url: 'https://a.test', title: 'A', snippet: 'snip' }],
}
const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider(result) })
@@ -228,8 +228,8 @@ describe('tool-web execution through the real registry', () => {
})
it('surfaces WEB_PROVIDER_AMBIGUOUS for multiple unconfigured providers', async () => {
const { ctx, fiber, call } = await mountTools({ search: searchProvider({ providerId: 'stub-search', query: 'q', sources: [], truncated: false }) })
ctx.web.registerSearchProvider({ id: 'other', status: () => available, search: () => Promise.resolve({ providerId: 'other', query: 'q', sources: [], truncated: false }) })
const { ctx, fiber, call } = await mountTools({ search: searchProvider({ sources: [], truncated: false }) })
ctx.web.registerSearchProvider({ id: 'other', available: () => available, search: () => Promise.resolve({ sources: [], truncated: false }) })
const out = await call('web_search', { query: 'q' })
expect(out.isError).toBe(true)
expect(out.error?.code).toBe('WEB_PROVIDER_AMBIGUOUS')
@@ -237,7 +237,7 @@ describe('tool-web execution through the real registry', () => {
})
it('rejects invalid arguments with a structured INVALID_ARGS error', async () => {
const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider({ providerId: 'stub-search', query: 'q', sources: [], truncated: false }) })
const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: searchProvider({ sources: [], truncated: false }) })
const out = await call('web_search', { query: 123 })
expect(out.isError).toBe(true)
expect(out.error?.code).toBe('INVALID_ARGS')
@@ -249,14 +249,14 @@ describe('tool-web execution through the real registry', () => {
})
it('executes web_fetch, forwarding the url (no timeout param) and the abort signal to the seam', async () => {
const seen: { request?: { url: string; timeoutMs?: number }; signal?: AbortSignal | undefined } = {}
const seen: { request?: { url: string }; signal?: AbortSignal | undefined } = {}
const fetchProvider = {
id: 'stub-fetch',
status: () => available,
fetch: (request: { url: string; timeoutMs?: number }, exec?: { signal?: AbortSignal }) => {
available: () => available,
fetch: (request: { url: string }, signal?: AbortSignal) => {
seen.request = request
seen.signal = exec?.signal
return Promise.resolve({ providerId: 'stub-fetch', url: request.url, statusCode: 200, body: { kind: 'text' as const, content: 'ok' }, truncated: false })
seen.signal = signal
return Promise.resolve({ url: request.url, statusCode: 200, body: { kind: 'text' as const, content: 'ok' }, truncated: false })
},
}
const { ctx, fiber } = await mountTools({ webConfig: { fetchProvider: 'stub-fetch' }, fetchProvider })
@@ -271,21 +271,21 @@ describe('tool-web execution through the real registry', () => {
})
it('executes web_fetch with no caller signal (forwards undefined to the seam)', async () => {
const seen: { signal?: AbortSignal | undefined; passedExec?: boolean } = {}
const seen: { signal?: AbortSignal | undefined; passedSignal?: boolean } = {}
const fetchProvider = {
id: 'stub-fetch',
status: () => available,
fetch: (request: { url: string }, exec?: { signal?: AbortSignal }) => {
seen.passedExec = exec !== undefined
seen.signal = exec?.signal
return Promise.resolve({ providerId: 'stub-fetch', url: request.url, statusCode: 200, body: { kind: 'text' as const, content: 'ok' }, truncated: false })
available: () => available,
fetch: (request: { url: string }, signal?: AbortSignal) => {
seen.passedSignal = signal !== undefined
seen.signal = signal
return Promise.resolve({ url: request.url, statusCode: 200, body: { kind: 'text' as const, content: 'ok' }, truncated: false })
},
}
const { ctx, fiber } = await mountTools({ webConfig: { fetchProvider: 'stub-fetch' }, fetchProvider })
// No signal on the execution: the tool passes `undefined` (not `{ signal: undefined }`).
// No signal on the execution: the tool passes `undefined`.
const out = await ctx.tools.execute({ callId: CallId('fetch-2'), name: 'web_fetch', arguments: { url: 'https://a.test' } })
expect(out.isError).toBe(false)
expect(seen.passedExec).toBe(false)
expect(seen.passedSignal).toBe(false)
expect(seen.signal).toBeUndefined()
await fiber.dispose()
})
@@ -294,8 +294,8 @@ describe('tool-web execution through the real registry', () => {
const seen: { signal?: AbortSignal | undefined } = {}
const provider: WebSearchProvider = {
id: 'stub-search',
status: () => available,
search: (_request, exec) => { seen.signal = exec?.signal; return Promise.resolve({ providerId: 'stub-search', query: 'q', sources: [], truncated: false }) },
available: () => available,
search: (_request, signal) => { seen.signal = signal; return Promise.resolve({ sources: [], truncated: false }) },
}
const { ctx, fiber } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider })
const controller = new AbortController()
@@ -310,8 +310,8 @@ describe('searchMaxResults is plugin config', () => {
const seen: { maxResults?: number | undefined } = {}
const provider: WebSearchProvider = {
id: 'stub-search',
status: () => available,
search: (request) => { seen.maxResults = request.maxResults; return Promise.resolve({ providerId: 'stub-search', query: 'q', sources: [], truncated: false }) },
available: () => available,
search: (request) => { seen.maxResults = request.maxResults; return Promise.resolve({ sources: [], truncated: false }) },
}
const { fiber, call } = await mountTools({ webConfig: { searchProvider: 'stub-search' }, search: provider })
await call('web_search', { query: 'q' })
@@ -323,8 +323,8 @@ describe('searchMaxResults is plugin config', () => {
const sources = Array.from({ length: 5 }, (_, i) => ({ url: `https://s${i}.test` }))
const provider: WebSearchProvider = {
id: 'stub-search',
status: () => available,
search: request => Promise.resolve({ providerId: 'stub-search', query: request.query, sources, truncated: false }),
available: () => available,
search: () => Promise.resolve({ sources, truncated: false }),
}
const { fiber, call } = await mountTools({ config: { searchMaxResults: 2 }, webConfig: { searchProvider: 'stub-search' }, search: provider })
const out = await call('web_search', { query: 'q' })

View File

@@ -8,7 +8,9 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, a resource-backstop timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource.
The provider's `timeoutMs`/`maxTimeoutMs` is a **resource backstop** for direct `ctx.web.fetch()` callers and misconfigured deployments — it is NOT the model-facing tool-call budget. The tool-call budget for `web_fetch` is deployment policy owned by [`@deepseek-ai/dsh-timeout-policy`](../../timeout/timeout-policy/README.md), which arms a per-call deadline on `exec.signal`. A shipped web-tool deployment sets the provider backstop **above** the `tool-timeout` budget, so the tool-call policy normally wins for model calls (returning `TOOL_TIMEOUT`); when the outer deadline signal reaches this provider first, it classifies as `WEB_ABORTED` and the outer wrapper replaces the result with `TOOL_TIMEOUT`. The provider's own `WEB_FETCH_TIMEOUT` only fires for a direct seam caller whose own budget elapsed.
The provider's `timeoutMs` is a resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments, not the model-facing tool-call budget. [`dsh-timeout-policy`](../../timeout/timeout-policy/README.md) owns the `web_fetch` tool-call budget by arming `exec.signal`.
A shipping web-tool deployment sets the provider backstop above the tool budget, so model calls normally return `TOOL_TIMEOUT`. If the outer deadline reaches the provider first, the provider reports `WEB_ABORTED` and the outer policy replaces it with `TOOL_TIMEOUT`. `WEB_FETCH_TIMEOUT` therefore identifies a direct seam caller whose provider budget elapsed.
## Transport hygiene
@@ -26,13 +28,18 @@ The provider's `timeoutMs`/`maxTimeoutMs` is a **resource backstop** for direct
| `maxUrlLength` | `2048` | Maximum accepted request URL length. |
| `maxResponseBytes` | `5_000_000` | Maximum response body size in bytes. |
| `maxBodyChars` | `100_000` | Maximum decoded body length in characters. |
| `timeoutMs` | `30_000` | Default fetch timeout — a resource backstop for direct `ctx.web.fetch()` callers, not the model-facing tool-call budget (that is `dsh-timeout-policy`). |
| `maxTimeoutMs` | `120_000` | Upper bound for a per-request timeout override (direct callers). |
| `timeoutMs` | `30_000` | Fetch timeout within Node's timer range — a resource backstop for direct `ctx.web.fetch()` callers, not the model-facing tool-call budget (that is `dsh-timeout-policy`). |
| `maxRedirects` | `5` | Maximum same-origin redirect hops (`0` follows none). |
| `userAgent` | `deepseek-harness/…` | `User-Agent` header. |
The numeric limits are validated at plugin construction: every cap except `maxRedirects` must be a positive finite number, and `maxRedirects` must be a non-negative integer. An invalid value throws rather than silently constructing a provider with nonsensical limits.
## Security note
## Model Experience
SSRF / private-network protection (blocking private, loopback, link-local, multicast, and otherwise non-public destinations, with DNS-resolve-then-validate and per-hop re-validation) is **deferred** — see the [web capability seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md). Until it lands, this provider is an SSRF primitive and **must not be enabled** in a deployment that can reach sensitive internal network targets.
Indirectly, through [`dsh-tool-web`](../tool-web/README.md), which places this provider's `maxBodyChars`-bounded decoded text or markdown-shaped HTML under its fetch-result wrapper and retains provider failures while redirects, headers, and transport mechanics remain hidden.
## Known Limitations and Deferred Work
- **SSRF / private-network protection is deferred** — no blocking of private, loopback, link-local, multicast, or otherwise non-public destinations, no DNS-resolve-then-validate, no per-hop re-validation (see [the web capability seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md)). Until it lands, this provider is an SSRF primitive and **must not be enabled** in a deployment that can reach sensitive internal network targets.
- **Only textual content decodes** — html/xhtml and `text/*`-plus-JSON/XML families; a missing `Content-Type` or any binary type throws `WEB_UNSUPPORTED_CONTENT_TYPE`, and text-extractable PDF decoding is named deferred work.
- **Charset comes only from the `Content-Type` header** (UTF-8 default) — an HTML `<meta charset>` declaration is ignored, and a declared-but-unrecognized charset label throws rather than falling back.

View File

@@ -24,7 +24,7 @@
"peerDependencies": {
"@deepseek-ai/dsh-timeout": "^0.0.1",
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
@@ -32,6 +32,6 @@
"devDependencies": {
"@deepseek-ai/dsh-timeout": "workspace:^",
"@deepseek-ai/dsh-web": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -13,13 +13,13 @@ import type {} from '@deepseek-ai/dsh-web'
import { LocalFetchProvider } from './provider.ts'
import type { LocalFetchLimits } from './provider.ts'
const MAX_NODE_TIMER_DELAY_MS = 2_147_483_647
export {
LOCAL_FETCH_PROVIDER_ID,
LocalFetchProvider,
} from './provider.ts'
export type { LocalFetchLimits } from './provider.ts'
export { classifyContentType, decoderForCharset, isSameOrigin, parseCharset, validateFetchUrl } from './policy.ts'
export type { FetchableKind } from './policy.ts'
/** Default `User-Agent`: an explicit product agent, never a browser disguise. */
export const DEFAULT_USER_AGENT = 'deepseek-harness/0.0.1 (+https://github.com/deepseek-ai)'
@@ -38,10 +38,8 @@ export interface Config {
maxResponseBytes?: number
/** Maximum decoded body length in characters. */
maxBodyChars?: number
/** Default fetch timeout in milliseconds. */
/** Default fetch timeout in milliseconds, within Node's timer range. */
timeoutMs?: number
/** Upper bound for a per-request timeout override. */
maxTimeoutMs?: number
/** Maximum number of same-origin redirect hops to follow. */
maxRedirects?: number
/** `User-Agent` header sent on every request. */
@@ -53,7 +51,6 @@ export const Config: z<Config> = z.object({
maxResponseBytes: z.number().default(5_000_000),
maxBodyChars: z.number().default(100_000),
timeoutMs: z.number().default(30_000),
maxTimeoutMs: z.number().default(120_000),
maxRedirects: z.number().default(5),
userAgent: z.string().default(DEFAULT_USER_AGENT),
})
@@ -68,6 +65,14 @@ function assertPositiveFinite(name: string, value: number): void {
}
}
/** Node coerces larger timer delays to 1 ms, so reject them at configuration time. */
function assertTimeoutMs(value: number): void {
assertPositiveFinite('timeoutMs', value)
if (value > MAX_NODE_TIMER_DELAY_MS) {
throw new Error(`web-fetch-local: timeoutMs must be no greater than ${MAX_NODE_TIMER_DELAY_MS}`)
}
}
/** The redirect hop cap must be a non-negative integer (0 follows no redirects). */
function assertNonNegativeInteger(name: string, value: number): void {
if (!Number.isInteger(value) || value < 0) {
@@ -82,15 +87,13 @@ export function apply(ctx: Context, config: Config): void {
assertPositiveFinite('maxUrlLength', resolved.maxUrlLength)
assertPositiveFinite('maxResponseBytes', resolved.maxResponseBytes)
assertPositiveFinite('maxBodyChars', resolved.maxBodyChars)
assertPositiveFinite('timeoutMs', resolved.timeoutMs)
assertPositiveFinite('maxTimeoutMs', resolved.maxTimeoutMs)
assertTimeoutMs(resolved.timeoutMs)
assertNonNegativeInteger('maxRedirects', resolved.maxRedirects)
const limits: LocalFetchLimits = {
maxUrlLength: resolved.maxUrlLength,
maxResponseBytes: resolved.maxResponseBytes,
maxBodyChars: resolved.maxBodyChars,
timeoutMs: resolved.timeoutMs,
maxTimeoutMs: resolved.maxTimeoutMs,
maxRedirects: resolved.maxRedirects,
userAgent: resolved.userAgent,
}

View File

@@ -1,27 +1,16 @@
/**
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public
* HTTP(S) URL with platform-native `fetch` at the repo's Node floor and returns a status
* code plus bounded decoded content. It owns SAFE RESOURCE RETRIEVAL — URL
* validation, redirect policy, timeout, abort, byte caps, charset decoding,
* content-type classification, binary rejection — but NOT presentation
* (HTML→markdown lives in `@deepseek-ai/dsh-tool-web`).
*
* Redirects are followed manually (`redirect: 'manual'`) so the provider can
* enforce a same-origin-only policy: a cross-origin redirect is refused with
* `WEB_REDIRECT_BLOCKED`, requiring a fresh tool call (Claude Code's WebFetch
* uses the same model). It does NOT carry browser cookies, editor/git
* credentials, or implicit access to private services.
*
* SSRF / private-network protection is DEFERRED (see the package RFC); until it
* lands this provider is an SSRF primitive and must not be enabled where it can
* reach sensitive internal targets.
* Safe HTTP(S) retrieval for `ctx.web`: validates URLs, follows only same-origin redirects,
* enforces time and size limits, classifies and decodes text, and leaves presentation to
* `@deepseek-ai/dsh-tool-web`. Requests carry no browser cookies or ambient credentials.
*
* Private-network and SSRF protection is not implemented; do not enable this provider where
* it can reach sensitive internal targets.
* @module @deepseek-ai/dsh-web-fetch-local/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type { WebFetchBody, WebFetchProvider, WebFetchRequest, WebFetchResult, WebProviderStatus } from '@deepseek-ai/dsh-web'
import { clampTimeout, deadline, timeoutOf } from '@deepseek-ai/dsh-timeout'
import type { WebFetchBody, WebFetchProvider, WebFetchRequest, WebFetchResult } from '@deepseek-ai/dsh-web'
import { deadline, timeoutOf } from '@deepseek-ai/dsh-timeout'
import { classifyContentType, decoderForCharset, isSameOrigin, parseCharset, validateFetchUrl } from './policy.ts'
/** Resolved provider limits (the plugin's schemastery Config supplies defaults). */
@@ -34,8 +23,6 @@ export interface LocalFetchLimits {
maxBodyChars: number
/** Default fetch timeout in milliseconds. */
timeoutMs: number
/** Upper bound for a per-request timeout override. */
maxTimeoutMs: number
/** Maximum number of (same-origin) redirect hops to follow. */
maxRedirects: number
/** `User-Agent` header sent on every request. */
@@ -52,20 +39,16 @@ export class LocalFetchProvider implements WebFetchProvider {
constructor(private readonly limits: LocalFetchLimits) {}
/** No credentials to check — an anonymous public fetcher is always usable. */
status(): WebProviderStatus {
return { available: true }
available(): boolean {
return true
}
async fetch(request: WebFetchRequest, exec?: { readonly signal?: AbortSignal }): Promise<WebFetchResult> {
if (exec?.signal?.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED')
const timeoutMs = clampTimeout(request.timeoutMs, this.limits.timeoutMs, this.limits.maxTimeoutMs)
async fetch(request: WebFetchRequest, signal?: AbortSignal): Promise<WebFetchResult> {
if (signal?.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED')
// One deadline signal fuses the caller's abort with our own timeout, so the
// network request and the streaming read both stop on either. The timeout
// abort carries a TimeoutReason we recover afterward to classify the cause
// (translateAbortOrNetwork), instead of hand-rolling a controller + timer +
// reason-recovery dance.
using d = deadline(exec?.signal, timeoutMs, 'WEB_FETCH_TIMEOUT')
// One signal stops both the request and body read. The deadline's TimeoutReason later
// distinguishes this provider's timeout from caller or outer-deadline cancellation.
using d = deadline(signal, this.limits.timeoutMs, 'WEB_FETCH_TIMEOUT')
return await this.followAndRead(request.url, d.signal)
}
@@ -78,11 +61,7 @@ export class LocalFetchProvider implements WebFetchProvider {
const response = await this.requestOnce(currentUrl, signal)
if (isRedirectStatus(response.status)) {
// The redirect budget is enforced BEFORE this hop's target is resolved
// or origin-checked, so `maxRedirects: N` follows at most N redirects
// exactly: the (N+1)th redirect is refused as "exceeded" regardless of
// where it points (a same-origin/cross-origin distinction on a hop we
// are not allowed to follow would be the wrong diagnosis).
// Enforce the redirect budget before resolving or validating the next hop.
if (redirectsFollowed >= this.limits.maxRedirects) {
await response.body?.cancel()
throw new WebError(`exceeded the maximum of ${this.limits.maxRedirects} redirects`, 'WEB_REDIRECT_BLOCKED')
@@ -95,10 +74,9 @@ export class LocalFetchProvider implements WebFetchProvider {
throw new WebError(`redirect response (HTTP ${response.status}) without a Location header`, 'WEB_PROVIDER_ERROR')
}
const target = resolveRedirect(location, currentUrl)
// Re-validate the target against the same transport hygiene a direct
// request gets: a redirect must not be a back door to a credentialed,
// non-http(s), or over-long URL that validateFetchUrl would reject. A
// rejection here must still cancel the body first (see below).
// Re-validate the target against the same transport hygiene a direct request gets: a
// redirect must not be a back door to a credentialed, non-http(s), or over-long URL
// that validateFetchUrl would reject.
let validatedTarget: URL
try {
validatedTarget = validateFetchUrl(target.toString(), this.limits.maxUrlLength)
@@ -161,7 +139,6 @@ export class LocalFetchProvider implements WebFetchProvider {
const body: WebFetchBody = kind === 'html' ? { kind: 'html', content } : { kind: 'text', content }
return {
providerId: this.id,
url: finalUrl.toString(),
statusCode: response.status,
body,

View File

@@ -3,16 +3,16 @@ import { createServer, type IncomingMessage, type Server, type ServerResponse }
import { AddressInfo } from 'node:net'
import { Context } from 'cordis'
import WebService from '@deepseek-ai/dsh-web'
import { LocalFetchProvider, LOCAL_FETCH_PROVIDER_ID, classifyContentType, decoderForCharset, isSameOrigin, parseCharset, validateFetchUrl } from '@deepseek-ai/dsh-web-fetch-local'
import { LocalFetchProvider, LOCAL_FETCH_PROVIDER_ID } from '@deepseek-ai/dsh-web-fetch-local'
import type { LocalFetchLimits } from '@deepseek-ai/dsh-web-fetch-local'
import * as fetchPlugin from '@deepseek-ai/dsh-web-fetch-local'
import { classifyContentType, decoderForCharset, isSameOrigin, parseCharset, validateFetchUrl } from '../src/policy.ts'
const limits: LocalFetchLimits = {
maxUrlLength: 2048,
maxResponseBytes: 5_000_000,
maxBodyChars: 100_000,
timeoutMs: 5_000,
maxTimeoutMs: 10_000,
maxRedirects: 5,
userAgent: 'test-agent/1.0',
}
@@ -82,7 +82,7 @@ describe('LocalFetchProvider success', () => {
it('fetches a text body', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('hello world') }
const result = await provider().fetch({ url: base })
expect(result.providerId).toBe(LOCAL_FETCH_PROVIDER_ID)
expect(provider().available()).toBe(true)
expect(result.statusCode).toBe(200)
expect(result.body).toEqual({ kind: 'text', content: 'hello world' })
expect(result.truncated).toBe(false)
@@ -287,14 +287,14 @@ describe('LocalFetchProvider invalid URLs and abort', () => {
it('honors a pre-aborted signal', async () => {
const controller = new AbortController()
controller.abort()
await expect(provider().fetch({ url: base }, { signal: controller.signal }))
await expect(provider().fetch({ url: base }, controller.signal))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
it('aborts an in-flight fetch via the signal', async () => {
handler = (_req, _res) => { /* never responds */ }
const controller = new AbortController()
const promise = provider().fetch({ url: base }, { signal: controller.signal })
const promise = provider().fetch({ url: base }, controller.signal)
controller.abort()
await expect(promise).rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
@@ -325,11 +325,6 @@ describe('LocalFetchProvider invalid URLs and abort', () => {
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('caps the per-request timeout at maxTimeoutMs', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('ok') }
const result = await provider({ maxTimeoutMs: 10_000 }).fetch({ url: base, timeoutMs: 999_999 })
expect(result.statusCode).toBe(200)
})
})
describe('LocalFetchProvider body cancellation on error paths', () => {
@@ -378,7 +373,7 @@ describe('web-fetch-local plugin registration', () => {
await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID })
const fiber = await ctx.plugin(fetchPlugin, {})
await expect(ctx.web.fetch({ url: `${base}/` }))
.resolves.toMatchObject({ providerId: LOCAL_FETCH_PROVIDER_ID, statusCode: 200 })
.resolves.toMatchObject({ statusCode: 200 })
await fiber.dispose()
await expect(ctx.web.fetch({ url: `${base}/` }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))
@@ -402,6 +397,13 @@ describe('web-fetch-local plugin registration', () => {
.rejects.toThrow(/timeoutMs must be a positive finite number/)
})
it('rejects a timeout beyond Node timer range at construction', async () => {
const ctx = new Context()
await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID })
await expect(ctx.plugin(fetchPlugin, { timeoutMs: 2_147_483_648 }))
.rejects.toThrow(/timeoutMs must be no greater than 2147483647/)
})
it('rejects a fractional redirect cap at construction', async () => {
const ctx = new Context()
await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID })
@@ -421,7 +423,7 @@ describe('web-fetch-local plugin registration', () => {
await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID })
const fiber = await ctx.plugin(fetchPlugin, { maxRedirects: 0 })
await expect(ctx.web.fetch({ url: `${base}/` }))
.resolves.toMatchObject({ providerId: LOCAL_FETCH_PROVIDER_ID, statusCode: 200 })
.resolves.toMatchObject({ statusCode: 200 })
await fiber.dispose()
})
})

View File

@@ -16,8 +16,8 @@ It reuses `$DEEPSEEK_API_KEY` (no new secret) but **not** `$DEEPSEEK_BASE_URL`:
| Key | Default | Meaning |
|---|---|---|
| `apiKey` | `$DEEPSEEK_API_KEY` | DeepSeek API key. Empty/absent provider `status()` reports `missing-credential`. Sent as both `x-api-key` and `Authorization: Bearer` (official vs Anthropic-compatible proxy). |
| `baseURL` | `https://api.deepseek.com/anthropic/v1` | Anthropic-compatible endpoint base; `/messages` is appended. Use a separate env var such as `$DEEPSEEK_SEARCH_BASE_URL` when overriding it; do not reuse `$DEEPSEEK_BASE_URL`, which belongs to the chat-completions LLM adapter. An unparseable value makes `status()` report `misconfigured`. |
| `apiKey` | `$DEEPSEEK_API_KEY` | DeepSeek API key. Empty/absent makes the provider unavailable. Sent as both `x-api-key` and `Authorization: Bearer` (official vs Anthropic-compatible proxy). |
| `baseURL` | `https://api.deepseek.com/anthropic/v1` | Anthropic-compatible endpoint base; `/messages` is appended. Use a separate env var such as `$DEEPSEEK_SEARCH_BASE_URL` when overriding it; do not reuse `$DEEPSEEK_BASE_URL`, which belongs to the chat-completions LLM adapter. An unparseable value makes the provider unavailable. |
| `model` | `deepseek-v4-flash` | Anthropic-format model name. |
| `apiVersion` | `2023-06-01` | `anthropic-version` header value. |
| `maxTokens` | `4096` | Positive-integer upper bound on generated tokens for the Messages request. |
@@ -33,4 +33,29 @@ It reuses `$DEEPSEEK_API_KEY` (no new secret) but **not** `$DEEPSEEK_BASE_URL`:
## Mapping
DeepSeek returns no provider-generated answer surface this provider trusts as `content`, so `content` is omitted. `sources[]` is built from the `web_search_result` items inside `web_search_tool_result` blocks: `url``url`, `title``title`, `publishedAt``page_age`. The per-source `snippet` lives separately in a `text` block's `citations[]` (a `cited_text` keyed by `url`), so the provider joins the two — a result with no citation excerpt simply has no `snippet`. Results are deduped by `url` (a `maxUses > 1` request can surface the same URL across searches). DeepSeek's `web_search` has no result-count knob (only `maxUses`), so `maxResults` is enforced by the seam (truncating `sources[]` and setting `truncated`). Provider failures surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`.
DeepSeek returns no provider-generated answer surface this provider trusts as `content`, so `content` is omitted. `sources[]` comes from `web_search_result` items inside `web_search_tool_result` blocks: `url``url`, `title``title`, and `publishedAt``page_age`. Snippets live separately as URL-keyed `cited_text` entries in a text block's `citations[]`; the provider joins them, leaving `snippet` absent when no excerpt exists.
Results are deduplicated by URL because one request may surface the same page across searches. DeepSeek exposes `maxUses`, not a result-count knob, so the seam enforces `maxResults` by truncating `sources[]` and setting `truncated`.
Provider failures become `WEB_PROVIDER_ERROR`; caller cancellation becomes `WEB_ABORTED`.
## Model Experience
### Auxiliary DeepSeek search request
**What the model sees**: A separate DeepSeek model receives exactly `Perform a web search for the query: <query>` as its user text and one native `web_search` server-tool definition. This request is not part of the conversation model's context.
**Token effect**: Separate provider input and output tokens are incurred for each search; `maxTokens` caps generated output and `maxUses` caps native search uses.
### Conversation tool result, indirectly
**What the model sees**: Through [`dsh-tool-web`](../tool-web/README.md), the conversation model sees deduplicated URLs, titles, dates, and citation snippets from structured search blocks; provider prose is not trusted as an answer. This provider's exact failures are `DeepSeek search aborted`, `DeepSeek search request failed: <error>`, `DeepSeek returned no web_search_tool_result blocks; the request may not have triggered native web search`, and `DeepSeek returned an unprocessable response body: <error>`; HTTP failures preserve the provider message. The consumer owns the error wrapper.
**Token effect**: Zero direct conversation tokens from registration. Result tokens scale with returned sources and snippets, then the seam enforces the requested source bound.
## Known Limitations and Deferred Work
- **One search costs a full Messages model turn** — latency plus generated tokens, with up to `maxUses` server-side searches; DeepSeek exposes no dedicated retrieval endpoint.
- **Over-returned sources still cost tokens** — with no result-count knob on the wire, `maxResults` is enforced only post-hoc by seam truncation.
- **Uncited results carry no `snippet`** — a source gains one only when a `text` block citation (`cited_text`) matches its URL.
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.

View File

@@ -23,13 +23,13 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-web": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -1,15 +1,7 @@
/**
* `@deepseek-ai/dsh-web-search-deepseek`: registers a DeepSeek-backed
* `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a
* default-export service): it registers INTO the seam's provider registry, like
* `@deepseek-ai/dsh-llm-deepseek` registers an adapter into `ctx.llm`.
*
* The provider talks to DeepSeek's Anthropic-compatible Messages API with the
* native `web_search_20250305` server tool. It reuses `$DEEPSEEK_API_KEY` (no
* new secret) but NOT `$DEEPSEEK_BASE_URL` — the search endpoint is the
* Anthropic-compatible base, distinct from the chat-completions base the LLM
* adapter uses.
*
* Register a DeepSeek-backed provider in `ctx.web`. It calls the Anthropic-compatible Messages API
* with native `web_search_20250305`. The provider reuses `DEEPSEEK_API_KEY` but not
* `DEEPSEEK_BASE_URL`, because search and chat-completions use different bases.
* @module @deepseek-ai/dsh-web-search-deepseek
*/
@@ -33,8 +25,6 @@ export {
DEEPSEEK_DEFAULT_MAX_USES,
DEEPSEEK_DEFAULT_MODEL,
DEEPSEEK_PROVIDER_ID,
citationSnippets,
mapAnthropicResponse,
} from './provider.ts'
export type { DeepSeekSearchProviderOptions } from './provider.ts'

View File

@@ -1,28 +1,13 @@
/**
* `DeepSeekSearchProvider`: a `WebSearchProvider` backed by DeepSeek's
* Anthropic-compatible Messages API with the native `web_search_20250305` server
* tool enabled.
*
* Unlike a dedicated search endpoint (Exa's `POST /search`, Perplexity's
* `/chat/completions`), this issues a FULL Messages model call carrying a server
* tool, so a search costs a complete model turn in latency and tokens. In return
* DeepSeek runs the search server-side and returns STRUCTURED
* `web_search_tool_result` blocks — this provider parses those blocks and never
* scrapes URLs out of model prose. Strict mode: if the response carries no
* `web_search_tool_result` block (native search did not trigger), it throws
* `WEB_PROVIDER_ERROR` rather than degrading to prose-scraping.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
* The Anthropic wire shape is a provider-private detail and does NOT make this
* provider depend on `ctx.llm`.
*
* DeepSeek search through an Anthropic-compatible Messages model call with the native
* `web_search_20250305` server tool. Each search costs a model turn, but returns structured
* result blocks; absence of those blocks is an error rather than a prose-scraping fallback.
* The wire format and native `fetch` client are provider-private and do not use `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-deepseek/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type {
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
@@ -64,7 +49,7 @@ const USER_AGENT = 'deepseek-harness/0.0.1'
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
export interface DeepSeekSearchProviderOptions {
/** DeepSeek API key. Empty/absent → `status()` reports `missing-credential`. */
/** DeepSeek API key. Empty/absent makes the provider unavailable. */
apiKey: string
/** Endpoint base; `/messages` is appended. */
baseURL: string
@@ -101,21 +86,17 @@ export function citationSnippets(blocks: readonly ContentBlock[]): Map<string, s
}
/**
* Map a DeepSeek Anthropic Messages response to a normalized search result.
* Walks `web_search_tool_result` blocks for citeable `web_search_result` items,
* joins each to its citation excerpt as `snippet`, and dedupes by `url` (a
* `max_uses > 1` request can surface the same URL across searches). The seam
* owns the final `maxResults` truncation, so `truncated` is always `false` here.
* Map a DeepSeek Anthropic Messages response to a normalized search result. Walks
* `web_search_tool_result` blocks for citeable `web_search_result` items, joins each to its
* citation excerpt as `snippet`, and dedupes by `url` (a `max_uses > 1` request can surface
* the same URL across searches). The seam owns the final `maxResults` truncation, so
* `truncated` is always `false` here.
*
* Throws `WEB_PROVIDER_ERROR` (strict mode) when no `web_search_tool_result`
* block is present — native search did not trigger, and prose-scraping is not a
* fallback.
*
* @param query - the original request query, echoed on the result.
* @param response - the parsed Messages response body.
* @returns the normalized result with deduped, snippet-joined sources.
* @throws {@link WebError} when native search produced no result block.
*/
export function mapAnthropicResponse(query: string, response: AnthropicResponse): WebSearchResult {
export function mapAnthropicResponse(response: AnthropicResponse): WebSearchResult {
const blocks = response.content ?? []
const resultBlocks = blocks.filter(
(block): block is WebSearchToolResultBlock => block.type === 'web_search_tool_result',
@@ -143,7 +124,7 @@ export function mapAnthropicResponse(query: string, response: AnthropicResponse)
})
}
}
return { providerId: DEEPSEEK_PROVIDER_ID, query, sources, truncated: false }
return { sources, truncated: false }
}
/** The DeepSeek-backed search provider. */
@@ -152,14 +133,14 @@ export class DeepSeekSearchProvider implements WebSearchProvider {
constructor(private readonly options: DeepSeekSearchProviderOptions) {}
status(): WebProviderStatus {
if (this.options.apiKey.length === 0) return { available: false, reason: 'missing-credential' }
if (!URL.canParse(this.options.baseURL)) return { available: false, reason: 'misconfigured' }
if (!isPositiveInteger(this.options.maxTokens) || !isPositiveInteger(this.options.maxUses)) return { available: false, reason: 'misconfigured' }
return { available: true }
available(): boolean {
return this.options.apiKey.length > 0
&& URL.canParse(this.options.baseURL)
&& isPositiveInteger(this.options.maxTokens)
&& isPositiveInteger(this.options.maxUses)
}
async search(request: WebSearchRequest, exec?: { readonly signal?: AbortSignal }): Promise<WebSearchResult> {
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
let response: Response
try {
response = await fetch(`${this.options.baseURL}/messages`, {
@@ -183,7 +164,7 @@ export class DeepSeekSearchProvider implements WebSearchProvider {
}],
tools: [{ type: 'web_search_20250305', name: 'web_search', max_uses: this.options.maxUses }],
}),
...exec?.signal ? { signal: exec.signal } : {},
...signal !== undefined ? { signal } : {},
})
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('DeepSeek search aborted', 'WEB_ABORTED', { cause: error })
@@ -211,7 +192,7 @@ export class DeepSeekSearchProvider implements WebSearchProvider {
try {
const payload = await response.json() as AnthropicResponse
return mapAnthropicResponse(request.query, payload)
return mapAnthropicResponse(payload)
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('DeepSeek search aborted', 'WEB_ABORTED', { cause: error })
if (error instanceof WebError) throw error

View File

@@ -1,16 +1,7 @@
/**
* Wire types for DeepSeek's Anthropic-compatible Messages API
* (`POST {baseURL}/messages`) with the native `web_search_20250305` server tool
* enabled. Types only — no runtime code.
*
* DeepSeek returns structured content blocks: `web_search_tool_result` blocks
* carry the citeable `web_search_result` items (`url`/`title`/`page_age`), while
* the snippet/excerpt for a URL lives separately in a `text` block's
* `citations[]` (a `cited_text` keyed by `url`). The provider joins the two.
*
* The Anthropic wire shape is a provider-private detail; it does not make this
* provider depend on `ctx.llm`.
*
* Provider-private wire types for DeepSeek's Anthropic-compatible Messages API. Citeable
* result items and citation excerpts arrive in separate blocks; the provider joins them by
* URL. These types do not create a dependency on `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-deepseek/types
*/

View File

@@ -29,7 +29,6 @@ maybe('DeepSeekSearchProvider real API', () => {
maxUses: DEEPSEEK_DEFAULT_MAX_USES,
})
const result = await provider.search({ query: 'What is the DeepSeek Harness SDK?', maxResults: 5 })
expect(result.providerId).toBe('deepseek')
expect(result.sources.length).toBeGreaterThan(0)
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
}, 60_000)

View File

@@ -4,11 +4,10 @@ import Loader from '@cordisjs/plugin-loader'
import WebService from '@deepseek-ai/dsh-web'
import {
DeepSeekSearchProvider,
citationSnippets,
mapAnthropicResponse,
DEEPSEEK_PROVIDER_ID,
} from '@deepseek-ai/dsh-web-search-deepseek'
import * as deepseekPlugin from '@deepseek-ai/dsh-web-search-deepseek'
import { citationSnippets, mapAnthropicResponse } from '../src/provider.ts'
import type { AnthropicResponse } from '@deepseek-ai/dsh-web-search-deepseek/src/types.ts'
const options = {
@@ -64,10 +63,8 @@ describe('citationSnippets', () => {
describe('mapAnthropicResponse', () => {
it('joins result items to citation snippets and maps page_age to publishedAt', () => {
const result = mapAnthropicResponse('q', searchResponse())
const result = mapAnthropicResponse(searchResponse())
expect(result).toEqual({
providerId: DEEPSEEK_PROVIDER_ID,
query: 'q',
sources: [
{ url: 'https://a.test', title: 'A', snippet: 'excerpt for A', publishedAt: '2026-02-02' },
{ url: 'https://b.test', title: 'B' },
@@ -77,7 +74,7 @@ describe('mapAnthropicResponse', () => {
})
it('dedupes repeated urls across result blocks (first wins)', () => {
const result = mapAnthropicResponse('q', {
const result = mapAnthropicResponse({
content: [
{ type: 'web_search_tool_result', content: [{ type: 'web_search_result', url: 'https://a.test', title: 'first' }] },
{ type: 'web_search_tool_result', content: [{ type: 'web_search_result', url: 'https://a.test', title: 'second' }] },
@@ -87,7 +84,7 @@ describe('mapAnthropicResponse', () => {
})
it('skips non-result items and items with an empty url', () => {
const result = mapAnthropicResponse('q', {
const result = mapAnthropicResponse({
content: [{
type: 'web_search_tool_result',
content: [
@@ -101,14 +98,14 @@ describe('mapAnthropicResponse', () => {
})
it('omits optional fields when absent or empty', () => {
const result = mapAnthropicResponse('q', {
const result = mapAnthropicResponse({
content: [{ type: 'web_search_tool_result', content: [{ type: 'web_search_result', url: 'https://a.test', title: '', page_age: '' }] }],
})
expect(result.sources).toEqual([{ url: 'https://a.test' }])
})
it('tolerates a text block with no citations', () => {
const result = mapAnthropicResponse('q', {
const result = mapAnthropicResponse({
content: [
{ type: 'text', text: 'no citations here' },
{ type: 'web_search_tool_result', content: [{ type: 'web_search_result', url: 'https://a.test', title: 'A' }] },
@@ -118,7 +115,7 @@ describe('mapAnthropicResponse', () => {
})
it('tolerates a result block with no content array', () => {
const result = mapAnthropicResponse('q', {
const result = mapAnthropicResponse({
content: [
{ type: 'web_search_tool_result' },
{ type: 'web_search_tool_result', content: [{ type: 'web_search_result', url: 'https://a.test' }] },
@@ -128,38 +125,33 @@ describe('mapAnthropicResponse', () => {
})
it('throws WEB_PROVIDER_ERROR (strict mode) when no result block is present', () => {
expect(() => mapAnthropicResponse('q', { content: [{ type: 'text', text: 'just prose, no search' }] }))
expect(() => mapAnthropicResponse({ content: [{ type: 'text', text: 'just prose, no search' }] }))
.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('throws WEB_PROVIDER_ERROR when content is absent entirely', () => {
expect(() => mapAnthropicResponse('q', {}))
expect(() => mapAnthropicResponse({}))
.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
})
describe('DeepSeekSearchProvider status', () => {
describe('DeepSeekSearchProvider availability', () => {
it('is unavailable without a key', () => {
expect(new DeepSeekSearchProvider({ ...options, apiKey: '' }).status())
.toEqual({ available: false, reason: 'missing-credential' })
expect(new DeepSeekSearchProvider({ ...options, apiKey: '' }).available()).toBe(false)
})
it('is available with a key', () => {
expect(new DeepSeekSearchProvider(options).status()).toEqual({ available: true })
expect(new DeepSeekSearchProvider(options).available()).toBe(true)
})
it('is misconfigured when the base URL is unparseable', () => {
expect(new DeepSeekSearchProvider({ ...options, baseURL: 'not a url' }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new DeepSeekSearchProvider({ ...options, baseURL: 'not a url' }).available()).toBe(false)
})
it('is misconfigured when request limits are not positive integers', () => {
expect(new DeepSeekSearchProvider({ ...options, maxTokens: 0 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new DeepSeekSearchProvider({ ...options, maxUses: 0 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new DeepSeekSearchProvider({ ...options, maxUses: 1.5 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new DeepSeekSearchProvider({ ...options, maxTokens: 0 }).available()).toBe(false)
expect(new DeepSeekSearchProvider({ ...options, maxUses: 0 }).available()).toBe(false)
expect(new DeepSeekSearchProvider({ ...options, maxUses: 1.5 }).available()).toBe(false)
})
})
@@ -186,7 +178,7 @@ describe('DeepSeekSearchProvider request mapping', () => {
const fetchMock = vi.fn(async () => jsonResponse(searchResponse()))
vi.stubGlobal('fetch', fetchMock)
const controller = new AbortController()
await new DeepSeekSearchProvider(options).search({ query: 'q' }, { signal: controller.signal })
await new DeepSeekSearchProvider(options).search({ query: 'q' }, controller.signal)
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
expect(init.signal).toBe(controller.signal)
})
@@ -268,7 +260,7 @@ describe('web-search-deepseek plugin registration', () => {
const ctx = new Context()
await ctx.plugin(WebService, { searchProvider: DEEPSEEK_PROVIDER_ID })
const fiber = await ctx.plugin(deepseekPlugin, { apiKey: 'ds-key' })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: DEEPSEEK_PROVIDER_ID })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ truncated: false })
await fiber.dispose()
await expect(ctx.web.search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))
@@ -300,14 +292,8 @@ describe('web-search-deepseek plugin registration', () => {
})
it('survives the real Loader unwrapExports path keeping name/inject/Config', () => {
// A stray `export default apply` would make the cordis Loader's
// unwrapExports (`exports.default ?? exports`) collapse the module to the
// bare `apply` function, DROPPING `inject: ['web']` — the plugin would then
// read ctx.web without injecting it and throw "cannot get property … without
// inject" the moment it loads. A hand-built ctx.plugin(namespace) mount
// bypasses unwrapExports and cannot catch that, so drive the real path.
// Prove it bites: add `export default apply` to src/index.ts, watch this go
// red, revert.
// A default export would make `unwrapExports` collapse the namespace and drop `inject: ['web']`.
// Drive the real Loader path because hand-built namespace mounting cannot expose that failure.
const loader = Object.create(Loader.prototype) as Loader
const unwrapped = loader.unwrapExports(deepseekPlugin) as Record<string, unknown>
expect(unwrapped).toBe(deepseekPlugin)
@@ -324,7 +310,7 @@ describe('web-search-deepseek plugin registration', () => {
const unwrapped = loader.unwrapExports(deepseekPlugin) as Parameters<Context['plugin']>[0]
// A collapsed export shape (dropped inject) would throw "without inject" here.
const fiber = await ctx.plugin(unwrapped, { apiKey: 'ds-key' })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: DEEPSEEK_PROVIDER_ID })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ truncated: false })
await fiber.dispose()
})

View File

@@ -8,8 +8,8 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
| Key | Default | Meaning |
|---|---|---|
| `apiKey` | `$EXA_API_KEY` | Exa API key. Empty/absent → provider `status()` reports `missing-credential` (the seam reports `configured-unavailable`/`none`). |
| `baseURL` | `https://api.exa.ai` | Endpoint base; `/search` is appended. An unparseable value makes `status()` report `misconfigured`. |
| `apiKey` | `$EXA_API_KEY` | Exa API key. Empty/absent makes the provider unavailable. |
| `baseURL` | `https://api.exa.ai` | Endpoint base; `/search` is appended. An unparseable value makes the provider unavailable. |
| `searchType` | `auto` | Retrieval mode sent as Exa's `type`: `auto` (Exa decides), `keyword`, or `neural`. |
| `numResults` | (unset) | Default result count when a request carries no `maxResults`. Unset sends no default. Must be a positive integer. |
| `highlightsPerResult` | `1` | Highlight sentences requested per result (Exa's `highlightsPerUrl`). Must be a positive integer. |
@@ -24,3 +24,13 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
## Mapping
Exa returns a flat `results[]` and no generated answer, so `content` is omitted. Each result maps to a `WebSearchSource`: `url``url`, `title``title`, `snippet` ← the first non-empty `highlights[]` entry (a result with no highlight has no portable snippet and is dropped), `publishedAt``publishedDate`. A request's `maxResults` wins over the configured `numResults` default and is sent as Exa's `numResults` for a cost/latency optimization; the final bound is enforced by the seam. Provider failures (HTTP errors, network failure, unparseable or wrong-shape bodies) surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`.
## Model Experience
Indirectly, through [`dsh-tool-web`](../tool-web/README.md), which retains this provider's `maxResults`-bounded URLs, titles, first highlights, and publication dates or its exact `Exa search aborted`, `Exa search request failed: <error>`, and `Exa returned an unprocessable response body: <error>` failures under the consumer's error wrapper while generated answers and provider-private fields remain outside context.
## Known Limitations and Deferred Work
- **A result with no non-blank highlight is dropped entirely** — no portable snippet to map, so fewer sources than the requested count can return.
- **Only `searchType`/`numResults`/`highlightsPerResult` are exposed** — Exa's other controls (livecrawl, category, domain/date filters, full-text contents) wait on provider-neutral seam fields ([seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md)).
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.

View File

@@ -23,13 +23,13 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-web": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -24,8 +24,6 @@ export {
EXA_DEFAULT_SEARCH_TYPE,
EXA_PROVIDER_ID,
ExaSearchProvider,
mapExaResponse,
mapExaResult,
} from './provider.ts'
export type { ExaSearchProviderOptions } from './provider.ts'

View File

@@ -1,20 +1,13 @@
/**
* `ExaSearchProvider`: a `WebSearchProvider` backed by the Exa search API
* (`POST /search` with highlight contents). Maps Exa's flat `results[]` into the
* seam's normalized `WebSearchResult`. Exa returns no provider-generated answer,
* so `content` is omitted; each result maps to a `WebSearchSource` with `url`,
* `title`, the first highlight as `snippet`, and `publishedDate` as
* `publishedAt`.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
*
* `ExaSearchProvider`: a `WebSearchProvider` backed by the Exa search API (`POST /search` with
* highlight contents). It maps the first non-blank highlight to `snippet`, maps
* `publishedDate` to `publishedAt`, drops entries without a snippet, and omits `content`
* because Exa returns no generated answer.
* @module @deepseek-ai/dsh-web-search-exa/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type {
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
@@ -39,7 +32,7 @@ const USER_AGENT = 'deepseek-harness/0.0.1'
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
export interface ExaSearchProviderOptions {
/** Exa API key. Empty/absent → `status()` reports `missing-credential`. */
/** Exa API key. Empty/absent makes the provider unavailable. */
apiKey: string
/** Endpoint base; `/search` is appended. */
baseURL: string
@@ -74,18 +67,17 @@ export function mapExaResult(result: ExaResult): WebSearchSource | undefined {
/**
* Map an Exa response envelope to a normalized search result.
*
* @param query - the original request query, echoed on the result.
* @param response - the parsed `POST /search` response body.
* @returns the normalized result; snippet-less entries are dropped
* ({@link mapExaResult}).
*/
export function mapExaResponse(query: string, response: ExaSearchResponse): WebSearchResult {
export function mapExaResponse(response: ExaSearchResponse): WebSearchResult {
const sources = (response.results ?? [])
.map(mapExaResult)
.filter((source): source is WebSearchSource => source !== undefined)
// Exa returns no generated answer, so `content` is omitted. The seam owns the
// final `maxResults` truncation, so this provider reports `truncated: false`.
return { providerId: EXA_PROVIDER_ID, query, sources, truncated: false }
return { sources, truncated: false }
}
/** The Exa-backed search provider. */
@@ -94,15 +86,14 @@ export class ExaSearchProvider implements WebSearchProvider {
constructor(private readonly options: ExaSearchProviderOptions) {}
status(): WebProviderStatus {
if (this.options.apiKey.length === 0) return { available: false, reason: 'missing-credential' }
if (!isValidBaseUrl(this.options.baseURL)) return { available: false, reason: 'misconfigured' }
if (!isPositiveInteger(this.options.highlightsPerResult)) return { available: false, reason: 'misconfigured' }
if (this.options.numResults !== undefined && !isPositiveInteger(this.options.numResults)) return { available: false, reason: 'misconfigured' }
return { available: true }
available(): boolean {
return this.options.apiKey.length > 0
&& isValidBaseUrl(this.options.baseURL)
&& isPositiveInteger(this.options.highlightsPerResult)
&& (this.options.numResults === undefined || isPositiveInteger(this.options.numResults))
}
async search(request: WebSearchRequest, exec?: { readonly signal?: AbortSignal }): Promise<WebSearchResult> {
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
// A per-request bound wins over the configured default; either may be absent.
const numResults = request.maxResults ?? this.options.numResults
let response: Response
@@ -121,7 +112,7 @@ export class ExaSearchProvider implements WebSearchProvider {
contents: { highlights: { highlightsPerUrl: this.options.highlightsPerResult } },
...numResults !== undefined ? { numResults } : {},
}),
...exec?.signal ? { signal: exec.signal } : {},
...signal !== undefined ? { signal } : {},
})
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('Exa search aborted', 'WEB_ABORTED', { cause: error })
@@ -149,7 +140,7 @@ export class ExaSearchProvider implements WebSearchProvider {
try {
const payload = await response.json() as ExaSearchResponse
return mapExaResponse(request.query, payload)
return mapExaResponse(payload)
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('Exa search aborted', 'WEB_ABORTED', { cause: error })
throw new WebError(`Exa returned an unprocessable response body: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })

View File

@@ -17,7 +17,6 @@ maybe('ExaSearchProvider real API', () => {
highlightsPerResult: EXA_DEFAULT_HIGHLIGHTS_PER_RESULT,
})
const result = await provider.search({ query: 'DeepSeek Harness SDK', maxResults: 5 })
expect(result.providerId).toBe('exa')
expect(result.sources.length).toBeGreaterThan(0)
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
}, 30_000)

View File

@@ -1,8 +1,9 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import WebService from '@deepseek-ai/dsh-web'
import { ExaSearchProvider, mapExaResponse, mapExaResult, EXA_PROVIDER_ID } from '@deepseek-ai/dsh-web-search-exa'
import { ExaSearchProvider, EXA_PROVIDER_ID } from '@deepseek-ai/dsh-web-search-exa'
import * as exaPlugin from '@deepseek-ai/dsh-web-search-exa'
import { mapExaResponse, mapExaResult } from '../src/provider.ts'
const options = { apiKey: 'exa-key', baseURL: 'https://api.exa.test', searchType: 'auto' as const, highlightsPerResult: 1 }
@@ -38,7 +39,7 @@ describe('Exa result mapping', () => {
})
it('maps a response to a result with no content and filtered sources', () => {
const result = mapExaResponse('q', {
const result = mapExaResponse({
results: [
{ url: 'https://a.test', highlights: ['one'] },
{ url: 'https://b.test' },
@@ -46,8 +47,6 @@ describe('Exa result mapping', () => {
],
})
expect(result).toEqual({
providerId: EXA_PROVIDER_ID,
query: 'q',
sources: [
{ url: 'https://a.test', snippet: 'one' },
{ url: 'https://c.test', title: 'C', snippet: 'three' },
@@ -58,36 +57,31 @@ describe('Exa result mapping', () => {
})
it('tolerates a missing results array', () => {
expect(mapExaResponse('q', {}).sources).toEqual([])
expect(mapExaResponse({}).sources).toEqual([])
})
})
describe('ExaSearchProvider status', () => {
describe('ExaSearchProvider availability', () => {
it('is unavailable without a key', () => {
expect(new ExaSearchProvider({ ...options, apiKey: '' }).status())
.toEqual({ available: false, reason: 'missing-credential' })
expect(new ExaSearchProvider({ ...options, apiKey: '' }).available()).toBe(false)
})
it('is available with a key', () => {
expect(new ExaSearchProvider(options).status()).toEqual({ available: true })
expect(new ExaSearchProvider(options).available()).toBe(true)
})
it('is misconfigured when the base URL is unparseable', () => {
expect(new ExaSearchProvider({ ...options, baseURL: 'not a url' }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new ExaSearchProvider({ ...options, baseURL: 'not a url' }).available()).toBe(false)
})
it('is misconfigured when highlightsPerResult is not a positive integer', () => {
expect(new ExaSearchProvider({ ...options, highlightsPerResult: 0 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new ExaSearchProvider({ ...options, highlightsPerResult: 1.5 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new ExaSearchProvider({ ...options, highlightsPerResult: 0 }).available()).toBe(false)
expect(new ExaSearchProvider({ ...options, highlightsPerResult: 1.5 }).available()).toBe(false)
})
it('is misconfigured when numResults is set but not a positive integer', () => {
expect(new ExaSearchProvider({ ...options, numResults: -1 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new ExaSearchProvider({ ...options, numResults: -1 }).available()).toBe(false)
})
})
@@ -139,7 +133,7 @@ describe('ExaSearchProvider request mapping', () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
const controller = new AbortController()
await new ExaSearchProvider(options).search({ query: 'q' }, { signal: controller.signal })
await new ExaSearchProvider(options).search({ query: 'q' }, controller.signal)
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
expect(init.signal).toBe(controller.signal)
})
@@ -209,7 +203,7 @@ describe('web-search-exa plugin registration', () => {
const ctx = new Context()
await ctx.plugin(WebService, { searchProvider: EXA_PROVIDER_ID })
const fiber = await ctx.plugin(exaPlugin, { apiKey: 'exa-key' })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: EXA_PROVIDER_ID })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ sources: [], truncated: false })
await fiber.dispose()
await expect(ctx.web.search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))

View File

@@ -8,8 +8,8 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
| Key | Default | Meaning |
|---|---|---|
| `apiKey` | `$PERPLEXITY_API_KEY` | Perplexity API key. Empty/absent provider `status()` reports `missing-credential`. |
| `baseURL` | `https://api.perplexity.ai` | Endpoint base; `/chat/completions` is appended. An unparseable value makes `status()` report `misconfigured`. |
| `apiKey` | `$PERPLEXITY_API_KEY` | Perplexity API key. Empty/absent makes the provider unavailable. |
| `baseURL` | `https://api.perplexity.ai` | Endpoint base; `/chat/completions` is appended. An unparseable value makes the provider unavailable. |
| `model` | `sonar` | Search model name. |
| `maxTokens` | `1024` | Upper bound on generated answer tokens (`max_tokens`). Must be a positive integer. |
| `searchRecency` | (unset) | Recency window sent as `search_recency_filter`: `day`, `week`, `month`, or `year`. Unset sends no filter. |
@@ -24,3 +24,24 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
## Mapping
`content``choices[0].message.content` (the generated answer). `sources[]` prefers the structured `search_results[]` (`url`, `title`, `snippet`, `publishedAt``date`), falling back to the URL-only `citations[]` array only when `search_results` is absent — those sources carry just a `url`, which is why `title`/`snippet`/`publishedAt` are optional on the seam. Provider failures surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`. Perplexity has no result-count control, so `maxResults` is enforced by the seam (truncating `sources[]` and setting `truncated`).
## Model Experience
### Auxiliary Perplexity request
**What the model sees**: A separate Perplexity model receives `<query>` verbatim as its sole user message through the chat-completions endpoint. This request is not part of the conversation model's context.
**Token effect**: Separate provider tokens are incurred per search; `maxTokens` caps the generated answer.
### Conversation tool result, indirectly
**What the model sees**: Through [`dsh-tool-web`](../tool-web/README.md), the conversation model sees the generated answer plus structured result metadata or URL-only citations. This provider's exact failures are `Perplexity search aborted`, `Perplexity search request failed: <error>`, and `Perplexity returned an unprocessable response body: <error>`; HTTP failures preserve the provider message. The consumer owns the error wrapper.
**Token effect**: Zero direct conversation tokens from registration. Answer and source tokens are data-dependent, source count is seam-bounded, and the retained result or error is resent until compaction.
## Known Limitations and Deferred Work
- **Citation-fallback sources are URL-only** — when Perplexity omits structured `search_results[]`, sources carry no `title`/`snippet`/`publishedAt`, so the tool renders bare hostname labels.
- **Over-returned sources still cost tokens and latency** — with no result-count control on the wire, `maxResults` is enforced only post-hoc by seam truncation.
- **Only `model`/`maxTokens`/`searchRecency` are exposed** — Perplexity's other search controls (domain filters, `web_search_options` context size, images) wait on provider-neutral seam fields ([seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md)).
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.

View File

@@ -23,13 +23,13 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-web": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -18,8 +18,6 @@ export {
PERPLEXITY_DEFAULT_MODEL,
PERPLEXITY_PROVIDER_ID,
PerplexitySearchProvider,
mapPerplexityResponse,
mapPerplexityResult,
} from './provider.ts'
export type { PerplexityRecency, PerplexitySearchProviderOptions } from './provider.ts'

View File

@@ -1,21 +1,13 @@
/**
* `PerplexitySearchProvider`: a `WebSearchProvider` backed by the Perplexity
* search API (an OpenAI-compatible `POST /chat/completions`). Maps the generated
* answer (`choices[0].message.content`) into `content`, and prefers the
* structured `search_results[]` for `sources[]`, falling back to the URL-only
* `citations[]` when `search_results` is absent.
*
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter. The OpenAI-compatible request shape
* is a provider-private detail and does NOT make this provider depend on
* Perplexity search over its OpenAI-compatible chat-completions endpoint. The generated answer
* becomes `content`; sources prefer structured `search_results[]` and fall back to URL-only
* `citations[]`. The wire format and native `fetch` client are provider-private and do not use
* `ctx.llm`.
*
* @module @deepseek-ai/dsh-web-search-perplexity/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type {
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
@@ -43,7 +35,7 @@ const USER_AGENT = 'deepseek-harness/0.0.1'
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
export interface PerplexitySearchProviderOptions {
/** Perplexity API key. Empty/absent → `status()` reports `missing-credential`. */
/** Perplexity API key. Empty/absent makes the provider unavailable. */
apiKey: string
/** Endpoint base; `/chat/completions` is appended. */
baseURL: string
@@ -75,18 +67,15 @@ export function mapPerplexityResult(result: PerplexitySearchResult): WebSearchSo
* structured `search_results[]`; falls back to URL-only `citations[]` (those
* sources carry just a `url`) only when `search_results` is absent.
*
* @param query - the original request query, echoed on the result.
* @param response - the parsed chat-completions response body.
* @returns the normalized result; `content` is omitted when the answer is empty.
*/
export function mapPerplexityResponse(query: string, response: PerplexityResponse): WebSearchResult {
export function mapPerplexityResponse(response: PerplexityResponse): WebSearchResult {
const content = response.choices?.[0]?.message?.content
const sources: WebSearchSource[] = response.search_results !== undefined
? response.search_results.map(mapPerplexityResult)
: (response.citations ?? []).map(url => ({ url }))
return {
providerId: PERPLEXITY_PROVIDER_ID,
query,
...content != null && content.length > 0 ? { content } : {},
sources,
truncated: false,
@@ -102,15 +91,14 @@ export class PerplexitySearchProvider implements WebSearchProvider {
// Availability checks stay beside each provider's distinct config contract;
// a shared base class would obscure which fields make this backend usable.
/* jscpd:ignore-start */
status(): WebProviderStatus {
if (this.options.apiKey.length === 0) return { available: false, reason: 'missing-credential' }
if (!URL.canParse(this.options.baseURL)) return { available: false, reason: 'misconfigured' }
if (!isPositiveInteger(this.options.maxTokens)) return { available: false, reason: 'misconfigured' }
return { available: true }
available(): boolean {
return this.options.apiKey.length > 0
&& URL.canParse(this.options.baseURL)
&& isPositiveInteger(this.options.maxTokens)
}
/* jscpd:ignore-end */
async search(request: WebSearchRequest, exec?: { readonly signal?: AbortSignal }): Promise<WebSearchResult> {
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
let response: Response
try {
response = await fetch(`${this.options.baseURL}/chat/completions`, {
@@ -127,7 +115,7 @@ export class PerplexitySearchProvider implements WebSearchProvider {
messages: [{ role: 'user', content: request.query }],
...this.options.searchRecency !== undefined ? { search_recency_filter: this.options.searchRecency } : {},
}),
...exec?.signal ? { signal: exec.signal } : {},
...signal !== undefined ? { signal } : {},
})
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('Perplexity search aborted', 'WEB_ABORTED', { cause: error })
@@ -155,7 +143,7 @@ export class PerplexitySearchProvider implements WebSearchProvider {
try {
const payload = await response.json() as PerplexityResponse
return mapPerplexityResponse(request.query, payload)
return mapPerplexityResponse(payload)
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('Perplexity search aborted', 'WEB_ABORTED', { cause: error })
throw new WebError(`Perplexity returned an unprocessable response body: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })

View File

@@ -1,13 +1,7 @@
/**
* Wire types for the Perplexity search API
* (`POST https://api.perplexity.ai/chat/completions`, an OpenAI-compatible chat
* shape). Types only — no runtime code. Perplexity returns a generated answer in
* `choices[0].message.content` plus citation surfaces: a structured
* `search_results[]` (preferred) and a URL-only `citations[]` fallback.
*
* The OpenAI-compatible wire shape is a provider-private detail; it does not make
* this provider depend on `ctx.llm`.
*
* Wire types for the Perplexity search API (`POST https://api.perplexity.ai/chat/completions`,
* an OpenAI-compatible chat shape). Results prefer structured `search_results` and fall back to
* URL-only `citations`; the provider-private wire shape does not depend on `ctx.llm`.
* @module @deepseek-ai/dsh-web-search-perplexity/types
*/

View File

@@ -17,7 +17,6 @@ maybe('PerplexitySearchProvider real API', () => {
maxTokens: PERPLEXITY_DEFAULT_MAX_TOKENS,
})
const result = await provider.search({ query: 'What is the DeepSeek Harness SDK?', maxResults: 5 })
expect(result.providerId).toBe('perplexity')
expect(result.content ?? '').not.toBe('')
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
}, 30_000)

View File

@@ -3,10 +3,10 @@ import { Context } from 'cordis'
import WebService from '@deepseek-ai/dsh-web'
import {
PerplexitySearchProvider,
mapPerplexityResponse,
PERPLEXITY_PROVIDER_ID,
} from '@deepseek-ai/dsh-web-search-perplexity'
import * as perplexityPlugin from '@deepseek-ai/dsh-web-search-perplexity'
import { mapPerplexityResponse } from '../src/provider.ts'
const options = { apiKey: 'pplx-key', baseURL: 'https://api.perplexity.test', model: 'sonar', maxTokens: 1024 }
@@ -20,7 +20,7 @@ afterEach(() => {
describe('Perplexity response mapping', () => {
it('maps the answer and prefers structured search_results', () => {
const result = mapPerplexityResponse('q', {
const result = mapPerplexityResponse({
choices: [{ message: { content: 'the answer' } }],
search_results: [
{ url: 'https://a.test', title: 'A', snippet: 'snip', date: '2026-02-02' },
@@ -29,8 +29,6 @@ describe('Perplexity response mapping', () => {
citations: ['https://ignored.test'],
})
expect(result).toEqual({
providerId: PERPLEXITY_PROVIDER_ID,
query: 'q',
content: 'the answer',
sources: [
{ url: 'https://a.test', title: 'A', snippet: 'snip', publishedAt: '2026-02-02' },
@@ -41,7 +39,7 @@ describe('Perplexity response mapping', () => {
})
it('falls back to URL-only citations when search_results is absent', () => {
const result = mapPerplexityResponse('q', {
const result = mapPerplexityResponse({
choices: [{ message: { content: 'answer' } }],
citations: ['https://a.test', 'https://b.test'],
})
@@ -49,43 +47,39 @@ describe('Perplexity response mapping', () => {
})
it('omits content when the answer is empty or missing', () => {
expect(mapPerplexityResponse('q', { citations: [] }).content).toBeUndefined()
expect(mapPerplexityResponse('q', { choices: [{ message: { content: '' } }] }).content).toBeUndefined()
expect(mapPerplexityResponse('q', { choices: [{ message: { content: null } }] }).content).toBeUndefined()
expect(mapPerplexityResponse({ citations: [] }).content).toBeUndefined()
expect(mapPerplexityResponse({ choices: [{ message: { content: '' } }] }).content).toBeUndefined()
expect(mapPerplexityResponse({ choices: [{ message: { content: null } }] }).content).toBeUndefined()
})
it('omits null/empty optional source fields', () => {
const result = mapPerplexityResponse('q', {
const result = mapPerplexityResponse({
search_results: [{ url: 'https://a.test', title: null, snippet: '', date: null }],
})
expect(result.sources).toEqual([{ url: 'https://a.test' }])
})
it('yields no sources when neither search_results nor citations are present', () => {
expect(mapPerplexityResponse('q', { choices: [{ message: { content: 'a' } }] }).sources).toEqual([])
expect(mapPerplexityResponse({ choices: [{ message: { content: 'a' } }] }).sources).toEqual([])
})
})
describe('PerplexitySearchProvider status', () => {
describe('PerplexitySearchProvider availability', () => {
it('is unavailable without a key', () => {
expect(new PerplexitySearchProvider({ ...options, apiKey: '' }).status())
.toEqual({ available: false, reason: 'missing-credential' })
expect(new PerplexitySearchProvider({ ...options, apiKey: '' }).available()).toBe(false)
})
it('is available with a key', () => {
expect(new PerplexitySearchProvider(options).status()).toEqual({ available: true })
expect(new PerplexitySearchProvider(options).available()).toBe(true)
})
it('is misconfigured when the base URL is unparseable', () => {
expect(new PerplexitySearchProvider({ ...options, baseURL: 'not a url' }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new PerplexitySearchProvider({ ...options, baseURL: 'not a url' }).available()).toBe(false)
})
it('is misconfigured when maxTokens is not a positive integer', () => {
expect(new PerplexitySearchProvider({ ...options, maxTokens: 0 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new PerplexitySearchProvider({ ...options, maxTokens: 1.5 }).status())
.toEqual({ available: false, reason: 'misconfigured' })
expect(new PerplexitySearchProvider({ ...options, maxTokens: 0 }).available()).toBe(false)
expect(new PerplexitySearchProvider({ ...options, maxTokens: 1.5 }).available()).toBe(false)
})
})
@@ -114,7 +108,7 @@ describe('PerplexitySearchProvider request mapping', () => {
const fetchMock = vi.fn(async () => jsonResponse({ citations: [] }))
vi.stubGlobal('fetch', fetchMock)
const controller = new AbortController()
await new PerplexitySearchProvider(options).search({ query: 'q' }, { signal: controller.signal })
await new PerplexitySearchProvider(options).search({ query: 'q' }, controller.signal)
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
expect(init.signal).toBe(controller.signal)
})
@@ -190,7 +184,7 @@ describe('web-search-perplexity plugin registration', () => {
const ctx = new Context()
await ctx.plugin(WebService, { searchProvider: PERPLEXITY_PROVIDER_ID })
const fiber = await ctx.plugin(perplexityPlugin, { apiKey: 'pplx-key' })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: PERPLEXITY_PROVIDER_ID })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ content: 'a', sources: [] })
await fiber.dispose()
await expect(ctx.web.search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))

View File

@@ -19,8 +19,8 @@ Search and fetch share no request schema and no business logic, but they are del
| Member | Semantics |
|---|---|
| `registerSearchProvider(provider)` / `registerFetchProvider(provider)` | Register a backend. Throws `WebError` `WEB_DUPLICATE_PROVIDER` on a duplicate id within that capability kind. Returns a disposer. Disposed with the calling fiber. |
| `search(request, exec?)` | Resolve the search provider and run one search. Enforces `request.maxResults` on the result (truncates `sources[]`, sets `truncated`). Throws `WebError` when the capability cannot run. |
| `fetch(request, exec?)` | Resolve the fetch provider and retrieve one URL. A non-2xx response is a result, not a throw. Throws `WebError` for failures to safely retrieve or represent the resource. |
| `search(request, signal?)` | Resolve the search provider and run one search. Enforces `request.maxResults` on the result (truncates `sources[]`, sets `truncated`). Throws `WebError` when the capability cannot run. |
| `fetch(request, signal?)` | Resolve the fetch provider and retrieve one URL. A non-2xx response is a result, not a throw. Throws `WebError` for failures to safely retrieve or represent the resource. |
Providers register **capabilities**, not tools. `dsh-tool-web` is the only owner of model-facing names, descriptions, prompt guidance, JSON schemas, and presentation.
@@ -30,15 +30,26 @@ Selection never depends on registration, config, or HMR order. A capability has
| Situation | Execution |
|---|---|
| configured id registered and `status().available` | runs that provider |
| configured id registered and `available()` | runs that provider |
| configured id not registered | `WEB_PROVIDER_CONFIGURED_MISSING` |
| configured id registered but unavailable | `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` |
| no id, exactly one registered usable provider | runs it |
| no id, no usable provider | `WEB_PROVIDER_UNAVAILABLE` |
| no id, multiple usable providers | `WEB_PROVIDER_AMBIGUOUS` |
The failure branches throw `WebError`, whose structured code (plus message detail — the missing id, the ambiguous candidate set) is the surface callers route on. A provider's own `status()` is a cheap local check (credential presence, parseable config) that feeds this execution-time selection and **must not make network calls**; `dsh-tool-web` never calls a provider's `status()` — it executes through `ctx.web.search()`/`fetch()` and routes on the thrown codes, so provider selection has one owner.
The failure branches throw `WebError`, whose structured code (plus message detail — the missing id, the ambiguous candidate set) is the surface callers route on. A provider's own `available()` is a cheap local check (credential presence, parseable config) that feeds this execution-time selection and **must not make network calls**; `dsh-tool-web` never calls it — the tool executes through `ctx.web.search()`/`fetch()` and routes on the thrown codes, so provider selection has one owner.
## Vocabulary
`WebSearchRequest` (`query`, `maxResults?`) → `WebSearchResult` (`providerId`, `query`, `content?`, `sources[]`, `truncated`); each `WebSearchSource` has a required `url` and optional `title`/`snippet`/`publishedAt` (Perplexity citations may be URL-only). `WebFetchRequest` (`url`, `timeoutMs?`) → `WebFetchResult` (`providerId`, final `url`, `statusCode`, `body`, `truncated`); `WebFetchBody` is a CLOSED discriminated union (`html` | `text`) owned here — consumers `switch` to exhaustiveness so a new kind breaks their compilation until handled. See `src/types.ts` for the full contracts and the `WebError` code taxonomy.
`WebSearchRequest` (`query`, `maxResults?`) → `WebSearchResult` (`content?`, `sources[]`, `truncated`); each `WebSearchSource` has a required `url` and optional `title`/`snippet`/`publishedAt` (Perplexity citations may be URL-only). `WebFetchRequest` (`url`) → `WebFetchResult` (final `url`, `statusCode`, `body`, `truncated`); cancellation is a direct optional `AbortSignal` argument to `search()`/`fetch()`. `WebFetchBody` is a CLOSED discriminated union (`html` | `text`) owned here — consumers `switch` to exhaustiveness so a new kind breaks their compilation until handled. See `src/types.ts` for the full contracts and the `WebError` code taxonomy.
## Model Experience
Indirectly, through `dsh-tool-web`, which retains bounded normalized provider data or the exact configured-provider, unavailable-provider, no-provider, multiple-provider, and `Error: <message>` failures while this registry contributes no prompt or schema itself.
## Known Limitations and Deferred Work
- **No observation surface** — no provider-change event and no capability-status query; availability is observed only by executing `search()`/`fetch()` and routing the thrown `WebError` codes, and the no-provider failure is the generic `WEB_PROVIDER_UNAVAILABLE` with no per-provider reason enumeration ([RFC](../../../docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md)).
- **`WebSearchRequest` carries only `query` + `maxResults`** — provider-neutral controls (recency, domain filters, regional hints, search depth) are deferred until Exa and Perplexity can both honor them honestly ([seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md)).
- **`WebFetchBody` has no `pdf` arm** — text-extractable PDF support is named deferred work; the closed union makes adding it a compile-enforced change across the three web packages.
- **Provider-backed page extraction is out of scope of `fetch()`** — a Firecrawl/Tavily-style `web_extract` capability is deferred rather than widening the fetch seam.

View File

@@ -23,13 +23,13 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-llm": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -1,28 +1,17 @@
/**
* The web access seam (`ctx.web`): a provider registry plus a provider-selecting
* execution surface for two capabilities — search and fetch. Provider packages
* register concrete backends with `registerSearchProvider` /
* `registerFetchProvider`; the model-facing consumer
* (`@deepseek-ai/dsh-tool-web`) executes through `search()` / `fetch()` and
* routes on the structured {@link WebError} codes selection throws.
*
* The registry half stays close to `LlmService`: a `Map<id, provider>` per
* capability kind, register methods that return disposers, duplicate ids that
* throw, and execution-time resolution that throws when the selected provider is
* absent or unusable — with selection rules that never depend on registration
* order.
*
* The web access seam (`ctx.web`): registries and provider-selecting execution for search and
* fetch. Duplicate ids are rejected. At execution time, a configured provider must exist and
* be usable; without one, exactly one usable provider is required, so selection never depends
* on registration order.
* @module @deepseek-ai/dsh-web
*/
import { Context, Service } from 'cordis'
import z from 'schemastery'
import type {
WebExecContext,
WebFetchProvider,
WebFetchRequest,
WebFetchResult,
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
@@ -33,12 +22,10 @@ export {
WebError,
} from './types.ts'
export type {
WebExecContext,
WebFetchBody,
WebFetchProvider,
WebFetchRequest,
WebFetchResult,
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
@@ -76,7 +63,7 @@ export interface WebServiceConfig {
* The web access service. Registered as `ctx.web` (one instance per context).
*
* Selection semantics (resolved at execution time, never order-dependent):
* - A configured id that is registered and `status().available` → that provider.
* - A configured id that is registered and `available()` → that provider.
* - A configured id not registered → `WEB_PROVIDER_CONFIGURED_MISSING`.
* - A configured id registered but unavailable →
* `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`.
@@ -147,15 +134,15 @@ export class WebService extends Service {
* capability cannot run. The seam enforces `request.maxResults` on the result:
* if the provider over-returns, `sources[]` is truncated and `truncated` set.
* @param request - the query plus result-shaping options.
* @param exec - the tool-execution context, forwarded to the provider.
* @param signal - optional cancellation signal forwarded to the provider.
* @returns the provider's results, capped to `request.maxResults`.
*/
async search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult> {
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
const provider = resolveProvider({
providers: this.searchProviders,
...this.searchProviderId !== undefined ? { configuredId: this.searchProviderId } : {},
})
const result = await provider.search(request, exec)
const result = await provider.search(request, signal)
return capSources(result, request.maxResults)
}
@@ -164,21 +151,21 @@ export class WebService extends Service {
* call time with the selection rules above; throws {@link WebError} when the
* capability cannot run. A non-2xx response is a result, not a throw.
* @param request - the URL plus retrieval options.
* @param exec - the tool-execution context, forwarded to the provider.
* @param signal - optional cancellation signal forwarded to the provider.
* @returns the retrieval outcome; non-2xx responses resolve descriptively.
*/
async fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult> {
async fetch(request: WebFetchRequest, signal?: AbortSignal): Promise<WebFetchResult> {
const provider = resolveProvider({
providers: this.fetchProviders,
...this.fetchProviderId !== undefined ? { configuredId: this.fetchProviderId } : {},
})
return provider.fetch(request, exec)
return provider.fetch(request, signal)
}
}
interface ResolvableProvider {
readonly id: string
status(): WebProviderStatus
available(): boolean
}
/** Resolve the selected provider or throw the matching {@link WebError}. */
@@ -189,12 +176,12 @@ function resolveProvider<P extends ResolvableProvider>(selection: Selection<P>):
if (!provider) {
throw new WebError(`configured web provider "${configuredId}" is not registered`, 'WEB_PROVIDER_CONFIGURED_MISSING')
}
if (!provider.status().available) {
if (!provider.available()) {
throw new WebError(`configured web provider "${configuredId}" is registered but unavailable`, 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE')
}
return provider
}
const usable = [...providers.values()].filter(provider => provider.status().available)
const usable = [...providers.values()].filter(provider => provider.available())
const [single] = usable
if (single === undefined) {
throw new WebError('no usable web provider is registered', 'WEB_PROVIDER_UNAVAILABLE')

View File

@@ -1,37 +1,12 @@
/**
* Vocabulary for the web capability seam (`ctx.web`): the search/fetch
* request/result shapes providers produce and consumers format, the provider
* status discriminant selection reads, the execution-control context, and the
* typed error taxonomy.
*
* These types are shared by every provider backend
* (`@deepseek-ai/dsh-web-search-exa`, `@deepseek-ai/dsh-web-search-perplexity`,
* `@deepseek-ai/dsh-web-fetch-local`, and future backends) and by the
* model-facing consumer (`@deepseek-ai/dsh-tool-web`). Search and fetch share no
* request schema and no business logic, but they are deliberately one seam:
* `ctx.web` is a single web-access middle layer with one provider-selection
* policy, one abort/error vocabulary, and one product-facing configuration
* point. The cost is the parallel `Search`/`Fetch` shapes below; that
* parallelism is intentional.
*
* Vocabulary for the web capability seam (`ctx.web`). Search and fetch deliberately share one
* seam so provider selection, cancellation, errors, and product configuration have one owner,
* while retaining separate request and result shapes.
* @module @deepseek-ai/dsh-web/types
*/
import { HarnessError } from '@deepseek-ai/dsh-llm'
/**
* Execution control threaded from the tool layer through the seam into a
* provider's network requests, stream readers, and expensive decoding. It is
* NOT business input: the first version carries only `signal` so `tool-web` can
* propagate turn cancellation, tool timeout, and agent disposal. It deliberately
* does NOT carry `ToolExecution`, which would make `dsh-web` depend on
* `dsh-tools`.
*/
export interface WebExecContext {
/** Abort signal a provider must honor for its network/decoding work. */
readonly signal?: AbortSignal
}
/**
* What one search-capable backend can return. The model-facing argument is just
* a query; `maxResults` is a `dsh-tool-web`-layer bound passed through unchanged
@@ -56,10 +31,6 @@ export interface WebSearchRequest {
* when it cut `sources[]` down to `maxResults`.
*/
export interface WebSearchResult {
/** Id of the provider that produced this result. */
readonly providerId: string
/** Echo of the query the provider answered. */
readonly query: string
/** Optional provider-generated answer text, search context, or summary. */
readonly content?: string
/** Citeable sources, already truncated to the request's `maxResults`. */
@@ -83,14 +54,13 @@ export interface WebSearchSource {
}
/**
* What one fetch-capable backend is asked to retrieve. `timeoutMs` is an
* optional positive hint the provider caps. The request deliberately omits
* `format`, `prompt`, and extraction controls — those are presentation or
* higher-level LLM concerns, not safe-retrieval inputs.
* What one fetch-capable backend is asked to retrieve. The request deliberately
* omits timeout, format, prompt, and extraction controls: cancellation is a
* direct execution argument, while presentation and higher-level LLM concerns
* belong outside safe retrieval.
*/
export interface WebFetchRequest {
readonly url: string
readonly timeoutMs?: number
}
/**
@@ -100,8 +70,6 @@ export interface WebFetchRequest {
* represent the resource.
*/
export interface WebFetchResult {
/** Id of the provider that produced this result. */
readonly providerId: string
/** The final URL after allowed redirects (the request URL is in the request). */
readonly url: string
/** HTTP status code of the fetched response. */
@@ -125,18 +93,6 @@ export type WebFetchBody =
| { readonly kind: 'html'; readonly content: string }
| { readonly kind: 'text'; readonly content: string }
/**
* Whether one concrete provider implementation is usable, by cheap local checks
* only (credential presence, parseable endpoint config). A provider `status()`
* must NOT make network calls. It is an input to execution-time selection, not
* a health system: `WebService.search()`/`fetch()` read it to pick a usable
* provider, and selection failure surfaces as the structured {@link WebError}
* codes callers route on.
*/
export type WebProviderStatus =
| { readonly available: true }
| { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' }
/**
* A search-capable backend. Registered with `ctx.web.registerSearchProvider`.
* `id` is a stable string, unique within the search capability kind.
@@ -144,9 +100,9 @@ export type WebProviderStatus =
export interface WebSearchProvider {
readonly id: string
/** Cheap local usability check; must not make network calls. */
status(): WebProviderStatus
/** Run one search; honor `exec.signal` for cancellation. */
search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult>
available(): boolean
/** Run one search; honor `signal` for cancellation. */
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>
}
/**
@@ -156,45 +112,17 @@ export interface WebSearchProvider {
export interface WebFetchProvider {
readonly id: string
/** Cheap local usability check; must not make network calls. */
status(): WebProviderStatus
/** Retrieve one URL; honor `exec.signal` for cancellation. */
fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult>
available(): boolean
/** Retrieve one URL; honor `signal` for cancellation. */
fetch(request: WebFetchRequest, signal?: AbortSignal): Promise<WebFetchResult>
}
/**
* Typed web error. Extends {@link HarnessError} so it carries a stable,
* machine-routable `code` (a `string`, like every other seam's error) and
* chains `cause`. `ToolRegistry.execute()` converts a thrown `WebError` into an
* error tool result whose structured metadata exposes the code, so callers
* (hooks, tests, UI) route on it.
*
* The `code` is an open `string`, NOT a closed union: a provider may raise its
* own codes without editing this package, and a consumer must tolerate an
* unknown code (a future provider will introduce ones this file never named).
* The codes split by who owns them — seam-neutral codes any provider may see,
* versus codes specific to a single implementation:
*
* Seam-neutral (raised by `WebService` selection and the shared contract):
* - `WEB_PROVIDER_UNAVAILABLE`: no provider configured and none usable.
* - `WEB_PROVIDER_CONFIGURED_MISSING`: a configured id is not registered.
* - `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`: a configured id is registered but its
* `status()` reports unavailable.
* - `WEB_PROVIDER_AMBIGUOUS`: no id configured and multiple usable providers
* exist (selection refuses to pick by registration order).
* - `WEB_DUPLICATE_PROVIDER`: a registration-time programming error — an id is
* already registered for that capability kind.
* - `WEB_ABORTED`: the operation was aborted via `WebExecContext.signal`.
* - `WEB_PROVIDER_ERROR`: catch-all for a provider's own failure surfaced
* through the seam, including network/transport failure (DNS, connection
* refused, TLS).
*
* Fetch-transport codes (owned by the `dsh-web-fetch-local` implementation; a
* different fetch backend need not raise these and may raise its own):
* - `WEB_INVALID_URL`: the fetch URL is malformed or not http(s).
* - `WEB_BLOCKED_URL`: the fetch URL is rejected by policy (credentials in URL).
* - `WEB_REDIRECT_BLOCKED`: a cross-origin redirect was refused.
* - `WEB_FETCH_TOO_LARGE`: the response exceeded the byte/character cap.
* - `WEB_FETCH_TIMEOUT`: the fetch exceeded its timeout.
* - `WEB_UNSUPPORTED_CONTENT_TYPE`: the response content type cannot be decoded.
* Typed web error with a machine-routable, open-string `code` and chained `cause`.
* Consumers must tolerate provider-specific codes. Shared codes cover unavailable,
* missing, unusable, ambiguous, or duplicate providers, cancellation, and provider failure;
* the local fetch provider additionally distinguishes invalid or blocked URLs, redirects,
* size and timeout limits, and unsupported content types. Tool execution exposes the code in
* structured error metadata.
*/
export class WebError extends HarnessError {}

View File

@@ -4,7 +4,6 @@ import WebService, {
WebError,
type WebFetchProvider,
type WebFetchResult,
type WebProviderStatus,
type WebSearchProvider,
type WebSearchRequest,
type WebSearchResult,
@@ -13,25 +12,25 @@ import WebService, {
/** A scripted search provider for contract tests. */
function makeSearchProvider(
id: string,
status: WebProviderStatus,
available: boolean,
search: (request: WebSearchRequest) => Promise<WebSearchResult>,
): WebSearchProvider {
return { id, status: () => status, search: request => search(request) }
return { id, available: () => available, search: request => search(request) }
}
function makeFetchProvider(id: string, status: WebProviderStatus, result: WebFetchResult): WebFetchProvider {
return { id, status: () => status, fetch: () => Promise.resolve(result) }
function makeFetchProvider(id: string, available: boolean, result: WebFetchResult): WebFetchProvider {
return { id, available: () => available, fetch: () => Promise.resolve(result) }
}
const available: WebProviderStatus = { available: true }
const unavailable: WebProviderStatus = { available: false, reason: 'missing-credential' }
const available = true
const unavailable = false
function searchResult(providerId: string, overrides: Partial<WebSearchResult> = {}): WebSearchResult {
return { providerId, query: 'q', sources: [], truncated: false, ...overrides }
function searchResult(marker: string, overrides: Partial<WebSearchResult> = {}): WebSearchResult {
return { content: marker, sources: [], truncated: false, ...overrides }
}
function fetchResult(providerId: string): WebFetchResult {
return { providerId, url: 'https://example.com', statusCode: 200, body: { kind: 'text', content: 'hi' }, truncated: false }
function fetchResult(marker: string): WebFetchResult {
return { url: 'https://example.com', statusCode: 200, body: { kind: 'text', content: marker }, truncated: false }
}
/** Mount a WebService on a fresh root context with the given config. */
@@ -46,7 +45,7 @@ describe('WebService registration', () => {
const { web } = await mountWeb()
const dispose = web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa'))))
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' })
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ content: 'exa' })
dispose()
await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' }))
@@ -70,7 +69,7 @@ describe('WebService registration', () => {
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
inner.web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa'))))
}, { inject: ['web'] }))
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' })
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ content: 'exa' })
await fiber.dispose()
await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' }))
})
@@ -111,26 +110,26 @@ describe('WebService execution resolution', () => {
const { web } = await mountWeb({ searchProvider: 'perplexity' })
web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa'))))
web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity'))))
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' })
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ content: 'perplexity' })
})
it('ignores unusable providers when auto-selecting', async () => {
const { web } = await mountWeb()
web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa'))))
web.registerSearchProvider(makeSearchProvider('perplexity', unavailable, () => Promise.resolve(searchResult('perplexity'))))
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' })
await expect(web.search({ query: 'q' })).resolves.toMatchObject({ content: 'exa' })
})
it('does not let registration order change auto-selection', async () => {
const a = await mountWeb()
a.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa'))))
a.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity'))))
await expect(a.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' })
await expect(a.web.search({ query: 'q' })).resolves.toMatchObject({ content: 'perplexity' })
const b = await mountWeb()
b.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity'))))
b.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa'))))
await expect(b.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' })
await expect(b.web.search({ query: 'q' })).resolves.toMatchObject({ content: 'perplexity' })
})
it('runs the selected provider and returns its result', async () => {
@@ -139,7 +138,6 @@ describe('WebService execution resolution', () => {
searchResult('exa', { content: 'answer', sources: [{ url: 'https://a' }] }),
)))
const result = await web.search({ query: 'q' })
expect(result.providerId).toBe('exa')
expect(result.content).toBe('answer')
expect(result.sources).toEqual([{ url: 'https://a' }])
})
@@ -149,11 +147,11 @@ describe('WebService execution resolution', () => {
const seen: (AbortSignal | undefined)[] = []
web.registerSearchProvider({
id: 'exa',
status: () => available,
search: (_request, exec) => { seen.push(exec?.signal); return Promise.resolve(searchResult('exa')) },
available: () => available,
search: (_request, signal) => { seen.push(signal); return Promise.resolve(searchResult('exa')) },
})
const controller = new AbortController()
await web.search({ query: 'q' }, { signal: controller.signal })
await web.search({ query: 'q' }, controller.signal)
expect(seen[0]).toBe(controller.signal)
})
})
@@ -195,7 +193,7 @@ describe('WebService fetch capability', () => {
const { web } = await mountWeb()
web.registerFetchProvider(makeFetchProvider('local-http', available, fetchResult('local-http')))
const result = await web.fetch({ url: 'https://example.com' })
expect(result.providerId).toBe('local-http')
expect(result.body.content).toBe('local-http')
expect(result.statusCode).toBe(200)
})