Add web capability seam: ctx.web, search/fetch providers, web tools

Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.

Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.

Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
This commit is contained in:
Dudu-0223
2026-06-25 15:04:12 +08:00
parent a4091daa3d
commit d01f5f73b7
50 changed files with 3610 additions and 8 deletions

View File

@@ -0,0 +1,270 @@
/**
* The web access seam (`ctx.web`): a provider registry plus a provider-selecting
* execution surface for two capabilities — search and fetch. Provider packages
* register concrete backends with `registerSearchProvider` /
* `registerFetchProvider`; the model-facing consumer
* (`@deepseek-ai/dsh-tool-web`) reads capability status and executes through
* `search()` / `fetch()`.
*
* The registry half stays close to `LlmService`: a `Map<id, provider>` per
* capability kind, register methods that return disposers, duplicate ids that
* throw, and execution-time resolution that throws when the selected provider is
* absent or unusable. On top of that sits one small selection-status layer so
* diagnostics and execution can explain why a capability can or cannot run,
* independent of registration order.
*
* @module @deepseek-ai/dsh-web
*/
import { Context, Service } from 'cordis'
import z from 'schemastery'
import type {
WebCapabilityStatus,
WebExecContext,
WebFetchProvider,
WebFetchRequest,
WebFetchResult,
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
} from './types.ts'
import { WebError } from './types.ts'
export {
WebError,
} from './types.ts'
export type {
WebCapabilityStatus,
WebErrorCode,
WebExecContext,
WebFetchBody,
WebFetchProvider,
WebFetchRequest,
WebFetchResult,
WebProviderStatus,
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
WebSearchSource,
} from './types.ts'
declare module 'cordis' {
interface Context {
web: WebService
}
interface Events {
/**
* Fired after the provider registry changes — a search or fetch provider was
* registered or disposed. Carries no payload and no capability graph: it
* means only "the provider registry changed; observers may recompute status
* from `ctx.web`". `searchStatus()` / `fetchStatus()` stay derived, not
* stored.
* @mode emit
*/
'web/providers-change'(this: WebService): void
}
}
/** Selection inputs shared by the status query and execution resolution. */
interface Selection<P> {
/** The configured provider id for this capability, if any. */
readonly configuredId?: string
/** Providers registered for this capability kind. */
readonly providers: ReadonlyMap<string, P>
}
/**
* Config for the web seam. `searchProvider` / `fetchProvider` pin which provider
* wins for each capability; both are optional (a single registered usable
* provider auto-selects). Operational overrides such as environment variables
* must feed these same fields rather than introduce a hidden priority chain.
*/
export interface WebServiceConfig {
/** Explicit search provider id. Omitted = auto-select when exactly one usable. */
readonly searchProvider?: string
/** Explicit fetch provider id. Omitted = auto-select when exactly one usable. */
readonly fetchProvider?: string
}
/**
* The web access service. Registered as `ctx.web` (one instance per context).
*
* Selection semantics (identical for status and execution, never order-
* dependent):
* - A configured id that is registered and `status().available` → that provider.
* - A configured id not registered → `configured-missing` /
* `WEB_PROVIDER_CONFIGURED_MISSING`.
* - A configured id registered but unavailable → `configured-unavailable` /
* `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`.
* - No id configured, exactly one registered usable provider → that provider.
* - No id configured, multiple usable providers → `ambiguous` /
* `WEB_PROVIDER_AMBIGUOUS`.
* - No id configured, no usable provider → `none` / `WEB_PROVIDER_UNAVAILABLE`.
*/
export class WebService extends Service {
/**
* Provider selection config. Operational env overrides feed the SAME fields:
* `$DSH_WEB_SEARCH_PROVIDER` / `$DSH_WEB_FETCH_PROVIDER` are equivalent to
* `searchProvider` / `fetchProvider` and are NOT a hidden priority chain.
*/
static Config: z<WebServiceConfig> = z.object({
searchProvider: z.string(),
fetchProvider: z.string(),
})
private searchProviders = new Map<string, WebSearchProvider>()
private fetchProviders = new Map<string, WebFetchProvider>()
private readonly searchProviderId: string | undefined
private readonly fetchProviderId: string | undefined
constructor(ctx: Context, config: WebServiceConfig = {}) {
super(ctx, 'web')
this.searchProviderId = config.searchProvider ?? process.env.DSH_WEB_SEARCH_PROVIDER
this.fetchProviderId = config.fetchProvider ?? process.env.DSH_WEB_FETCH_PROVIDER
}
/**
* Register a search provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER`
* if its id is already registered for search. Returns a disposer; emits
* `web/providers-change` after a successful register and again on dispose.
* Disposed with the calling fiber.
*/
registerSearchProvider(provider: WebSearchProvider): () => void {
return this.registerProvider(this.searchProviders, provider)
}
/**
* Register a fetch provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER`
* if its id is already registered for fetch. Returns a disposer; emits
* `web/providers-change` after a successful register and again on dispose.
* Disposed with the calling fiber.
*/
registerFetchProvider(provider: WebFetchProvider): () => void {
return this.registerProvider(this.fetchProviders, provider)
}
private registerProvider<P extends { readonly id: string }>(store: Map<string, P>, provider: P): () => void {
if (store.has(provider.id)) {
throw new WebError(`a web provider with id "${provider.id}" is already registered`, 'WEB_DUPLICATE_PROVIDER')
}
const dispose = this.ctx.effect(function* (this: WebService) {
store.set(provider.id, provider)
// Yield the rollback BEFORE emitting `web/providers-change`: the generator
// effect collects each yielded disposer before the next step runs, so a
// throwing change listener removes the just-added provider instead of
// leaking it into the registry.
yield () => {
store.delete(provider.id)
this.ctx.emit('web/providers-change')
}
this.ctx.emit('web/providers-change')
}.bind(this), 'web.registerProvider()')
// ctx.effect's disposer returns Promise<void>; our disposer API is
// synchronous fire-and-forget — discard the (always-resolved) promise.
return () => void dispose()
}
/** Search-capability selection status, derived live (never stored). */
searchStatus(): WebCapabilityStatus {
return resolveStatus({
providers: this.searchProviders,
...this.searchProviderId !== undefined ? { configuredId: this.searchProviderId } : {},
})
}
/** Fetch-capability selection status, derived live (never stored). */
fetchStatus(): WebCapabilityStatus {
return resolveStatus({
providers: this.fetchProviders,
...this.fetchProviderId !== undefined ? { configuredId: this.fetchProviderId } : {},
})
}
/**
* Run one search through the selected provider. Resolves the provider at call
* time with the selection rules above; throws {@link WebError} when the
* capability cannot run. The seam enforces `request.maxResults` on the result:
* if the provider over-returns, `sources[]` is truncated and `truncated` set.
*/
async search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult> {
const provider = resolveProvider({
providers: this.searchProviders,
...this.searchProviderId !== undefined ? { configuredId: this.searchProviderId } : {},
})
const result = await provider.search(request, exec)
return capSources(result, request.maxResults)
}
/**
* Retrieve one URL through the selected provider. Resolves the provider at
* call time with the selection rules above; throws {@link WebError} when the
* capability cannot run. A non-2xx response is a result, not a throw.
*/
async fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult> {
const provider = resolveProvider({
providers: this.fetchProviders,
...this.fetchProviderId !== undefined ? { configuredId: this.fetchProviderId } : {},
})
return provider.fetch(request, exec)
}
}
interface ResolvableProvider {
readonly id: string
status(): WebProviderStatus
}
/** Compute the capability status from configured id + registered providers. */
function resolveStatus<P extends ResolvableProvider>(selection: Selection<P>): WebCapabilityStatus {
const { configuredId, providers } = selection
if (configuredId !== undefined) {
const provider = providers.get(configuredId)
if (!provider) return { available: false, reason: 'configured-missing' }
if (!provider.status().available) return { available: false, reason: 'configured-unavailable' }
return { available: true, providerId: configuredId }
}
const usable = [...providers.values()].filter(provider => provider.status().available)
const [single] = usable
if (single === undefined) return { available: false, reason: 'none' }
if (usable.length > 1) return { available: false, reason: 'ambiguous' }
return { available: true, providerId: single.id }
}
/**
* Resolve the selected provider or throw the matching {@link WebError}. Shares
* the selection rules with {@link resolveStatus} so status and execution can
* never disagree.
*/
function resolveProvider<P extends ResolvableProvider>(selection: Selection<P>): P {
const { configuredId, providers } = selection
if (configuredId !== undefined) {
const provider = providers.get(configuredId)
if (!provider) {
throw new WebError(`configured web provider "${configuredId}" is not registered`, 'WEB_PROVIDER_CONFIGURED_MISSING')
}
if (!provider.status().available) {
throw new WebError(`configured web provider "${configuredId}" is registered but unavailable`, 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE')
}
return provider
}
const usable = [...providers.values()].filter(provider => provider.status().available)
const [single] = usable
if (single === undefined) {
throw new WebError('no usable web provider is registered', 'WEB_PROVIDER_UNAVAILABLE')
}
if (usable.length > 1) {
const ids = usable.map(provider => provider.id).join(', ')
throw new WebError(`multiple usable web providers are registered (${ids}); configure one explicitly`, 'WEB_PROVIDER_AMBIGUOUS')
}
return single
}
/** Enforce `maxResults` on a search result: truncate `sources[]` and flag it. */
function capSources(result: WebSearchResult, maxResults: number | undefined): WebSearchResult {
if (maxResults === undefined || result.sources.length <= maxResults) return result
return { ...result, sources: result.sources.slice(0, maxResults), truncated: true }
}
export default WebService

View File

@@ -0,0 +1,225 @@
/**
* Vocabulary for the web capability seam (`ctx.web`): the search/fetch
* request/result shapes providers produce and consumers format, the provider
* and capability status discriminants selection reports, the execution-control
* context, and the typed error taxonomy.
*
* These types are shared by every provider backend
* (`@deepseek-ai/dsh-web-search-exa`, `@deepseek-ai/dsh-web-search-perplexity`,
* `@deepseek-ai/dsh-web-fetch-local`, and future backends) and by the
* model-facing consumer (`@deepseek-ai/dsh-tool-web`). Search and fetch share no
* request schema and no business logic, but they are deliberately one seam:
* `ctx.web` is a single web-access middle layer with one provider-selection
* policy, one abort/error vocabulary, and one product-facing configuration
* point. The cost is the parallel `Search`/`Fetch` shapes below; that
* parallelism is intentional.
*
* @module @deepseek-ai/dsh-web/types
*/
import { HarnessError } from '@deepseek-ai/dsh-llm'
/**
* Execution control threaded from the tool layer through the seam into a
* provider's network requests, stream readers, and expensive decoding. It is
* NOT business input: the first version carries only `signal` so `tool-web` can
* propagate turn cancellation, tool timeout, and agent disposal. It deliberately
* does NOT carry `ToolExecution`, which would make `dsh-web` depend on
* `dsh-tools`.
*/
export interface WebExecContext {
/** Abort signal a provider must honor for its network/decoding work. */
readonly signal?: AbortSignal
}
/**
* What one search-capable backend can return. The model-facing argument is just
* a query; `maxResults` is a `dsh-tool-web`-layer bound passed through unchanged
* and enforced on the way back by the seam (see {@link WebSearchResult}).
*/
export interface WebSearchRequest {
readonly query: string
/**
* Upper bound on returned sources; the seam truncates to it. Omitted = no
* bound. `dsh-tool-web` always sets it. A provider whose API supports a
* result-count control (Exa's `numResults`) should apply it at the request
* layer as a cost/latency optimization; the seam enforces the bound
* regardless.
*/
readonly maxResults?: number
}
/**
* Normalized search outcome. `content` is optional provider-generated answer
* text or summary (Exa returns none; Perplexity returns a generated answer).
* `sources[]` is the portable citation surface. `truncated` is set by the seam
* when it cut `sources[]` down to `maxResults`.
*/
export interface WebSearchResult {
/** Id of the provider that produced this result. */
readonly providerId: string
/** Echo of the query the provider answered. */
readonly query: string
/** Optional provider-generated answer text, search context, or summary. */
readonly content?: string
/** Citeable sources, already truncated to the request's `maxResults`. */
readonly sources: readonly WebSearchSource[]
/** True when the seam dropped sources to honor `maxResults`. */
readonly truncated: boolean
}
/**
* One citeable source. A source always has a URL; `title`, `snippet`, and
* `publishedAt` are optional because not every provider returns them — forcing
* adapters to invent them would make the seam lie (Perplexity citations may be
* URL-only). `dsh-tool-web` renders `title ?? hostname(url)` for display.
*/
export interface WebSearchSource {
readonly url: string
readonly title?: string
readonly snippet?: string
/** Publication/crawl timestamp as a provider-supplied ISO-8601 string. */
readonly publishedAt?: string
}
/**
* What one fetch-capable backend is asked to retrieve. `timeoutMs` is an
* optional positive hint the provider caps. The request deliberately omits
* `format`, `prompt`, and extraction controls — those are presentation or
* higher-level LLM concerns, not safe-retrieval inputs.
*/
export interface WebFetchRequest {
readonly url: string
readonly timeoutMs?: number
}
/**
* Normalized fetch outcome. A successful network fetch of a non-2xx response is
* a result, not an error: the status code is part of the fetched resource
* state. {@link WebError} is reserved for failures to safely retrieve or
* represent the resource.
*/
export interface WebFetchResult {
/** Id of the provider that produced this result. */
readonly providerId: string
/** The final URL after allowed redirects (the request URL is in the request). */
readonly url: string
/** HTTP status code of the fetched response. */
readonly statusCode: number
/** Decoded body, classified by content kind. */
readonly body: WebFetchBody
/** True when the provider capped the decoded body. */
readonly truncated: boolean
}
/**
* The decoded body of a fetched resource. A CLOSED discriminated union owned by
* `dsh-web`: the provider decodes the kind and `dsh-tool-web` renders it, so a
* new kind is a coordinated change across known packages, not a plugin
* extension. Consumers `switch` on `kind` ending in `default: assertNever(...)`
* so adding a kind breaks compilation at every consumer until handled. Each arm
* stays its own object literal even where fields coincide today, leaving room
* for arm-specific fields later (a `pdf` body's `pageCount`).
*/
export type WebFetchBody =
| { readonly kind: 'html'; readonly content: string }
| { readonly kind: 'text'; readonly content: string }
/**
* Whether one concrete provider implementation is usable, by cheap local checks
* only (credential presence, parseable endpoint config). A provider `status()`
* must NOT make network calls. It is an input to selection, not a health system.
*/
export type WebProviderStatus =
| { readonly available: true }
| { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' }
/**
* Whether a capability (search or fetch) has a selected usable provider, or the
* broad category in which selection fails. Intentionally small: it carries the
* winning `providerId` on the available branch (so diagnostics can report which
* provider won) but NOT the per-reason payload (the missing id, the ambiguous
* candidate set). That branchable detail lives in the {@link WebError} thrown at
* execution time — the surface callers route on — so the same fact does not get
* two homes that can disagree.
*/
export type WebCapabilityStatus =
| { readonly available: true; readonly providerId: string }
| { readonly available: false; readonly reason: 'none' | 'configured-missing' | 'configured-unavailable' | 'ambiguous' }
/**
* A search-capable backend. Registered with `ctx.web.registerSearchProvider`.
* `id` is a stable string, unique within the search capability kind.
*/
export interface WebSearchProvider {
readonly id: string
/** Cheap local usability check; must not make network calls. */
status(): WebProviderStatus
/** Run one search; honor `exec.signal` for cancellation. */
search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult>
}
/**
* A fetch-capable backend. Registered with `ctx.web.registerFetchProvider`.
* `id` is a stable string, unique within the fetch capability kind.
*/
export interface WebFetchProvider {
readonly id: string
/** Cheap local usability check; must not make network calls. */
status(): WebProviderStatus
/** Retrieve one URL; honor `exec.signal` for cancellation. */
fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult>
}
/**
* Stable codes for {@link WebError}. Callers (hooks, tests, UI) route on these.
*
* - `WEB_PROVIDER_UNAVAILABLE`: no provider configured and none usable.
* - `WEB_PROVIDER_CONFIGURED_MISSING`: a configured id is not registered.
* - `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`: a configured id is registered but its
* `status()` reports unavailable.
* - `WEB_PROVIDER_AMBIGUOUS`: no id configured and multiple usable providers
* exist (selection refuses to pick by registration order).
* - `WEB_DUPLICATE_PROVIDER`: a registration-time programming error — an id is
* already registered for that capability kind.
* - `WEB_INVALID_URL`: the fetch URL is malformed or not http(s).
* - `WEB_BLOCKED_URL`: the fetch URL is rejected by policy (credentials in URL).
* - `WEB_REDIRECT_BLOCKED`: a cross-origin redirect was refused.
* - `WEB_FETCH_TOO_LARGE`: the response exceeded the byte/character cap.
* - `WEB_FETCH_TIMEOUT`: the fetch exceeded its timeout.
* - `WEB_ABORTED`: the operation was aborted via `WebExecContext.signal`.
* - `WEB_UNSUPPORTED_CONTENT_TYPE`: the response content type cannot be decoded.
* - `WEB_PROVIDER_ERROR`: catch-all for a provider's own failure surfaced through
* the seam, including network/transport failure (DNS, connection refused, TLS).
*/
export type WebErrorCode =
| 'WEB_PROVIDER_UNAVAILABLE'
| 'WEB_PROVIDER_CONFIGURED_MISSING'
| 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE'
| 'WEB_PROVIDER_AMBIGUOUS'
| 'WEB_DUPLICATE_PROVIDER'
| 'WEB_INVALID_URL'
| 'WEB_BLOCKED_URL'
| 'WEB_REDIRECT_BLOCKED'
| 'WEB_FETCH_TOO_LARGE'
| 'WEB_FETCH_TIMEOUT'
| 'WEB_ABORTED'
| 'WEB_UNSUPPORTED_CONTENT_TYPE'
| 'WEB_PROVIDER_ERROR'
/**
* Typed web error. Extends {@link HarnessError} so it carries a stable
* {@link WebErrorCode} and chains `cause`. `dsh-web` owns this vocabulary so
* providers, the seam, and the tool layer raise the same codes instead of each
* inventing message strings. `ToolRegistry.execute()` converts a thrown
* `WebError` into an error tool result whose structured metadata exposes the
* code.
*/
export class WebError extends HarnessError {
override readonly code: WebErrorCode
constructor(message: string, code: WebErrorCode, options?: ErrorOptions) {
super(message, code, options)
this.code = code
}
}