Add web capability seam: ctx.web, search/fetch providers, web tools

Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.

Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.

Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
This commit is contained in:
Dudu-0223
2026-06-25 15:04:12 +08:00
parent a4091daa3d
commit d01f5f73b7
50 changed files with 3610 additions and 8 deletions

View File

@@ -0,0 +1,34 @@
# @deepseek-ai/dsh-web-fetch-local
An anonymous public HTTP(S) `WebFetchProvider` for the harness [web capability seam](../web/README.md) (`ctx.web`). It retrieves a concrete URL and returns a status code plus bounded decoded content.
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the key and it does not register a model-facing tool. It is a function/namespace plugin (`inject: ['web']`).
## Responsibility split
The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource.
## Transport hygiene
- Accepts only `http:` and `https:` URLs; rejects credentials in URLs (`WEB_BLOCKED_URL`) and over-long/malformed URLs (`WEB_INVALID_URL`).
- Enforces a max URL length, response byte cap (`WEB_FETCH_TOO_LARGE`), decoded body character cap, timeout (`WEB_FETCH_TIMEOUT`), and redirect hop cap.
- Propagates the caller's abort signal (`WEB_ABORTED`) into the network request and the streaming read.
- Follows only **same-origin** redirects; a cross-origin redirect fails with `WEB_REDIRECT_BLOCKED`, requiring a fresh tool call (the model of Claude Code's WebFetch).
- Sends an explicit product `User-Agent`, never a browser disguise.
- Rejects unsupported (e.g. binary) content types with `WEB_UNSUPPORTED_CONTENT_TYPE`.
## Config
| Key | Default | Meaning |
|---|---|---|
| `maxUrlLength` | `2048` | Maximum accepted request URL length. |
| `maxResponseBytes` | `5_000_000` | Maximum response body size in bytes. |
| `maxBodyChars` | `100_000` | Maximum decoded body length in characters. |
| `timeoutMs` | `30_000` | Default fetch timeout. |
| `maxTimeoutMs` | `120_000` | Upper bound for a per-request timeout override. |
| `maxRedirects` | `5` | Maximum same-origin redirect hops. |
| `userAgent` | `deepseek-harness/…` | `User-Agent` header. |
## Security note
SSRF / private-network protection (blocking private, loopback, link-local, multicast, and otherwise non-public destinations, with DNS-resolve-then-validate and per-hop re-validation) is **deferred** — see the [web capability seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md). Until it lands, this provider is an SSRF primitive and **must not be enabled** in a deployment that can reach sensitive internal network targets.

View File

@@ -0,0 +1,33 @@
{
"name": "@deepseek-ai/dsh-web-fetch-local",
"description": "Anonymous public HTTP(S) fetch provider for the DeepSeek Harness web capability seam (ctx.web)",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/index.d.ts",
"exports": {
".": {
"types": "./lib/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-web": "^0.0.1",
"cordis": "^4.0.0-rc.6"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-web": "workspace:^",
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,77 @@
/**
* `@deepseek-ai/dsh-web-fetch-local`: registers an anonymous public HTTP(S)
* `WebFetchProvider` with `ctx.web`. A function/namespace plugin (NOT a
* default-export service): it registers INTO the seam's fetch registry, like the
* search providers register into the search registry.
*
* @module @deepseek-ai/dsh-web-fetch-local
*/
import type { Context } from 'cordis'
import z from 'schemastery'
import type {} from '@deepseek-ai/dsh-web'
import { LocalFetchProvider } from './provider.ts'
import type { LocalFetchLimits } from './provider.ts'
export {
LOCAL_FETCH_PROVIDER_ID,
LocalFetchProvider,
} from './provider.ts'
export type { LocalFetchLimits } from './provider.ts'
export { classifyContentType, isSameOrigin, validateFetchUrl } from './policy.ts'
export type { FetchableKind } from './policy.ts'
/** Default `User-Agent`: an explicit product agent, never a browser disguise. */
export const DEFAULT_USER_AGENT = 'deepseek-harness/0.0.1 (+https://github.com/deepseek-ai)'
/** Cordis plugin name used by loader diagnostics. */
export const name = 'web-fetch-local'
/** The web seam this provider registers into. */
export const inject = ['web']
export interface Config {
/** Maximum accepted request URL length. */
maxUrlLength?: number
/** Maximum response body size in bytes. */
maxResponseBytes?: number
/** Maximum decoded body length in characters. */
maxBodyChars?: number
/** Default fetch timeout in milliseconds. */
timeoutMs?: number
/** Upper bound for a per-request timeout override. */
maxTimeoutMs?: number
/** Maximum number of same-origin redirect hops to follow. */
maxRedirects?: number
/** `User-Agent` header sent on every request. */
userAgent?: string
}
export const Config: z<Config> = z.object({
maxUrlLength: z.number().default(2048),
maxResponseBytes: z.number().default(5_000_000),
maxBodyChars: z.number().default(100_000),
timeoutMs: z.number().default(30_000),
maxTimeoutMs: z.number().default(120_000),
maxRedirects: z.number().default(5),
userAgent: z.string().default(DEFAULT_USER_AGENT),
})
/** The shape after schemastery applies its defaults to every field. */
type ResolvedConfig = Required<Config>
/** Register the local HTTP(S) fetch provider with `ctx.web`. */
export function apply(ctx: Context, config: Config): void {
// schemastery (Config) has already filled every defaulted field.
const resolved = config as ResolvedConfig
const limits: LocalFetchLimits = {
maxUrlLength: resolved.maxUrlLength,
maxResponseBytes: resolved.maxResponseBytes,
maxBodyChars: resolved.maxBodyChars,
timeoutMs: resolved.timeoutMs,
maxTimeoutMs: resolved.maxTimeoutMs,
maxRedirects: resolved.maxRedirects,
userAgent: resolved.userAgent,
}
ctx.web.registerFetchProvider(new LocalFetchProvider(limits))
}

View File

@@ -0,0 +1,59 @@
/**
* URL validation and content-type classification for the local HTTP(S) fetch
* provider — the pure, network-free half. The provider's `fetch()` composes
* these with transport (redirect following, byte caps, decoding).
*
* @module @deepseek-ai/dsh-web-fetch-local/policy
*/
import { WebError } from '@deepseek-ai/dsh-web'
/** The body kinds this provider decodes. */
export type FetchableKind = 'html' | 'text'
/**
* Validate a request URL against the basic transport hygiene the provider
* enforces before any network access: http(s) only, no embedded credentials,
* bounded length. Returns the parsed `URL`. Throws {@link WebError} otherwise.
* (SSRF / private-network blocking is deferred — see the package RFC.)
*/
export function validateFetchUrl(input: string, maxUrlLength: number): URL {
if (input.length > maxUrlLength) {
throw new WebError(`URL exceeds the maximum length of ${maxUrlLength}`, 'WEB_INVALID_URL')
}
let url: URL
try {
url = new URL(input)
} catch (error: unknown) {
throw new WebError(`invalid URL: ${input}`, 'WEB_INVALID_URL', { cause: error })
}
if (url.protocol !== 'http:' && url.protocol !== 'https:') {
throw new WebError(`unsupported URL scheme "${url.protocol}" (only http and https are allowed)`, 'WEB_INVALID_URL')
}
if (url.username.length > 0 || url.password.length > 0) {
throw new WebError('credentials in URLs are not allowed', 'WEB_BLOCKED_URL')
}
return url
}
/**
* Two URLs are same-origin when scheme, hostname, and port match. A redirect
* that crosses origins is refused so each new origin requires a fresh tool call
* (and thus a fresh provider/permission decision).
*/
export function isSameOrigin(a: URL, b: URL): boolean {
return a.protocol === b.protocol && a.hostname === b.hostname && a.port === b.port
}
/**
* Classify a response `Content-Type` into a decodable body kind, or `undefined`
* for an unsupported (e.g. binary) type. `text/html` and `application/xhtml+xml`
* are `html`; other `text/*` plus a few structured text types are `text`.
*/
export function classifyContentType(contentType: string | null): FetchableKind | undefined {
const mime = (contentType ?? '').replace(/;.*$/s, '').trim().toLowerCase()
if (mime === 'text/html' || mime === 'application/xhtml+xml') return 'html'
if (mime.startsWith('text/')) return 'text'
if (mime === 'application/json' || mime === 'application/xml' || mime.endsWith('+json') || mime.endsWith('+xml')) return 'text'
return undefined
}

View File

@@ -0,0 +1,233 @@
/**
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public
* HTTP(S) URL with the platform-native `fetch` (Node 24) and returns a status
* code plus bounded decoded content. It owns SAFE RESOURCE RETRIEVAL — URL
* validation, redirect policy, timeout, abort, byte caps, charset decoding,
* content-type classification, binary rejection — but NOT presentation
* (HTML→markdown lives in `@deepseek-ai/dsh-tool-web`).
*
* Redirects are followed manually (`redirect: 'manual'`) so the provider can
* enforce a same-origin-only policy: a cross-origin redirect is refused with
* `WEB_REDIRECT_BLOCKED`, requiring a fresh tool call (Claude Code's WebFetch
* uses the same model). It does NOT carry browser cookies, editor/git
* credentials, or implicit access to private services.
*
* SSRF / private-network protection is DEFERRED (see the package RFC); until it
* lands this provider is an SSRF primitive and must not be enabled where it can
* reach sensitive internal targets.
*
* @module @deepseek-ai/dsh-web-fetch-local/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type { WebFetchBody, WebFetchProvider, WebFetchRequest, WebFetchResult, WebProviderStatus } from '@deepseek-ai/dsh-web'
import { classifyContentType, isSameOrigin, validateFetchUrl } from './policy.ts'
/** Resolved provider limits (the plugin's schemastery Config supplies defaults). */
export interface LocalFetchLimits {
/** Maximum accepted request URL length. */
maxUrlLength: number
/** Maximum response body size in bytes (read is aborted past this). */
maxResponseBytes: number
/** Maximum decoded body length in characters (truncated past this). */
maxBodyChars: number
/** Default fetch timeout in milliseconds. */
timeoutMs: number
/** Upper bound for a per-request timeout override. */
maxTimeoutMs: number
/** Maximum number of (same-origin) redirect hops to follow. */
maxRedirects: number
/** `User-Agent` header sent on every request. */
userAgent: string
}
/** Stable id this provider registers under. */
export const LOCAL_FETCH_PROVIDER_ID = 'local-http'
/** The anonymous public HTTP(S) fetch provider. */
export class LocalFetchProvider implements WebFetchProvider {
readonly id = LOCAL_FETCH_PROVIDER_ID
constructor(private readonly limits: LocalFetchLimits) {}
/** No credentials to check — an anonymous public fetcher is always usable. */
status(): WebProviderStatus {
return { available: true }
}
async fetch(request: WebFetchRequest, exec?: { readonly signal?: AbortSignal }): Promise<WebFetchResult> {
const timeoutMs = request.timeoutMs !== undefined
? Math.min(request.timeoutMs, this.limits.maxTimeoutMs)
: this.limits.timeoutMs
// One controller drives both the caller's abort and our own timeout, so the
// network request and the streaming read both stop on either.
const controller = new AbortController()
const onAbort = (): void => { controller.abort() }
if (exec?.signal !== undefined) {
if (exec.signal.aborted) throw new WebError('web fetch aborted', 'WEB_ABORTED')
exec.signal.addEventListener('abort', onAbort, { once: true })
}
const timer = setTimeout(() => { controller.abort(new WebError('web fetch timed out', 'WEB_FETCH_TIMEOUT')) }, timeoutMs)
try {
return await this.followAndRead(request.url, controller, timeoutMs)
} finally {
clearTimeout(timer)
if (exec?.signal !== undefined) exec.signal.removeEventListener('abort', onAbort)
}
}
/** Follow same-origin redirects up to the hop cap, then read the final response. */
private async followAndRead(initialUrl: string, controller: AbortController, timeoutMs: number): Promise<WebFetchResult> {
let currentUrl = validateFetchUrl(initialUrl, this.limits.maxUrlLength)
for (let hop = 0; hop <= this.limits.maxRedirects; hop++) {
const response = await this.requestOnce(currentUrl, controller, timeoutMs)
if (isRedirectStatus(response.status)) {
const location = response.headers.get('location')
if (location === null) {
// A redirect status with no Location is not a usable resource.
throw new WebError(`redirect response (HTTP ${response.status}) without a Location header`, 'WEB_PROVIDER_ERROR')
}
const target = resolveRedirect(location, currentUrl)
if (!isSameOrigin(target, currentUrl)) {
throw new WebError(
`cross-origin redirect to ${target.origin} is not followed automatically; retry against that URL directly`,
'WEB_REDIRECT_BLOCKED',
)
}
await response.body?.cancel()
currentUrl = target
continue
}
return await this.readBody(response, currentUrl)
}
throw new WebError(`exceeded the maximum of ${this.limits.maxRedirects} redirects`, 'WEB_REDIRECT_BLOCKED')
}
private async requestOnce(url: URL, controller: AbortController, _timeoutMs: number): Promise<Response> {
try {
return await fetch(url, {
method: 'GET',
redirect: 'manual',
headers: { 'user-agent': this.limits.userAgent, 'accept': 'text/html,application/xhtml+xml,text/*;q=0.9,application/json;q=0.8' },
signal: controller.signal,
})
} catch (error: unknown) {
throw translateAbortOrNetwork(error)
}
}
/** Read, byte-cap, classify, and decode the final response body. */
private async readBody(response: Response, finalUrl: URL): Promise<WebFetchResult> {
const kind = classifyContentType(response.headers.get('content-type'))
if (kind === undefined) {
await response.body?.cancel()
throw new WebError(`unsupported content type "${response.headers.get('content-type') ?? 'unknown'}"`, 'WEB_UNSUPPORTED_CONTENT_TYPE')
}
const { bytes, truncatedByBytes } = await this.readCapped(response)
const decoded = new TextDecoder('utf-8').decode(bytes)
const truncatedByChars = decoded.length > this.limits.maxBodyChars
const content = truncatedByChars ? decoded.slice(0, this.limits.maxBodyChars) : decoded
const body: WebFetchBody = kind === 'html' ? { kind: 'html', content } : { kind: 'text', content }
return {
providerId: this.id,
url: finalUrl.toString(),
statusCode: response.status,
body,
truncated: truncatedByBytes || truncatedByChars,
}
}
/**
* Read the response stream up to `maxResponseBytes`. A `Content-Length` over
* the cap rejects immediately with `WEB_FETCH_TOO_LARGE`; a stream that grows
* past the cap is cut short (`truncatedByBytes`) rather than rejected, so a
* server that under-reports still yields a bounded usable body.
*/
private async readCapped(response: Response): Promise<{ bytes: Uint8Array; truncatedByBytes: boolean }> {
const declared = response.headers.get('content-length')
if (declared !== null) {
const length = Number(declared)
if (Number.isFinite(length) && length > this.limits.maxResponseBytes) {
await response.body?.cancel()
throw new WebError(`response exceeds the maximum of ${this.limits.maxResponseBytes} bytes`, 'WEB_FETCH_TOO_LARGE')
}
}
/* v8 ignore next -- a 2xx Response from fetch always exposes a body stream; the null guard is defensive. */
if (response.body === null) return { bytes: new Uint8Array(0), truncatedByBytes: false }
const chunks: Uint8Array[] = []
let total = 0
let truncatedByBytes = false
const reader = response.body.getReader()
try {
for (;;) {
const { done, value } = await reader.read()
if (done) break
const remaining = this.limits.maxResponseBytes - total
if (value.byteLength >= remaining) {
chunks.push(value.subarray(0, remaining))
total += remaining
truncatedByBytes = true
break
}
chunks.push(value)
total += value.byteLength
}
} catch (error: unknown) {
/* v8 ignore next -- mid-stream read fault needs a network drop after headers; translate path covered by request-phase tests. */
throw translateAbortOrNetwork(error)
} finally {
/* v8 ignore next 4 -- cancel() after a completed/broken read settles without rejecting; unobserved best-effort cleanup. */
await reader.cancel().catch(() => {
// Cancel after a successful read (or after we broke past the cap) is
// best-effort cleanup; the bytes we need are already collected.
})
}
const bytes = new Uint8Array(total)
let offset = 0
for (const chunk of chunks) {
bytes.set(chunk, offset)
offset += chunk.byteLength
}
return { bytes, truncatedByBytes }
}
}
/** HTTP redirect status codes that carry a `Location`. */
function isRedirectStatus(status: number): boolean {
return status === 301 || status === 302 || status === 303 || status === 307 || status === 308
}
/** Resolve a (possibly relative) `Location` against the current URL. */
function resolveRedirect(location: string, base: URL): URL {
try {
return new URL(location, base)
} catch (error: unknown) {
/* v8 ignore next 2 -- URL resolution against a valid absolute base effectively never throws; defensive guard. */
throw new WebError(`invalid redirect Location "${location}"`, 'WEB_PROVIDER_ERROR', { cause: error })
}
}
/**
* Translate a thrown fetch/stream error into a `WebError`. Our own
* `WEB_FETCH_TIMEOUT` (passed to `controller.abort(reason)`) and any other
* already-typed `WebError` pass through; an `AbortError` becomes `WEB_ABORTED`;
* anything else is a transport/network failure (`WEB_PROVIDER_ERROR`).
*/
function translateAbortOrNetwork(error: unknown): WebError {
if (error instanceof WebError) return error
if (error instanceof DOMException && error.name === 'AbortError') {
return new WebError('web fetch aborted', 'WEB_ABORTED', { cause: error })
}
return new WebError(`web fetch failed: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
}

View File

@@ -0,0 +1,239 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { createServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http'
import { AddressInfo } from 'node:net'
import { Context } from 'cordis'
import WebService from '@deepseek-ai/dsh-web'
import { LocalFetchProvider, LOCAL_FETCH_PROVIDER_ID, classifyContentType, isSameOrigin, validateFetchUrl } from '@deepseek-ai/dsh-web-fetch-local'
import type { LocalFetchLimits } from '@deepseek-ai/dsh-web-fetch-local'
import * as fetchPlugin from '@deepseek-ai/dsh-web-fetch-local'
const limits: LocalFetchLimits = {
maxUrlLength: 2048,
maxResponseBytes: 5_000_000,
maxBodyChars: 100_000,
timeoutMs: 5_000,
maxTimeoutMs: 10_000,
maxRedirects: 5,
userAgent: 'test-agent/1.0',
}
type Handler = (req: IncomingMessage, res: ServerResponse) => void
let server: Server
let base: string
let handler: Handler
beforeEach(async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('default') }
server = createServer((req, res) => { handler(req, res) })
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
const { port } = server.address() as AddressInfo
base = `http://127.0.0.1:${port}`
})
afterEach(async () => {
await new Promise<void>(resolve => server.close(() => { resolve() }))
})
function provider(overrides: Partial<LocalFetchLimits> = {}): LocalFetchProvider {
return new LocalFetchProvider({ ...limits, ...overrides })
}
describe('policy helpers', () => {
it('validates scheme, credentials, and length', () => {
expect(validateFetchUrl('https://example.com/x', 2048).hostname).toBe('example.com')
expect(() => validateFetchUrl('ftp://example.com', 2048)).toThrow(expect.objectContaining({ code: 'WEB_INVALID_URL' }))
expect(() => validateFetchUrl('not a url', 2048)).toThrow(expect.objectContaining({ code: 'WEB_INVALID_URL' }))
expect(() => validateFetchUrl('https://user:pass@example.com', 2048)).toThrow(expect.objectContaining({ code: 'WEB_BLOCKED_URL' }))
expect(() => validateFetchUrl(`https://example.com/${'a'.repeat(3000)}`, 2048)).toThrow(expect.objectContaining({ code: 'WEB_INVALID_URL' }))
})
it('classifies content types', () => {
expect(classifyContentType('text/html; charset=utf-8')).toBe('html')
expect(classifyContentType('application/xhtml+xml')).toBe('html')
expect(classifyContentType('text/plain')).toBe('text')
expect(classifyContentType('application/json')).toBe('text')
expect(classifyContentType('image/png')).toBeUndefined()
expect(classifyContentType(null)).toBeUndefined()
})
it('compares origins', () => {
expect(isSameOrigin(new URL('https://a.com/x'), new URL('https://a.com/y'))).toBe(true)
expect(isSameOrigin(new URL('https://a.com'), new URL('https://b.com'))).toBe(false)
expect(isSameOrigin(new URL('http://a.com'), new URL('https://a.com'))).toBe(false)
})
})
describe('LocalFetchProvider success', () => {
it('fetches a text body', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('hello world') }
const result = await provider().fetch({ url: base })
expect(result.providerId).toBe(LOCAL_FETCH_PROVIDER_ID)
expect(result.statusCode).toBe(200)
expect(result.body).toEqual({ kind: 'text', content: 'hello world' })
expect(result.truncated).toBe(false)
})
it('fetches an html body and classifies it as html', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/html' }); res.end('<h1>hi</h1>') }
const result = await provider().fetch({ url: base })
expect(result.body).toEqual({ kind: 'html', content: '<h1>hi</h1>' })
})
it('sends the configured user agent', async () => {
let seen: string | undefined
handler = (req, res) => { seen = req.headers['user-agent']; res.writeHead(200, { 'content-type': 'text/plain' }); res.end('ok') }
await provider().fetch({ url: base })
expect(seen).toBe('test-agent/1.0')
})
it('returns a non-2xx response as a result, not an error', async () => {
handler = (_req, res) => { res.writeHead(404, { 'content-type': 'text/plain' }); res.end('nope') }
const result = await provider().fetch({ url: base })
expect(result.statusCode).toBe(404)
expect(result.body).toEqual({ kind: 'text', content: 'nope' })
})
})
describe('LocalFetchProvider caps', () => {
it('rejects an over-cap Content-Length with WEB_FETCH_TOO_LARGE', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain', 'content-length': '999999' }); res.end('x'.repeat(999999)) }
await expect(provider({ maxResponseBytes: 10 }).fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_FETCH_TOO_LARGE' }))
})
it('truncates a stream that grows past the byte cap', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('abcdefghij') }
const result = await provider({ maxResponseBytes: 4 }).fetch({ url: base })
expect(result.body.content).toBe('abcd')
expect(result.truncated).toBe(true)
})
it('truncates a decoded body past the character cap', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('abcdefghij') }
const result = await provider({ maxBodyChars: 3 }).fetch({ url: base })
expect(result.body.content).toBe('abc')
expect(result.truncated).toBe(true)
})
it('rejects an unsupported content type', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('binary') }
await expect(provider().fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_UNSUPPORTED_CONTENT_TYPE' }))
})
it('rejects a response with no content type at all', async () => {
handler = (_req, res) => { res.writeHead(200); res.end('no type') }
await expect(provider().fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_UNSUPPORTED_CONTENT_TYPE' }))
})
it('accepts a declared content-length within the cap', async () => {
handler = (_req, res) => { const body = 'sized'; res.writeHead(200, { 'content-type': 'text/plain', 'content-length': String(body.length) }); res.end(body) }
const result = await provider().fetch({ url: base })
expect(result.body.content).toBe('sized')
})
})
describe('LocalFetchProvider redirects', () => {
it('follows a same-origin redirect and reports the final URL', async () => {
handler = (req, res) => {
if (req.url === '/start') { res.writeHead(302, { location: '/end' }); res.end() }
else { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('arrived') }
}
const result = await provider().fetch({ url: `${base}/start` })
expect(result.body.content).toBe('arrived')
expect(result.url).toBe(`${base}/end`)
})
it('blocks a cross-origin redirect with WEB_REDIRECT_BLOCKED', async () => {
handler = (_req, res) => { res.writeHead(302, { location: 'https://example.com/' }); res.end() }
await expect(provider().fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_REDIRECT_BLOCKED' }))
})
it('rejects exceeding the redirect hop cap', async () => {
handler = (req, res) => {
const n = Number(new URL(req.url ?? '/', base).searchParams.get('n') ?? '0')
res.writeHead(302, { location: `/?n=${n + 1}` })
res.end()
}
await expect(provider({ maxRedirects: 2 }).fetch({ url: `${base}/?n=0` }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_REDIRECT_BLOCKED' }))
})
it('treats a redirect without a Location header as a provider error', async () => {
handler = (_req, res) => { res.writeHead(302); res.end() }
await expect(provider().fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('follows a relative same-origin redirect', async () => {
handler = (req, res) => {
if (req.url === '/a') { res.writeHead(301, { location: 'b' }); res.end() }
else { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('landed') }
}
const result = await provider().fetch({ url: `${base}/a` })
expect(result.body.content).toBe('landed')
})
})
describe('LocalFetchProvider invalid URLs and abort', () => {
it('rejects a non-http scheme before any network access', async () => {
await expect(provider().fetch({ url: 'ftp://example.com' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_INVALID_URL' }))
})
it('rejects credentials in the URL', async () => {
await expect(provider().fetch({ url: 'http://user:pass@127.0.0.1/' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_BLOCKED_URL' }))
})
it('honors a pre-aborted signal', async () => {
const controller = new AbortController()
controller.abort()
await expect(provider().fetch({ url: base }, { signal: controller.signal }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
it('aborts an in-flight fetch via the signal', async () => {
handler = (_req, _res) => { /* never responds */ }
const controller = new AbortController()
const promise = provider().fetch({ url: base }, { signal: controller.signal })
controller.abort()
await expect(promise).rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
it('times out a slow response with WEB_FETCH_TIMEOUT', async () => {
handler = (_req, _res) => { /* never responds */ }
await expect(provider({ timeoutMs: 50 }).fetch({ url: base }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_FETCH_TIMEOUT' }))
})
it('maps a connection failure to WEB_PROVIDER_ERROR', async () => {
// Port 1 on loopback is not listening: a real connection failure (not abort).
await expect(provider().fetch({ url: 'http://127.0.0.1:1/' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('caps the per-request timeout at maxTimeoutMs', async () => {
handler = (_req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('ok') }
const result = await provider({ maxTimeoutMs: 10_000 }).fetch({ url: base, timeoutMs: 999_999 })
expect(result.statusCode).toBe(200)
})
})
describe('web-fetch-local plugin registration', () => {
it('registers the provider into ctx.web (HMR-safe)', async () => {
const ctx = new Context()
await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID })
const fiber = await ctx.plugin(fetchPlugin, {})
expect(ctx.web.fetchStatus()).toEqual({ available: true, providerId: LOCAL_FETCH_PROVIDER_ID })
await fiber.dispose()
expect(ctx.web.fetchStatus()).toEqual({ available: false, reason: 'configured-missing' })
})
it('has no default export (namespace plugin export shape)', () => {
expect('default' in fetchPlugin).toBe(false)
})
})

View File

@@ -0,0 +1,24 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../../../vendor/schemastery"
},
{
"path": "../web"
}
]
}