feat: add searxng web-search provider, openrouter cost balance UI, offline scripts; update source-launch and docs
Some checks failed
CI / node 22.19 (push) Has been skipped
CI / node 26 (push) Has been skipped
CI / python 3.10 / keyless SDK (push) Has been skipped
CI / python runtime / release-shaped Linux x64 (push) Has been skipped
CI / windows node 24 / wine blocking (push) Has been skipped
CI / wine apt cache (push) Successful in 58s
CI / serial / linux (push) Has been skipped
Deploy documentation / build (push) Failing after 2m46s
Deploy documentation / deploy (push) Has been skipped
E2E (real DeepSeek API) / e2e (push) Failing after 1m18s
Sandbox / sandbox e2e (bwrap, ubuntu-latest) (push) Failing after 1m18s
Landlock Run / Matrix (push) Successful in 13s
Release (vendor) / Pack npm tarballs (push) Failing after 3m43s
Release (dsh) / Pack npm tarballs (push) Failing after 1m53s
Sandbox / sandbox e2e (landlock, ubuntu-24.04) (push) Failing after 1m51s
Release (vendor) / Publish to npm (push) Has been skipped
Release (dsh) / Publish to npm (push) Has been skipped
CI / node 24 / static (push) Has been cancelled
CI / node 24 / coverage (push) Has been cancelled
CI / node 24 / snapshots and artifacts (push) Has been cancelled
CI / windows node 24 / native complete (push) Has been cancelled
CI / serial / linux (self-hosted standby) (push) Has been cancelled
CI / serial / macos (push) Has been cancelled
CI / serial / windows (self-hosted standby) (push) Has been cancelled
CI / larger-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (16, windows, dsh-windows-2025-16core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (32, windows, dsh-windows-2025-32core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (4, windows, dsh-windows-2025-4core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (64, windows, dsh-windows-2025-64core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (8, windows, dsh-windows-2025-8core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (96, windows, dsh-windows-2025-96core, production-site) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, 16) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, windows, dsh-windows-2025-16core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, windows, dsh-windows-2025-32core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, 4) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, windows, dsh-windows-2025-4core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, windows, dsh-windows-2025-64core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, 8) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, windows, dsh-windows-2025-8core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, windows, dsh-windows-2025-96core, 2) (push) Has been cancelled
CI / all checks passed (push) Has been cancelled
Sandbox / sandbox e2e (seatbelt, macos-latest) (push) Has been cancelled
Sandbox / sandbox e2e (landlock, ubuntu-24.04-arm) (push) Has been cancelled
Landlock Run / ${{ matrix.platform }} (push) Has been cancelled
Landlock Run / darwin (no platform package — degradation proof) (push) Has been cancelled

This commit is contained in:
2026-08-20 13:01:40 +07:00
parent 99f6f02fec
commit ed152416d5
111 changed files with 5038 additions and 43 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/web/README.md
README.md: fc37d7cdead59138db149b5a86f0a0c031d40037
README.zh.md: 53fe673ddaed235bbb938db4757e9cd240928d61
README.md: 449c96c71b580964486968431c28f71dc466e8eb
README.zh.md: a7d7d6dc1a382abe481f3520aa42480ff0317976

View File

@@ -8,6 +8,7 @@ This family provides provider-neutral web search and fetch operations plus the m
|---|---|---|
| [`web/`](web/README.md) | Defines web provider registration, selection, and shared errors | `ctx.web` |
| [`web-search-exa/`](web-search-exa/README.md) | Provides web search through Exa | registers on `ctx.web` |
| [`web-search-searxng/`](web-search-searxng/README.md) | Provides web search through a SearXNG instance | registers on `ctx.web` |
| [`web-search-perplexity/`](web-search-perplexity/README.md) | Provides web search through Perplexity | registers on `ctx.web` |
| [`web-search-deepseek/`](web-search-deepseek/README.md) | Provides native DeepSeek web search | registers on `ctx.web` |
| [`web-fetch-http/`](web-fetch-http/README.md) | Fetches public HTTP and HTTPS resources | registers on `ctx.web` |

View File

@@ -8,6 +8,7 @@
|---|---|---|
| [`web/`](web/README.md) | 定义 web 提供方注册、选择和共享错误 | `ctx.web` |
| [`web-search-exa/`](web-search-exa/README.md) | 通过 Exa 提供 web 搜索 | 注册到 `ctx.web` |
| [`web-search-searxng/`](web-search-searxng/README.md) | 通过 SearXNG 实例提供 web 搜索 | 注册到 `ctx.web` |
| [`web-search-perplexity/`](web-search-perplexity/README.md) | 通过 Perplexity 提供 web 搜索 | 注册到 `ctx.web` |
| [`web-search-deepseek/`](web-search-deepseek/README.md) | 提供 DeepSeek 原生 web 搜索 | 注册到 `ctx.web` |
| [`web-fetch-http/`](web-fetch-http/README.md) | 抓取公共 HTTP 和 HTTPS 资源 | 注册到 `ctx.web` |

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/web/web-search-searxng/README.md
README.md: 0e0c76fc884eb5968618673cc58f6ae5f88112cc
README.zh.md: aa550a67c49458c9488c4b3bf24532935db99e01

View File

@@ -0,0 +1,52 @@
# @deepseek-ai/dsh-web-search-searxng
English | [中文](README.zh.md)
A [SearXNG](https://docs.searxng.org)-backed `WebSearchProvider` for the harness [web capability seam](../web/README.md) (`ctx.web`). It calls a SearXNG instance's JSON API (`GET /search?format=json`) and maps the flat `results[]` into the seam's normalized `WebSearchResult`. It targets a self-hosted or private SearXNG: there is no single canonical public instance, so `baseURL` has no default, and the provider deliberately carries no API key or `Authorization` header.
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the `ctx.web` key and it does not register a model-facing tool (that is `@deepseek-ai/dsh-tool-web`). Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`) that registers its backend, not a default-export service.
## Config
| Key | Default | Meaning |
|---|---|---|
| `baseURL` | (none) | SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. |
| `language` | `auto` | SearXNG `language` request value; `auto` lets the instance decide per user preferences. |
| `timeRange` | (unset) | SearXNG `time_range` recency filter: `day`, `week`, `month`, or `year`. Omitted sends no filter. |
```yaml
- id: web-search-searxng
name: '@deepseek-ai/dsh-web-search-searxng'
config:
baseURL: https://searx.example
```
The entry above is the base layer of the `web-search-searxng` Settings section: a user layer over it (a `settings.yaml` section or an in-session settings edit) reaches the NEXT search, because the provider projects the section per call rather than capturing it at registration. The seam's provider selection therefore never flickers when the instance or a filter changes. A section without `baseURL` still passes the schema but leaves the provider unavailable — no endpoint is guessed.
```yaml
# $DSH_HOME/settings.yaml
web-search-searxng:
baseURL: https://searx.example
language: auto
timeRange: day
```
SearXNG exposes no per-request result-count control — page size is instance configuration — so no `numResults` option exists; the seam enforces `maxResults` on the result.
## Mapping
SearXNG returns a flat `results[]` and no generated answer, so `content` is omitted. Each result maps to a `WebSearchSource`: `url` ← `url`, `title` ← `title`, `snippet` ← `content`, `publishedAt` ← `publishedDate`. Unlike the Exa provider it does **not** drop snippet-less entries: URL and title are still useful, so every result is kept. Provider failures (HTTP errors, network failure, unparseable or wrong-shape bodies) surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`. HTTP redirects are rejected before the `Location` target is contacted and surface as `WEB_PROVIDER_ERROR`. The request carries no `Authorization` header, so no credential can leak to a redirect target.
## Model Experience
Indirectly, through [`dsh-tool-web`](../tool-web/README.md), which retains this provider's `maxResults`-bounded URLs, titles, snippets, and publication dates or its exact `SearXNG search aborted`, `SearXNG search request failed: <error>`, and `SearXNG returned an unprocessable response body: <error>` failures under the consumer's error wrapper while generated answers and provider-private fields remain outside context.
#### KV Cache effect
No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **Result count is not bounded at the request** — SearXNG page size is instance configuration, so the provider sends no count and the seam truncates on return; a single page (typically ~20 results) is fetched.
- **No aggregate-answer or infobox content is surfaced** — SearXNG `answers` and `infoboxes` are not mapped into `content`. No generated answer is trusted.
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.

View File

@@ -0,0 +1,52 @@
# @deepseek-ai/dsh-web-search-searxng
[English](README.md) | 中文
由 [SearXNG](https://docs.searxng.org) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它调用 SearXNG 实例的 JSON API(`GET /search?format=json`),把扁平 `results[]` 映射为 seam 规范化的 `WebSearchResult`。此提供方面向自托管或私有 SearXNG:不存在统一的公共实例,因此 `baseURL` 没有默认值,提供方刻意不携带 API 密钥或 `Authorization` 头。
这是一个**实现**包:它向 `ctx.web` 注册提供方,不拥有 `ctx.web` 键,也不注册面向模型的工具(后者属于 `@deepseek-ai/dsh-tool-web`)。与 `@deepseek-ai/dsh-llm-deepseek` 一样,它是函数/命名空间插件(`inject: ['web']`),负责注册后端,而非默认导出服务。
## 配置
| 配置键 | 默认值 | 含义 |
|---|---|---|
| `baseURL` | (无) | SearXNG 实例基址;追加 `/search`。为空或无法解析时提供方不可用。 |
| `language` | `auto` | SearXNG `language` 请求值;`auto` 让实例按用户偏好决定。 |
| `timeRange` | (未设置) | SearXNG `time_range` 时效过滤器:`day`、`week`、`month` 或 `year`。省略时不发送过滤器。 |
```yaml
- id: web-search-searxng
name: '@deepseek-ai/dsh-web-search-searxng'
config:
baseURL: https://searx.example
```
上面的条目是 `web-search-searxng` Settings 分节的基础层:覆盖在它之上的用户层(`settings.yaml` 分节或会话内设置的编辑)会作用于**下一次**搜索,因为提供方按调用投影分节,而非在注册时捕获。因此 seam 的提供方选择在实例或过滤器变化时不会闪烁。不含 `baseURL` 的分节仍能通过 schema,但提供方保持不可用——不会猜测任何端点。
```yaml
# $DSH_HOME/settings.yaml
web-search-searxng:
baseURL: https://searx.example
language: auto
timeRange: day
```
SearXNG 不提供按请求控制结果数量的方式——页面大小属于实例配置——因此没有 `numResults` 选项;seam 会在结果返回时强制执行 `maxResults`。
## 映射
SearXNG 返回扁平 `results[]`,不返回生成答案,因此省略 `content`。每项结果映射为 `WebSearchSource`:`url` ← `url`、`title` ← `title`、`snippet` ← `content`、`publishedAt` ← `publishedDate`。与 Exa 提供方不同,此提供方**不会**丢弃无 snippet 的结果:URL 与标题仍然有用,因此保留每项结果。提供方失败(HTTP 错误、网络失败、响应体无法解析或结构不符)以 `WebError` `WEB_PROVIDER_ERROR` 呈现;中止请求以 `WEB_ABORTED` 呈现。HTTP 重定向会在访问 `Location` 指向的目标之前被拒绝,并以 `WEB_PROVIDER_ERROR` 呈现。请求不携带 `Authorization` 头,因此不会有凭据泄露给重定向目标。
## 模型体验
通过 [`dsh-tool-web`](../tool-web/README.md) 间接影响;该工具保留此提供方经 `maxResults` 限制的 URL、标题、snippet 与发布日期,或将确切的错误消息 `SearXNG search aborted`、`SearXNG search request failed: <error>` 和 `SearXNG returned an unprocessable response body: <error>` 置于消费方的错误包装层内;生成答案与提供方私有字段不进入上下文。
#### KV Cache 影响
不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。
## 已知限制与暂缓事项
- **请求端不限制结果数量**:SearXNG 页面大小属于实例配置,因此提供方不发送数量,由 seam 在返回时截断;只抓取单页(通常约 20 条结果)。
- **不呈现聚合答案或 infobox 内容**:SearXNG 的 `answers` 与 `infoboxes` 不映射进 `content`。不信任任何生成答案。
- **按错误形状分类中止**:只有 `DOMException` 且名为 `AbortError` 时才映射为 `WEB_ABORTED`;携带自定义原因的中止(例如 `dsh-timeout` 的 `TimeoutReason`)会呈现为 `WEB_PROVIDER_ERROR`。

View File

@@ -0,0 +1,49 @@
{
"name": "@deepseek-ai/dsh-web-search-searxng",
"description": "SearXNG-backed search provider for the DeepSeek Harness web capability seam (ctx.web)",
"version": "0.1.0-rc.7",
"publishConfig": {
"access": "public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/deepseek-ai/deepseek-harness.git",
"directory": "packages/web/web-search-searxng"
},
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./invariant": {
"types": "./lib/types/invariant.d.ts",
"default": "./lib/invariant.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/invariant.js",
"lib/types/**/*.d.ts"
],
"license": "MIT",
"peerDependencies": {
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-settings": "workspace:^",
"@deepseek-ai/dsh-web": "workspace:^",
"@deepseek-ai/cordis": "workspace:^"
},
"dependencies": {
"@deepseek-ai/schemastery": "workspace:^"
},
"devDependencies": {
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-settings": "workspace:^",
"@deepseek-ai/dsh-web": "workspace:^",
"@deepseek-ai/cordis": "workspace:^"
}
}

View File

@@ -0,0 +1,79 @@
/**
* `@deepseek-ai/dsh-web-search-searxng`: registers a SearXNG-backed
* `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a
* default-export service): a search provider does not own the `ctx.web` key —
* it registers INTO the seam's provider registry, exactly as
* `@deepseek-ai/dsh-web-search-exa` does. The key is owned by
* `@deepseek-ai/dsh-web`.
*
* @module @deepseek-ai/dsh-web-search-searxng
*/
import type { Context } from '@deepseek-ai/cordis'
import z from '@deepseek-ai/schemastery'
import { installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import type {} from '@deepseek-ai/dsh-web'
import { SearXngSearchProvider } from './provider.ts'
export {
SEARXNG_DEFAULT_LANGUAGE,
SEARXNG_NO_TIME_RANGE,
SEARXNG_PROVIDER_ID,
SEARXNG_TIME_RANGES,
SearXngSearchProvider,
} from './provider.ts'
export type {
SearXngSearchProviderOptions,
SearXngSearchProviderSource,
SearXngTimeRange,
} from './provider.ts'
/** Cordis plugin name used by loader diagnostics. */
export const name = 'web-search-searxng'
/** The web seam this provider registers into. */
export const inject = ['web']
/** Plugin config. `baseURL` is optional in the schema so an omitted value
* surfaces as an unavailable provider (the seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`)
* rather than failing boot; `apply` fills constant defaults. */
export interface Config {
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
baseURL?: string
/** SearXNG `language` to request. Defaults to `auto`. */
language?: string
/** SearXNG `time_range` recency filter. Omitted = no filter. */
timeRange?: 'day' | 'week' | 'month' | 'year'
}
export const Config: z<Config> = z.object({
baseURL: z.string(),
language: z.string(),
timeRange: z.union(['day', 'week', 'month', 'year'] as const),
})
/** Settings namespace carrying the SearXNG instance and any per-search filters. */
export const SEARXNG_SETTINGS_NAMESPACE = settingsNamespace('web-search-searxng')
/** Register the SearXNG search provider with `ctx.web`, reading the live section
* per search so an in-session settings edit applies without re-registration. */
export function apply(ctx: Context, config: Config): void {
let current: () => Config = () => config
installSettingsSection(ctx, SEARXNG_SETTINGS_NAMESPACE, Config, config, {
setSource: (source) => {
current = source
},
// The registration carries no resolved value: the provider projects the
// section per search, so a committed change needs no re-registration.
onChange: () => {},
})
ctx.web.registerSearchProvider(new SearXngSearchProvider(() => ({
// SearXNG exposes no uniform public endpoint and the product requires
// evidence over an unsupported default, so `baseURL` has no constant
// fallback: an omitted value surfaces as an unavailable provider (the
// seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`), not a silent endpoint.
baseURL: current().baseURL ?? '',
...current().language !== undefined ? { language: current().language } : {},
...current().timeRange !== undefined ? { timeRange: current().timeRange } : {},
})))
}

View File

@@ -0,0 +1,30 @@
/**
* Package-owned invariant companion for `@deepseek-ai/dsh-web-search-searxng`.
* @module @deepseek-ai/dsh-web-search-searxng/invariant
*/
/* jscpd:ignore-start */
import type { Context } from '@deepseek-ai/cordis'
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
const PACKAGE_NAME = '@deepseek-ai/dsh-web-search-searxng'
/** Cordis companion plugin name. */
export const name = 'web-search-searxng-invariant'
/** Service required before the companion can reserve package ownership. */
export const inject = ['invariants']
/**
* No runtime invariant: this package exposes no independent event sequence or mutable data relation
* beyond contracts enforced at its owning seam.
*/
const install: InvariantInstaller = () => {}
/**
* Register this package's invariant companion.
* @param ctx - Cordis context carrying the invariant service.
* @returns the installed registration's disposer after setup succeeds.
*/
export const apply = (ctx: Context): Promise<() => void> =>
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
/* jscpd:ignore-end */

View File

@@ -0,0 +1,166 @@
/**
* `SearXngSearchProvider`: a `WebSearchProvider` backed by a SearXNG instance's
* JSON API (`GET /search?format=json`). It maps against the flat `results[]`,
* keeps entries even without a snippet (URL and title remain useful), and
* omits `content` because SearXNG returns no generated answer. The provider
* carries no credentials, so a request carries no `Authorization` header.
* @module @deepseek-ai/dsh-web-search-searxng/provider
*/
import { WebError } from '@deepseek-ai/dsh-web'
import type {
WebSearchProvider,
WebSearchRequest,
WebSearchResult,
WebSearchSource,
} from '@deepseek-ai/dsh-web'
import type { SearXngError, SearXngResult, SearXngSearchResponse } from './types.ts'
/** Stable id this provider registers under. */
export const SEARXNG_PROVIDER_ID = 'searxng'
/** Default language: let SearXNG decide per user preferences. */
export const SEARXNG_DEFAULT_LANGUAGE = 'auto'
/** `time_range` sentinel meaning "no recency filter". */
export const SEARXNG_NO_TIME_RANGE = 'none'
/** Valid `time_range` values SearXNG accepts. */
export const SEARXNG_TIME_RANGES = ['day', 'week', 'month', 'year'] as const
/** One of SearXNG's `time_range` values, passed through to the endpoint. */
export type SearXngTimeRange = (typeof SEARXNG_TIME_RANGES)[number]
/** Attribution header sent on every request. Bump with the package version. */
const USER_AGENT = 'deepseek-harness/0.0.1'
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
export interface SearXngSearchProviderOptions {
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
baseURL: string
/** SearXNG `language` to request. `auto` lets the instance decide. */
language?: string
/** SearXNG `time_range` recency filter. */
timeRange?: SearXngTimeRange
}
/** A snapshot or a per-search resolver (settings live-reload hands the latter). */
export type SearXngSearchProviderSource = SearXngSearchProviderOptions | (() => SearXngSearchProviderOptions)
/**
* Map one SearXNG result to a normalized source. The URL is always carried; a
* blank title or snippet is omitted rather than emitted as empty.
*
* @param result - one entry of SearXNG's `results[]`.
* @returns the normalized source.
*/
export function mapSearXngResult(result: SearXngResult): WebSearchSource {
return {
url: result.url,
...result.title != null && result.title.trim().length > 0 ? { title: result.title } : {},
...result.content != null && result.content.trim().length > 0 ? { snippet: result.content } : {},
...result.publishedDate != null && result.publishedDate.length > 0 ? { publishedAt: result.publishedDate } : {},
}
}
/**
* Map a SearXNG response envelope to a normalized search result.
*
* @param response - the parsed `format=json` response body.
* @returns the normalized result; `content` is omitted (no generated answer).
*/
export function mapSearXngResponse(response: SearXngSearchResponse): WebSearchResult {
// SearXNG cannot bound per-request result count (page size is instance
// configuration), so the request carries no count and the web service owns
// the final `maxResults` truncation; this provider reports `truncated: false`.
return { sources: (response.results ?? []).map(mapSearXngResult), truncated: false }
}
/** The SearXNG-backed search provider; HTTP redirects fail as `WEB_PROVIDER_ERROR`. */
export class SearXngSearchProvider implements WebSearchProvider {
readonly id = SEARXNG_PROVIDER_ID
constructor(source: SearXngSearchProviderSource) {
this.source = source
}
private readonly source: SearXngSearchProviderSource
/** Read the current options: a snapshot stays fixed, a resolver reads live settings. */
private resolveOptions(): SearXngSearchProviderOptions {
return typeof this.source === 'function' ? this.source() : this.source
}
available(): boolean {
const options = this.resolveOptions()
return isValidBaseUrl(options.baseURL)
&& (options.language === undefined || options.language.length > 0)
&& (options.timeRange === undefined || SEARXNG_TIME_RANGES.includes(options.timeRange))
}
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
const options = this.resolveOptions()
const base = trimTrailingSlashes(options.baseURL)
const params = new URLSearchParams({ q: request.query, format: 'json' })
if (options.language !== undefined) params.set('language', options.language)
if (options.timeRange !== undefined) params.set('time_range', options.timeRange)
let response: Response
try {
response = await fetch(`${base}/search?${params}`, {
method: 'GET',
redirect: 'error',
headers: {
'accept': 'application/json',
'user-agent': USER_AGENT,
},
...signal !== undefined ? { signal } : {},
})
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
throw new WebError(`SearXNG search request failed: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
}
if (!response.ok) {
const status = response.status
let message = `SearXNG API error (HTTP ${status})`
try {
const parsed = await response.json() as SearXngError
const detail = parsed.error ?? parsed.message ?? parsed.content
if (detail !== undefined && detail.length > 0) message = detail
} catch (error: unknown) {
// An abort fired mid-body must surface as WEB_ABORTED, not be swallowed
// into a generic HTTP-error message — cancellation is not a provider
// error (the seam's cancellation contract).
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
// Otherwise: the HTTP status is already captured in `message` above; a
// malformed/non-JSON error body (normal for gateway 5xx/429s) can only
// cost a richer provider message, never the real error.
}
throw new WebError(message, 'WEB_PROVIDER_ERROR')
}
try {
const payload = await response.json() as SearXngSearchResponse
return mapSearXngResponse(payload)
} catch (error: unknown) {
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
throw new WebError(`SearXNG returned an unprocessable response body: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
}
}
}
/** True when `baseURL` parses as an absolute URL (a cheap local config check). */
function isValidBaseUrl(baseURL: string): boolean {
return URL.canParse(baseURL)
}
/** Strip a trailing slash so `${base}/search` never doubles a separator. */
function trimTrailingSlashes(baseURL: string): string {
return baseURL.replace(/\/+$/, '')
}
/** True for a fetch/`AbortSignal` abort, surfaced as `WEB_ABORTED`. */
function isAbortError(error: unknown): boolean {
return error instanceof DOMException && error.name === 'AbortError'
}

View File

@@ -0,0 +1,30 @@
/**
* Wire types for the SearXNG JSON API (`GET {base}/search?q=...&format=json`).
* Types only — no runtime code. SearXNG returns a flat `results[]`; each entry
* carries a URL, an optional title, an optional `content` snippet, and an
* optional `publishedDate`.
*
* @module @deepseek-ai/dsh-web-search-searxng/types
*/
/** One entry of SearXNG's flat `results[]`. */
export interface SearXngResult {
url: string
title?: string | null
/** The page snippet SearXNG derives for the result. */
content?: string | null
publishedDate?: string | null
}
/** SearXNG's JSON search response envelope. */
export interface SearXngSearchResponse {
results?: SearXngResult[]
}
/** SearXNG's error response envelope (best-effort; fields vary by failure). */
export interface SearXngError {
error?: string
message?: string
/** Some SearXNG error responses carry the message under `content`. */
content?: string
}

View File

@@ -0,0 +1,24 @@
import { describe, expect, it } from 'vitest'
import { SearXngSearchProvider } from '@deepseek-ai/dsh-web-search-searxng'
/**
* Real-API smoke for the SearXNG search provider against a live instance.
* Self-skips without `$SEARXNG_BASE_URL` (CI has no endpoint), per the
* with-key e2e policy in docs/testing.md.
*/
const baseURL = process.env.SEARXNG_BASE_URL
const maybe = baseURL !== undefined && baseURL.length > 0 ? describe : describe.skip
maybe('SearXngSearchProvider real API', () => {
it('returns sources for a live query', async () => {
const provider = new SearXngSearchProvider({
baseURL: baseURL!,
...process.env.SEARXNG_LANGUAGE !== undefined && process.env.SEARXNG_LANGUAGE.length > 0
? { language: process.env.SEARXNG_LANGUAGE }
: {},
})
const result = await provider.search({ query: 'DeepSeek Harness', maxResults: 5 })
expect(result.sources.length).toBeGreaterThan(0)
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
}, 30_000)
})

View File

@@ -0,0 +1,226 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from '@deepseek-ai/cordis'
import WebRuntime from '@deepseek-ai/dsh-web'
import { SearXngSearchProvider, SEARXNG_PROVIDER_ID } from '@deepseek-ai/dsh-web-search-searxng'
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
import { mapSearXngResponse, mapSearXngResult } from '../src/provider.ts'
const options = { baseURL: 'https://searx.test' }
function jsonResponse(body: unknown, init: ResponseInit = {}): Response {
return new Response(JSON.stringify(body), { status: 200, headers: { 'content-type': 'application/json' }, ...init })
}
afterEach(() => {
vi.unstubAllGlobals()
})
describe('SearXng result mapping', () => {
it('maps a full result entry', () => {
expect(mapSearXngResult({
url: 'https://a.test',
title: 'A',
content: 'an excerpt',
publishedDate: '2026-01-01',
})).toEqual({ url: 'https://a.test', title: 'A', snippet: 'an excerpt', publishedAt: '2026-01-01' })
})
it('keeps a URL-only result rather than dropping it', () => {
expect(mapSearXngResult({ url: 'https://a.test' })).toEqual({ url: 'https://a.test' })
expect(mapSearXngResult({ url: 'https://a.test', content: ' ' })).toEqual({ url: 'https://a.test' })
})
it('omits null/empty optional fields rather than emitting them', () => {
expect(mapSearXngResult({ url: 'https://a.test', title: null, content: null, publishedDate: null }))
.toEqual({ url: 'https://a.test' })
expect(mapSearXngResult({ url: 'https://a.test', title: '', content: '', publishedDate: '' }))
.toEqual({ url: 'https://a.test' })
})
it('maps a response to a result with no content and all sources kept', () => {
const result = mapSearXngResponse({
results: [
{ url: 'https://a.test', title: 'A', content: 'one' },
{ url: 'https://b.test' },
{ url: 'https://c.test', content: 'three' },
],
})
expect(result).toEqual({
sources: [
{ url: 'https://a.test', title: 'A', snippet: 'one' },
{ url: 'https://b.test' },
{ url: 'https://c.test', snippet: 'three' },
],
truncated: false,
})
expect(result.content).toBeUndefined()
})
it('tolerates a missing results array', () => {
expect(mapSearXngResponse({}).sources).toEqual([])
})
})
describe('SearXngSearchProvider availability', () => {
it('is unavailable without a base URL', () => {
expect(new SearXngSearchProvider({ baseURL: '' }).available()).toBe(false)
})
it('is available with a base URL', () => {
expect(new SearXngSearchProvider(options).available()).toBe(true)
})
it('is misconfigured when the base URL is unparseable', () => {
expect(new SearXngSearchProvider({ baseURL: 'not a url' }).available()).toBe(false)
})
it('is misconfigured when language is empty or timeRange is invalid', () => {
expect(new SearXngSearchProvider({ ...options, language: '' }).available()).toBe(false)
expect(new SearXngSearchProvider({ ...options, timeRange: 'decade' as never }).available()).toBe(false)
expect(new SearXngSearchProvider({ ...options, timeRange: 'week' }).available()).toBe(true)
})
})
describe('SearXngSearchProvider request mapping', () => {
it('issues a GET with query and json format and no authorization header', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [{ url: 'https://a.test' }] }))
vi.stubGlobal('fetch', fetchMock)
await new SearXngSearchProvider(options).search({ query: 'hello world' })
expect(fetchMock).toHaveBeenCalledOnce()
const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
expect(url).toBe('https://searx.test/search?q=hello+world&format=json')
expect(init.method).toBe('GET')
expect(init.redirect).toBe('error')
expect((init.headers as Record<string, string>)['authorization']).toBeUndefined()
})
it('sends language and time_range when configured', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
await new SearXngSearchProvider({ ...options, language: 'en', timeRange: 'week' }).search({ query: 'q' })
const [url] = fetchMock.mock.calls[0] as unknown as [string]
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
})
it('omits language and time_range when unset', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
await new SearXngSearchProvider(options).search({ query: 'q' })
const [url] = fetchMock.mock.calls[0] as unknown as [string]
expect(url).toBe('https://searx.test/search?q=q&format=json')
})
it('does not double the separator when baseURL carries a trailing slash', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
await new SearXngSearchProvider({ baseURL: 'https://searx.test/' }).search({ query: 'q' })
const [url] = fetchMock.mock.calls[0] as unknown as [string]
expect(url).toBe('https://searx.test/search?q=q&format=json')
})
it('forwards the abort signal', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
const controller = new AbortController()
await new SearXngSearchProvider(options).search({ query: 'q' }, controller.signal)
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
expect(init.signal).toBe(controller.signal)
})
})
describe('SearXngSearchProvider error handling', () => {
it('maps an HTTP error to WEB_PROVIDER_ERROR with the provider message', async () => {
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ error: 'missing instance' }, { status: 401 })))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'missing instance' }))
})
it('keeps a status-line message when the error body is not JSON', async () => {
vi.stubGlobal('fetch', vi.fn(async () => new Response('gateway down', { status: 502 })))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'SearXNG API error (HTTP 502)' }))
})
it('reads the message from content when present', async () => {
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ content: 'json disabled' }, { status: 403 })))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ message: 'json disabled' }))
})
it('maps a network failure to WEB_PROVIDER_ERROR', async () => {
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new TypeError('connection refused'))))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('maps an abort to WEB_ABORTED', async () => {
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new DOMException('aborted', 'AbortError'))))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
it('maps an unparseable success body to WEB_PROVIDER_ERROR', async () => {
vi.stubGlobal('fetch', vi.fn(async () => new Response('not json', { status: 200 })))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('maps a well-formed body of the wrong shape to WEB_PROVIDER_ERROR, not a raw TypeError', async () => {
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: {} }, { status: 200 })))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
})
it('surfaces an abort during success-body parse as WEB_ABORTED, not provider error', async () => {
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: true, status: 200 }
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
it('surfaces an abort during error-body parse as WEB_ABORTED', async () => {
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: false, status: 500 }
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
})
})
describe('web-search-searxng plugin registration', () => {
it('registers the provider into ctx.web (HMR-safe)', async () => {
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: [] })))
const ctx = new Context()
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL })
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ sources: [], truncated: false })
await fiber.dispose()
await expect(ctx.web.search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))
})
it('has no default export (namespace plugin export shape)', () => {
expect('default' in searxngPlugin).toBe(false)
})
it('threads language and timeRange config into the request', async () => {
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
vi.stubGlobal('fetch', fetchMock)
const ctx = new Context()
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL, language: 'en', timeRange: 'week' })
await ctx.web.search({ query: 'q' })
const [url] = fetchMock.mock.calls[0] as unknown as [string]
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
await fiber.dispose()
})
it('is unavailable when baseURL is omitted', async () => {
const ctx = new Context()
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
await ctx.plugin(searxngPlugin, { baseURL: '' })
await expect(ctx.web.search({ query: 'q' }))
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' }))
})
})

View File

@@ -0,0 +1,120 @@
/** The `web-search-searxng` settings section layered over the composition entry. */
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from '@deepseek-ai/cordis'
import type { Fiber } from '@deepseek-ai/cordis'
import { SettingsProvider } from '@deepseek-ai/dsh-settings'
import type { SettingsNamespace } from '@deepseek-ai/dsh-settings'
import WebRuntime from '@deepseek-ai/dsh-web'
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
import { SEARXNG_SETTINGS_NAMESPACE } from '@deepseek-ai/dsh-web-search-searxng'
/** The smallest real provider: one in-memory document, always writable. */
class MemorySettings extends SettingsProvider {
doc: Record<string, unknown> = {}
get writable(): boolean {
return true
}
protected load(): Promise<Record<string, unknown>> {
return Promise.resolve(structuredClone(this.doc))
}
protected persist(ns: SettingsNamespace, section: Record<string, unknown>): Promise<void> {
this.doc = { ...this.doc, [ns]: structuredClone(section) }
return Promise.resolve()
}
}
function jsonResponse(body: unknown): Response {
return new Response(JSON.stringify(body), {
status: 200,
headers: { 'content-type': 'application/json' },
})
}
/** The smallest SearXNG-shaped answer the provider accepts — enough to observe the request. */
const ONE_RESULT = {
results: [{ url: 'https://a.test', title: 'A', content: 'snip' }],
}
async function boot(): Promise<{ ctx: Context; settingsFiber: Fiber; pluginFiber: Fiber }> {
const ctx = new Context()
await ctx.plugin(WebRuntime, {})
const settingsFiber = ctx.plugin(MemorySettings)
await settingsFiber.await()
const pluginFiber = ctx.plugin(searxngPlugin, { baseURL: 'https://search.entry.test' })
await pluginFiber.await()
return { ctx, settingsFiber, pluginFiber }
}
afterEach(() => {
vi.restoreAllMocks()
})
/**
* Run one search and answer the endpoint it reached. A fresh `Response` per
* call because a body can only be read once, and the call history is cleared
* because repeated `spyOn` returns the same spy.
* @param ctx - context whose `ctx.web` serves the search.
* @returns the URL the provider fetched.
*/
async function searchOnce(ctx: Context): Promise<string> {
const fetchSpy = vi.spyOn(globalThis, 'fetch')
.mockImplementation(() => Promise.resolve(jsonResponse(ONE_RESULT)))
fetchSpy.mockClear()
await ctx.web.search({ query: 'anything' })
return String((fetchSpy.mock.calls.at(-1)?.[0] as URL | string | undefined) ?? '')
}
describe('web-search-searxng settings section', () => {
it('serves a stored endpoint to the next search without re-registering the provider', async () => {
const bench = await boot()
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
baseURL: 'https://search.stored.test',
})
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
await bench.ctx.fiber.dispose()
})
it('applies a stored language and time range to the next request', async () => {
const bench = await boot()
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
baseURL: 'https://search.entry.test',
language: 'de',
timeRange: 'week',
})
const url = new URL(await searchOnce(bench.ctx))
expect(url.searchParams.get('language')).toBe('de')
expect(url.searchParams.get('time_range')).toBe('week')
await bench.ctx.fiber.dispose()
})
it('falls back to the composition entry when the settings provider detaches', async () => {
const bench = await boot()
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
baseURL: 'https://search.stored.test',
})
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
await bench.settingsFiber.dispose()
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
await bench.ctx.fiber.dispose()
})
it('releases the namespace when the plugin unloads', async () => {
const bench = await boot()
expect(bench.ctx.settings.describe().map(row => String(row.ns))).toContain('web-search-searxng')
await bench.pluginFiber.dispose()
expect(bench.ctx.settings.describe().map(row => String(row.ns))).not.toContain('web-search-searxng')
await bench.ctx.fiber.dispose()
})
})

View File

@@ -0,0 +1,30 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../../../vendor/schemastery"
},
{
"path": "../web"
},
{
"path": "../../settings/settings"
},
{
"path": "../../runtime-diagnostics/invariants"
}
]
}

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/web/web/README.md
README.md: 8dfc7f032e25e40207bb3880777b814e67175a20
README.zh.md: 3354037bfcd77ca2b1a07da710b565065baf543b
README.md: 0071d12d36c4fb696c8b35bef35bf1c1d62b22f1
README.zh.md: 9146b78778dd9e8e208c2ae3d997fee72829c18a

View File

@@ -10,6 +10,7 @@ This package owns the Service Definition role of the web capability. Unlike shel
|---|---|
| `@deepseek-ai/dsh-web` (this) | Service Definition: the service, provider registries, selection policy, request/result vocabulary, the `WebError` taxonomy |
| `@deepseek-ai/dsh-web-search-exa` | Search provider: Exa |
| `@deepseek-ai/dsh-web-search-searxng` | Search provider: SearXNG |
| `@deepseek-ai/dsh-web-search-perplexity` | Search provider: Perplexity |
| `@deepseek-ai/dsh-web-fetch-http` | Fetch provider: anonymous public HTTP(S) |
| `@deepseek-ai/dsh-tool-web` | Consumer: the model-facing `web_search` / `web_fetch` tool schemas over `ctx.web` |

View File

@@ -10,6 +10,7 @@
|---|---|
| `@deepseek-ai/dsh-web`(本包) | Service Definition:服务、提供方注册表、选择策略、请求/结果词汇、`WebError` 分类体系 |
| `@deepseek-ai/dsh-web-search-exa` | 搜索提供方:Exa |
| `@deepseek-ai/dsh-web-search-searxng` | 搜索提供方:SearXNG |
| `@deepseek-ai/dsh-web-search-perplexity` | 搜索提供方:Perplexity |
| `@deepseek-ai/dsh-web-fetch-http` | 抓取提供方:匿名公共 HTTP(S) |
| `@deepseek-ai/dsh-tool-web` | Consumer:面向模型的 `web_search`/`web_fetch` 工具 schema,构建于 `ctx.web` 之上 |