feat: add searxng web-search provider, openrouter cost balance UI, offline scripts; update source-launch and docs
Some checks failed
CI / node 22.19 (push) Has been skipped
CI / node 26 (push) Has been skipped
CI / python 3.10 / keyless SDK (push) Has been skipped
CI / python runtime / release-shaped Linux x64 (push) Has been skipped
CI / windows node 24 / wine blocking (push) Has been skipped
CI / wine apt cache (push) Successful in 58s
CI / serial / linux (push) Has been skipped
Deploy documentation / build (push) Failing after 2m46s
Deploy documentation / deploy (push) Has been skipped
E2E (real DeepSeek API) / e2e (push) Failing after 1m18s
Sandbox / sandbox e2e (bwrap, ubuntu-latest) (push) Failing after 1m18s
Landlock Run / Matrix (push) Successful in 13s
Release (vendor) / Pack npm tarballs (push) Failing after 3m43s
Release (dsh) / Pack npm tarballs (push) Failing after 1m53s
Sandbox / sandbox e2e (landlock, ubuntu-24.04) (push) Failing after 1m51s
Release (vendor) / Publish to npm (push) Has been skipped
Release (dsh) / Publish to npm (push) Has been skipped
CI / node 24 / static (push) Has been cancelled
CI / node 24 / coverage (push) Has been cancelled
CI / node 24 / snapshots and artifacts (push) Has been cancelled
CI / windows node 24 / native complete (push) Has been cancelled
CI / serial / linux (self-hosted standby) (push) Has been cancelled
CI / serial / macos (push) Has been cancelled
CI / serial / windows (self-hosted standby) (push) Has been cancelled
CI / larger-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (16, windows, dsh-windows-2025-16core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (32, windows, dsh-windows-2025-32core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (4, windows, dsh-windows-2025-4core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (64, windows, dsh-windows-2025-64core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (8, windows, dsh-windows-2025-8core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (96, windows, dsh-windows-2025-96core, production-site) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, 16) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, windows, dsh-windows-2025-16core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, windows, dsh-windows-2025-32core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, 4) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, windows, dsh-windows-2025-4core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, windows, dsh-windows-2025-64core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, 8) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, windows, dsh-windows-2025-8core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, windows, dsh-windows-2025-96core, 2) (push) Has been cancelled
CI / all checks passed (push) Has been cancelled
Sandbox / sandbox e2e (seatbelt, macos-latest) (push) Has been cancelled
Sandbox / sandbox e2e (landlock, ubuntu-24.04-arm) (push) Has been cancelled
Landlock Run / ${{ matrix.platform }} (push) Has been cancelled
Landlock Run / darwin (no platform package — degradation proof) (push) Has been cancelled
Some checks failed
CI / node 22.19 (push) Has been skipped
CI / node 26 (push) Has been skipped
CI / python 3.10 / keyless SDK (push) Has been skipped
CI / python runtime / release-shaped Linux x64 (push) Has been skipped
CI / windows node 24 / wine blocking (push) Has been skipped
CI / wine apt cache (push) Successful in 58s
CI / serial / linux (push) Has been skipped
Deploy documentation / build (push) Failing after 2m46s
Deploy documentation / deploy (push) Has been skipped
E2E (real DeepSeek API) / e2e (push) Failing after 1m18s
Sandbox / sandbox e2e (bwrap, ubuntu-latest) (push) Failing after 1m18s
Landlock Run / Matrix (push) Successful in 13s
Release (vendor) / Pack npm tarballs (push) Failing after 3m43s
Release (dsh) / Pack npm tarballs (push) Failing after 1m53s
Sandbox / sandbox e2e (landlock, ubuntu-24.04) (push) Failing after 1m51s
Release (vendor) / Publish to npm (push) Has been skipped
Release (dsh) / Publish to npm (push) Has been skipped
CI / node 24 / static (push) Has been cancelled
CI / node 24 / coverage (push) Has been cancelled
CI / node 24 / snapshots and artifacts (push) Has been cancelled
CI / windows node 24 / native complete (push) Has been cancelled
CI / serial / linux (self-hosted standby) (push) Has been cancelled
CI / serial / macos (push) Has been cancelled
CI / serial / windows (self-hosted standby) (push) Has been cancelled
CI / larger-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (16, windows, dsh-windows-2025-16core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (32, windows, dsh-windows-2025-32core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (4, windows, dsh-windows-2025-4core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (64, windows, dsh-windows-2025-64core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (8, windows, dsh-windows-2025-8core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (96, windows, dsh-windows-2025-96core, production-site) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, 16) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, windows, dsh-windows-2025-16core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, windows, dsh-windows-2025-32core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, 4) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, windows, dsh-windows-2025-4core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, windows, dsh-windows-2025-64core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, 8) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, windows, dsh-windows-2025-8core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, windows, dsh-windows-2025-96core, 2) (push) Has been cancelled
CI / all checks passed (push) Has been cancelled
Sandbox / sandbox e2e (seatbelt, macos-latest) (push) Has been cancelled
Sandbox / sandbox e2e (landlock, ubuntu-24.04-arm) (push) Has been cancelled
Landlock Run / ${{ matrix.platform }} (push) Has been cancelled
Landlock Run / darwin (no platform package — degradation proof) (push) Has been cancelled
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/web/README.md
|
||||
README.md: fc37d7cdead59138db149b5a86f0a0c031d40037
|
||||
README.zh.md: 53fe673ddaed235bbb938db4757e9cd240928d61
|
||||
README.md: 449c96c71b580964486968431c28f71dc466e8eb
|
||||
README.zh.md: a7d7d6dc1a382abe481f3520aa42480ff0317976
|
||||
|
||||
@@ -8,6 +8,7 @@ This family provides provider-neutral web search and fetch operations plus the m
|
||||
|---|---|---|
|
||||
| [`web/`](web/README.md) | Defines web provider registration, selection, and shared errors | `ctx.web` |
|
||||
| [`web-search-exa/`](web-search-exa/README.md) | Provides web search through Exa | registers on `ctx.web` |
|
||||
| [`web-search-searxng/`](web-search-searxng/README.md) | Provides web search through a SearXNG instance | registers on `ctx.web` |
|
||||
| [`web-search-perplexity/`](web-search-perplexity/README.md) | Provides web search through Perplexity | registers on `ctx.web` |
|
||||
| [`web-search-deepseek/`](web-search-deepseek/README.md) | Provides native DeepSeek web search | registers on `ctx.web` |
|
||||
| [`web-fetch-http/`](web-fetch-http/README.md) | Fetches public HTTP and HTTPS resources | registers on `ctx.web` |
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
|---|---|---|
|
||||
| [`web/`](web/README.md) | 定义 web 提供方注册、选择和共享错误 | `ctx.web` |
|
||||
| [`web-search-exa/`](web-search-exa/README.md) | 通过 Exa 提供 web 搜索 | 注册到 `ctx.web` |
|
||||
| [`web-search-searxng/`](web-search-searxng/README.md) | 通过 SearXNG 实例提供 web 搜索 | 注册到 `ctx.web` |
|
||||
| [`web-search-perplexity/`](web-search-perplexity/README.md) | 通过 Perplexity 提供 web 搜索 | 注册到 `ctx.web` |
|
||||
| [`web-search-deepseek/`](web-search-deepseek/README.md) | 提供 DeepSeek 原生 web 搜索 | 注册到 `ctx.web` |
|
||||
| [`web-fetch-http/`](web-fetch-http/README.md) | 抓取公共 HTTP 和 HTTPS 资源 | 注册到 `ctx.web` |
|
||||
|
||||
6
packages/web/web-search-searxng/README.i18n.yaml
Normal file
6
packages/web/web-search-searxng/README.i18n.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/web/web-search-searxng/README.md
|
||||
README.md: 0e0c76fc884eb5968618673cc58f6ae5f88112cc
|
||||
README.zh.md: aa550a67c49458c9488c4b3bf24532935db99e01
|
||||
52
packages/web/web-search-searxng/README.md
Normal file
52
packages/web/web-search-searxng/README.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# @deepseek-ai/dsh-web-search-searxng
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
A [SearXNG](https://docs.searxng.org)-backed `WebSearchProvider` for the harness [web capability seam](../web/README.md) (`ctx.web`). It calls a SearXNG instance's JSON API (`GET /search?format=json`) and maps the flat `results[]` into the seam's normalized `WebSearchResult`. It targets a self-hosted or private SearXNG: there is no single canonical public instance, so `baseURL` has no default, and the provider deliberately carries no API key or `Authorization` header.
|
||||
|
||||
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the `ctx.web` key and it does not register a model-facing tool (that is `@deepseek-ai/dsh-tool-web`). Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`) that registers its backend, not a default-export service.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `baseURL` | (none) | SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. |
|
||||
| `language` | `auto` | SearXNG `language` request value; `auto` lets the instance decide per user preferences. |
|
||||
| `timeRange` | (unset) | SearXNG `time_range` recency filter: `day`, `week`, `month`, or `year`. Omitted sends no filter. |
|
||||
|
||||
```yaml
|
||||
- id: web-search-searxng
|
||||
name: '@deepseek-ai/dsh-web-search-searxng'
|
||||
config:
|
||||
baseURL: https://searx.example
|
||||
```
|
||||
|
||||
The entry above is the base layer of the `web-search-searxng` Settings section: a user layer over it (a `settings.yaml` section or an in-session settings edit) reaches the NEXT search, because the provider projects the section per call rather than capturing it at registration. The seam's provider selection therefore never flickers when the instance or a filter changes. A section without `baseURL` still passes the schema but leaves the provider unavailable — no endpoint is guessed.
|
||||
|
||||
```yaml
|
||||
# $DSH_HOME/settings.yaml
|
||||
web-search-searxng:
|
||||
baseURL: https://searx.example
|
||||
language: auto
|
||||
timeRange: day
|
||||
```
|
||||
|
||||
SearXNG exposes no per-request result-count control — page size is instance configuration — so no `numResults` option exists; the seam enforces `maxResults` on the result.
|
||||
|
||||
## Mapping
|
||||
|
||||
SearXNG returns a flat `results[]` and no generated answer, so `content` is omitted. Each result maps to a `WebSearchSource`: `url` ← `url`, `title` ← `title`, `snippet` ← `content`, `publishedAt` ← `publishedDate`. Unlike the Exa provider it does **not** drop snippet-less entries: URL and title are still useful, so every result is kept. Provider failures (HTTP errors, network failure, unparseable or wrong-shape bodies) surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`. HTTP redirects are rejected before the `Location` target is contacted and surface as `WEB_PROVIDER_ERROR`. The request carries no `Authorization` header, so no credential can leak to a redirect target.
|
||||
|
||||
## Model Experience
|
||||
|
||||
Indirectly, through [`dsh-tool-web`](../tool-web/README.md), which retains this provider's `maxResults`-bounded URLs, titles, snippets, and publication dates or its exact `SearXNG search aborted`, `SearXNG search request failed: <error>`, and `SearXNG returned an unprocessable response body: <error>` failures under the consumer's error wrapper while generated answers and provider-private fields remain outside context.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Result count is not bounded at the request** — SearXNG page size is instance configuration, so the provider sends no count and the seam truncates on return; a single page (typically ~20 results) is fetched.
|
||||
- **No aggregate-answer or infobox content is surfaced** — SearXNG `answers` and `infoboxes` are not mapped into `content`. No generated answer is trusted.
|
||||
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.
|
||||
52
packages/web/web-search-searxng/README.zh.md
Normal file
52
packages/web/web-search-searxng/README.zh.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# @deepseek-ai/dsh-web-search-searxng
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
由 [SearXNG](https://docs.searxng.org) 支持的 `WebSearchProvider`,用于 harness [web 能力 seam](../web/README.md)(`ctx.web`)。它调用 SearXNG 实例的 JSON API(`GET /search?format=json`),把扁平 `results[]` 映射为 seam 规范化的 `WebSearchResult`。此提供方面向自托管或私有 SearXNG:不存在统一的公共实例,因此 `baseURL` 没有默认值,提供方刻意不携带 API 密钥或 `Authorization` 头。
|
||||
|
||||
这是一个**实现**包:它向 `ctx.web` 注册提供方,不拥有 `ctx.web` 键,也不注册面向模型的工具(后者属于 `@deepseek-ai/dsh-tool-web`)。与 `@deepseek-ai/dsh-llm-deepseek` 一样,它是函数/命名空间插件(`inject: ['web']`),负责注册后端,而非默认导出服务。
|
||||
|
||||
## 配置
|
||||
|
||||
| 配置键 | 默认值 | 含义 |
|
||||
|---|---|---|
|
||||
| `baseURL` | (无) | SearXNG 实例基址;追加 `/search`。为空或无法解析时提供方不可用。 |
|
||||
| `language` | `auto` | SearXNG `language` 请求值;`auto` 让实例按用户偏好决定。 |
|
||||
| `timeRange` | (未设置) | SearXNG `time_range` 时效过滤器:`day`、`week`、`month` 或 `year`。省略时不发送过滤器。 |
|
||||
|
||||
```yaml
|
||||
- id: web-search-searxng
|
||||
name: '@deepseek-ai/dsh-web-search-searxng'
|
||||
config:
|
||||
baseURL: https://searx.example
|
||||
```
|
||||
|
||||
上面的条目是 `web-search-searxng` Settings 分节的基础层:覆盖在它之上的用户层(`settings.yaml` 分节或会话内设置的编辑)会作用于**下一次**搜索,因为提供方按调用投影分节,而非在注册时捕获。因此 seam 的提供方选择在实例或过滤器变化时不会闪烁。不含 `baseURL` 的分节仍能通过 schema,但提供方保持不可用——不会猜测任何端点。
|
||||
|
||||
```yaml
|
||||
# $DSH_HOME/settings.yaml
|
||||
web-search-searxng:
|
||||
baseURL: https://searx.example
|
||||
language: auto
|
||||
timeRange: day
|
||||
```
|
||||
|
||||
SearXNG 不提供按请求控制结果数量的方式——页面大小属于实例配置——因此没有 `numResults` 选项;seam 会在结果返回时强制执行 `maxResults`。
|
||||
|
||||
## 映射
|
||||
|
||||
SearXNG 返回扁平 `results[]`,不返回生成答案,因此省略 `content`。每项结果映射为 `WebSearchSource`:`url` ← `url`、`title` ← `title`、`snippet` ← `content`、`publishedAt` ← `publishedDate`。与 Exa 提供方不同,此提供方**不会**丢弃无 snippet 的结果:URL 与标题仍然有用,因此保留每项结果。提供方失败(HTTP 错误、网络失败、响应体无法解析或结构不符)以 `WebError` `WEB_PROVIDER_ERROR` 呈现;中止请求以 `WEB_ABORTED` 呈现。HTTP 重定向会在访问 `Location` 指向的目标之前被拒绝,并以 `WEB_PROVIDER_ERROR` 呈现。请求不携带 `Authorization` 头,因此不会有凭据泄露给重定向目标。
|
||||
|
||||
## 模型体验
|
||||
|
||||
通过 [`dsh-tool-web`](../tool-web/README.md) 间接影响;该工具保留此提供方经 `maxResults` 限制的 URL、标题、snippet 与发布日期,或将确切的错误消息 `SearXNG search aborted`、`SearXNG search request failed: <error>` 和 `SearXNG returned an unprocessable response body: <error>` 置于消费方的错误包装层内;生成答案与提供方私有字段不进入上下文。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **请求端不限制结果数量**:SearXNG 页面大小属于实例配置,因此提供方不发送数量,由 seam 在返回时截断;只抓取单页(通常约 20 条结果)。
|
||||
- **不呈现聚合答案或 infobox 内容**:SearXNG 的 `answers` 与 `infoboxes` 不映射进 `content`。不信任任何生成答案。
|
||||
- **按错误形状分类中止**:只有 `DOMException` 且名为 `AbortError` 时才映射为 `WEB_ABORTED`;携带自定义原因的中止(例如 `dsh-timeout` 的 `TimeoutReason`)会呈现为 `WEB_PROVIDER_ERROR`。
|
||||
49
packages/web/web-search-searxng/package.json
Normal file
49
packages/web/web-search-searxng/package.json
Normal file
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-web-search-searxng",
|
||||
"description": "SearXNG-backed search provider for the DeepSeek Harness web capability seam (ctx.web)",
|
||||
"version": "0.1.0-rc.7",
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/deepseek-ai/deepseek-harness.git",
|
||||
"directory": "packages/web/web-search-searxng"
|
||||
},
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts"
|
||||
],
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-settings": "workspace:^",
|
||||
"@deepseek-ai/dsh-web": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/schemastery": "workspace:^"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-settings": "workspace:^",
|
||||
"@deepseek-ai/dsh-web": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
}
|
||||
}
|
||||
79
packages/web/web-search-searxng/src/index.ts
Normal file
79
packages/web/web-search-searxng/src/index.ts
Normal file
@@ -0,0 +1,79 @@
|
||||
/**
|
||||
* `@deepseek-ai/dsh-web-search-searxng`: registers a SearXNG-backed
|
||||
* `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a
|
||||
* default-export service): a search provider does not own the `ctx.web` key —
|
||||
* it registers INTO the seam's provider registry, exactly as
|
||||
* `@deepseek-ai/dsh-web-search-exa` does. The key is owned by
|
||||
* `@deepseek-ai/dsh-web`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-web-search-searxng
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import type {} from '@deepseek-ai/dsh-web'
|
||||
import { SearXngSearchProvider } from './provider.ts'
|
||||
|
||||
export {
|
||||
SEARXNG_DEFAULT_LANGUAGE,
|
||||
SEARXNG_NO_TIME_RANGE,
|
||||
SEARXNG_PROVIDER_ID,
|
||||
SEARXNG_TIME_RANGES,
|
||||
SearXngSearchProvider,
|
||||
} from './provider.ts'
|
||||
export type {
|
||||
SearXngSearchProviderOptions,
|
||||
SearXngSearchProviderSource,
|
||||
SearXngTimeRange,
|
||||
} from './provider.ts'
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'web-search-searxng'
|
||||
|
||||
/** The web seam this provider registers into. */
|
||||
export const inject = ['web']
|
||||
|
||||
/** Plugin config. `baseURL` is optional in the schema so an omitted value
|
||||
* surfaces as an unavailable provider (the seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`)
|
||||
* rather than failing boot; `apply` fills constant defaults. */
|
||||
export interface Config {
|
||||
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
|
||||
baseURL?: string
|
||||
/** SearXNG `language` to request. Defaults to `auto`. */
|
||||
language?: string
|
||||
/** SearXNG `time_range` recency filter. Omitted = no filter. */
|
||||
timeRange?: 'day' | 'week' | 'month' | 'year'
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
baseURL: z.string(),
|
||||
language: z.string(),
|
||||
timeRange: z.union(['day', 'week', 'month', 'year'] as const),
|
||||
})
|
||||
|
||||
/** Settings namespace carrying the SearXNG instance and any per-search filters. */
|
||||
export const SEARXNG_SETTINGS_NAMESPACE = settingsNamespace('web-search-searxng')
|
||||
|
||||
/** Register the SearXNG search provider with `ctx.web`, reading the live section
|
||||
* per search so an in-session settings edit applies without re-registration. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
let current: () => Config = () => config
|
||||
installSettingsSection(ctx, SEARXNG_SETTINGS_NAMESPACE, Config, config, {
|
||||
setSource: (source) => {
|
||||
current = source
|
||||
},
|
||||
// The registration carries no resolved value: the provider projects the
|
||||
// section per search, so a committed change needs no re-registration.
|
||||
onChange: () => {},
|
||||
})
|
||||
ctx.web.registerSearchProvider(new SearXngSearchProvider(() => ({
|
||||
// SearXNG exposes no uniform public endpoint and the product requires
|
||||
// evidence over an unsupported default, so `baseURL` has no constant
|
||||
// fallback: an omitted value surfaces as an unavailable provider (the
|
||||
// seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`), not a silent endpoint.
|
||||
baseURL: current().baseURL ?? '',
|
||||
...current().language !== undefined ? { language: current().language } : {},
|
||||
...current().timeRange !== undefined ? { timeRange: current().timeRange } : {},
|
||||
})))
|
||||
}
|
||||
30
packages/web/web-search-searxng/src/invariant.ts
Normal file
30
packages/web/web-search-searxng/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-web-search-searxng`.
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'web-search-searxng-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this package exposes no independent event sequence or mutable data relation
|
||||
* beyond contracts enforced at its owning seam.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
166
packages/web/web-search-searxng/src/provider.ts
Normal file
166
packages/web/web-search-searxng/src/provider.ts
Normal file
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* `SearXngSearchProvider`: a `WebSearchProvider` backed by a SearXNG instance's
|
||||
* JSON API (`GET /search?format=json`). It maps against the flat `results[]`,
|
||||
* keeps entries even without a snippet (URL and title remain useful), and
|
||||
* omits `content` because SearXNG returns no generated answer. The provider
|
||||
* carries no credentials, so a request carries no `Authorization` header.
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/provider
|
||||
*/
|
||||
|
||||
import { WebError } from '@deepseek-ai/dsh-web'
|
||||
import type {
|
||||
WebSearchProvider,
|
||||
WebSearchRequest,
|
||||
WebSearchResult,
|
||||
WebSearchSource,
|
||||
} from '@deepseek-ai/dsh-web'
|
||||
import type { SearXngError, SearXngResult, SearXngSearchResponse } from './types.ts'
|
||||
|
||||
/** Stable id this provider registers under. */
|
||||
export const SEARXNG_PROVIDER_ID = 'searxng'
|
||||
|
||||
/** Default language: let SearXNG decide per user preferences. */
|
||||
export const SEARXNG_DEFAULT_LANGUAGE = 'auto'
|
||||
|
||||
/** `time_range` sentinel meaning "no recency filter". */
|
||||
export const SEARXNG_NO_TIME_RANGE = 'none'
|
||||
|
||||
/** Valid `time_range` values SearXNG accepts. */
|
||||
export const SEARXNG_TIME_RANGES = ['day', 'week', 'month', 'year'] as const
|
||||
|
||||
/** One of SearXNG's `time_range` values, passed through to the endpoint. */
|
||||
export type SearXngTimeRange = (typeof SEARXNG_TIME_RANGES)[number]
|
||||
|
||||
/** Attribution header sent on every request. Bump with the package version. */
|
||||
const USER_AGENT = 'deepseek-harness/0.0.1'
|
||||
|
||||
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
|
||||
export interface SearXngSearchProviderOptions {
|
||||
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
|
||||
baseURL: string
|
||||
/** SearXNG `language` to request. `auto` lets the instance decide. */
|
||||
language?: string
|
||||
/** SearXNG `time_range` recency filter. */
|
||||
timeRange?: SearXngTimeRange
|
||||
}
|
||||
|
||||
/** A snapshot or a per-search resolver (settings live-reload hands the latter). */
|
||||
export type SearXngSearchProviderSource = SearXngSearchProviderOptions | (() => SearXngSearchProviderOptions)
|
||||
|
||||
/**
|
||||
* Map one SearXNG result to a normalized source. The URL is always carried; a
|
||||
* blank title or snippet is omitted rather than emitted as empty.
|
||||
*
|
||||
* @param result - one entry of SearXNG's `results[]`.
|
||||
* @returns the normalized source.
|
||||
*/
|
||||
export function mapSearXngResult(result: SearXngResult): WebSearchSource {
|
||||
return {
|
||||
url: result.url,
|
||||
...result.title != null && result.title.trim().length > 0 ? { title: result.title } : {},
|
||||
...result.content != null && result.content.trim().length > 0 ? { snippet: result.content } : {},
|
||||
...result.publishedDate != null && result.publishedDate.length > 0 ? { publishedAt: result.publishedDate } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a SearXNG response envelope to a normalized search result.
|
||||
*
|
||||
* @param response - the parsed `format=json` response body.
|
||||
* @returns the normalized result; `content` is omitted (no generated answer).
|
||||
*/
|
||||
export function mapSearXngResponse(response: SearXngSearchResponse): WebSearchResult {
|
||||
// SearXNG cannot bound per-request result count (page size is instance
|
||||
// configuration), so the request carries no count and the web service owns
|
||||
// the final `maxResults` truncation; this provider reports `truncated: false`.
|
||||
return { sources: (response.results ?? []).map(mapSearXngResult), truncated: false }
|
||||
}
|
||||
|
||||
/** The SearXNG-backed search provider; HTTP redirects fail as `WEB_PROVIDER_ERROR`. */
|
||||
export class SearXngSearchProvider implements WebSearchProvider {
|
||||
readonly id = SEARXNG_PROVIDER_ID
|
||||
|
||||
constructor(source: SearXngSearchProviderSource) {
|
||||
this.source = source
|
||||
}
|
||||
|
||||
private readonly source: SearXngSearchProviderSource
|
||||
|
||||
/** Read the current options: a snapshot stays fixed, a resolver reads live settings. */
|
||||
private resolveOptions(): SearXngSearchProviderOptions {
|
||||
return typeof this.source === 'function' ? this.source() : this.source
|
||||
}
|
||||
|
||||
available(): boolean {
|
||||
const options = this.resolveOptions()
|
||||
return isValidBaseUrl(options.baseURL)
|
||||
&& (options.language === undefined || options.language.length > 0)
|
||||
&& (options.timeRange === undefined || SEARXNG_TIME_RANGES.includes(options.timeRange))
|
||||
}
|
||||
|
||||
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
|
||||
const options = this.resolveOptions()
|
||||
const base = trimTrailingSlashes(options.baseURL)
|
||||
const params = new URLSearchParams({ q: request.query, format: 'json' })
|
||||
if (options.language !== undefined) params.set('language', options.language)
|
||||
if (options.timeRange !== undefined) params.set('time_range', options.timeRange)
|
||||
|
||||
let response: Response
|
||||
try {
|
||||
response = await fetch(`${base}/search?${params}`, {
|
||||
method: 'GET',
|
||||
redirect: 'error',
|
||||
headers: {
|
||||
'accept': 'application/json',
|
||||
'user-agent': USER_AGENT,
|
||||
},
|
||||
...signal !== undefined ? { signal } : {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
throw new WebError(`SearXNG search request failed: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
const status = response.status
|
||||
let message = `SearXNG API error (HTTP ${status})`
|
||||
try {
|
||||
const parsed = await response.json() as SearXngError
|
||||
const detail = parsed.error ?? parsed.message ?? parsed.content
|
||||
if (detail !== undefined && detail.length > 0) message = detail
|
||||
} catch (error: unknown) {
|
||||
// An abort fired mid-body must surface as WEB_ABORTED, not be swallowed
|
||||
// into a generic HTTP-error message — cancellation is not a provider
|
||||
// error (the seam's cancellation contract).
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
// Otherwise: the HTTP status is already captured in `message` above; a
|
||||
// malformed/non-JSON error body (normal for gateway 5xx/429s) can only
|
||||
// cost a richer provider message, never the real error.
|
||||
}
|
||||
throw new WebError(message, 'WEB_PROVIDER_ERROR')
|
||||
}
|
||||
|
||||
try {
|
||||
const payload = await response.json() as SearXngSearchResponse
|
||||
return mapSearXngResponse(payload)
|
||||
} catch (error: unknown) {
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
throw new WebError(`SearXNG returned an unprocessable response body: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** True when `baseURL` parses as an absolute URL (a cheap local config check). */
|
||||
function isValidBaseUrl(baseURL: string): boolean {
|
||||
return URL.canParse(baseURL)
|
||||
}
|
||||
|
||||
/** Strip a trailing slash so `${base}/search` never doubles a separator. */
|
||||
function trimTrailingSlashes(baseURL: string): string {
|
||||
return baseURL.replace(/\/+$/, '')
|
||||
}
|
||||
|
||||
/** True for a fetch/`AbortSignal` abort, surfaced as `WEB_ABORTED`. */
|
||||
function isAbortError(error: unknown): boolean {
|
||||
return error instanceof DOMException && error.name === 'AbortError'
|
||||
}
|
||||
30
packages/web/web-search-searxng/src/types.ts
Normal file
30
packages/web/web-search-searxng/src/types.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Wire types for the SearXNG JSON API (`GET {base}/search?q=...&format=json`).
|
||||
* Types only — no runtime code. SearXNG returns a flat `results[]`; each entry
|
||||
* carries a URL, an optional title, an optional `content` snippet, and an
|
||||
* optional `publishedDate`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/types
|
||||
*/
|
||||
|
||||
/** One entry of SearXNG's flat `results[]`. */
|
||||
export interface SearXngResult {
|
||||
url: string
|
||||
title?: string | null
|
||||
/** The page snippet SearXNG derives for the result. */
|
||||
content?: string | null
|
||||
publishedDate?: string | null
|
||||
}
|
||||
|
||||
/** SearXNG's JSON search response envelope. */
|
||||
export interface SearXngSearchResponse {
|
||||
results?: SearXngResult[]
|
||||
}
|
||||
|
||||
/** SearXNG's error response envelope (best-effort; fields vary by failure). */
|
||||
export interface SearXngError {
|
||||
error?: string
|
||||
message?: string
|
||||
/** Some SearXNG error responses carry the message under `content`. */
|
||||
content?: string
|
||||
}
|
||||
24
packages/web/web-search-searxng/tests/searxng.e2e.ts
Normal file
24
packages/web/web-search-searxng/tests/searxng.e2e.ts
Normal file
@@ -0,0 +1,24 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SearXngSearchProvider } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/**
|
||||
* Real-API smoke for the SearXNG search provider against a live instance.
|
||||
* Self-skips without `$SEARXNG_BASE_URL` (CI has no endpoint), per the
|
||||
* with-key e2e policy in docs/testing.md.
|
||||
*/
|
||||
const baseURL = process.env.SEARXNG_BASE_URL
|
||||
const maybe = baseURL !== undefined && baseURL.length > 0 ? describe : describe.skip
|
||||
|
||||
maybe('SearXngSearchProvider real API', () => {
|
||||
it('returns sources for a live query', async () => {
|
||||
const provider = new SearXngSearchProvider({
|
||||
baseURL: baseURL!,
|
||||
...process.env.SEARXNG_LANGUAGE !== undefined && process.env.SEARXNG_LANGUAGE.length > 0
|
||||
? { language: process.env.SEARXNG_LANGUAGE }
|
||||
: {},
|
||||
})
|
||||
const result = await provider.search({ query: 'DeepSeek Harness', maxResults: 5 })
|
||||
expect(result.sources.length).toBeGreaterThan(0)
|
||||
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
|
||||
}, 30_000)
|
||||
})
|
||||
226
packages/web/web-search-searxng/tests/searxng.spec.ts
Normal file
226
packages/web/web-search-searxng/tests/searxng.spec.ts
Normal file
@@ -0,0 +1,226 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import WebRuntime from '@deepseek-ai/dsh-web'
|
||||
import { SearXngSearchProvider, SEARXNG_PROVIDER_ID } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import { mapSearXngResponse, mapSearXngResult } from '../src/provider.ts'
|
||||
|
||||
const options = { baseURL: 'https://searx.test' }
|
||||
|
||||
function jsonResponse(body: unknown, init: ResponseInit = {}): Response {
|
||||
return new Response(JSON.stringify(body), { status: 200, headers: { 'content-type': 'application/json' }, ...init })
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
describe('SearXng result mapping', () => {
|
||||
it('maps a full result entry', () => {
|
||||
expect(mapSearXngResult({
|
||||
url: 'https://a.test',
|
||||
title: 'A',
|
||||
content: 'an excerpt',
|
||||
publishedDate: '2026-01-01',
|
||||
})).toEqual({ url: 'https://a.test', title: 'A', snippet: 'an excerpt', publishedAt: '2026-01-01' })
|
||||
})
|
||||
|
||||
it('keeps a URL-only result rather than dropping it', () => {
|
||||
expect(mapSearXngResult({ url: 'https://a.test' })).toEqual({ url: 'https://a.test' })
|
||||
expect(mapSearXngResult({ url: 'https://a.test', content: ' ' })).toEqual({ url: 'https://a.test' })
|
||||
})
|
||||
|
||||
it('omits null/empty optional fields rather than emitting them', () => {
|
||||
expect(mapSearXngResult({ url: 'https://a.test', title: null, content: null, publishedDate: null }))
|
||||
.toEqual({ url: 'https://a.test' })
|
||||
expect(mapSearXngResult({ url: 'https://a.test', title: '', content: '', publishedDate: '' }))
|
||||
.toEqual({ url: 'https://a.test' })
|
||||
})
|
||||
|
||||
it('maps a response to a result with no content and all sources kept', () => {
|
||||
const result = mapSearXngResponse({
|
||||
results: [
|
||||
{ url: 'https://a.test', title: 'A', content: 'one' },
|
||||
{ url: 'https://b.test' },
|
||||
{ url: 'https://c.test', content: 'three' },
|
||||
],
|
||||
})
|
||||
expect(result).toEqual({
|
||||
sources: [
|
||||
{ url: 'https://a.test', title: 'A', snippet: 'one' },
|
||||
{ url: 'https://b.test' },
|
||||
{ url: 'https://c.test', snippet: 'three' },
|
||||
],
|
||||
truncated: false,
|
||||
})
|
||||
expect(result.content).toBeUndefined()
|
||||
})
|
||||
|
||||
it('tolerates a missing results array', () => {
|
||||
expect(mapSearXngResponse({}).sources).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider availability', () => {
|
||||
it('is unavailable without a base URL', () => {
|
||||
expect(new SearXngSearchProvider({ baseURL: '' }).available()).toBe(false)
|
||||
})
|
||||
|
||||
it('is available with a base URL', () => {
|
||||
expect(new SearXngSearchProvider(options).available()).toBe(true)
|
||||
})
|
||||
|
||||
it('is misconfigured when the base URL is unparseable', () => {
|
||||
expect(new SearXngSearchProvider({ baseURL: 'not a url' }).available()).toBe(false)
|
||||
})
|
||||
|
||||
it('is misconfigured when language is empty or timeRange is invalid', () => {
|
||||
expect(new SearXngSearchProvider({ ...options, language: '' }).available()).toBe(false)
|
||||
expect(new SearXngSearchProvider({ ...options, timeRange: 'decade' as never }).available()).toBe(false)
|
||||
expect(new SearXngSearchProvider({ ...options, timeRange: 'week' }).available()).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider request mapping', () => {
|
||||
it('issues a GET with query and json format and no authorization header', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [{ url: 'https://a.test' }] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
await new SearXngSearchProvider(options).search({ query: 'hello world' })
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledOnce()
|
||||
const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
|
||||
expect(url).toBe('https://searx.test/search?q=hello+world&format=json')
|
||||
expect(init.method).toBe('GET')
|
||||
expect(init.redirect).toBe('error')
|
||||
expect((init.headers as Record<string, string>)['authorization']).toBeUndefined()
|
||||
})
|
||||
|
||||
it('sends language and time_range when configured', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider({ ...options, language: 'en', timeRange: 'week' }).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
|
||||
})
|
||||
|
||||
it('omits language and time_range when unset', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider(options).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json')
|
||||
})
|
||||
|
||||
it('does not double the separator when baseURL carries a trailing slash', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider({ baseURL: 'https://searx.test/' }).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json')
|
||||
})
|
||||
|
||||
it('forwards the abort signal', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const controller = new AbortController()
|
||||
await new SearXngSearchProvider(options).search({ query: 'q' }, controller.signal)
|
||||
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
|
||||
expect(init.signal).toBe(controller.signal)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider error handling', () => {
|
||||
it('maps an HTTP error to WEB_PROVIDER_ERROR with the provider message', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ error: 'missing instance' }, { status: 401 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'missing instance' }))
|
||||
})
|
||||
|
||||
it('keeps a status-line message when the error body is not JSON', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => new Response('gateway down', { status: 502 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'SearXNG API error (HTTP 502)' }))
|
||||
})
|
||||
|
||||
it('reads the message from content when present', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ content: 'json disabled' }, { status: 403 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ message: 'json disabled' }))
|
||||
})
|
||||
|
||||
it('maps a network failure to WEB_PROVIDER_ERROR', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new TypeError('connection refused'))))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('maps an abort to WEB_ABORTED', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new DOMException('aborted', 'AbortError'))))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
|
||||
it('maps an unparseable success body to WEB_PROVIDER_ERROR', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => new Response('not json', { status: 200 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('maps a well-formed body of the wrong shape to WEB_PROVIDER_ERROR, not a raw TypeError', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: {} }, { status: 200 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('surfaces an abort during success-body parse as WEB_ABORTED, not provider error', async () => {
|
||||
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: true, status: 200 }
|
||||
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
|
||||
it('surfaces an abort during error-body parse as WEB_ABORTED', async () => {
|
||||
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: false, status: 500 }
|
||||
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
})
|
||||
|
||||
describe('web-search-searxng plugin registration', () => {
|
||||
it('registers the provider into ctx.web (HMR-safe)', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: [] })))
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL })
|
||||
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ sources: [], truncated: false })
|
||||
await fiber.dispose()
|
||||
await expect(ctx.web.search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))
|
||||
})
|
||||
|
||||
it('has no default export (namespace plugin export shape)', () => {
|
||||
expect('default' in searxngPlugin).toBe(false)
|
||||
})
|
||||
|
||||
it('threads language and timeRange config into the request', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL, language: 'en', timeRange: 'week' })
|
||||
await ctx.web.search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('is unavailable when baseURL is omitted', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
await ctx.plugin(searxngPlugin, { baseURL: '' })
|
||||
await expect(ctx.web.search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' }))
|
||||
})
|
||||
})
|
||||
120
packages/web/web-search-searxng/tests/settings.spec.ts
Normal file
120
packages/web/web-search-searxng/tests/settings.spec.ts
Normal file
@@ -0,0 +1,120 @@
|
||||
/** The `web-search-searxng` settings section layered over the composition entry. */
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import type { Fiber } from '@deepseek-ai/cordis'
|
||||
import { SettingsProvider } from '@deepseek-ai/dsh-settings'
|
||||
import type { SettingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import WebRuntime from '@deepseek-ai/dsh-web'
|
||||
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import { SEARXNG_SETTINGS_NAMESPACE } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/** The smallest real provider: one in-memory document, always writable. */
|
||||
class MemorySettings extends SettingsProvider {
|
||||
doc: Record<string, unknown> = {}
|
||||
|
||||
get writable(): boolean {
|
||||
return true
|
||||
}
|
||||
|
||||
protected load(): Promise<Record<string, unknown>> {
|
||||
return Promise.resolve(structuredClone(this.doc))
|
||||
}
|
||||
|
||||
protected persist(ns: SettingsNamespace, section: Record<string, unknown>): Promise<void> {
|
||||
this.doc = { ...this.doc, [ns]: structuredClone(section) }
|
||||
return Promise.resolve()
|
||||
}
|
||||
}
|
||||
|
||||
function jsonResponse(body: unknown): Response {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
})
|
||||
}
|
||||
|
||||
/** The smallest SearXNG-shaped answer the provider accepts — enough to observe the request. */
|
||||
const ONE_RESULT = {
|
||||
results: [{ url: 'https://a.test', title: 'A', content: 'snip' }],
|
||||
}
|
||||
|
||||
async function boot(): Promise<{ ctx: Context; settingsFiber: Fiber; pluginFiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, {})
|
||||
const settingsFiber = ctx.plugin(MemorySettings)
|
||||
await settingsFiber.await()
|
||||
const pluginFiber = ctx.plugin(searxngPlugin, { baseURL: 'https://search.entry.test' })
|
||||
await pluginFiber.await()
|
||||
return { ctx, settingsFiber, pluginFiber }
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
/**
|
||||
* Run one search and answer the endpoint it reached. A fresh `Response` per
|
||||
* call because a body can only be read once, and the call history is cleared
|
||||
* because repeated `spyOn` returns the same spy.
|
||||
* @param ctx - context whose `ctx.web` serves the search.
|
||||
* @returns the URL the provider fetched.
|
||||
*/
|
||||
async function searchOnce(ctx: Context): Promise<string> {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch')
|
||||
.mockImplementation(() => Promise.resolve(jsonResponse(ONE_RESULT)))
|
||||
fetchSpy.mockClear()
|
||||
await ctx.web.search({ query: 'anything' })
|
||||
return String((fetchSpy.mock.calls.at(-1)?.[0] as URL | string | undefined) ?? '')
|
||||
}
|
||||
|
||||
describe('web-search-searxng settings section', () => {
|
||||
it('serves a stored endpoint to the next search without re-registering the provider', async () => {
|
||||
const bench = await boot()
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
|
||||
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.stored.test',
|
||||
})
|
||||
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('applies a stored language and time range to the next request', async () => {
|
||||
const bench = await boot()
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.entry.test',
|
||||
language: 'de',
|
||||
timeRange: 'week',
|
||||
})
|
||||
|
||||
const url = new URL(await searchOnce(bench.ctx))
|
||||
expect(url.searchParams.get('language')).toBe('de')
|
||||
expect(url.searchParams.get('time_range')).toBe('week')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('falls back to the composition entry when the settings provider detaches', async () => {
|
||||
const bench = await boot()
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.stored.test',
|
||||
})
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
|
||||
|
||||
await bench.settingsFiber.dispose()
|
||||
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('releases the namespace when the plugin unloads', async () => {
|
||||
const bench = await boot()
|
||||
expect(bench.ctx.settings.describe().map(row => String(row.ns))).toContain('web-search-searxng')
|
||||
|
||||
await bench.pluginFiber.dispose()
|
||||
|
||||
expect(bench.ctx.settings.describe().map(row => String(row.ns))).not.toContain('web-search-searxng')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
30
packages/web/web-search-searxng/tsconfig.json
Normal file
30
packages/web/web-search-searxng/tsconfig.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../web"
|
||||
},
|
||||
{
|
||||
"path": "../../settings/settings"
|
||||
},
|
||||
{
|
||||
"path": "../../runtime-diagnostics/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/web/web/README.md
|
||||
README.md: 8dfc7f032e25e40207bb3880777b814e67175a20
|
||||
README.zh.md: 3354037bfcd77ca2b1a07da710b565065baf543b
|
||||
README.md: 0071d12d36c4fb696c8b35bef35bf1c1d62b22f1
|
||||
README.zh.md: 9146b78778dd9e8e208c2ae3d997fee72829c18a
|
||||
|
||||
@@ -10,6 +10,7 @@ This package owns the Service Definition role of the web capability. Unlike shel
|
||||
|---|---|
|
||||
| `@deepseek-ai/dsh-web` (this) | Service Definition: the service, provider registries, selection policy, request/result vocabulary, the `WebError` taxonomy |
|
||||
| `@deepseek-ai/dsh-web-search-exa` | Search provider: Exa |
|
||||
| `@deepseek-ai/dsh-web-search-searxng` | Search provider: SearXNG |
|
||||
| `@deepseek-ai/dsh-web-search-perplexity` | Search provider: Perplexity |
|
||||
| `@deepseek-ai/dsh-web-fetch-http` | Fetch provider: anonymous public HTTP(S) |
|
||||
| `@deepseek-ai/dsh-tool-web` | Consumer: the model-facing `web_search` / `web_fetch` tool schemas over `ctx.web` |
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
|---|---|
|
||||
| `@deepseek-ai/dsh-web`(本包) | Service Definition:服务、提供方注册表、选择策略、请求/结果词汇、`WebError` 分类体系 |
|
||||
| `@deepseek-ai/dsh-web-search-exa` | 搜索提供方:Exa |
|
||||
| `@deepseek-ai/dsh-web-search-searxng` | 搜索提供方:SearXNG |
|
||||
| `@deepseek-ai/dsh-web-search-perplexity` | 搜索提供方:Perplexity |
|
||||
| `@deepseek-ai/dsh-web-fetch-http` | 抓取提供方:匿名公共 HTTP(S) |
|
||||
| `@deepseek-ai/dsh-tool-web` | Consumer:面向模型的 `web_search`/`web_fetch` 工具 schema,构建于 `ctx.web` 之上 |
|
||||
|
||||
Reference in New Issue
Block a user