feat: add searxng web-search provider, openrouter cost balance UI, offline scripts; update source-launch and docs
Some checks failed
CI / node 22.19 (push) Has been skipped
CI / node 26 (push) Has been skipped
CI / python 3.10 / keyless SDK (push) Has been skipped
CI / python runtime / release-shaped Linux x64 (push) Has been skipped
CI / windows node 24 / wine blocking (push) Has been skipped
CI / wine apt cache (push) Successful in 58s
CI / serial / linux (push) Has been skipped
Deploy documentation / build (push) Failing after 2m46s
Deploy documentation / deploy (push) Has been skipped
E2E (real DeepSeek API) / e2e (push) Failing after 1m18s
Sandbox / sandbox e2e (bwrap, ubuntu-latest) (push) Failing after 1m18s
Landlock Run / Matrix (push) Successful in 13s
Release (vendor) / Pack npm tarballs (push) Failing after 3m43s
Release (dsh) / Pack npm tarballs (push) Failing after 1m53s
Sandbox / sandbox e2e (landlock, ubuntu-24.04) (push) Failing after 1m51s
Release (vendor) / Publish to npm (push) Has been skipped
Release (dsh) / Publish to npm (push) Has been skipped
CI / node 24 / static (push) Has been cancelled
CI / node 24 / coverage (push) Has been cancelled
CI / node 24 / snapshots and artifacts (push) Has been cancelled
CI / windows node 24 / native complete (push) Has been cancelled
CI / serial / linux (self-hosted standby) (push) Has been cancelled
CI / serial / macos (push) Has been cancelled
CI / serial / windows (self-hosted standby) (push) Has been cancelled
CI / larger-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (16, windows, dsh-windows-2025-16core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (32, windows, dsh-windows-2025-32core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (4, windows, dsh-windows-2025-4core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (64, windows, dsh-windows-2025-64core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (8, windows, dsh-windows-2025-8core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (96, windows, dsh-windows-2025-96core, production-site) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, 16) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, windows, dsh-windows-2025-16core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, windows, dsh-windows-2025-32core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, 4) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, windows, dsh-windows-2025-4core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, windows, dsh-windows-2025-64core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, 8) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, windows, dsh-windows-2025-8core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, windows, dsh-windows-2025-96core, 2) (push) Has been cancelled
CI / all checks passed (push) Has been cancelled
Sandbox / sandbox e2e (seatbelt, macos-latest) (push) Has been cancelled
Sandbox / sandbox e2e (landlock, ubuntu-24.04-arm) (push) Has been cancelled
Landlock Run / ${{ matrix.platform }} (push) Has been cancelled
Landlock Run / darwin (no platform package — degradation proof) (push) Has been cancelled
Some checks failed
CI / node 22.19 (push) Has been skipped
CI / node 26 (push) Has been skipped
CI / python 3.10 / keyless SDK (push) Has been skipped
CI / python runtime / release-shaped Linux x64 (push) Has been skipped
CI / windows node 24 / wine blocking (push) Has been skipped
CI / wine apt cache (push) Successful in 58s
CI / serial / linux (push) Has been skipped
Deploy documentation / build (push) Failing after 2m46s
Deploy documentation / deploy (push) Has been skipped
E2E (real DeepSeek API) / e2e (push) Failing after 1m18s
Sandbox / sandbox e2e (bwrap, ubuntu-latest) (push) Failing after 1m18s
Landlock Run / Matrix (push) Successful in 13s
Release (vendor) / Pack npm tarballs (push) Failing after 3m43s
Release (dsh) / Pack npm tarballs (push) Failing after 1m53s
Sandbox / sandbox e2e (landlock, ubuntu-24.04) (push) Failing after 1m51s
Release (vendor) / Publish to npm (push) Has been skipped
Release (dsh) / Publish to npm (push) Has been skipped
CI / node 24 / static (push) Has been cancelled
CI / node 24 / coverage (push) Has been cancelled
CI / node 24 / snapshots and artifacts (push) Has been cancelled
CI / windows node 24 / native complete (push) Has been cancelled
CI / serial / linux (self-hosted standby) (push) Has been cancelled
CI / serial / macos (push) Has been cancelled
CI / serial / windows (self-hosted standby) (push) Has been cancelled
CI / larger-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (16, windows, dsh-windows-2025-16core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (32, windows, dsh-windows-2025-32core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (4, windows, dsh-windows-2025-4core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (64, windows, dsh-windows-2025-64core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (8, windows, dsh-windows-2025-8core, production-site) (push) Has been cancelled
CI / larger-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, typecheck) (push) Has been cancelled
CI / larger-runner-benchmark (96, windows, dsh-windows-2025-96core, production-site) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, linux, dsh-ubuntu-24-04-16core, 16) (push) Has been cancelled
CI / consolidated-runner-benchmark (16, windows, dsh-windows-2025-16core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, linux, dsh-ubuntu-24-04-32core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (32, windows, dsh-windows-2025-32core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, linux, dsh-ubuntu-24-04-4core, 4) (push) Has been cancelled
CI / consolidated-runner-benchmark (4, windows, dsh-windows-2025-4core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, linux, dsh-ubuntu-24-04-64core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (64, windows, dsh-windows-2025-64core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, linux, dsh-ubuntu-24-04-8core, 8) (push) Has been cancelled
CI / consolidated-runner-benchmark (8, windows, dsh-windows-2025-8core, 2) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, linux, dsh-ubuntu-24-04-96core, 32) (push) Has been cancelled
CI / consolidated-runner-benchmark (96, windows, dsh-windows-2025-96core, 2) (push) Has been cancelled
CI / all checks passed (push) Has been cancelled
Sandbox / sandbox e2e (seatbelt, macos-latest) (push) Has been cancelled
Sandbox / sandbox e2e (landlock, ubuntu-24.04-arm) (push) Has been cancelled
Landlock Run / ${{ matrix.platform }} (push) Has been cancelled
Landlock Run / darwin (no platform package — degradation proof) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-19-searxng-web-search-provider.md
|
||||
2026-08-19-searxng-web-search-provider.md: 377dba2af133677b4fd1fd591cb3929ac2acda33
|
||||
2026-08-19-searxng-web-search-provider.zh.md: 29d406831ea947c383263dd5cdba840fa9e448a8
|
||||
@@ -0,0 +1,38 @@
|
||||
# Agent Note: SearXNG web search provider
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-19-searxng-web-search-provider.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The harness [web capability seam](2026-06-24-web-capability-seam.md) ships three search providers — Exa, Perplexity, and native DeepSeek web search — all of which are hosted API services. None fits a deployment that wants to self-host its own metasearch. SearXNG is a self-hostable metasearch engine with a JSON API, so it is the natural backend for such deployments. It differs from the hosted providers in two ways that shape the adapter: there is no single canonical SearXNG endpoint (each instance is its own server, so `baseURL` cannot have a sensible public default), and an engine does not expose a per-request result-count control (page size is instance configuration).
|
||||
|
||||
## Decision
|
||||
|
||||
Add `@deepseek-ai/dsh-web-search-searxng` (`packages/web/web-search-searxng`), a `WebSearchProvider` that registers under the stable id `searxng` and calls a SearXNG instance's JSON API (`GET {base}/search?q=...&format=json`).
|
||||
|
||||
- `baseURL` has **no default**; the schema keeps it optional so an omitted or unparseable value surfaces as an unavailable provider (`WEB_PROVIDER_CONFIGURED_UNAVAILABLE`) rather than a boot error. The product requires evidence before an unsupported default endpoint, and no single public SearXNG instance is authoritative.
|
||||
- Config flows through the `web-search-searxng` Settings namespace (layered over the cordis entry), and the provider projects the resolved section **per search**, so a settings edit applies to the next call without re-registration — the same projection topology as `dsh-web-search-deepseek`.
|
||||
- The provider carries **no API key and no `Authorization` header**: it targets open instances, so no credential can leak to a redirect target. HTTP redirects are still rejected with `redirect: 'error'`, mapping to `WEB_PROVIDER_ERROR`, matching the seam's redirect hygiene.
|
||||
- `language` (SearXNG `language`, default `auto`) and `timeRange` (`day`|`week`|`month`|`year`) are exposed; a request carries no result count because the API cannot bound one, so the seam's `maxResults` truncation is the sole bound. `content` is omitted (no generated answer).
|
||||
- Results map to `WebSearchSource`: `url` ← `url`, `title` ← `title`, `snippet` ← `content`. Unlike Exa, a result **without a snippet is kept** — URL and title are still useful — so sources are not dropped for missing snippets.
|
||||
|
||||
The package follows the provider topology of `dsh-web-search-exa`: a function/namespace plugin (`inject: ['web']`) that registers into `ctx.web`, `inject`-less in the sense that it registers a backend rather than owning the `ctx.web` key, with package-owned mapping and error translation and an empty package invariant companion.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Give `baseURL` a public-instance default
|
||||
|
||||
Rejected. A default like `https://searx.be` would pick an arbitrary, possibly unreachable or rate-limited public instance and silently send traffic to a third party without the operator choosing it. Requiring an explicit `baseURL` makes the provider unavailable until configured, which is the honest misconfiguration signal the seam already names (`WEB_PROVIDER_CONFIGURED_UNAVAILABLE`).
|
||||
|
||||
### Route SearXNG through an OpenAI-compatible gateway instead of a dedicated provider
|
||||
|
||||
Rejected. The harness search seam consumes `WebSearchProvider` backends, not arbitrary OpenAI-compatible endpoints; the LLM provider configuration surface does not feed `ctx.web`. A dedicated SearXNG `WebSearchProvider` is the only path that reaches the web search tools without a bespoke glue service.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A deployment gains a self-hosted search backend whose query goes to the operator's own instance, with optional language and recency filters but no per-query result-count control — single-page results are truncated by the seam.
|
||||
- The provider set grows to four search backends; `searchProvider` (or `$DSH_WEB_SEARCH_PROVIDER`) names `searxng` to select it, or auto-selection picks it when it is the only usable provider.
|
||||
- Redirect rejection means a misconfigured `baseURL` that answers 3xx surfaces as `WEB_PROVIDER_ERROR` rather than silently following; with no credentials on the request this is hygiene, not secret protection.
|
||||
- Snippet-less results are retained, so `tool-web` renders fallback URL/hostname labels for them rather than dropping the source.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Agent Note: SearXNG web search provider
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-19-searxng-web-search-provider.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
harness [web 能力 seam](2026-06-24-web-capability-seam.md) 自带了三个搜索提供方——Exa、Perplexity 与 DeepSeek 原生 web 搜索——它们都是托管 API 服务。它们都不适合希望自托管元搜索的部署场景。SearXNG 是具备 JSON API 的可自托管元搜索引擎,因此是这类部署的自然后端。它在这两个方面与托管提供方不同,进而塑造了适配器:不存在统一的 SearXNG 端点(每个实例都是独立服务器,因此 `baseURL` 无法有合理的公共默认值),且引擎不提供按请求控制结果数量的方式(页面大小属于实例配置)。
|
||||
|
||||
## 决策
|
||||
|
||||
新增 `@deepseek-ai/dsh-web-search-searxng`(`packages/web/web-search-searxng`),一个以稳定 id `searxng` 注册、调用 SearXNG 实例 JSON API(`GET {base}/search?q=...&format=json`)的 `WebSearchProvider`。
|
||||
|
||||
- `baseURL` **无默认值**;schema 保持其为可选,因此省略或无法解析的值呈现为不可用提供方(`WEB_PROVIDER_CONFIGURED_UNAVAILABLE`),而非启动错误。产品要求先有证据再采用未经验证的默认端点,且不存在权威的公共 SearXNG 实例。
|
||||
- 配置经由 `web-search-searxng` Settings 命名空间流入(叠加在 cordis 条目之上),提供方**按搜索**投影解析后的分节,因此设置变更在下次调用即生效,无需重新注册——与 `dsh-web-search-deepseek` 的投影拓扑一致。
|
||||
- 提供方**不携带 API 密钥或 `Authorization` 头**:它面向开放实例,因此不会有凭据泄露给重定向目标。HTTP 重定向仍以 `redirect: 'error'` 拒绝,映射为 `WEB_PROVIDER_ERROR`,与 seam 的重定向卫生一致。
|
||||
- 公开 `language`(SearXNG `language`,默认 `auto`)与 `timeRange`(`day`|`week`|`month`|`year`);请求不携带结果数量,因为 API 无法限制数量,因此 seam 的 `maxResults` 截断是唯一上限。省略 `content`(无生成答案)。
|
||||
- 结果映射为 `WebSearchSource`:`url` ← `url`、`title` ← `title`、`snippet` ← `content`。与 Exa 不同,**没有 snippet 的结果会被保留**——URL 与标题仍然有用——因此不会因缺少 snippet 而丢弃来源。
|
||||
|
||||
该包遵循 `dsh-web-search-exa` 的提供方拓扑:函数/命名空间插件(`inject: ['web']`)向 `ctx.web` 注册后端而非拥有 `ctx.web` 键,拥有包级映射与错误转换,并带一个空的包级 invariant 伴生插件。
|
||||
|
||||
## 备选方案
|
||||
|
||||
### 为 `baseURL` 提供公共实例默认值
|
||||
|
||||
拒绝。类似 `https://searx.be` 的默认值会挑选任意、可能不可达或被限速的公共实例,并在未由操作员选择的情况下把流量悄悄发给第三方。要求显式 `baseURL` 会使提供方在配置前不可用,这正是 seam 已命名的诚实误配置信号(`WEB_PROVIDER_CONFIGURED_UNAVAILABLE`)。
|
||||
|
||||
### 通过 OpenAI 兼容网关路由 SearXNG,而非专用提供方
|
||||
|
||||
拒绝。harness 搜索 seam 消费的是 `WebSearchProvider` 后端,而非任意 OpenAI 兼容端点;LLM 提供方配置面并不供给 `ctx.web`。专用 SearXNG `WebSearchProvider` 是不借助自制胶水服务而触达 web 搜索工具的唯一路径。
|
||||
|
||||
## 后果
|
||||
|
||||
- 部署获得自托管搜索后端,其查询发往操作员自己的实例,支持可选的语言与时效过滤器,但没有按查询的结果数量控制——单页结果由 seam 截断。
|
||||
- 提供方集合增至四个搜索后端;`searchProvider`(或 `$DSH_WEB_SEARCH_PROVIDER`)指定 `searxng` 以选择它,或当其是唯一可用提供方时自动选择。
|
||||
- 拒绝重定向意味着返回 3xx 的误配置 `baseURL` 呈现为 `WEB_PROVIDER_ERROR` 而非静默跟随;由于请求无凭据,这是卫生而非密钥保护。
|
||||
- 无 snippet 的结果被保留,因此 `tool-web` 为它们渲染回退的 URL/主机名标签,而非丢弃来源。
|
||||
Reference in New Issue
Block a user