chore: isolate shared dsh plugins into independent monorepo
This commit is contained in:
8
packages/telegram-remote/.gitignore
vendored
Normal file
8
packages/telegram-remote/.gitignore
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
node_modules/
|
||||
*.log
|
||||
.DS_Store
|
||||
*.token
|
||||
*.lock
|
||||
*state.json
|
||||
.env
|
||||
|
||||
135
packages/telegram-remote/README.md
Normal file
135
packages/telegram-remote/README.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# 🤖 dsh-telegram-remote
|
||||
|
||||
**Full remote control + live state visibility for the [DeepSeek Harness](https://github.com/deepseek-ai/dsh) — from Telegram.**
|
||||
|
||||
Turn any Telegram chat into a pocket terminal for your harness: chat with your AI, watch replies stream live (thinking → tools → text), run commands, manage chats/subagents/goals/jobs, and control every harness feature — all through one bot with zero external services (direct Bot API long polling).
|
||||
|
||||
---
|
||||
|
||||
## ✨ Features
|
||||
|
||||
- 💬 **Chat from Telegram** — send a message, get a live-streamed structured reply (thinking, tools, final answer) edited in place
|
||||
- 🎛 **Full harness control** — 49 commands: chats, models, sessions, subagents, goals, jobs, files, PowerShell, exports, presets, skills, settings, credentials, permissions, and more
|
||||
- 📡 **Live state** — status, running turns, queued messages (steer / edit / remove), background jobs
|
||||
- 🔐 **Permission-aware** — sandbox read/write/full control per chat, approval buttons for risky tools
|
||||
- 🔔 **Notifications** — per-chat on/off/all, background activity pushed to you
|
||||
- 🔄 **Hot reload** — plugin edits hot-reload with no restart (HMR)
|
||||
- 🔒 **Single-instance** — a lock file guarantees exactly one poller per bot token (no Telegram 409s)
|
||||
- 🖥 **Works everywhere** — long polling means no public IP / no webhooks / works behind NAT
|
||||
|
||||
## 📦 Requirements
|
||||
|
||||
- Node.js **>= 22** (uses global `fetch`)
|
||||
- A [DeepSeek Harness](https://github.com/deepseek-ai/dsh) installation with a profile (e.g. `web`)
|
||||
- A Telegram bot token from [@BotFather](https://t.me/BotFather)
|
||||
|
||||
## 🚀 Installation
|
||||
|
||||
1. **Install the plugin** into your harness profiles:
|
||||
|
||||
```powershell
|
||||
# from your harness profile dir (e.g. ~/.dsh/profiles/web)
|
||||
npm install link:path/to/dsh-telegram-remote
|
||||
```
|
||||
|
||||
Or add it to the profile's `package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": {
|
||||
"dsh-telegram-remote": "link:C:/path/to/dsh-telegram-remote"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Add the plugin to the profile patch** (`cordis.patch.yml`):
|
||||
|
||||
```yaml
|
||||
- insert:
|
||||
- id: telegram-remote
|
||||
name: 'dsh-telegram-remote'
|
||||
config:
|
||||
tokenFile: 'C:\path\to\telegram.token' # file containing the bot token
|
||||
ownerChatId: 123456789 # YOUR telegram user id (get it from /whoami)
|
||||
workspaceRoot: 'C:\path\to\workspace'
|
||||
allowEval: true
|
||||
notifyOnStartup: true
|
||||
```
|
||||
|
||||
3. **Create the token file** (or set `tokenEnv` instead):
|
||||
|
||||
```powershell
|
||||
Set-Content -Path telegram.token -Value "123456:ABC-your-bot-token" -NoNewline
|
||||
```
|
||||
|
||||
4. **Start the harness**, open the chat with your bot, and send `/start`.
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
| Key | Default | Description |
|
||||
|---|---|---|
|
||||
| `botToken` | `""` | Token directly (alternative to a file/env) |
|
||||
| `tokenEnv` | `"TELEGRAM_BOT_TOKEN"` | Env var holding the token |
|
||||
| `tokenFile` | `""` | File containing the token |
|
||||
| `ownerChatId` | `undefined` | Your Telegram user id — full access |
|
||||
| `allowedUserIds` | `[]` | Extra users allowed to chat |
|
||||
| `workspaceRoot` | `process.cwd()` | Where `/new` chats start |
|
||||
| `allowEval` | `true` | Enable `/eval` (runs JS in the harness) |
|
||||
| `notifyOnStartup` | `true` | Send a "bot online" message on boot |
|
||||
| `stateFile` / `logFile` | `~/.dsh/...` | Override runtime state / log paths |
|
||||
|
||||
## 💬 Usage
|
||||
|
||||
Just **type a message** — it goes to your AI and the reply streams back:
|
||||
|
||||
```
|
||||
💭 Thinking
|
||||
<live reasoning…>
|
||||
|
||||
🔧 Tools
|
||||
⋯ write ✅ read
|
||||
|
||||
🤖 Reply
|
||||
<live streaming text…>
|
||||
|
||||
─────── ⋆⋅☆⋅⋆ ───────
|
||||
⏳ 34s
|
||||
```
|
||||
|
||||
Tap the **/start** keyboard or type `/help` for the full command list. Key commands:
|
||||
|
||||
- `/chats` · `/new` · `/open` — manage chats (`/chats` shows 10 at a time, `/chats 2` pages; subagent chats are hidden)
|
||||
- `/model` · `/models` — switch AI models (`/model max` for deepest thinking)
|
||||
- `/status` — what's happening right now
|
||||
- `/queue` `/steer` `/edit` `/remove` — control queued messages
|
||||
- `/cmd` `/fs` `/mkdir` — run commands and manage files
|
||||
- `/agents` `/send` `/interrupt` — subagents
|
||||
- `/goal` — long-running objectives
|
||||
- `/jobs` `/kill` — background tasks
|
||||
- `/export` `/search` `/archive` — conversation history
|
||||
- `/permission` — sandbox mode per chat
|
||||
- `/eval` `/raw` `/api` — power-user harness access
|
||||
- `/reboot` `/shutdown` — harness lifecycle (graceful)
|
||||
|
||||
## 🔒 Security notes
|
||||
|
||||
- The bot can control your computer — **only add users you trust** (`ownerChatId` / `allowedUserIds`).
|
||||
- Every request is checked against the per-chat `/permission` mode; risky tools request approval.
|
||||
- `/eval` and `/cmd` are powerful — consider `allowEval: false` unless you need them.
|
||||
- No telemetry, no external services: the bot talks to Telegram and your harness only.
|
||||
|
||||
## 🔄 Hot reload (development)
|
||||
|
||||
With the harness HMR enabled, editing this plugin's `lib/*.js` files reloads it in ~2s:
|
||||
|
||||
```yaml
|
||||
- id: hmr
|
||||
disabled: false
|
||||
config:
|
||||
root: ['C:\path\to\dsh-telegram-remote']
|
||||
ignored: []
|
||||
```
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT
|
||||
38
packages/telegram-remote/package.json
Normal file
38
packages/telegram-remote/package.json
Normal file
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-telegram-remote",
|
||||
"version": "0.1.0",
|
||||
"description": "Remote control + full state visibility for the DeepSeek Harness over Telegram Bot API (long polling, no external services).",
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"exports": {
|
||||
".": "./lib/index.js",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"README.md"
|
||||
],
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=22.19"
|
||||
},
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"registry": "https://git.byte-mate.ru/api/packages/Coder/npm/",
|
||||
"tag": "latest"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://git.byte-mate.ru/Coder/dsh-plugins.git",
|
||||
"directory": "packages/telegram-remote"
|
||||
},
|
||||
"keywords": [
|
||||
"deepseek",
|
||||
"dsh",
|
||||
"telegram",
|
||||
"bot",
|
||||
"remote-control",
|
||||
"cordis",
|
||||
"plugin"
|
||||
]
|
||||
}
|
||||
64
packages/tool-lab/package.json
Normal file
64
packages/tool-lab/package.json
Normal file
@@ -0,0 +1,64 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-tool-lab",
|
||||
"description": "Model-facing laboratory tools over the LAN media lab: ComfyUI image generation, Docling PDF OCR, and Whishper speech-to-text",
|
||||
"version": "0.1.0-rc.7",
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"registry": "https://git.byte-mate.ru/api/packages/Coder/npm/",
|
||||
"tag": "rc"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://git.byte-mate.ru/Coder/dsh-plugins.git",
|
||||
"directory": "packages/tool-lab"
|
||||
},
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib"
|
||||
],
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=22.19"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.json",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/cordis": "^4.0.1",
|
||||
"@deepseek-ai/dsh-fs": "^0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-invariants": "^0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-timeout": "^0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-tools": "^0.1.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/schemastery": "^3.18.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/cordis": "4.0.1",
|
||||
"@deepseek-ai/schemastery": "3.18.1",
|
||||
"@deepseek-ai/dsh-fs": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-invariants": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-timeout": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-tools": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-llm": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-sandbox": "0.1.0-rc.7",
|
||||
"typescript": "^6.0.3",
|
||||
"vitest": "^4.1.8",
|
||||
"@types/node": "^22.20.0"
|
||||
}
|
||||
}
|
||||
203
packages/tool-lab/src/comfy.ts
Normal file
203
packages/tool-lab/src/comfy.ts
Normal file
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* ComfyUI image-generation tool. Submits an SDXL-turbo workflow to a shared
|
||||
* ComfyUI instance, polls `/history/{prompt_id}` until the SaveImage node
|
||||
* produced a file, and returns the served PNG URL. Services are unauthenticated.
|
||||
* @module @deepseek-ai/dsh-tool-lab/comfy
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import { HttpError, deadline, sleep } from './helpers.ts'
|
||||
|
||||
/** Default SDXL-turbo checkpoint on the lab server. */
|
||||
export const DEFAULT_MODEL = 'sdxl_turbo.safetensors'
|
||||
/** Default negative prompt applied unless the caller overrides it. */
|
||||
export const DEFAULT_NEGATIVE = 'negative prompt'
|
||||
/** Default output filename prefix for SaveImage. */
|
||||
export const DEFAULT_FILENAME_PREFIX = 'dsh'
|
||||
/** Default step count for the SDXL-turbo Euler-normal pass. */
|
||||
export const DEFAULT_STEPS = 4
|
||||
/** Default generation dimensions when neither axis is supplied. */
|
||||
export const DEFAULT_SIZE = 512
|
||||
/** One history poll interval (ms) while awaiting the rendered image. */
|
||||
export const POLL_INTERVAL_MS = 1_000
|
||||
|
||||
/** Schema-validated arguments for `lab_generate_image`. */
|
||||
export interface GenerateImageArgs {
|
||||
prompt: string
|
||||
negative?: string
|
||||
width?: number
|
||||
height?: number
|
||||
steps?: number
|
||||
seed?: number
|
||||
model?: string
|
||||
filename_prefix?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and resolve optional image arguments against defaults. Positive
|
||||
* integer constraints the schema DSL cannot express are enforced here.
|
||||
*/
|
||||
export function resolveImageArgs(args: GenerateImageArgs): Required<GenerateImageArgs> {
|
||||
const resolved = {
|
||||
prompt: args.prompt,
|
||||
negative: args.negative ?? DEFAULT_NEGATIVE,
|
||||
width: args.width ?? DEFAULT_SIZE,
|
||||
height: args.height ?? DEFAULT_SIZE,
|
||||
steps: args.steps ?? DEFAULT_STEPS,
|
||||
seed: args.seed ?? 123,
|
||||
model: args.model ?? DEFAULT_MODEL,
|
||||
filename_prefix: args.filename_prefix ?? DEFAULT_FILENAME_PREFIX,
|
||||
}
|
||||
for (const [name, value] of [
|
||||
['width', resolved.width],
|
||||
['height', resolved.height],
|
||||
['steps', resolved.steps],
|
||||
['seed', resolved.seed],
|
||||
] as const) {
|
||||
if (!Number.isInteger(value) || value < 1) throw new Error(`lab_generate_image: ${name} must be a positive integer`)
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
/**
|
||||
* Build an SDXL-turbo ComfyUI workflow for a 4-step Euler-normal pass. The
|
||||
* node graph is the lab's validated shape; only the seed, dimensions, step
|
||||
* count, prompts, and output prefix vary per call.
|
||||
*/
|
||||
export function buildWorkflow(opts: {
|
||||
prompt: string
|
||||
negative: string
|
||||
width: number
|
||||
height: number
|
||||
steps: number
|
||||
seed: number
|
||||
model: string
|
||||
filenamePrefix: string
|
||||
}): Record<string, unknown> {
|
||||
const { prompt, negative, width, height, steps, seed, model, filenamePrefix } = opts
|
||||
return {
|
||||
'3': {
|
||||
class_type: 'KSampler',
|
||||
inputs: {
|
||||
seed,
|
||||
steps,
|
||||
cfg: 1.0,
|
||||
sampler_name: 'euler',
|
||||
scheduler: 'normal',
|
||||
denoise: 1.0,
|
||||
model: ['4', 0],
|
||||
positive: ['6', 0],
|
||||
negative: ['7', 0],
|
||||
latent_image: ['5', 0],
|
||||
},
|
||||
},
|
||||
'4': { class_type: 'CheckpointLoaderSimple', inputs: { ckpt_name: model } },
|
||||
'5': { class_type: 'EmptyLatentImage', inputs: { width, height, batch_size: 1 } },
|
||||
'6': { class_type: 'CLIPTextEncode', inputs: { text: prompt, clip: ['4', 1] } },
|
||||
'7': { class_type: 'CLIPTextEncode', inputs: { text: negative, clip: ['4', 1] } },
|
||||
'8': { class_type: 'VAEDecode', inputs: { samples: ['3', 0], vae: ['4', 2] } },
|
||||
'9': { class_type: 'SaveImage', inputs: { filename_prefix: filenamePrefix, images: ['8', 0] } },
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute one SDXL-turbo generation against a ComfyUI server: queue the
|
||||
* workflow, poll history for the rendered image, and return its `GET /view` URL.
|
||||
*
|
||||
* @param baseUrl - the ComfyUI server base URL, e.g. `http://192.168.31.240:8188`.
|
||||
* @param resolved - schema-validated, default-resolved generation arguments.
|
||||
* @param signal - the executor's cancellation signal (fused into the deadline).
|
||||
* @param timeoutMs - cooperative timeout budget for the whole call.
|
||||
* @returns the served PNG URL.
|
||||
*/
|
||||
export async function runComfyImage(
|
||||
baseUrl: string,
|
||||
resolved: Required<GenerateImageArgs>,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
): Promise<string> {
|
||||
using d = deadline(signal, timeoutMs, 'LAB_TOOL_TIMEOUT')
|
||||
|
||||
let response = await fetch(`${baseUrl}/prompt`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt: buildWorkflow({
|
||||
prompt: resolved.prompt,
|
||||
negative: resolved.negative,
|
||||
width: resolved.width,
|
||||
height: resolved.height,
|
||||
steps: resolved.steps,
|
||||
seed: resolved.seed,
|
||||
model: resolved.model,
|
||||
filenamePrefix: resolved.filename_prefix,
|
||||
}),
|
||||
client_id: 'dsh',
|
||||
}),
|
||||
signal: d.signal,
|
||||
})
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_generate_image: ComfyUI prompt failed (HTTP ${response.status})`, response.status, 'LAB_COMFY_HTTP')
|
||||
}
|
||||
const queued = await response.json() as { prompt_id?: string; node_errors?: Record<string, unknown> }
|
||||
const nodeErrors = queued.node_errors ?? {}
|
||||
if (Object.keys(nodeErrors).length > 0) throw new Error(`lab_generate_image: workflow rejected: ${JSON.stringify(nodeErrors)}`)
|
||||
const promptId = queued.prompt_id
|
||||
if (promptId === undefined) throw new Error('lab_generate_image: /prompt returned no prompt_id')
|
||||
|
||||
while (true) {
|
||||
d.signal.throwIfAborted()
|
||||
response = await fetch(`${baseUrl}/history/${encodeURIComponent(promptId)}`, { signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_generate_image: ComfyUI history failed (HTTP ${response.status})`, response.status, 'LAB_COMFY_HTTP')
|
||||
}
|
||||
const history = await response.json() as Record<string, { outputs?: Record<string, { images?: { filename?: string }[] }> }>
|
||||
const images = history[promptId]?.outputs?.['9']?.images ?? []
|
||||
const filename = images[0]?.filename
|
||||
if (filename !== undefined) {
|
||||
return `${baseUrl}/view?filename=${encodeURIComponent(filename)}&subfolder=&type=output`
|
||||
}
|
||||
await sleep(POLL_INTERVAL_MS, d.signal)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `lab_generate_image` tool.
|
||||
*
|
||||
* @param ctx - context whose `tools` registry receives the definition
|
||||
* (effect-scoped; unregistered on plugin dispose).
|
||||
* @param baseUrl - the ComfyUI server base URL.
|
||||
* @param timeoutMs - cooperative timeout budget attached as the tool's `timeoutMs`.
|
||||
* @param maxOutputChars - output cap; image generation returns a URL and ignores this.
|
||||
*/
|
||||
export function registerLabComfyTool(ctx: Context, baseUrl: string, timeoutMs: number, maxOutputChars: number): void {
|
||||
void maxOutputChars
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'lab_generate_image',
|
||||
description:
|
||||
'Generate an image with the shared ComfyUI lab server (SDXL-turbo, 4 steps, euler/normal, no auth). '
|
||||
+ 'Returns the served PNG URL. Polls the render up to the tool timeout; use the default size for fastest results.',
|
||||
parameters: {
|
||||
prompt: { type: 'string', required: true, description: 'The positive text prompt describing the image.' },
|
||||
negative: { type: 'string', description: `Negative prompt; defaults to "${DEFAULT_NEGATIVE}".` },
|
||||
width: { type: 'integer', description: `Output width; defaults to ${DEFAULT_SIZE}.` },
|
||||
height: { type: 'integer', description: `Output height; defaults to ${DEFAULT_SIZE}.` },
|
||||
steps: { type: 'integer', description: `Sampling steps; defaults to ${DEFAULT_STEPS} (SDXL-turbo).` },
|
||||
seed: { type: 'integer', description: 'Sampling seed; defaults to 123.' },
|
||||
model: { type: 'string', description: `Checkpoint name on the server; defaults to "${DEFAULT_MODEL}".` },
|
||||
filename_prefix: { type: 'string', description: `Output filename prefix; defaults to "${DEFAULT_FILENAME_PREFIX}".` },
|
||||
},
|
||||
output: {
|
||||
schema: { type: 'string' },
|
||||
render: (_args, value) => [{ type: 'text', text: String(value) }],
|
||||
},
|
||||
timeoutMs,
|
||||
// A shared GPU render is idempotent from the caller's perspective: sibling
|
||||
// calls mutate no parent-owned state beyond the lab queue.
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
return runComfyImage(baseUrl, resolveImageArgs(args as GenerateImageArgs), exec.signal, timeoutMs)
|
||||
},
|
||||
}))
|
||||
}
|
||||
150
packages/tool-lab/src/docling.ts
Normal file
150
packages/tool-lab/src/docling.ts
Normal file
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Docling PDF-OCR tool. Uploads a PDF through Docling's multipart async
|
||||
* conversion endpoint, polls the task status, and returns the extracted
|
||||
* markdown. Services are unauthenticated.
|
||||
* @module @deepseek-ai/dsh-tool-lab/docling
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { FsTarget } from '@deepseek-ai/dsh-fs'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import { HttpError, deadline, sleep } from './helpers.ts'
|
||||
|
||||
/** Docling `/v1/status/poll` wait hint (seconds) per poll. */
|
||||
export const DOCLING_POLL_WAIT_S = 2
|
||||
/** One transport retry interval (ms) between status polls. */
|
||||
export const DOCLING_POLL_INTERVAL_MS = 2_000
|
||||
|
||||
/** Schema-validated arguments for `lab_ocr_pdf`. */
|
||||
export interface OcrPdfArgs {
|
||||
file_path: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Upload a PDF blob and poll the Docling task until it succeeds, returning
|
||||
* the extracted markdown (`document.md_content`).
|
||||
*
|
||||
* @param baseUrl - the Docling server base URL, e.g. `http://192.168.31.159:5001`.
|
||||
* @param bytes - the PDF file bytes read through `ctx.fs.readBytes`.
|
||||
* @param filename - the file name sent in the multipart upload.
|
||||
* @param signal - the executor's cancellation signal (deadline-fused).
|
||||
* @param timeoutMs - cooperative timeout budget for the whole call.
|
||||
* @returns the extracted markdown text.
|
||||
*/
|
||||
export async function runDoclingOcr(
|
||||
baseUrl: string,
|
||||
bytes: Uint8Array,
|
||||
filename: string,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
): Promise<string> {
|
||||
using d = deadline(signal, timeoutMs, 'LAB_TOOL_TIMEOUT')
|
||||
|
||||
const form = new FormData()
|
||||
form.append('files', new Blob([bytes as BlobPart], { type: 'application/pdf' }), filename)
|
||||
|
||||
let response = await fetch(`${baseUrl}/v1/convert/file/async`, { method: 'POST', body: form, signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_ocr_pdf: Docling convert failed (HTTP ${response.status})`, response.status, 'LAB_DOCLING_HTTP')
|
||||
}
|
||||
const queued = await response.json() as { task_id?: string; task_status?: string }
|
||||
const taskId = queued.task_id
|
||||
if (taskId === undefined) throw new Error('lab_ocr_pdf: /v1/convert/file/async returned no task_id')
|
||||
|
||||
while (true) {
|
||||
d.signal.throwIfAborted()
|
||||
response = await fetch(`${baseUrl}/v1/status/poll/${encodeURIComponent(taskId)}?wait=${DOCLING_POLL_WAIT_S}`, { signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_ocr_pdf: Docling status failed (HTTP ${response.status})`, response.status, 'LAB_DOCLING_HTTP')
|
||||
}
|
||||
const status = await response.json() as { task_status?: string }
|
||||
if (status.task_status === 'success') break
|
||||
if (status.task_status === 'failure' || status.task_status === 'cancelled' || status.task_status === 'error') {
|
||||
throw new Error(`lab_ocr_pdf: Docling task ended with status "${status.task_status}"`)
|
||||
}
|
||||
await sleep(DOCLING_POLL_INTERVAL_MS, d.signal)
|
||||
}
|
||||
|
||||
d.signal.throwIfAborted()
|
||||
response = await fetch(`${baseUrl}/v1/result/${encodeURIComponent(taskId)}`, { signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_ocr_pdf: Docling result failed (HTTP ${response.status})`, response.status, 'LAB_DOCLING_HTTP')
|
||||
}
|
||||
const result = await response.json() as { status?: string; errors?: unknown[]; document?: { md_content?: string } }
|
||||
if (result.status !== 'success') throw new Error(`lab_ocr_pdf: Docling result status "${result.status ?? 'unknown'}"`)
|
||||
const markdown = result.document?.md_content
|
||||
if (markdown === undefined) throw new Error('lab_ocr_pdf: Docling result carried no document.md_content')
|
||||
return markdown
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `lab_ocr_pdf` tool. Reads the file as bytes via `ctx.fs`
|
||||
* (`resolve` then `readBytes`, bounded by `maxBytes`) and uploads it.
|
||||
*
|
||||
* @param ctx - context whose `tools` registry receives the definition and whose
|
||||
* `fs` provides file reads.
|
||||
* @param baseUrl - the Docling server base URL.
|
||||
* @param timeoutMs - cooperative timeout budget attached as the tool's `timeoutMs`.
|
||||
* @param maxBytes - inclusive cap on the PDF bytes read from disk.
|
||||
* @param maxOutputChars - cap on the returned markdown; longer output is truncated.
|
||||
*/
|
||||
export function registerLabDoclingTool(
|
||||
ctx: Context,
|
||||
baseUrl: string,
|
||||
timeoutMs: number,
|
||||
maxBytes: number,
|
||||
maxOutputChars: number,
|
||||
): void {
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'lab_ocr_pdf',
|
||||
description:
|
||||
'OCR a PDF file and return the extracted markdown text via the shared Docling server (no auth). '
|
||||
+ `The file is read from disk up to ${formatBytes(maxBytes)} and uploaded as multipart; the tool polls until OCR succeeds or times out.`,
|
||||
parameters: {
|
||||
file_path: { type: 'string', required: true, description: 'Path to the PDF file to OCR.' },
|
||||
},
|
||||
output: {
|
||||
schema: { type: 'string' },
|
||||
render: (_args, value) => [{ type: 'text', text: String(value) }],
|
||||
},
|
||||
timeoutMs,
|
||||
// Reads and an upload to the shared OCR queue: no parent-owned mutation.
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
const target = await ctx.fs.resolve(args.file_path, { signal: exec.signal })
|
||||
return runDoclingOcrFromTarget(
|
||||
baseUrl,
|
||||
ctx,
|
||||
target,
|
||||
args.file_path,
|
||||
exec.signal,
|
||||
timeoutMs,
|
||||
maxBytes,
|
||||
maxOutputChars,
|
||||
)
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
/** Read the resolved target bytes and run the Docling OCR flow. */
|
||||
async function runDoclingOcrFromTarget(
|
||||
baseUrl: string,
|
||||
ctx: Context,
|
||||
target: FsTarget,
|
||||
displayPath: string,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
maxBytes: number,
|
||||
maxOutputChars: number,
|
||||
): Promise<string> {
|
||||
const bytes = await ctx.fs.readBytes(target, signal, maxBytes)
|
||||
const filename = displayPath.split(/[\\/]/).pop() ?? 'input.pdf'
|
||||
const markdown = await runDoclingOcr(baseUrl, bytes, filename, signal, timeoutMs)
|
||||
if (markdown.length <= maxOutputChars) return markdown
|
||||
return `${markdown.slice(0, maxOutputChars)}\n\n(OCR output truncated.)`
|
||||
}
|
||||
|
||||
/** Human-readable byte bound for the tool description. */
|
||||
function formatBytes(bytes: number): string {
|
||||
return bytes >= 1024 * 1024 ? `${Math.floor(bytes / (1024 * 1024))}MB` : `${bytes}B`
|
||||
}
|
||||
68
packages/tool-lab/src/helpers.ts
Normal file
68
packages/tool-lab/src/helpers.ts
Normal file
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Shared cooperative utilities for the lab tools: a signal-fused deadline, an
|
||||
* abortable sleep, and a structured HTTP/upstream error carrier. Deadline and
|
||||
* sleep cooperate with the caller's cancellation signal so a timed-out or
|
||||
* cancelled call unwinds promptly instead of blocking on a hung server.
|
||||
* @module @deepseek-ai/dsh-tool-lab/helpers
|
||||
*/
|
||||
|
||||
import { deadline as makeDeadline } from '@deepseek-ai/dsh-timeout'
|
||||
import type { Deadline } from '@deepseek-ai/dsh-timeout'
|
||||
|
||||
/**
|
||||
* Cooperative classification error for lab-service HTTP failures. Carries the
|
||||
* HTTP status (or `undefined` for transport failures) and a stable code so
|
||||
* callers and diagnostics can distinguish a rejected workflow, an unreachable
|
||||
* server, and a timed-out request without reparsing the message.
|
||||
*/
|
||||
export class HttpError extends Error {
|
||||
override name = 'HttpError'
|
||||
|
||||
constructor(
|
||||
message: string,
|
||||
readonly status: number | undefined,
|
||||
readonly code: string,
|
||||
) {
|
||||
super(message)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fuse the executor's cancellation signal with a per-call timeout budget.
|
||||
* The returned deadline aborts on upstream cancellation OR on timeout (the
|
||||
* timeout carries a {@link TimeoutReason} with the given code). `using` clears
|
||||
* the timer at scope exit.
|
||||
*
|
||||
* @param signal - the executor's cancellation signal, if any.
|
||||
* @param timeoutMs - cooperative timeout budget in milliseconds.
|
||||
* @param code - capability-owned code stamped onto the timeout reason.
|
||||
* @returns the fused deadline (signal + timer cleanup).
|
||||
*/
|
||||
export function deadline(signal: AbortSignal | undefined, timeoutMs: number, code: string): Deadline {
|
||||
return makeDeadline(signal, timeoutMs, code)
|
||||
}
|
||||
|
||||
/**
|
||||
* Suspend the current coroutine for `ms` milliseconds, returning early when
|
||||
* the signal aborts. Throws the abort once registered; cooperative callers
|
||||
* check `signal.aborted` or let the next upstream call raise.
|
||||
*
|
||||
* @param ms - sleep duration in milliseconds.
|
||||
* @param signal - optional signal to observe.
|
||||
*/
|
||||
export async function sleep(ms: number, signal: AbortSignal | undefined): Promise<void> {
|
||||
if (ms <= 0) return
|
||||
if (signal !== undefined && signal.aborted) signal.throwIfAborted()
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
setTimeout(resolve, ms)
|
||||
return
|
||||
}
|
||||
const onAbort = (): void => reject(signal.reason)
|
||||
signal.addEventListener('abort', onAbort, { once: true })
|
||||
setTimeout(() => {
|
||||
signal.removeEventListener('abort', onAbort)
|
||||
resolve()
|
||||
}, ms)
|
||||
})
|
||||
}
|
||||
90
packages/tool-lab/src/index.ts
Normal file
90
packages/tool-lab/src/index.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* Model-facing lab tools over the LAN media lab: ComfyUI image generation,
|
||||
* Docling PDF OCR, and Whishper speech-to-text. This package owns schemas,
|
||||
* validation, defaults, and presentation; each service module owns its HTTP
|
||||
* flow. Enablement is config-driven; the three tools register together.
|
||||
* @module @deepseek-ai/dsh-tool-lab
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { registerLabComfyTool } from './comfy.ts'
|
||||
import { registerLabDoclingTool } from './docling.ts'
|
||||
import { registerLabWhishTool } from './whish.ts'
|
||||
|
||||
export { DEFAULT_MODEL as COMFY_DEFAULT_MODEL, DEFAULT_NEGATIVE as COMFY_DEFAULT_NEGATIVE } from './comfy.ts'
|
||||
export type { GenerateImageArgs as ComfyGenerateImageArgs } from './comfy.ts'
|
||||
export type { OcrPdfArgs as DoclingOcrPdfArgs } from './docling.ts'
|
||||
export type { TranscribeAudioArgs as WhishTranscribeAudioArgs } from './whish.ts'
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'lab'
|
||||
|
||||
/** Services required by the lab tool suite. */
|
||||
export const inject = ['tools', 'fs']
|
||||
|
||||
/** Default ComfyUI server base URL. */
|
||||
export const DEFAULT_COMFY_BASE_URL = 'http://192.168.31.240:8188'
|
||||
/** Default Docling server base URL. */
|
||||
export const DEFAULT_DOCLING_BASE_URL = 'http://192.168.31.159:5001'
|
||||
/** Default Whishper server base URL. */
|
||||
export const DEFAULT_WHISH_BASE_URL = 'http://192.168.31.159:8082'
|
||||
/** Default per-call cooperative timeout budget. */
|
||||
export const DEFAULT_TIMEOUT_MS = 120_000
|
||||
/** Default cap on one uploaded file. */
|
||||
export const DEFAULT_MAX_UPLOAD_BYTES = 20 * 1024 * 1024
|
||||
/** Default cap on one tool's text output. */
|
||||
export const DEFAULT_MAX_OUTPUT_CHARS = 200_000
|
||||
|
||||
/** Plugin config: lab service endpoints and the shared tool bounds. */
|
||||
export interface Config {
|
||||
/** ComfyUI server base URL. */
|
||||
comfyBaseUrl?: string
|
||||
/** Docling server base URL. */
|
||||
doclingBaseUrl?: string
|
||||
/** Whishper server base URL. */
|
||||
whishBaseUrl?: string
|
||||
/** Cooperative timeout budget (ms) for one lab tool call. */
|
||||
timeoutMs?: number
|
||||
/** Inclusive cap on one uploaded file (bytes). */
|
||||
maxUploadBytes?: number
|
||||
/** Cap on one tool's text output (characters). */
|
||||
maxOutputChars?: number
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
comfyBaseUrl: z.string().default(DEFAULT_COMFY_BASE_URL),
|
||||
doclingBaseUrl: z.string().default(DEFAULT_DOCLING_BASE_URL),
|
||||
whishBaseUrl: z.string().default(DEFAULT_WHISH_BASE_URL),
|
||||
timeoutMs: z.number().default(DEFAULT_TIMEOUT_MS),
|
||||
maxUploadBytes: z.number().default(DEFAULT_MAX_UPLOAD_BYTES),
|
||||
maxOutputChars: z.number().default(DEFAULT_MAX_OUTPUT_CHARS),
|
||||
})
|
||||
|
||||
/** Complete config after schemastery applies every field default. */
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/** Configured timeouts and caps must be positive integers. */
|
||||
function assertPositiveInteger(name: string, value: number): void {
|
||||
if (!Number.isInteger(value) || value < 1) throw new Error(`tool-lab: ${name} must be a positive integer`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the three lab tools. Each tool's cooperative timeout budget
|
||||
* (`timeoutMs`) and file/output bounds come from config and are attached to
|
||||
* the definition for `@deepseek-ai/dsh-tool-call-timeout-policy` to enforce.
|
||||
* The effect-based registry cleanup means no manual teardown is needed.
|
||||
*/
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const resolved = config as ResolvedConfig
|
||||
for (const [key, value] of [
|
||||
['timeoutMs', resolved.timeoutMs],
|
||||
['maxUploadBytes', resolved.maxUploadBytes],
|
||||
['maxOutputChars', resolved.maxOutputChars],
|
||||
] as const) {
|
||||
assertPositiveInteger(key, value)
|
||||
}
|
||||
registerLabComfyTool(ctx, resolved.comfyBaseUrl, resolved.timeoutMs, resolved.maxOutputChars)
|
||||
registerLabDoclingTool(ctx, resolved.doclingBaseUrl, resolved.timeoutMs, resolved.maxUploadBytes, resolved.maxOutputChars)
|
||||
registerLabWhishTool(ctx, resolved.whishBaseUrl, resolved.timeoutMs, resolved.maxUploadBytes, resolved.maxOutputChars)
|
||||
}
|
||||
30
packages/tool-lab/src/invariant.ts
Normal file
30
packages/tool-lab/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-tool-lab`.
|
||||
* @module @deepseek-ai/dsh-tool-lab/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-tool-lab'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'tool-lab-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this model-facing adapter has no independent lifecycle
|
||||
* stream; execution relations are owned by the capability seams it calls.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
153
packages/tool-lab/src/whish.ts
Normal file
153
packages/tool-lab/src/whish.ts
Normal file
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* Whishper speech-to-text tool. Uploads an audio file through Whishper's
|
||||
* multipart `/api/transcriptions` endpoint, polls until a transcript is ready,
|
||||
* and returns `result.text`. Services are unauthenticated.
|
||||
* @module @deepseek-ai/dsh-tool-lab/whish
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { FsTarget } from '@deepseek-ai/dsh-fs'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import { HttpError, deadline, sleep } from './helpers.ts'
|
||||
|
||||
/** One transcription status poll interval (ms). */
|
||||
export const WHISH_POLL_INTERVAL_MS = 2_000
|
||||
/** Default Whishper model size when the caller omits it. */
|
||||
export const WHISH_DEFAULT_MODEL = 'base'
|
||||
|
||||
/** Schema-validated arguments for `lab_transcribe_audio`. */
|
||||
export interface TranscribeAudioArgs {
|
||||
file_path: string
|
||||
model_size?: string
|
||||
language?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Upload an audio blob and poll Whishper until the transcript text is ready,
|
||||
* returning it. A queued item reports `status: -1`; once `status >= 0` AND
|
||||
* `result.text` is non-empty the transcript is done.
|
||||
*
|
||||
* @param baseUrl - the Whishper server base URL, e.g. `http://192.168.31.159:8082`.
|
||||
* @param bytes - the audio bytes read through `ctx.fs.readBytes`.
|
||||
* @param filename - the file name sent in the multipart upload.
|
||||
* @param modelSize - optional model size hint.
|
||||
* @param language - optional spoken-language hint.
|
||||
* @param signal - the executor's cancellation signal (deadline-fused).
|
||||
* @param timeoutMs - cooperative timeout budget for the whole call.
|
||||
* @returns the transcribed text.
|
||||
*/
|
||||
export async function runWhishTranscribe(
|
||||
baseUrl: string,
|
||||
bytes: Uint8Array,
|
||||
filename: string,
|
||||
modelSize: string | undefined,
|
||||
language: string | undefined,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
): Promise<string> {
|
||||
using d = deadline(signal, timeoutMs, 'LAB_TOOL_TIMEOUT')
|
||||
|
||||
const form = new FormData()
|
||||
form.append('files', new Blob([bytes as BlobPart]), filename)
|
||||
if (modelSize !== undefined) form.append('model_size', modelSize)
|
||||
if (language !== undefined) form.append('language', language)
|
||||
|
||||
let response = await fetch(`${baseUrl}/api/transcriptions`, { method: 'POST', body: form, signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_transcribe_audio: Whishper upload failed (HTTP ${response.status})`, response.status, 'LAB_WHISH_HTTP')
|
||||
}
|
||||
const queued = await response.json() as { id?: string; status?: number }
|
||||
const id = queued.id
|
||||
if (id === undefined) throw new Error('lab_transcribe_audio: /api/transcriptions returned no id')
|
||||
|
||||
while (true) {
|
||||
d.signal.throwIfAborted()
|
||||
response = await fetch(`${baseUrl}/api/transcriptions/${encodeURIComponent(String(id))}`, { signal: d.signal })
|
||||
if (!response.ok) {
|
||||
throw new HttpError(`lab_transcribe_audio: Whishper status failed (HTTP ${response.status})`, response.status, 'LAB_WISH_HTTP')
|
||||
}
|
||||
const state = await response.json() as { status?: number; result?: { text?: string } }
|
||||
const status = state.status ?? 0
|
||||
const text = state.result?.text
|
||||
if (status >= 0 && text !== undefined && text.trim().length > 0) return text
|
||||
await sleep(WHISH_POLL_INTERVAL_MS, d.signal)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the `lab_transcribe_audio` tool. Reads the audio file as bytes via
|
||||
* `ctx.fs` (`resolve` then `readBytes`, bounded by `maxBytes`) and uploads it.
|
||||
*
|
||||
* @param ctx - context whose `tools` registry receives the definition and whose
|
||||
* `fs` provides file reads.
|
||||
* @param baseUrl - the Whishper server base URL.
|
||||
* @param timeoutMs - cooperative timeout budget attached as the tool's `timeoutMs`.
|
||||
* @param maxBytes - inclusive cap on the audio bytes read from disk.
|
||||
* @param maxOutputChars - cap on the returned transcript; longer text is truncated.
|
||||
*/
|
||||
export function registerLabWhishTool(
|
||||
ctx: Context,
|
||||
baseUrl: string,
|
||||
timeoutMs: number,
|
||||
maxBytes: number,
|
||||
maxOutputChars: number,
|
||||
): void {
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'lab_transcribe_audio',
|
||||
description:
|
||||
'Transcribe speech to text from an audio file via the shared Whishper server (no auth). '
|
||||
+ `The file is read from disk up to ${formatBytes(maxBytes)} and uploaded as multipart; the tool polls until the transcript is ready or times out.`,
|
||||
parameters: {
|
||||
file_path: { type: 'string', required: true, description: 'Path to the audio file to transcribe.' },
|
||||
model_size: { type: 'string', description: `Optional model size hint; defaults to "${WHISH_DEFAULT_MODEL}".` },
|
||||
language: { type: 'string', description: 'Optional spoken-language hint (e.g. "en" or "ru").' },
|
||||
},
|
||||
output: {
|
||||
schema: { type: 'string' },
|
||||
render: (_args, value) => [{ type: 'text', text: String(value) }],
|
||||
},
|
||||
timeoutMs,
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
const target = await ctx.fs.resolve(args.file_path, { signal: exec.signal })
|
||||
return runWhishFromTarget(
|
||||
baseUrl,
|
||||
ctx,
|
||||
target,
|
||||
args.file_path,
|
||||
args.model_size,
|
||||
args.language,
|
||||
exec.signal,
|
||||
timeoutMs,
|
||||
maxBytes,
|
||||
maxOutputChars,
|
||||
)
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
/** Read the resolved target bytes and run the Whishper flow. */
|
||||
async function runWhishFromTarget(
|
||||
baseUrl: string,
|
||||
ctx: Context,
|
||||
target: FsTarget,
|
||||
displayPath: string,
|
||||
modelSize: string | undefined,
|
||||
language: string | undefined,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
maxBytes: number,
|
||||
maxOutputChars: number,
|
||||
): Promise<string> {
|
||||
const bytes = await ctx.fs.readBytes(target, signal, maxBytes)
|
||||
const filename = displayPath.split(/[\\/]/).pop() ?? 'audio.bin'
|
||||
const model = modelSize !== undefined && modelSize.trim().length > 0 ? modelSize.trim() : WHISH_DEFAULT_MODEL
|
||||
const text = await runWhishTranscribe(baseUrl, bytes, filename, model, language, signal, timeoutMs)
|
||||
if (text.length <= maxOutputChars) return text
|
||||
return `${text.slice(0, maxOutputChars)}\n\n(Transcript truncated.)`
|
||||
}
|
||||
|
||||
/** Human-readable byte bound for the tool description. */
|
||||
function formatBytes(bytes: number): string {
|
||||
return bytes >= 1024 * 1024 ? `${Math.floor(bytes / (1024 * 1024))}MB` : `${bytes}B`
|
||||
}
|
||||
162
packages/tool-lab/tests/tool-lab.spec.ts
Normal file
162
packages/tool-lab/tests/tool-lab.spec.ts
Normal file
@@ -0,0 +1,162 @@
|
||||
import { describe, expect, it, vi, afterEach } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import ToolRuntime from '@deepseek-ai/dsh-tools'
|
||||
import { FileSystem } from '@deepseek-ai/dsh-fs'
|
||||
import type { FsTarget, FsDirEntry, FsEditOutcome, FsEditRequest, FsInfo, FsPathInfo, FsWriteIntent, FsWriteOutcome, FsVersion, FsTargetKey } from '@deepseek-ai/dsh-fs'
|
||||
import * as ToolLab from '@deepseek-ai/dsh-tool-lab'
|
||||
import { runComfyImage, resolveImageArgs, buildWorkflow } from '@deepseek-ai/dsh-tool-lab/src/comfy.ts'
|
||||
import { runDoclingOcr } from '@deepseek-ai/dsh-tool-lab/src/docling.ts'
|
||||
import { runWhishTranscribe } from '@deepseek-ai/dsh-tool-lab/src/whish.ts'
|
||||
import type { SandboxExecutionPolicy, SandboxMode } from '@deepseek-ai/dsh-sandbox'
|
||||
|
||||
const testToolSignal = new AbortController().signal
|
||||
|
||||
/** In-memory fake ctx.fs backend for upload tools. */
|
||||
class FakeFs extends FileSystem {
|
||||
files = new Map<string, Uint8Array>()
|
||||
override async resolve(path: string): Promise<FsTarget> {
|
||||
return { targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` }
|
||||
}
|
||||
override processPath(target: FsTarget): string { return String(target.targetKey) }
|
||||
override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` }
|
||||
override contains(parent: FsTarget, child: FsTarget): boolean {
|
||||
return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`)
|
||||
}
|
||||
override async stat(): Promise<FsInfo | undefined> { return undefined }
|
||||
override async lstat(): Promise<FsPathInfo | undefined> { return undefined }
|
||||
override async readText(target: FsTarget): Promise<string> {
|
||||
return new TextDecoder().decode(this.files.get(target.targetKey) ?? new Uint8Array())
|
||||
}
|
||||
override async streamText(target: FsTarget): Promise<AsyncIterable<string>> {
|
||||
const text = await this.readText(target)
|
||||
return (async function* () { yield text })()
|
||||
}
|
||||
override async readBytes(target: FsTarget, _signal: AbortSignal | undefined, maxBytes: number): Promise<Uint8Array> {
|
||||
const bytes = this.files.get(target.targetKey) ?? new Uint8Array()
|
||||
if (bytes.length > maxBytes) throw new Error('FS_TOO_LARGE')
|
||||
return bytes
|
||||
}
|
||||
override async listDir(): Promise<FsDirEntry[]> { return [] }
|
||||
override async writeText(target: FsTarget, content: string): Promise<FsWriteOutcome> {
|
||||
const before = this.files.get(target.targetKey) ?? null
|
||||
this.files.set(target.targetKey, new TextEncoder().encode(content))
|
||||
return { operation: before !== null ? 'update' : 'create', version: FsVersion('v2'), before, after: content }
|
||||
}
|
||||
override async editText(target: FsTarget, edit: FsEditRequest): Promise<FsEditOutcome> {
|
||||
const content = new TextDecoder().decode(this.files.get(target.targetKey) ?? new Uint8Array())
|
||||
const after = content.split(edit.oldString).join(edit.newString)
|
||||
this.files.set(target.targetKey, new TextEncoder().encode(after))
|
||||
return { version: FsVersion('v3'), before: content, after }
|
||||
}
|
||||
}
|
||||
|
||||
async function mount(opts: { files?: Record<string, Uint8Array> } = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(ToolRuntime)
|
||||
const fs = new FakeFs()
|
||||
if (opts.files) for (const [k, v] of Object.entries(opts.files)) fs.files.set(`key:${k}`, v)
|
||||
await ctx.plugin(fs, {})
|
||||
await ctx.plugin(ToolLab, {})
|
||||
let counter = 0
|
||||
const call = (name: string, args: unknown) => ctx.tools.execute({
|
||||
signal: testToolSignal,
|
||||
callId: CallId(`call-${++counter}`),
|
||||
name,
|
||||
arguments: args,
|
||||
})
|
||||
return { ctx, fs, call }
|
||||
}
|
||||
|
||||
function jsonResponse(body: unknown, { status = 200, headers }: { status?: number; headers?: Record<string, string> } = {}): Response {
|
||||
return new Response(JSON.stringify(body), { status, headers: { 'content-type': 'application/json', ...headers } })
|
||||
}
|
||||
|
||||
const pngBytes = new Uint8Array([137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13])
|
||||
|
||||
afterEach(() => { vi.unstubAllGlobals() })
|
||||
|
||||
describe('lab_generate_image (ComfyUI)', () => {
|
||||
it('queues the workflow and returns the served PNG URL after polling history', async () => {
|
||||
const calls: string[] = []
|
||||
vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL) => {
|
||||
const url = String(input)
|
||||
calls.push(url)
|
||||
if (url.endsWith('/prompt')) return jsonResponse({ prompt_id: 'p-1', node_errors: {} })
|
||||
if (url.includes('/history/p-1')) return jsonResponse({ 'p-1': { outputs: { '9': { images: [{ filename: 'dsh_00001_.png' }] } } } })
|
||||
throw new Error(`unexpected fetch ${url}`)
|
||||
}))
|
||||
const { fs } = await mountTarget({ files: { 'x.png': pngBytes } })
|
||||
void fs
|
||||
|
||||
const out = await runComfyImage('http://192.168.31.240:8188', resolveImageArgs({ prompt: 'a cat' }), undefined, 30_000)
|
||||
expect(out).toBe('http://192.168.31.240:8188/view?filename=dsh_00001_.png&subfolder=&type=output')
|
||||
expect(calls.some(u => u.endsWith('/prompt'))).toBe(true)
|
||||
})
|
||||
|
||||
it('rejects a workflow with non-empty node_errors', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ prompt_id: 'p-2', node_errors: { '3': ['bad'] } })))
|
||||
await expect(runComfyImage('http://x', resolveImageArgs({ prompt: 'x' }), undefined, 30_000))
|
||||
.rejects.toThrow(/workflow rejected/)
|
||||
})
|
||||
|
||||
it('rejects non-positive integer dimensions and steps', () => {
|
||||
expect(() => resolveImageArgs({ prompt: 'x', width: 0 })).toThrow(/width must be a positive integer/)
|
||||
expect(() => resolveImageArgs({ prompt: 'x', steps: -1 })).toThrow(/steps must be a positive integer/)
|
||||
})
|
||||
|
||||
it('uses the documented SDXL-turbo defaults in the workflow', () => {
|
||||
const wf = buildWorkflow({ prompt: 'p', negative: 'negative prompt', width: 512, height: 512, steps: 4, seed: 123, model: 'sdxl_turbo.safetensors', filenamePrefix: 'dsh' })
|
||||
const sampler = wf['3'] as { inputs: Record<string, unknown> }
|
||||
expect(sampler.inputs.cfg).toBe(1.0)
|
||||
expect(sampler.inputs.sampler_name).toBe('euler')
|
||||
expect(sampler.inputs.scheduler).toBe('normal')
|
||||
expect(sampler.inputs.steps).toBe(4)
|
||||
})
|
||||
})
|
||||
|
||||
describe('lab_ocr_pdf (Docling)', () => {
|
||||
it('uploads a PDF and returns document.md_content after polling', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL) => {
|
||||
const url = String(input)
|
||||
if (url.includes('/v1/convert/file/async')) return jsonResponse({ task_id: 't-1', task_status: 'pending' })
|
||||
if (url.includes('/v1/status/poll/t-1')) return jsonResponse({ task_status: 'success' })
|
||||
if (url.includes('/v1/result/t-1')) return jsonResponse({ status: 'success', errors: [], document: { filename: 'a.pdf', md_content: '# OCRed' } })
|
||||
throw new Error(`unexpected ${url}`)
|
||||
}))
|
||||
const out = await runDoclingOcr('http://192.168.31.159:5001', new TextEncoder().encode('%PDF-1.4'), 'a.pdf', undefined, 30_000)
|
||||
expect(out).toBe('# OCRed')
|
||||
expect(globalThis.fetch).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects an empty task id', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ task_id: undefined })))
|
||||
await expect(runDoclingOcr('http://x', new Uint8Array(), 'a.pdf', undefined, 30_000)).rejects.toThrow(/no task_id/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('lab_transcribe_audio (Whishper)', () => {
|
||||
it('polls until result.text is non-empty', async () => {
|
||||
let polls = 0
|
||||
vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/api/transcriptions')) return jsonResponse({ id: 'w-1', status: -1 })
|
||||
polls += 1
|
||||
if (polls === 1) return jsonResponse({ id: 'w-1', status: -1, result: { text: '' } })
|
||||
return jsonResponse({ id: 'w-1', status: 0, result: { text: 'hello world', language: 'en', duration: 1.2 } })
|
||||
}))
|
||||
const out = await runWhishTranscribe('http://192.168.31.159:8082', new TextEncoder().encode('wav'), 'a.wav', undefined, undefined, undefined, 30_000)
|
||||
expect(out).toBe('hello world')
|
||||
})
|
||||
|
||||
it('rejects an upload with no id', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ id: undefined })))
|
||||
await expect(runWhishTranscribe('http://x', new Uint8Array(), 'a.wav', undefined, undefined, undefined, 30_000))
|
||||
.rejects.toThrow(/no id/)
|
||||
})
|
||||
})
|
||||
|
||||
// keep imports referenced for the fake implementation; these types come from
|
||||
// @deepseek-ai/dsh-fs and are used by FakeFs above.
|
||||
type _ = SandboxExecutionPolicy | SandboxMode
|
||||
export { FakeFs }
|
||||
10
packages/tool-lab/tsconfig.json
Normal file
10
packages/tool-lab/tsconfig.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
]
|
||||
}
|
||||
6
packages/web-search-searxng/README.i18n.yaml
Normal file
6
packages/web-search-searxng/README.i18n.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/web/web-search-searxng/README.md
|
||||
README.md: 0e0c76fc884eb5968618673cc58f6ae5f88112cc
|
||||
README.zh.md: aa550a67c49458c9488c4b3bf24532935db99e01
|
||||
52
packages/web-search-searxng/README.md
Normal file
52
packages/web-search-searxng/README.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# @deepseek-ai/dsh-web-search-searxng
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
A [SearXNG](https://docs.searxng.org)-backed `WebSearchProvider` for the harness [web capability seam](../web/README.md) (`ctx.web`). It calls a SearXNG instance's JSON API (`GET /search?format=json`) and maps the flat `results[]` into the seam's normalized `WebSearchResult`. It targets a self-hosted or private SearXNG: there is no single canonical public instance, so `baseURL` has no default, and the provider deliberately carries no API key or `Authorization` header.
|
||||
|
||||
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the `ctx.web` key and it does not register a model-facing tool (that is `@deepseek-ai/dsh-tool-web`). Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`) that registers its backend, not a default-export service.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `baseURL` | (none) | SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. |
|
||||
| `language` | `auto` | SearXNG `language` request value; `auto` lets the instance decide per user preferences. |
|
||||
| `timeRange` | (unset) | SearXNG `time_range` recency filter: `day`, `week`, `month`, or `year`. Omitted sends no filter. |
|
||||
|
||||
```yaml
|
||||
- id: web-search-searxng
|
||||
name: '@deepseek-ai/dsh-web-search-searxng'
|
||||
config:
|
||||
baseURL: https://searx.example
|
||||
```
|
||||
|
||||
The entry above is the base layer of the `web-search-searxng` Settings section: a user layer over it (a `settings.yaml` section or an in-session settings edit) reaches the NEXT search, because the provider projects the section per call rather than capturing it at registration. The seam's provider selection therefore never flickers when the instance or a filter changes. A section without `baseURL` still passes the schema but leaves the provider unavailable — no endpoint is guessed.
|
||||
|
||||
```yaml
|
||||
# $DSH_HOME/settings.yaml
|
||||
web-search-searxng:
|
||||
baseURL: https://searx.example
|
||||
language: auto
|
||||
timeRange: day
|
||||
```
|
||||
|
||||
SearXNG exposes no per-request result-count control — page size is instance configuration — so no `numResults` option exists; the seam enforces `maxResults` on the result.
|
||||
|
||||
## Mapping
|
||||
|
||||
SearXNG returns a flat `results[]` and no generated answer, so `content` is omitted. Each result maps to a `WebSearchSource`: `url` ← `url`, `title` ← `title`, `snippet` ← `content`, `publishedAt` ← `publishedDate`. Unlike the Exa provider it does **not** drop snippet-less entries: URL and title are still useful, so every result is kept. Provider failures (HTTP errors, network failure, unparseable or wrong-shape bodies) surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`. HTTP redirects are rejected before the `Location` target is contacted and surface as `WEB_PROVIDER_ERROR`. The request carries no `Authorization` header, so no credential can leak to a redirect target.
|
||||
|
||||
## Model Experience
|
||||
|
||||
Indirectly, through [`dsh-tool-web`](../tool-web/README.md), which retains this provider's `maxResults`-bounded URLs, titles, snippets, and publication dates or its exact `SearXNG search aborted`, `SearXNG search request failed: <error>`, and `SearXNG returned an unprocessable response body: <error>` failures under the consumer's error wrapper while generated answers and provider-private fields remain outside context.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Result count is not bounded at the request** — SearXNG page size is instance configuration, so the provider sends no count and the seam truncates on return; a single page (typically ~20 results) is fetched.
|
||||
- **No aggregate-answer or infobox content is surfaced** — SearXNG `answers` and `infoboxes` are not mapped into `content`. No generated answer is trusted.
|
||||
- **Abort classification is error-shape-based** — only a `DOMException` named `AbortError` maps to `WEB_ABORTED`; an abort carrying a custom reason (e.g. `dsh-timeout`'s `TimeoutReason`) surfaces as `WEB_PROVIDER_ERROR`.
|
||||
52
packages/web-search-searxng/README.zh.md
Normal file
52
packages/web-search-searxng/README.zh.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# @deepseek-ai/dsh-web-search-searxng
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
由 [SearXNG](https://docs.searxng.org) 支持的 `WebSearchProvider`，用于 harness [web 能力 seam](../web/README.md)（`ctx.web`）。它调用 SearXNG 实例的 JSON API（`GET /search?format=json`），把扁平 `results[]` 映射为 seam 规范化的 `WebSearchResult`。此提供方面向自托管或私有 SearXNG：不存在统一的公共实例，因此 `baseURL` 没有默认值，提供方刻意不携带 API 密钥或 `Authorization` 头。
|
||||
|
||||
这是一个**实现**包：它向 `ctx.web` 注册提供方，不拥有 `ctx.web` 键，也不注册面向模型的工具（后者属于 `@deepseek-ai/dsh-tool-web`）。与 `@deepseek-ai/dsh-llm-deepseek` 一样，它是函数／命名空间插件（`inject: ['web']`），负责注册后端，而非默认导出服务。
|
||||
|
||||
## 配置
|
||||
|
||||
| 配置键 | 默认值 | 含义 |
|
||||
|---|---|---|
|
||||
| `baseURL` | （无） | SearXNG 实例基址；追加 `/search`。为空或无法解析时提供方不可用。 |
|
||||
| `language` | `auto` | SearXNG `language` 请求值；`auto` 让实例按用户偏好决定。 |
|
||||
| `timeRange` | （未设置） | SearXNG `time_range` 时效过滤器：`day`、`week`、`month` 或 `year`。省略时不发送过滤器。 |
|
||||
|
||||
```yaml
|
||||
- id: web-search-searxng
|
||||
name: '@deepseek-ai/dsh-web-search-searxng'
|
||||
config:
|
||||
baseURL: https://searx.example
|
||||
```
|
||||
|
||||
上面的条目是 `web-search-searxng` Settings 分节的基础层：覆盖在它之上的用户层（`settings.yaml` 分节或会话内设置的编辑）会作用于**下一次**搜索，因为提供方按调用投影分节，而非在注册时捕获。因此 seam 的提供方选择在实例或过滤器变化时不会闪烁。不含 `baseURL` 的分节仍能通过 schema，但提供方保持不可用——不会猜测任何端点。
|
||||
|
||||
```yaml
|
||||
# $DSH_HOME/settings.yaml
|
||||
web-search-searxng:
|
||||
baseURL: https://searx.example
|
||||
language: auto
|
||||
timeRange: day
|
||||
```
|
||||
|
||||
SearXNG 不提供按请求控制结果数量的方式——页面大小属于实例配置——因此没有 `numResults` 选项；seam 会在结果返回时强制执行 `maxResults`。
|
||||
|
||||
## 映射
|
||||
|
||||
SearXNG 返回扁平 `results[]`，不返回生成答案，因此省略 `content`。每项结果映射为 `WebSearchSource`：`url` ← `url`、`title` ← `title`、`snippet` ← `content`、`publishedAt` ← `publishedDate`。与 Exa 提供方不同，此提供方**不会**丢弃无 snippet 的结果：URL 与标题仍然有用，因此保留每项结果。提供方失败（HTTP 错误、网络失败、响应体无法解析或结构不符）以 `WebError` `WEB_PROVIDER_ERROR` 呈现；中止请求以 `WEB_ABORTED` 呈现。HTTP 重定向会在访问 `Location` 指向的目标之前被拒绝，并以 `WEB_PROVIDER_ERROR` 呈现。请求不携带 `Authorization` 头，因此不会有凭据泄露给重定向目标。
|
||||
|
||||
## 模型体验
|
||||
|
||||
通过 [`dsh-tool-web`](../tool-web/README.md) 间接影响；该工具保留此提供方经 `maxResults` 限制的 URL、标题、snippet 与发布日期，或将确切的错误消息 `SearXNG search aborted`、`SearXNG search request failed: <error>` 和 `SearXNG returned an unprocessable response body: <error>` 置于消费方的错误包装层内；生成答案与提供方私有字段不进入上下文。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
不会直接导致 KV Cache 失效；请求前缀变更由上述消费方负责。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **请求端不限制结果数量**：SearXNG 页面大小属于实例配置，因此提供方不发送数量，由 seam 在返回时截断；只抓取单页（通常约 20 条结果）。
|
||||
- **不呈现聚合答案或 infobox 内容**：SearXNG 的 `answers` 与 `infoboxes` 不映射进 `content`。不信任任何生成答案。
|
||||
- **按错误形状分类中止**：只有 `DOMException` 且名为 `AbortError` 时才映射为 `WEB_ABORTED`；携带自定义原因的中止（例如 `dsh-timeout` 的 `TimeoutReason`）会呈现为 `WEB_PROVIDER_ERROR`。
|
||||
60
packages/web-search-searxng/package.json
Normal file
60
packages/web-search-searxng/package.json
Normal file
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-web-search-searxng",
|
||||
"description": "SearXNG-backed search provider for the DeepSeek Harness web capability seam (ctx.web)",
|
||||
"version": "0.1.0-rc.7",
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"registry": "https://git.byte-mate.ru/api/packages/Coder/npm/",
|
||||
"tag": "rc"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://git.byte-mate.ru/Coder/dsh-plugins.git",
|
||||
"directory": "packages/web-search-searxng"
|
||||
},
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib"
|
||||
],
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=22.19"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.json",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/cordis": "^4.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-settings": "^0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-web": "^0.1.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/schemastery": "^3.18.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/cordis": "4.0.1",
|
||||
"@deepseek-ai/schemastery": "3.18.1",
|
||||
"@deepseek-ai/dsh-invariants": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-settings": "0.1.0-rc.7",
|
||||
"@deepseek-ai/dsh-web": "0.1.0-rc.7",
|
||||
"typescript": "^6.0.3",
|
||||
"vitest": "^4.1.8",
|
||||
"@types/node": "^22.20.0"
|
||||
}
|
||||
}
|
||||
79
packages/web-search-searxng/src/index.ts
Normal file
79
packages/web-search-searxng/src/index.ts
Normal file
@@ -0,0 +1,79 @@
|
||||
/**
|
||||
* `@deepseek-ai/dsh-web-search-searxng`: registers a SearXNG-backed
|
||||
* `WebSearchProvider` with `ctx.web`. A function/namespace plugin (NOT a
|
||||
* default-export service): a search provider does not own the `ctx.web` key —
|
||||
* it registers INTO the seam's provider registry, exactly as
|
||||
* `@deepseek-ai/dsh-web-search-exa` does. The key is owned by
|
||||
* `@deepseek-ai/dsh-web`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-web-search-searxng
|
||||
*/
|
||||
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import type {} from '@deepseek-ai/dsh-web'
|
||||
import { SearXngSearchProvider } from './provider.ts'
|
||||
|
||||
export {
|
||||
SEARXNG_DEFAULT_LANGUAGE,
|
||||
SEARXNG_NO_TIME_RANGE,
|
||||
SEARXNG_PROVIDER_ID,
|
||||
SEARXNG_TIME_RANGES,
|
||||
SearXngSearchProvider,
|
||||
} from './provider.ts'
|
||||
export type {
|
||||
SearXngSearchProviderOptions,
|
||||
SearXngSearchProviderSource,
|
||||
SearXngTimeRange,
|
||||
} from './provider.ts'
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'web-search-searxng'
|
||||
|
||||
/** The web seam this provider registers into. */
|
||||
export const inject = ['web']
|
||||
|
||||
/** Plugin config. `baseURL` is optional in the schema so an omitted value
|
||||
* surfaces as an unavailable provider (the seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`)
|
||||
* rather than failing boot; `apply` fills constant defaults. */
|
||||
export interface Config {
|
||||
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
|
||||
baseURL?: string
|
||||
/** SearXNG `language` to request. Defaults to `auto`. */
|
||||
language?: string
|
||||
/** SearXNG `time_range` recency filter. Omitted = no filter. */
|
||||
timeRange?: 'day' | 'week' | 'month' | 'year'
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
baseURL: z.string(),
|
||||
language: z.string(),
|
||||
timeRange: z.union(['day', 'week', 'month', 'year'] as const),
|
||||
})
|
||||
|
||||
/** Settings namespace carrying the SearXNG instance and any per-search filters. */
|
||||
export const SEARXNG_SETTINGS_NAMESPACE = settingsNamespace('web-search-searxng')
|
||||
|
||||
/** Register the SearXNG search provider with `ctx.web`, reading the live section
|
||||
* per search so an in-session settings edit applies without re-registration. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
let current: () => Config = () => config
|
||||
installSettingsSection(ctx, SEARXNG_SETTINGS_NAMESPACE, Config, config, {
|
||||
setSource: (source) => {
|
||||
current = source
|
||||
},
|
||||
// The registration carries no resolved value: the provider projects the
|
||||
// section per search, so a committed change needs no re-registration.
|
||||
onChange: () => {},
|
||||
})
|
||||
ctx.web.registerSearchProvider(new SearXngSearchProvider(() => ({
|
||||
// SearXNG exposes no uniform public endpoint and the product requires
|
||||
// evidence over an unsupported default, so `baseURL` has no constant
|
||||
// fallback: an omitted value surfaces as an unavailable provider (the
|
||||
// seam's `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`), not a silent endpoint.
|
||||
baseURL: current().baseURL ?? '',
|
||||
...current().language !== undefined ? { language: current().language } : {},
|
||||
...current().timeRange !== undefined ? { timeRange: current().timeRange } : {},
|
||||
})))
|
||||
}
|
||||
30
packages/web-search-searxng/src/invariant.ts
Normal file
30
packages/web-search-searxng/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-web-search-searxng`.
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'web-search-searxng-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this package exposes no independent event sequence or mutable data relation
|
||||
* beyond contracts enforced at its owning seam.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
166
packages/web-search-searxng/src/provider.ts
Normal file
166
packages/web-search-searxng/src/provider.ts
Normal file
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* `SearXngSearchProvider`: a `WebSearchProvider` backed by a SearXNG instance's
|
||||
* JSON API (`GET /search?format=json`). It maps against the flat `results[]`,
|
||||
* keeps entries even without a snippet (URL and title remain useful), and
|
||||
* omits `content` because SearXNG returns no generated answer. The provider
|
||||
* carries no credentials, so a request carries no `Authorization` header.
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/provider
|
||||
*/
|
||||
|
||||
import { WebError } from '@deepseek-ai/dsh-web'
|
||||
import type {
|
||||
WebSearchProvider,
|
||||
WebSearchRequest,
|
||||
WebSearchResult,
|
||||
WebSearchSource,
|
||||
} from '@deepseek-ai/dsh-web'
|
||||
import type { SearXngError, SearXngResult, SearXngSearchResponse } from './types.ts'
|
||||
|
||||
/** Stable id this provider registers under. */
|
||||
export const SEARXNG_PROVIDER_ID = 'searxng'
|
||||
|
||||
/** Default language: let SearXNG decide per user preferences. */
|
||||
export const SEARXNG_DEFAULT_LANGUAGE = 'auto'
|
||||
|
||||
/** `time_range` sentinel meaning "no recency filter". */
|
||||
export const SEARXNG_NO_TIME_RANGE = 'none'
|
||||
|
||||
/** Valid `time_range` values SearXNG accepts. */
|
||||
export const SEARXNG_TIME_RANGES = ['day', 'week', 'month', 'year'] as const
|
||||
|
||||
/** One of SearXNG's `time_range` values, passed through to the endpoint. */
|
||||
export type SearXngTimeRange = (typeof SEARXNG_TIME_RANGES)[number]
|
||||
|
||||
/** Attribution header sent on every request. Bump with the package version. */
|
||||
const USER_AGENT = 'deepseek-harness/0.0.1'
|
||||
|
||||
/** Resolved provider options (the plugin's `apply` supplies env-var and constant defaults). */
|
||||
export interface SearXngSearchProviderOptions {
|
||||
/** SearXNG instance base; `/search` is appended. Empty/unparseable makes the provider unavailable. */
|
||||
baseURL: string
|
||||
/** SearXNG `language` to request. `auto` lets the instance decide. */
|
||||
language?: string
|
||||
/** SearXNG `time_range` recency filter. */
|
||||
timeRange?: SearXngTimeRange
|
||||
}
|
||||
|
||||
/** A snapshot or a per-search resolver (settings live-reload hands the latter). */
|
||||
export type SearXngSearchProviderSource = SearXngSearchProviderOptions | (() => SearXngSearchProviderOptions)
|
||||
|
||||
/**
|
||||
* Map one SearXNG result to a normalized source. The URL is always carried; a
|
||||
* blank title or snippet is omitted rather than emitted as empty.
|
||||
*
|
||||
* @param result - one entry of SearXNG's `results[]`.
|
||||
* @returns the normalized source.
|
||||
*/
|
||||
export function mapSearXngResult(result: SearXngResult): WebSearchSource {
|
||||
return {
|
||||
url: result.url,
|
||||
...result.title != null && result.title.trim().length > 0 ? { title: result.title } : {},
|
||||
...result.content != null && result.content.trim().length > 0 ? { snippet: result.content } : {},
|
||||
...result.publishedDate != null && result.publishedDate.length > 0 ? { publishedAt: result.publishedDate } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a SearXNG response envelope to a normalized search result.
|
||||
*
|
||||
* @param response - the parsed `format=json` response body.
|
||||
* @returns the normalized result; `content` is omitted (no generated answer).
|
||||
*/
|
||||
export function mapSearXngResponse(response: SearXngSearchResponse): WebSearchResult {
|
||||
// SearXNG cannot bound per-request result count (page size is instance
|
||||
// configuration), so the request carries no count and the web service owns
|
||||
// the final `maxResults` truncation; this provider reports `truncated: false`.
|
||||
return { sources: (response.results ?? []).map(mapSearXngResult), truncated: false }
|
||||
}
|
||||
|
||||
/** The SearXNG-backed search provider; HTTP redirects fail as `WEB_PROVIDER_ERROR`. */
|
||||
export class SearXngSearchProvider implements WebSearchProvider {
|
||||
readonly id = SEARXNG_PROVIDER_ID
|
||||
|
||||
constructor(source: SearXngSearchProviderSource) {
|
||||
this.source = source
|
||||
}
|
||||
|
||||
private readonly source: SearXngSearchProviderSource
|
||||
|
||||
/** Read the current options: a snapshot stays fixed, a resolver reads live settings. */
|
||||
private resolveOptions(): SearXngSearchProviderOptions {
|
||||
return typeof this.source === 'function' ? this.source() : this.source
|
||||
}
|
||||
|
||||
available(): boolean {
|
||||
const options = this.resolveOptions()
|
||||
return isValidBaseUrl(options.baseURL)
|
||||
&& (options.language === undefined || options.language.length > 0)
|
||||
&& (options.timeRange === undefined || SEARXNG_TIME_RANGES.includes(options.timeRange))
|
||||
}
|
||||
|
||||
async search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult> {
|
||||
const options = this.resolveOptions()
|
||||
const base = trimTrailingSlashes(options.baseURL)
|
||||
const params = new URLSearchParams({ q: request.query, format: 'json' })
|
||||
if (options.language !== undefined) params.set('language', options.language)
|
||||
if (options.timeRange !== undefined) params.set('time_range', options.timeRange)
|
||||
|
||||
let response: Response
|
||||
try {
|
||||
response = await fetch(`${base}/search?${params}`, {
|
||||
method: 'GET',
|
||||
redirect: 'error',
|
||||
headers: {
|
||||
'accept': 'application/json',
|
||||
'user-agent': USER_AGENT,
|
||||
},
|
||||
...signal !== undefined ? { signal } : {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
throw new WebError(`SearXNG search request failed: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
const status = response.status
|
||||
let message = `SearXNG API error (HTTP ${status})`
|
||||
try {
|
||||
const parsed = await response.json() as SearXngError
|
||||
const detail = parsed.error ?? parsed.message ?? parsed.content
|
||||
if (detail !== undefined && detail.length > 0) message = detail
|
||||
} catch (error: unknown) {
|
||||
// An abort fired mid-body must surface as WEB_ABORTED, not be swallowed
|
||||
// into a generic HTTP-error message — cancellation is not a provider
|
||||
// error (the seam's cancellation contract).
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
// Otherwise: the HTTP status is already captured in `message` above; a
|
||||
// malformed/non-JSON error body (normal for gateway 5xx/429s) can only
|
||||
// cost a richer provider message, never the real error.
|
||||
}
|
||||
throw new WebError(message, 'WEB_PROVIDER_ERROR')
|
||||
}
|
||||
|
||||
try {
|
||||
const payload = await response.json() as SearXngSearchResponse
|
||||
return mapSearXngResponse(payload)
|
||||
} catch (error: unknown) {
|
||||
if (isAbortError(error)) throw new WebError('SearXNG search aborted', 'WEB_ABORTED', { cause: error })
|
||||
throw new WebError(`SearXNG returned an unprocessable response body: ${String(error)}`, 'WEB_PROVIDER_ERROR', { cause: error })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** True when `baseURL` parses as an absolute URL (a cheap local config check). */
|
||||
function isValidBaseUrl(baseURL: string): boolean {
|
||||
return URL.canParse(baseURL)
|
||||
}
|
||||
|
||||
/** Strip a trailing slash so `${base}/search` never doubles a separator. */
|
||||
function trimTrailingSlashes(baseURL: string): string {
|
||||
return baseURL.replace(/\/+$/, '')
|
||||
}
|
||||
|
||||
/** True for a fetch/`AbortSignal` abort, surfaced as `WEB_ABORTED`. */
|
||||
function isAbortError(error: unknown): boolean {
|
||||
return error instanceof DOMException && error.name === 'AbortError'
|
||||
}
|
||||
30
packages/web-search-searxng/src/types.ts
Normal file
30
packages/web-search-searxng/src/types.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Wire types for the SearXNG JSON API (`GET {base}/search?q=...&format=json`).
|
||||
* Types only — no runtime code. SearXNG returns a flat `results[]`; each entry
|
||||
* carries a URL, an optional title, an optional `content` snippet, and an
|
||||
* optional `publishedDate`.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-web-search-searxng/types
|
||||
*/
|
||||
|
||||
/** One entry of SearXNG's flat `results[]`. */
|
||||
export interface SearXngResult {
|
||||
url: string
|
||||
title?: string | null
|
||||
/** The page snippet SearXNG derives for the result. */
|
||||
content?: string | null
|
||||
publishedDate?: string | null
|
||||
}
|
||||
|
||||
/** SearXNG's JSON search response envelope. */
|
||||
export interface SearXngSearchResponse {
|
||||
results?: SearXngResult[]
|
||||
}
|
||||
|
||||
/** SearXNG's error response envelope (best-effort; fields vary by failure). */
|
||||
export interface SearXngError {
|
||||
error?: string
|
||||
message?: string
|
||||
/** Some SearXNG error responses carry the message under `content`. */
|
||||
content?: string
|
||||
}
|
||||
24
packages/web-search-searxng/tests/searxng.e2e.ts
Normal file
24
packages/web-search-searxng/tests/searxng.e2e.ts
Normal file
@@ -0,0 +1,24 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SearXngSearchProvider } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/**
|
||||
* Real-API smoke for the SearXNG search provider against a live instance.
|
||||
* Self-skips without `$SEARXNG_BASE_URL` (CI has no endpoint), per the
|
||||
* with-key e2e policy in docs/testing.md.
|
||||
*/
|
||||
const baseURL = process.env.SEARXNG_BASE_URL
|
||||
const maybe = baseURL !== undefined && baseURL.length > 0 ? describe : describe.skip
|
||||
|
||||
maybe('SearXngSearchProvider real API', () => {
|
||||
it('returns sources for a live query', async () => {
|
||||
const provider = new SearXngSearchProvider({
|
||||
baseURL: baseURL!,
|
||||
...process.env.SEARXNG_LANGUAGE !== undefined && process.env.SEARXNG_LANGUAGE.length > 0
|
||||
? { language: process.env.SEARXNG_LANGUAGE }
|
||||
: {},
|
||||
})
|
||||
const result = await provider.search({ query: 'DeepSeek Harness', maxResults: 5 })
|
||||
expect(result.sources.length).toBeGreaterThan(0)
|
||||
for (const source of result.sources) expect(source.url).toMatch(/^https?:\/\//)
|
||||
}, 30_000)
|
||||
})
|
||||
226
packages/web-search-searxng/tests/searxng.spec.ts
Normal file
226
packages/web-search-searxng/tests/searxng.spec.ts
Normal file
@@ -0,0 +1,226 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import WebRuntime from '@deepseek-ai/dsh-web'
|
||||
import { SearXngSearchProvider, SEARXNG_PROVIDER_ID } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import { mapSearXngResponse, mapSearXngResult } from '../src/provider.ts'
|
||||
|
||||
const options = { baseURL: 'https://searx.test' }
|
||||
|
||||
function jsonResponse(body: unknown, init: ResponseInit = {}): Response {
|
||||
return new Response(JSON.stringify(body), { status: 200, headers: { 'content-type': 'application/json' }, ...init })
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
describe('SearXng result mapping', () => {
|
||||
it('maps a full result entry', () => {
|
||||
expect(mapSearXngResult({
|
||||
url: 'https://a.test',
|
||||
title: 'A',
|
||||
content: 'an excerpt',
|
||||
publishedDate: '2026-01-01',
|
||||
})).toEqual({ url: 'https://a.test', title: 'A', snippet: 'an excerpt', publishedAt: '2026-01-01' })
|
||||
})
|
||||
|
||||
it('keeps a URL-only result rather than dropping it', () => {
|
||||
expect(mapSearXngResult({ url: 'https://a.test' })).toEqual({ url: 'https://a.test' })
|
||||
expect(mapSearXngResult({ url: 'https://a.test', content: ' ' })).toEqual({ url: 'https://a.test' })
|
||||
})
|
||||
|
||||
it('omits null/empty optional fields rather than emitting them', () => {
|
||||
expect(mapSearXngResult({ url: 'https://a.test', title: null, content: null, publishedDate: null }))
|
||||
.toEqual({ url: 'https://a.test' })
|
||||
expect(mapSearXngResult({ url: 'https://a.test', title: '', content: '', publishedDate: '' }))
|
||||
.toEqual({ url: 'https://a.test' })
|
||||
})
|
||||
|
||||
it('maps a response to a result with no content and all sources kept', () => {
|
||||
const result = mapSearXngResponse({
|
||||
results: [
|
||||
{ url: 'https://a.test', title: 'A', content: 'one' },
|
||||
{ url: 'https://b.test' },
|
||||
{ url: 'https://c.test', content: 'three' },
|
||||
],
|
||||
})
|
||||
expect(result).toEqual({
|
||||
sources: [
|
||||
{ url: 'https://a.test', title: 'A', snippet: 'one' },
|
||||
{ url: 'https://b.test' },
|
||||
{ url: 'https://c.test', snippet: 'three' },
|
||||
],
|
||||
truncated: false,
|
||||
})
|
||||
expect(result.content).toBeUndefined()
|
||||
})
|
||||
|
||||
it('tolerates a missing results array', () => {
|
||||
expect(mapSearXngResponse({}).sources).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider availability', () => {
|
||||
it('is unavailable without a base URL', () => {
|
||||
expect(new SearXngSearchProvider({ baseURL: '' }).available()).toBe(false)
|
||||
})
|
||||
|
||||
it('is available with a base URL', () => {
|
||||
expect(new SearXngSearchProvider(options).available()).toBe(true)
|
||||
})
|
||||
|
||||
it('is misconfigured when the base URL is unparseable', () => {
|
||||
expect(new SearXngSearchProvider({ baseURL: 'not a url' }).available()).toBe(false)
|
||||
})
|
||||
|
||||
it('is misconfigured when language is empty or timeRange is invalid', () => {
|
||||
expect(new SearXngSearchProvider({ ...options, language: '' }).available()).toBe(false)
|
||||
expect(new SearXngSearchProvider({ ...options, timeRange: 'decade' as never }).available()).toBe(false)
|
||||
expect(new SearXngSearchProvider({ ...options, timeRange: 'week' }).available()).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider request mapping', () => {
|
||||
it('issues a GET with query and json format and no authorization header', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [{ url: 'https://a.test' }] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
|
||||
await new SearXngSearchProvider(options).search({ query: 'hello world' })
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledOnce()
|
||||
const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
|
||||
expect(url).toBe('https://searx.test/search?q=hello+world&format=json')
|
||||
expect(init.method).toBe('GET')
|
||||
expect(init.redirect).toBe('error')
|
||||
expect((init.headers as Record<string, string>)['authorization']).toBeUndefined()
|
||||
})
|
||||
|
||||
it('sends language and time_range when configured', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider({ ...options, language: 'en', timeRange: 'week' }).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
|
||||
})
|
||||
|
||||
it('omits language and time_range when unset', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider(options).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json')
|
||||
})
|
||||
|
||||
it('does not double the separator when baseURL carries a trailing slash', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
await new SearXngSearchProvider({ baseURL: 'https://searx.test/' }).search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json')
|
||||
})
|
||||
|
||||
it('forwards the abort signal', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const controller = new AbortController()
|
||||
await new SearXngSearchProvider(options).search({ query: 'q' }, controller.signal)
|
||||
const [, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]
|
||||
expect(init.signal).toBe(controller.signal)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SearXngSearchProvider error handling', () => {
|
||||
it('maps an HTTP error to WEB_PROVIDER_ERROR with the provider message', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ error: 'missing instance' }, { status: 401 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'missing instance' }))
|
||||
})
|
||||
|
||||
it('keeps a status-line message when the error body is not JSON', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => new Response('gateway down', { status: 502 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR', message: 'SearXNG API error (HTTP 502)' }))
|
||||
})
|
||||
|
||||
it('reads the message from content when present', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ content: 'json disabled' }, { status: 403 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ message: 'json disabled' }))
|
||||
})
|
||||
|
||||
it('maps a network failure to WEB_PROVIDER_ERROR', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new TypeError('connection refused'))))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('maps an abort to WEB_ABORTED', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(() => Promise.reject(new DOMException('aborted', 'AbortError'))))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
|
||||
it('maps an unparseable success body to WEB_PROVIDER_ERROR', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => new Response('not json', { status: 200 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('maps a well-formed body of the wrong shape to WEB_PROVIDER_ERROR, not a raw TypeError', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: {} }, { status: 200 })))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_ERROR' }))
|
||||
})
|
||||
|
||||
it('surfaces an abort during success-body parse as WEB_ABORTED, not provider error', async () => {
|
||||
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: true, status: 200 }
|
||||
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
|
||||
it('surfaces an abort during error-body parse as WEB_ABORTED', async () => {
|
||||
const body = { json: () => Promise.reject(new DOMException('aborted', 'AbortError')), ok: false, status: 500 }
|
||||
vi.stubGlobal('fetch', vi.fn(async () => body as unknown as Response))
|
||||
await expect(new SearXngSearchProvider(options).search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_ABORTED' }))
|
||||
})
|
||||
})
|
||||
|
||||
describe('web-search-searxng plugin registration', () => {
|
||||
it('registers the provider into ctx.web (HMR-safe)', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: [] })))
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL })
|
||||
await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ sources: [], truncated: false })
|
||||
await fiber.dispose()
|
||||
await expect(ctx.web.search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' }))
|
||||
})
|
||||
|
||||
it('has no default export (namespace plugin export shape)', () => {
|
||||
expect('default' in searxngPlugin).toBe(false)
|
||||
})
|
||||
|
||||
it('threads language and timeRange config into the request', async () => {
|
||||
const fetchMock = vi.fn(async () => jsonResponse({ results: [] }))
|
||||
vi.stubGlobal('fetch', fetchMock)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
const fiber = await ctx.plugin(searxngPlugin, { baseURL: options.baseURL, language: 'en', timeRange: 'week' })
|
||||
await ctx.web.search({ query: 'q' })
|
||||
const [url] = fetchMock.mock.calls[0] as unknown as [string]
|
||||
expect(url).toBe('https://searx.test/search?q=q&format=json&language=en&time_range=week')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('is unavailable when baseURL is omitted', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, { searchProvider: SEARXNG_PROVIDER_ID })
|
||||
await ctx.plugin(searxngPlugin, { baseURL: '' })
|
||||
await expect(ctx.web.search({ query: 'q' }))
|
||||
.rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' }))
|
||||
})
|
||||
})
|
||||
120
packages/web-search-searxng/tests/settings.spec.ts
Normal file
120
packages/web-search-searxng/tests/settings.spec.ts
Normal file
@@ -0,0 +1,120 @@
|
||||
/** The `web-search-searxng` settings section layered over the composition entry. */
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import type { Fiber } from '@deepseek-ai/cordis'
|
||||
import { SettingsProvider } from '@deepseek-ai/dsh-settings'
|
||||
import type { SettingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import WebRuntime from '@deepseek-ai/dsh-web'
|
||||
import * as searxngPlugin from '@deepseek-ai/dsh-web-search-searxng'
|
||||
import { SEARXNG_SETTINGS_NAMESPACE } from '@deepseek-ai/dsh-web-search-searxng'
|
||||
|
||||
/** The smallest real provider: one in-memory document, always writable. */
|
||||
class MemorySettings extends SettingsProvider {
|
||||
doc: Record<string, unknown> = {}
|
||||
|
||||
get writable(): boolean {
|
||||
return true
|
||||
}
|
||||
|
||||
protected load(): Promise<Record<string, unknown>> {
|
||||
return Promise.resolve(structuredClone(this.doc))
|
||||
}
|
||||
|
||||
protected persist(ns: SettingsNamespace, section: Record<string, unknown>): Promise<void> {
|
||||
this.doc = { ...this.doc, [ns]: structuredClone(section) }
|
||||
return Promise.resolve()
|
||||
}
|
||||
}
|
||||
|
||||
function jsonResponse(body: unknown): Response {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
})
|
||||
}
|
||||
|
||||
/** The smallest SearXNG-shaped answer the provider accepts — enough to observe the request. */
|
||||
const ONE_RESULT = {
|
||||
results: [{ url: 'https://a.test', title: 'A', content: 'snip' }],
|
||||
}
|
||||
|
||||
async function boot(): Promise<{ ctx: Context; settingsFiber: Fiber; pluginFiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(WebRuntime, {})
|
||||
const settingsFiber = ctx.plugin(MemorySettings)
|
||||
await settingsFiber.await()
|
||||
const pluginFiber = ctx.plugin(searxngPlugin, { baseURL: 'https://search.entry.test' })
|
||||
await pluginFiber.await()
|
||||
return { ctx, settingsFiber, pluginFiber }
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
/**
|
||||
* Run one search and answer the endpoint it reached. A fresh `Response` per
|
||||
* call because a body can only be read once, and the call history is cleared
|
||||
* because repeated `spyOn` returns the same spy.
|
||||
* @param ctx - context whose `ctx.web` serves the search.
|
||||
* @returns the URL the provider fetched.
|
||||
*/
|
||||
async function searchOnce(ctx: Context): Promise<string> {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch')
|
||||
.mockImplementation(() => Promise.resolve(jsonResponse(ONE_RESULT)))
|
||||
fetchSpy.mockClear()
|
||||
await ctx.web.search({ query: 'anything' })
|
||||
return String((fetchSpy.mock.calls.at(-1)?.[0] as URL | string | undefined) ?? '')
|
||||
}
|
||||
|
||||
describe('web-search-searxng settings section', () => {
|
||||
it('serves a stored endpoint to the next search without re-registering the provider', async () => {
|
||||
const bench = await boot()
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
|
||||
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.stored.test',
|
||||
})
|
||||
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('applies a stored language and time range to the next request', async () => {
|
||||
const bench = await boot()
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.entry.test',
|
||||
language: 'de',
|
||||
timeRange: 'week',
|
||||
})
|
||||
|
||||
const url = new URL(await searchOnce(bench.ctx))
|
||||
expect(url.searchParams.get('language')).toBe('de')
|
||||
expect(url.searchParams.get('time_range')).toBe('week')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('falls back to the composition entry when the settings provider detaches', async () => {
|
||||
const bench = await boot()
|
||||
await bench.ctx.settings.update(SEARXNG_SETTINGS_NAMESPACE, {
|
||||
baseURL: 'https://search.stored.test',
|
||||
})
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.stored.test')
|
||||
|
||||
await bench.settingsFiber.dispose()
|
||||
|
||||
expect(await searchOnce(bench.ctx)).toContain('https://search.entry.test')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('releases the namespace when the plugin unloads', async () => {
|
||||
const bench = await boot()
|
||||
expect(bench.ctx.settings.describe().map(row => String(row.ns))).toContain('web-search-searxng')
|
||||
|
||||
await bench.pluginFiber.dispose()
|
||||
|
||||
expect(bench.ctx.settings.describe().map(row => String(row.ns))).not.toContain('web-search-searxng')
|
||||
await bench.ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
10
packages/web-search-searxng/tsconfig.json
Normal file
10
packages/web-search-searxng/tsconfig.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user