Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: address model experience review

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-12 02:55:26 +08:00
parent a08485fc80
commit b5cd511f35
64 changed files with 443 additions and 410 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
packages/web/tool-web/README.md
View File

@@ -4,14 +4,6 @@ The model-facing web tool suite — `web_search` and `web_fetch` — over the [w
Each tool is registered independently; a product that wants only one disables the other via config (`{ search: false }` / `{ fetch: false }`).
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| System prompt | Each enabled tool adds one short section: search guidance says to discover current sources and follow with fetch; fetch guidance says to retrieve a specific HTTP(S) URL and cite it. | Fixed guidance cost per request for each enabled tool. |
| Tool schemas | According to config, the model sees `web_search(query)`, `web_fetch(url)`, or both. Result-count and timeout budgets are deployment settings, not model arguments. | Fixed schema cost per request; disabling a tool removes its schema and guidance. |
| Tool-call history and results | Search returns an optional answer and bounded source entries; fetch returns status plus decoded text or markdown-shaped HTML, or a structured error. Queries and URLs remain in call history. | Data-dependent results are resent until compaction. Search sources are capped by `searchMaxResults`; fetch providers cap body size, and timeout policy can replace a late result with a short error. |
## Tools
| Tool | Args | Behavior |
@@ -42,6 +34,14 @@ Tool registration follows product **enablement**, not backend availability. A to
The tool never calls a provider's `status()` and never enumerates providers — its only execution path is `ctx.web.search()` / `ctx.web.fetch()`, and provider unavailability reaches it as the structured `WebError` codes selection throws at execution time. Provider selection stays entirely inside the seam, with one owner.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| System prompt | Each enabled tool adds one short section: search guidance says to discover current sources and follow with fetch; fetch guidance says to retrieve a specific HTTP(S) URL and cite it. | Fixed guidance cost per request for each enabled tool. |
| Tool schemas | According to config, the model sees `web_search(query)`, `web_fetch(url)`, or both. Result-count and timeout budgets are deployment settings, not model arguments. | Fixed schema cost per request; disabling a tool removes its schema and guidance. |
| Tool-call history and results | Search returns an optional answer and bounded source entries; fetch returns status plus decoded text or markdown-shaped HTML, or a structured error. Queries and URLs remain in call history. | Data-dependent results are resent until compaction. Search sources are capped by `searchMaxResults`; fetch providers cap body size, and timeout policy can replace a late result with a short error. |
## Known Limitations and Deferred Work
- **`htmlToMarkdown` is a minimal regex converter, not an HTML parser** — it strips script/style/noscript, keeps headings/bullets/links, and decodes about a dozen named entities; tables, images, and nested formatting are lost.

12
packages/web/web-fetch-local/README.md
View File

@@ -4,12 +4,6 @@ An anonymous public HTTP(S) `WebFetchProvider` for the harness [web capability s
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the key and it does not register a model-facing tool. It is a function/namespace plugin (`inject: ['web']`).
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Web fetch result, indirectly | Through `dsh-tool-web`, the conversation model sees the final URL, HTTP status, and decoded text or markdown-shaped HTML, or a structured retrieval error. Redirects, headers, and transport mechanics are not added to context unless reflected in an error. | Zero direct tokens. `maxBodyChars` bounds decoded result length before the tool records it; the retained result is resent until compaction. |
## Responsibility split
The provider owns **safe resource retrieval**: URL validation, HTTP transport, redirect policy, a resource-backstop timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `@deepseek-ai/dsh-tool-web` owns **presentation** (HTML→markdown, truncation formatting). A non-2xx HTTP response is a *result* (status code + decoded body), not an error; `WebError` is reserved for failures to safely retrieve or represent the resource.
@@ -39,6 +33,12 @@ The provider's `timeoutMs`/`maxTimeoutMs` is a **resource backstop** for direct
The numeric limits are validated at plugin construction: every cap except `maxRedirects` must be a positive finite number, and `maxRedirects` must be a non-negative integer. An invalid value throws rather than silently constructing a provider with nonsensical limits.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Web fetch result, indirectly | Through `dsh-tool-web`, the conversation model sees the final URL, HTTP status, and decoded text or markdown-shaped HTML, or a structured retrieval error. Redirects, headers, and transport mechanics are not added to context unless reflected in an error. | Zero direct tokens. `maxBodyChars` bounds decoded result length before the tool records it; the retained result is resent until compaction. |
## Known Limitations and Deferred Work
- **SSRF / private-network protection is deferred** — no blocking of private, loopback, link-local, multicast, or otherwise non-public destinations, no DNS-resolve-then-validate, no per-hop re-validation (see [the web capability seam RFC](../../../docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md)). Until it lands, this provider is an SSRF primitive and **must not be enabled** in a deployment that can reach sensitive internal network targets.

14
packages/web/web-search-deepseek/README.md
View File

@@ -4,13 +4,6 @@ A [DeepSeek](https://deepseek.com)-backed `WebSearchProvider` for the harness [w
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the key and it does not register a model-facing tool. Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`). The Anthropic wire shape is a provider-private detail — it does **not** make this provider depend on `ctx.llm`.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Auxiliary DeepSeek search request | A separate DeepSeek model receives the search query and native `web_search` server-tool definition. This request is not part of the conversation model's context. | Separate provider input and output tokens are incurred for each search; `maxTokens` caps generated output and `maxUses` caps native search uses. |
| Conversation tool result, indirectly | Through `dsh-tool-web`, the conversation model sees deduplicated URLs, titles, dates, and citation snippets from structured search blocks; provider prose is not trusted as an answer. | Zero direct conversation tokens from registration. Result tokens scale with returned sources and snippets, then the seam enforces the requested source bound. |
## How it differs from a dedicated search endpoint
Exa and Perplexity expose dedicated search endpoints; DeepSeek does not. Instead this provider issues a **full Messages model call** carrying the `web_search` server tool, so one search costs a complete model turn in latency and tokens — heavier than a pure retrieval endpoint. DeepSeek runs the search server-side and returns **structured** `web_search_tool_result` blocks; the provider parses those blocks and **never scrapes URLs out of model prose**.
@@ -42,6 +35,13 @@ It reuses `$DEEPSEEK_API_KEY` (no new secret) but **not** `$DEEPSEEK_BASE_URL`:
DeepSeek returns no provider-generated answer surface this provider trusts as `content`, so `content` is omitted. `sources[]` is built from the `web_search_result` items inside `web_search_tool_result` blocks: `url` ← `url`, `title` ← `title`, `publishedAt` ← `page_age`. The per-source `snippet` lives separately in a `text` block's `citations[]` (a `cited_text` keyed by `url`), so the provider joins the two — a result with no citation excerpt simply has no `snippet`. Results are deduped by `url` (a `maxUses > 1` request can surface the same URL across searches). DeepSeek's `web_search` has no result-count knob (only `maxUses`), so `maxResults` is enforced by the seam (truncating `sources[]` and setting `truncated`). Provider failures surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Auxiliary DeepSeek search request | A separate DeepSeek model receives the search query and native `web_search` server-tool definition. This request is not part of the conversation model's context. | Separate provider input and output tokens are incurred for each search; `maxTokens` caps generated output and `maxUses` caps native search uses. |
| Conversation tool result, indirectly | Through `dsh-tool-web`, the conversation model sees deduplicated URLs, titles, dates, and citation snippets from structured search blocks; provider prose is not trusted as an answer. | Zero direct conversation tokens from registration. Result tokens scale with returned sources and snippets, then the seam enforces the requested source bound. |
## Known Limitations and Deferred Work
- **One search costs a full Messages model turn** — latency plus generated tokens, with up to `maxUses` server-side searches; DeepSeek exposes no dedicated retrieval endpoint.

12
packages/web/web-search-exa/README.md
View File

@@ -4,12 +4,6 @@ An [Exa](https://exa.ai)-backed `WebSearchProvider` for the harness [web capabil
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the `ctx.web` key and it does not register a model-facing tool (that is `@deepseek-ai/dsh-tool-web`). Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`) that registers its backend, not a default-export service.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Web search result, indirectly | Through `dsh-tool-web`, the conversation model sees Exa result URLs, titles, first highlight snippets, and publication dates. No generated answer or provider-private response fields enter the tool result. | Zero direct harness-model tokens. Result size scales with the bounded source list and snippets; the seam enforces `maxResults`, and retained results remain until compaction. |
## Config
| Key | Default | Meaning |
@@ -31,6 +25,12 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
Exa returns a flat `results[]` and no generated answer, so `content` is omitted. Each result maps to a `WebSearchSource`: `url` ← `url`, `title` ← `title`, `snippet` ← the first non-empty `highlights[]` entry (a result with no highlight has no portable snippet and is dropped), `publishedAt` ← `publishedDate`. A request's `maxResults` wins over the configured `numResults` default and is sent as Exa's `numResults` for a cost/latency optimization; the final bound is enforced by the seam. Provider failures (HTTP errors, network failure, unparseable or wrong-shape bodies) surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Web search result, indirectly | Through `dsh-tool-web`, the conversation model sees Exa result URLs, titles, first highlight snippets, and publication dates. No generated answer or provider-private response fields enter the tool result. | Zero direct harness-model tokens. Result size scales with the bounded source list and snippets; the seam enforces `maxResults`, and retained results remain until compaction. |
## Known Limitations and Deferred Work
- **A result with no non-blank highlight is dropped entirely** — no portable snippet to map, so fewer sources than the requested count can return.

14
packages/web/web-search-perplexity/README.md
View File

@@ -4,13 +4,6 @@ A [Perplexity](https://perplexity.ai)-backed `WebSearchProvider` for the harness
This is an **implementation** package: it registers a provider into `ctx.web`, it does not own the key and it does not register a model-facing tool. Like `@deepseek-ai/dsh-llm-deepseek`, it is a function/namespace plugin (`inject: ['web']`). The OpenAI-compatible wire shape is a provider-private detail — it does **not** make this provider depend on `ctx.llm`.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Auxiliary Perplexity request | A separate Perplexity model receives the search query through its chat-completions endpoint. This request is not part of the conversation model's context. | Separate provider tokens are incurred per search; `maxTokens` caps the generated answer. |
| Conversation tool result, indirectly | Through `dsh-tool-web`, the conversation model sees the generated answer plus structured result metadata or URL-only citations. | Zero direct conversation tokens from registration. Answer and source tokens are data-dependent, source count is seam-bounded, and the retained result is resent until compaction. |
## Config
| Key | Default | Meaning |
@@ -32,6 +25,13 @@ This is an **implementation** package: it registers a provider into `ctx.web`, i
`content` ← `choices[0].message.content` (the generated answer). `sources[]` prefers the structured `search_results[]` (`url`, `title`, `snippet`, `publishedAt` ← `date`), falling back to the URL-only `citations[]` array only when `search_results` is absent — those sources carry just a `url`, which is why `title`/`snippet`/`publishedAt` are optional on the seam. Provider failures surface as `WebError` `WEB_PROVIDER_ERROR`; an aborted request surfaces as `WEB_ABORTED`. Perplexity has no result-count control, so `maxResults` is enforced by the seam (truncating `sources[]` and setting `truncated`).
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Auxiliary Perplexity request | A separate Perplexity model receives the search query through its chat-completions endpoint. This request is not part of the conversation model's context. | Separate provider tokens are incurred per search; `maxTokens` caps the generated answer. |
| Conversation tool result, indirectly | Through `dsh-tool-web`, the conversation model sees the generated answer plus structured result metadata or URL-only citations. | Zero direct conversation tokens from registration. Answer and source tokens are data-dependent, source count is seam-bounded, and the retained result is resent until compaction. |
## Known Limitations and Deferred Work
- **Citation-fallback sources are URL-only** — when Perplexity omits structured `search_results[]`, sources carry no `title`/`snippet`/`publishedAt`, so the tool renders bare hostname labels.

12
packages/web/web/README.md
View File

@@ -14,12 +14,6 @@ This package is the interface third of the web capability. Unlike bash/fs it spa
Search and fetch share no request schema and no business logic, but they are deliberately one seam: `ctx.web` is a single web-access middle layer with one provider-selection policy owner, one abort/error vocabulary, and one product-facing "how this harness reaches the web" config surface. The cost is the parallel `Search`/`Fetch` method pairs; that parallelism is intentional, not a missed extraction.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| None directly | The seam registers providers, not tools or prompt text. `dsh-tool-web` renders normalized search answers, sources, fetched bodies, and structured `WebError` values. Provider selection details stay internal except for an execution error. | Zero direct tokens. The seam indirectly bounds search result tokens by truncating sources to `maxResults`; all rendered size comes through a consumer. |
## Service API (`ctx.web`)
| Member | Semantics |
@@ -49,6 +43,12 @@ The failure branches throw `WebError`, whose structured code (plus message detai
`WebSearchRequest` (`query`, `maxResults?`) → `WebSearchResult` (`providerId`, `query`, `content?`, `sources[]`, `truncated`); each `WebSearchSource` has a required `url` and optional `title`/`snippet`/`publishedAt` (Perplexity citations may be URL-only). `WebFetchRequest` (`url`, `timeoutMs?`) → `WebFetchResult` (`providerId`, final `url`, `statusCode`, `body`, `truncated`); `WebFetchBody` is a CLOSED discriminated union (`html` | `text`) owned here — consumers `switch` to exhaustiveness so a new kind breaks their compilation until handled. See `src/types.ts` for the full contracts and the `WebError` code taxonomy.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| None directly | The seam registers providers, not tools or prompt text. `dsh-tool-web` renders normalized search answers, sources, fetched bodies, and structured `WebError` values. Provider selection details stay internal except for an execution error. | Zero direct tokens. The seam indirectly bounds search result tokens by truncating sources to `maxResults`; all rendered size comes through a consumer. |
## Known Limitations and Deferred Work
- **No observation surface** — no provider-change event and no capability-status query; availability is observed only by executing `search()`/`fetch()` and routing the thrown `WebError` codes, and the no-provider failure is the generic `WEB_PROVIDER_UNAVAILABLE` with no per-provider reason enumeration ([RFC](../../../docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md)).