Merge remote-tracking branch 'origin/master' into worktree/web-multimodal-image-input
# Conflicts: # docs/architecture.i18n.yaml # docs/architecture.md # docs/architecture.zh.md # docs/config-catalog.md # docs/core-data-structures/core.i18n.yaml # docs/core-data-structures/llm-streaming.i18n.yaml # docs/module-graph.md # examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl # packages/README.i18n.yaml # packages/client/connection/src/client/fixture.ts # packages/client/connection/src/index.ts # packages/client/runtime/README.i18n.yaml # packages/client/runtime/README.md # packages/client/runtime/README.zh.md # packages/client/runtime/src/client/sessions/conversation.ts # packages/client/ui-conversation/README.i18n.yaml # packages/client/ui-conversation/src/client/apply.ts # packages/client/ui-conversation/src/client/chat/ChatView.tsx # packages/client/ui-conversation/src/client/chat/MessageItem.tsx # packages/client/ui-conversation/src/client/contract/slots.ts # packages/client/ui-trajectory/tests/views.spec.tsx # packages/compact/compact-basic/README.i18n.yaml # packages/cordis/tool-cordis/src/api-catalog.ts # packages/host/apiproxy/src/api-proxy.ts # packages/host/apiproxy/src/api/index.ts # packages/host/apiproxy/src/api/sessions.ts # packages/host/apiproxy/src/index.ts # packages/host/apiproxy/tests/fetch-carrier.spec.ts # packages/llm/llm-deepseek/src/adapter.ts # packages/llm/llm-deepseek/tests/adapter.spec.ts # packages/llm/llm-deepseek/tests/serialize.spec.ts # packages/llm/llm-pi-ai/README.i18n.yaml # packages/llm/llm-pi-ai/src/adapter.ts # packages/llm/llm-pi-ai/src/index.ts # packages/llm/llm-pi-ai/tests/adapter.spec.ts # packages/llm/llm/src/types.ts # packages/ui/tui/README.i18n.yaml # packages/ui/tui/src/index.ts # packages/ui/tui/tests/tui.spec.ts
This commit is contained in:
Yichen Jiang
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 0278a4a582e535125d001e09736b89f13be72a0c
|
||||
README.zh.md: e3e2b9559d69e4be10cd4d373bbda2dd47396b72
|
||||
# pnpm run verify-translation-pairing --write packages/llm/README.md
|
||||
README.md: 66b7beabd73cc3fec7230f209a9da0da48a37c95
|
||||
README.zh.md: 48c54358ce3e8e21e33a6ef5b75a7e095b6581d5
|
||||
|
||||
@@ -8,8 +8,8 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|
||||
|---|---|---|
|
||||
| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` |
|
||||
| `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | Bounded transient request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-retry/` | Exact-provider normal or unbounded request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API adapter (direct fetch + eventsource-parser SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
|
||||
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the interface or consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter supplies retry policy and resolves available exact-model identity, context capacity, and reasoning metadata; the retry executor and token meter remain provider-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
|
||||
@@ -8,8 +8,8 @@ LLM seam 及其提供方适配器。接口包（`llm`）拥有抽象服务、内
|
||||
|---|---|---|
|
||||
| `llm/` | 抽象 LLM 服务 + 内容块词汇 + 分片组装器 | `ctx.llm` |
|
||||
| `token-meter/` | 感知回放的请求与表层 token 测量 | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | 有界的暂时性请求重试策略 | （监听 `agent/request-error`） |
|
||||
| `llm-deepseek/` | DeepSeek API 适配器（手写 fetch/SSE） | （注册到 `ctx.llm`） |
|
||||
| `llm-retry/` | 确切提供方的 normal 或无界请求重试策略 | （监听 `agent/request-error`） |
|
||||
| `llm-deepseek/` | DeepSeek API 适配器（直接 fetch + eventsource-parser SSE） | （注册到 `ctx.llm`） |
|
||||
| `llm-pi-ai/` | 通过 `@earendil-works/pi-ai` 实现的多提供方适配器 | （注册到 `ctx.llm`） |
|
||||
|
||||
接口位于 `llm/llm/`；适配器、重试策略和可复用的 token 计量器都是该分组下的扁平兄弟包。请求按 `provider` 路由，而 `model` 会原样传给选中的适配器。拥有路由的适配器可以解析精确的提供方／模型上下文容量；token 计量器仍与模型无关。新的提供方适配器只需在 `ctx.llm` 上注册一个或多个提供方路由，无需改动接口或消费方。两个已交付实现见[双生 LLM 适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)，测量归属见[回放 token 计量器 Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md)，容量与压缩策略归属见[路由模型上下文 Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
接口位于 `llm/llm/`；适配器、重试策略和可复用的 token 计量器都是该分组下的扁平兄弟包。请求按 `provider` 路由，而 `model` 会原样传给选中的适配器。拥有路由的适配器提供重试策略，并解析可用的确切模型身份、上下文容量和推理元数据；重试执行器与 token 计量器仍与提供方无关。新的提供方适配器只需在 `ctx.llm` 上注册一个或多个提供方路由，无需改动消费方。两个已交付实现见[双生 LLM 适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)，测量归属见[回放 token 计量器 Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md)，容量与压缩策略归属见[路由模型上下文 Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: e191f3fcd265a6ca9cec3a8dae5f730ce27accf1
|
||||
README.zh.md: 268096e5f1a145e8d5cf6469524d36fe48984617
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 7a2314ea2ca0fb6606310a4961fcbc658240a7b8
|
||||
README.zh.md: 523bbbfd29b4598c024a1fff4a7121a7cb88bf41
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
DeepSeek chat-completions adapter for the harness LLM seam: hand-rolled `fetch` + SSE translation from the official wire format (source of truth: the API docs — guides/thinking_mode, guides/tool_calls, api/create-chat-completion) into the `StreamChunk` protocol.
|
||||
DeepSeek chat-completions adapter for the harness LLM seam: direct `fetch` + SSE (framed by `eventsource-parser`) translating the official wire format (source of truth: the API docs — guides/thinking_mode, guides/tool_calls, api/create-chat-completion) into the `StreamChunk` protocol.
|
||||
|
||||
A second, library-backed implementation of the same seam exists in `@deepseek-ai/dsh-llm-pi-ai`. This package always owns the `deepseek` provider route; mounting a pi-ai profile with `provider: deepseek` in the same context throws `LlmError('DUPLICATE_ADAPTER')` by design.
|
||||
|
||||
@@ -17,26 +17,32 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY # or rely on the env fallback
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
|
||||
reasoningEffort: high # optional; off | high | max — omitted ⇒ high
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
retryPolicy: # optional; omission uses bounded normal defaults
|
||||
mode: always # normal | always
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
defaultContextWindow: 256000 # optional positive-integer fallback for models without an exact value
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
name: DeepSeek V4 Flash
|
||||
name: DeepSeek-V4-Flash
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
```
|
||||
|
||||
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for UI selectors and deployment introspection, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
The plugin registers the single provider route `deepseek` together with its resolved `retryPolicy`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` as `DeepSeek-V4-Flash` and `deepseek-v4-pro` as `DeepSeek-V4-Pro`, each with a 256,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists it returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists, `context` is absent without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` wire field is not sent and the server applies its own default for the model. The only accepted values are `high` and `max` (DeepSeek's official effort levels). It is meaningful only with thinking enabled (the provider default).
|
||||
The same exact-model result exposes ordered `off`, `high`, and `max` efforts under `reasoning` for every pass-through model when deployment policy permits thinking. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header`. `high` and `max` enable thinking and serialize as the official top-level `reasoning_effort`; adapter-owned `off` instead serializes `thinking.type: disabled` and omits `reasoning_effort`. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O.
|
||||
|
||||
`thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral. A request with `GenerateOptions.purpose: 'session-title'` forces thinking disabled and omits `reasoning_effort`, reserving its bounded output for visible title text without changing conversation or compaction defaults.
|
||||
`thinking: disabled` is a deployment lock that publishes only `off` with `off` as its default. Omitting `reasoningEffort` or configuring it as `off` is valid; configuring `high` or `max` fails plugin loading, and a direct per-request attempt to enable thinking fails before network I/O. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults.
|
||||
|
||||
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; agent-level retry is a separate plugin policy.
|
||||
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; it registers the configured policy as provider metadata, and `dsh-llm-retry` separately executes it at durable agent-step boundaries.
|
||||
|
||||
## App attribution
|
||||
|
||||
@@ -45,6 +51,7 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
## Wire-format notes (verified live + against the official docs)
|
||||
|
||||
- Streaming only (`stream_options.include_usage` always on). `usage` may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to `[DONE]`, so `usage` always precedes `finish` and nothing follows `finish`.
|
||||
- The adapter-owned `off` effort maps to `thinking: {type: 'disabled'}` and never crosses the wire as `reasoning_effort: 'off'`.
|
||||
- The first thinking-mode chunk carries `reasoning_content: ""` — handled (no spurious reasoning block).
|
||||
- **Reasoning passback rule**: on assistant turns that carried tool calls, `reasoning_content` is serialized back in history (required by the API in thinking mode); on tool-call-free turns it is dropped (ignored anyway — saves tokens).
|
||||
- Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write metric.
|
||||
@@ -55,7 +62,7 @@ Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA`
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites run against a local `node:http` mock SSE server (no network), including structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
|
||||
Unit suites run against a local `node:http` mock SSE server (no network), including dynamic `high`/`off`/`max` selection, structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -81,7 +88,7 @@ Reasoning, text, and raw-string tool arguments are translated into harness chunk
|
||||
|
||||
#### Token effect
|
||||
|
||||
Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input.
|
||||
Generated tokens follow the request's logged reasoning effort and `maxTokens`; only loop-retained blocks affect later input.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
harness LLM seam 的 DeepSeek chat-completions 适配器：手写 `fetch` + SSE，将官方协议格式（真源：API 文档 guides/thinking_mode、guides/tool_calls、api/create-chat-completion）转换为 `StreamChunk` 协议。
|
||||
harness LLM seam 的 DeepSeek chat-completions 适配器：直接 `fetch` + SSE（由 `eventsource-parser` 分帧），将官方协议格式（真源：API 文档 guides/thinking_mode、guides/tool_calls、api/create-chat-completion）转换为 `StreamChunk` 协议。
|
||||
|
||||
同一 seam 的第二个库支持实现位于 `@deepseek-ai/dsh-llm-pi-ai`。本包始终拥有 `deepseek` 提供方路由；在同一上下文中装载 `provider: deepseek` 的 pi-ai profile 会按设计抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
@@ -17,26 +17,32 @@ harness LLM seam 的 DeepSeek chat-completions 适配器：手写 `fetch` + SSE
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY # or rely on the env fallback
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
|
||||
reasoningEffort: high # optional; off | high | max — omitted ⇒ high
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
retryPolicy: # optional; omission uses bounded normal defaults
|
||||
mode: always # normal | always
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
defaultContextWindow: 256000 # optional positive-integer fallback for models without an exact value
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
name: DeepSeek V4 Flash
|
||||
name: DeepSeek-V4-Flash
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
```
|
||||
|
||||
该插件注册唯一提供方路由 `deepseek`。请求使用 `provider: deepseek` 选择该路由；其 `model` 会作为协议 `model` 字符串原样传递，因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash` 和 `deepseek-v4-pro`，两者的上下文窗口均为 128,000 token；显式列表会替换这些默认值，`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek')` 公开给 UI selector 与部署自省，但仍只提供建议：未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。
|
||||
该插件注册唯一提供方路由 `deepseek`，同时注册解析后的 `retryPolicy`。请求使用 `provider: deepseek` 选择该路由；其 `model` 会作为协议 `model` 字符串原样传递，因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`（名称为 `DeepSeek-V4-Flash`）和 `deepseek-v4-pro`（名称为 `DeepSeek-V4-Pro`），两者的上下文窗口均为 256,000 token；显式列表会替换这些默认值，`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek')` 公开给 ACP（Agent Client Protocol）编辑器和 Web 选择器等客户端，但仍只提供建议：未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。
|
||||
|
||||
`contextWindow` 对每个已配置模型都可选，不会通过建议 catalog 公开。`ctx.llm.resolveModelContext('deepseek', model)` 先返回精确模型值，再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。两者都不存在时返回 `undefined`，不会使路由失效。因此，压力敏感插件可以获得部署拥有的容量，不会将模型 selector 视为权威。为 `deepseek` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
`contextWindow` 对每个已配置模型都可选，不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek', model).context` 先返回精确模型值，再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。两者都不存在时，`context` 字段缺失但不会使路由失效。因此，压力敏感插件可以获得部署拥有的容量，不会将模型 selector 视为权威。为 `deepseek` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
`reasoningEffort` 默认**省略**：未设置时，不发送 `reasoning_effort` 协议字段，服务器会为模型应用自身默认值。只接受 `high` 和 `max`（DeepSeek 官方 effort 级别）。只有在启用 thinking 时才有意义（提供方默认启用）。
|
||||
同一确切模型结果会在部署策略允许思考时，为每个原样传递模型在 `reasoning` 下公开有序的 `off`、`high` 和 `max` 推理强度。`reasoningEffort` 选择部署默认值，省略时回退为 `high`。`agent/request` 可以在每个会话步骤替换它；解析后的值会记录在 `request/header`。`high` 和 `max` 会启用思考，并序列化为官方顶层 `reasoning_effort`；适配器持有的 `off` 则序列化为 `thinking.type: disabled`，且省略 `reasoning_effort`。不支持的值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
`thinking`／`reasoningEffort` 是适配器级请求默认值，序列化为官方顶层 `thinking: {type}`／`reasoning_effort` 协议字段。它们位于适配器配置中（而非 `GenerateOptions`），以保持核心词汇与提供方无关。携带 `GenerateOptions.purpose: 'session-title'` 的请求会强制禁用 thinking 并省略 `reasoning_effort`，将有界输出保留给可见标题文本，不改变会话或压缩默认值。
|
||||
`thinking: disabled` 是部署锁定：它只公布 `off`，并以 `off` 为默认值。省略 `reasoningEffort` 或将其配置为 `off` 均有效；配置 `high` 或 `max` 会使插件加载失败，直接按请求启用思考也会在网络 I/O 前失败。携带 `GenerateOptions.purpose: 'session-title'` 的请求也会强制禁用思考并省略已解析的推理强度，将有界输出保留给可见标题文本，不改变会话或压缩默认值。
|
||||
|
||||
`streamIdleTimeoutMs` 会限制每次未完成提供方读取，包括初始 `fetch`，但不计入消费方在 chunk 间花费的时间。一个稳定 abort 信号会在整个调用中达到请求与 body reader；过期会停止传输并抛出 `LlmError('TIMEOUT')`，较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用精确发起一次提供方请求；agent 级重试是独立插件策略。
|
||||
`streamIdleTimeoutMs` 会限制每次未完成提供方读取，包括初始 `fetch`，但不计入消费方在 chunk 间花费的时间。一个稳定 abort 信号会在整个调用中达到请求与 body reader；过期会停止传输并抛出 `LlmError('TIMEOUT')`，较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用精确发起一次提供方请求；它把已配置策略注册为提供方元数据，再由 `dsh-llm-retry` 在持久 agent 步骤边界单独执行该策略。
|
||||
|
||||
## 应用归因
|
||||
|
||||
@@ -45,6 +51,7 @@ harness LLM seam 的 DeepSeek chat-completions 适配器：手写 `fetch` + SSE
|
||||
## 协议格式说明（已通过实时请求与官方文档验证）
|
||||
|
||||
- 只支持流式输出（`stream_options.include_usage` 始终开启）。`usage` 可能附着在 finish chunk 上，也可能作为尾随仅 usage chunk 到达；转换器会将两者都延迟到 `[DONE]`，因此 `usage` 始终位于 `finish` 之前，`finish` 之后不会出现任何内容。
|
||||
- 适配器持有的 `off` 推理强度映射为 `thinking: {type: 'disabled'}`，绝不会以 `reasoning_effort: 'off'` 跨越协议。
|
||||
- 第一个 thinking 模式 chunk 携带 `reasoning_content: ""`，系统会处理它（不会产生多余 reasoning 块）。
|
||||
- **Reasoning 回传规则**：对携带工具调用的 assistant 轮次，会将 `reasoning_content` 序列化回历史（thinking 模式 API 必需）；对不含工具调用的轮次，它会被丢弃（不会使用，可节省 token）。
|
||||
- Cache 计量：`cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`；DeepSeek 不报告 cache-write 指标。
|
||||
@@ -55,7 +62,7 @@ harness LLM seam 的 DeepSeek chat-completions 适配器：手写 `fetch` + SSE
|
||||
|
||||
## 测试
|
||||
|
||||
单元套件使用本地 `node:http` mock SSE 服务器（无网络），覆盖结构化 HTTP 事实、格式错误／截断流、调用方 abort、连接失败，以及 idle 超时确实会 abort 实际 body 的证明。真实 API 覆盖位于 `tests/adapter.e2e.ts`（`pnpm run test:e2e`，由 key 调节）：V4 Flash + V4 Pro，覆盖 thinking 启用／禁用与两种官方 effort 级别，包括 thinking + 工具往返与 reasoning 回传。
|
||||
单元套件使用本地 `node:http` mock SSE 服务器（无网络），覆盖动态 `high`／`off`／`max` 选择、结构化 HTTP 事实、格式错误／截断流、调用方 abort、连接失败，以及 idle 超时确实会 abort 实际 body 的证明。真实 API 覆盖位于 `tests/adapter.e2e.ts`（`pnpm run test:e2e`，由 key 调节）：V4 Flash + V4 Pro，覆盖 thinking 启用／禁用与两种官方 effort 级别，包括 thinking + 工具往返与 reasoning 回传。
|
||||
|
||||
## 模型体验
|
||||
|
||||
@@ -81,7 +88,7 @@ Reasoning、文本与原始字符串工具参数会转换为 harness chunk，供
|
||||
|
||||
#### Token 影响
|
||||
|
||||
生成 token 遵循提供方 thinking 与 effort 设置及请求的 `maxTokens`；只有 loop 保留的块会影响后续输入。
|
||||
生成 token 遵循请求中已记录的推理强度和 `maxTokens`；只有 loop 保留的块会影响后续输入。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"eventsource-parser": "^3.1.0",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -5,12 +5,14 @@
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
ResolvedRetryPolicy,
|
||||
RetryPolicyConfig,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
@@ -20,7 +22,7 @@ import { parseSse } from './sse.ts'
|
||||
import { translate } from './translate.ts'
|
||||
import type { WireError } from './types.ts'
|
||||
|
||||
/** One optional model entry advertised by the hand-written adapter. */
|
||||
/** One optional model entry advertised by the direct-fetch adapter. */
|
||||
export interface DeepSeekCatalogModel {
|
||||
/** Wire model id accepted by the configured endpoint. */
|
||||
id: string
|
||||
@@ -46,11 +48,35 @@ export interface DeepSeekAdapterOptions {
|
||||
models?: readonly DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT'
|
||||
const OFF_REASONING_EFFORT = ReasoningEffortId('off')
|
||||
const HIGH_REASONING_EFFORT = ReasoningEffortId('high')
|
||||
const MAX_REASONING_EFFORT = ReasoningEffortId('max')
|
||||
const REASONING_EFFORTS = [
|
||||
{ id: OFF_REASONING_EFFORT, name: 'Off' },
|
||||
{ id: HIGH_REASONING_EFFORT, name: 'High' },
|
||||
{ id: MAX_REASONING_EFFORT, name: 'Max' },
|
||||
] as const
|
||||
const OFF_ONLY_REASONING_EFFORTS = [
|
||||
{ id: OFF_REASONING_EFFORT, name: 'Off' },
|
||||
] as const
|
||||
|
||||
function modelInfo(provider: string, model: DeepSeekCatalogModel): LlmModelInfo {
|
||||
return {
|
||||
provider,
|
||||
id: model.id,
|
||||
name: model.name ?? model.id,
|
||||
...model.description === undefined ? {} : { description: model.description },
|
||||
inputModalities: ['text'],
|
||||
outputModalities: ['text'],
|
||||
}
|
||||
}
|
||||
|
||||
function providerRetryAfterMs(value: string | null): number | undefined {
|
||||
if (value === null) return undefined
|
||||
@@ -95,9 +121,15 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin
|
||||
*/
|
||||
export class DeepSeekAdapter extends LlmAdapter {
|
||||
private readonly streamIdleTimeoutMs: number
|
||||
private readonly retryPolicy: ResolvedRetryPolicy
|
||||
|
||||
constructor(private readonly options: DeepSeekAdapterOptions) {
|
||||
super()
|
||||
if (options.defaults?.thinking === 'disabled'
|
||||
&& options.defaults.reasoningEffort !== undefined
|
||||
&& options.defaults.reasoningEffort !== 'off') {
|
||||
throw new Error('llm-deepseek: only reasoningEffort "off" can be configured when thinking is disabled')
|
||||
}
|
||||
if (options.defaultContextWindow !== undefined
|
||||
&& (!Number.isInteger(options.defaultContextWindow) || options.defaultContextWindow <= 0)) {
|
||||
throw new Error('llm-deepseek: defaultContextWindow must be a positive integer')
|
||||
@@ -110,30 +142,52 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
`llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
this.retryPolicy = resolveRetryPolicy(options.retryPolicy, 'llm-deepseek: retryPolicy')
|
||||
}
|
||||
|
||||
override providerInfo(provider: string): LlmProviderInfo {
|
||||
return { id: provider, name: 'DeepSeek' }
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve((this.options.models ?? []).map(model => ({
|
||||
provider,
|
||||
id: model.id,
|
||||
name: model.name ?? model.id,
|
||||
...model.description === undefined ? {} : { description: model.description },
|
||||
inputModalities: ['text'],
|
||||
outputModalities: ['text'],
|
||||
})))
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve((this.options.models ?? []).map(model => modelInfo(provider, model)))
|
||||
}
|
||||
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const contextWindow = this.options.models?.find(entry => entry.id === model)?.contextWindow
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const configured = this.options.models?.find(entry => entry.id === model)
|
||||
const contextWindow = configured?.contextWindow
|
||||
?? this.options.defaultContextWindow
|
||||
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
|
||||
return Promise.resolve({
|
||||
...configured === undefined
|
||||
? { provider, id: model, name: model }
|
||||
: modelInfo(provider, configured),
|
||||
...contextWindow === undefined ? {} : { context: { contextWindow } },
|
||||
...this.options.defaults?.thinking === 'disabled'
|
||||
? {
|
||||
reasoning: {
|
||||
efforts: OFF_ONLY_REASONING_EFFORTS,
|
||||
defaultEffort: OFF_REASONING_EFFORT,
|
||||
},
|
||||
}
|
||||
: {
|
||||
reasoning: {
|
||||
efforts: REASONING_EFFORTS,
|
||||
defaultEffort: this.options.defaults?.reasoningEffort === 'off'
|
||||
? OFF_REASONING_EFFORT
|
||||
: this.options.defaults?.reasoningEffort === 'max'
|
||||
? MAX_REASONING_EFFORT
|
||||
: HIGH_REASONING_EFFORT,
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
|
||||
import type { DeepSeekCatalogModel } from './adapter.ts'
|
||||
@@ -21,31 +22,34 @@ export const name = 'llm-deepseek'
|
||||
export const inject = ['llm']
|
||||
|
||||
const DEFAULT_MODELS: DeepSeekCatalogModel[] = [
|
||||
{ id: 'deepseek-v4-flash', contextWindow: 128_000 },
|
||||
{ id: 'deepseek-v4-pro', contextWindow: 128_000 },
|
||||
{ id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', contextWindow: 256_000 },
|
||||
{ id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', contextWindow: 256_000 },
|
||||
]
|
||||
|
||||
/**
|
||||
* Plugin config, validated by the same-named schemastery schema. Every field
|
||||
* is optional in yml: credentials/endpoint fall back to the environment (a
|
||||
* missing API key fails plugin load, not the first call), and omitted
|
||||
* thinking fields send nothing on the wire, so the provider default applies.
|
||||
* missing API key fails plugin load, not the first call), omitted thinking
|
||||
* mode uses the provider default, and omitted reasoning effort resolves to
|
||||
* `high`.
|
||||
*/
|
||||
export interface Config {
|
||||
/** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */
|
||||
apiKey?: string
|
||||
/** Endpoint base; falls back to $DEEPSEEK_BASE_URL, then the public API. */
|
||||
baseURL?: string
|
||||
/** Thinking-mode default for every request (provider default: enabled). */
|
||||
/** Deployment thinking policy; `disabled` limits every conversation request to `off`. */
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
/** Thinking effort (only meaningful with thinking enabled). */
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
/** Default thinking effort (default `high`); `off` disables thinking per request. */
|
||||
reasoningEffort?: 'off' | 'high' | 'max'
|
||||
/** Positive context capacity used when the selected model has no exact value. */
|
||||
defaultContextWindow?: number
|
||||
/** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
const catalogModel: z<DeepSeekCatalogModel> = z.object({
|
||||
@@ -59,10 +63,11 @@ export const Config: z<Config> = z.object({
|
||||
apiKey: z.string(),
|
||||
baseURL: z.string(),
|
||||
thinking: z.union(['enabled', 'disabled']),
|
||||
reasoningEffort: z.union(['high', 'max']),
|
||||
reasoningEffort: z.union(['off', 'high', 'max']),
|
||||
defaultContextWindow: z.number().step(1).min(1),
|
||||
models: z.array(catalogModel).default(DEFAULT_MODELS),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
retryPolicy: RetryPolicySchema,
|
||||
})
|
||||
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
@@ -94,6 +99,11 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee
|
||||
}
|
||||
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
if (config.thinking === 'disabled'
|
||||
&& config.reasoningEffort !== undefined
|
||||
&& config.reasoningEffort !== 'off') {
|
||||
throw new Error('llm-deepseek: only reasoningEffort "off" can be configured when thinking is disabled')
|
||||
}
|
||||
const apiKey = config.apiKey ?? process.env.DEEPSEEK_API_KEY
|
||||
if (apiKey === undefined || apiKey.length === 0) {
|
||||
throw new Error('llm-deepseek: an API key is required (Config.apiKey or $DEEPSEEK_API_KEY)')
|
||||
@@ -111,5 +121,6 @@ export function apply(ctx: Context, config: Config): void {
|
||||
: { defaultContextWindow: config.defaultContextWindow },
|
||||
models: resolveModels(config.models),
|
||||
streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
...config.retryPolicy === undefined ? {} : { retryPolicy: config.retryPolicy },
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -14,7 +14,42 @@ import type { WireMessage, WireRequest, WireTool } from './types.ts'
|
||||
/** Adapter-level request defaults (from plugin config). */
|
||||
export interface RequestDefaults {
|
||||
thinking?: 'enabled' | 'disabled' | undefined
|
||||
reasoningEffort?: 'high' | 'max' | undefined
|
||||
reasoningEffort?: 'off' | 'high' | 'max' | undefined
|
||||
}
|
||||
|
||||
interface ResolvedThinking {
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
}
|
||||
|
||||
/** Validate the adapter-owned effort before resolving its DeepSeek wire fields. */
|
||||
function reasoningEffort(effort: NonNullable<GenerateOptions['reasoningEffort']>): 'off' | 'high' | 'max' {
|
||||
if (effort === 'off' || effort === 'high' || effort === 'max') {
|
||||
return effort as 'off' | 'high' | 'max'
|
||||
}
|
||||
throw new LlmError(
|
||||
`DeepSeek does not support reasoning effort "${effort}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
|
||||
/** Resolve one legal thinking/effort pair without exposing `off` as a wire effort. */
|
||||
function resolveThinking(options: GenerateOptions, defaults: RequestDefaults): ResolvedThinking {
|
||||
if (options.purpose === 'session-title') return { thinking: 'disabled' }
|
||||
const effort = options.reasoningEffort === undefined
|
||||
? defaults.reasoningEffort
|
||||
: reasoningEffort(options.reasoningEffort)
|
||||
if (defaults.thinking === 'disabled' && effort !== undefined && effort !== 'off') {
|
||||
throw new LlmError(
|
||||
`DeepSeek deployment does not support reasoning effort "${effort}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
if (effort === 'off') return { thinking: 'disabled' }
|
||||
if (effort === 'high' || effort === 'max') {
|
||||
return { thinking: 'enabled', reasoningEffort: effort }
|
||||
}
|
||||
return defaults.thinking === undefined ? {} : { thinking: defaults.thinking }
|
||||
}
|
||||
|
||||
/** Join the text blocks of a message (used for user/tool-result content). */
|
||||
@@ -133,16 +168,17 @@ export function serializeRequest(options: GenerateOptions, defaults: RequestDefa
|
||||
}))
|
||||
// A short title budget must produce visible text; conversation and
|
||||
// compaction calls continue to inherit the adapter's thinking defaults.
|
||||
const thinking = options.purpose === 'session-title' ? 'disabled' : defaults.thinking
|
||||
const reasoningEffort = options.purpose === 'session-title' ? undefined : defaults.reasoningEffort
|
||||
const resolvedThinking = resolveThinking(options, defaults)
|
||||
|
||||
return {
|
||||
model: options.model,
|
||||
messages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...thinking !== undefined ? { thinking: { type: thinking } } : {},
|
||||
...reasoningEffort !== undefined ? { reasoning_effort: reasoningEffort } : {},
|
||||
...resolvedThinking.thinking !== undefined ? { thinking: { type: resolvedThinking.thinking } } : {},
|
||||
...resolvedThinking.reasoningEffort !== undefined
|
||||
? { reasoning_effort: resolvedThinking.reasoningEffort }
|
||||
: {},
|
||||
...tools !== undefined && tools.length > 0 ? { tools } : {},
|
||||
...options.temperature !== undefined ? { temperature: options.temperature } : {},
|
||||
...options.maxTokens !== undefined ? { max_tokens: options.maxTokens } : {},
|
||||
|
||||
@@ -1,65 +1,33 @@
|
||||
/**
|
||||
* Decode an SSE byte stream into event `data` payloads. Network reads may split UTF-8 or lines;
|
||||
* CRLF, comments, non-data fields, and multi-data events are handled per SSE rules. The literal
|
||||
* `[DONE]` is yielded so the caller owns final flushing, and EOF before it raises {@link LlmError}.
|
||||
* Decode an SSE byte stream into event `data` payloads. Framing — chunk
|
||||
* reassembly, UTF-8/CRLF/BOM handling, comment and non-data field skipping,
|
||||
* multi-`data:` joining — is `eventsource-parser`'s; this module keeps only
|
||||
* the DeepSeek protocol: the literal `[DONE]` is yielded so the caller owns
|
||||
* final flushing, and EOF before it raises {@link LlmError}. Framing is
|
||||
* spec-strict: an event dispatches only on its blank-line terminator, so an
|
||||
* unterminated tail at EOF is truncation, not a flushable payload.
|
||||
*
|
||||
* Minimal SSE (text/event-stream) parser for the chat-completions stream.
|
||||
* @module dsh-llm-deepseek/sse
|
||||
*/
|
||||
|
||||
import { EventSourceParserStream } from 'eventsource-parser/stream'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** The terminal payload DeepSeek (and OpenAI) send after the last chunk. */
|
||||
export const DONE = '[DONE]'
|
||||
|
||||
/** Extract the joined data payload from one raw SSE event block. */
|
||||
function eventData(block: string): string | undefined {
|
||||
const data: string[] = []
|
||||
for (const rawLine of block.split('\n')) {
|
||||
const line = rawLine.endsWith('\r') ? rawLine.slice(0, -1) : rawLine
|
||||
if (line.startsWith('data:')) {
|
||||
// The spec strips ONE leading space after the colon.
|
||||
data.push(line.startsWith('data: ') ? line.slice(6) : line.slice(5))
|
||||
}
|
||||
// Comments (':…') and other fields (event:, id:, retry:) are ignored.
|
||||
}
|
||||
if (data.length === 0) return undefined
|
||||
return data.join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a byte stream into SSE data payloads. Yields `[DONE]` as the final
|
||||
* Parse an SSE byte stream into data payloads. Yields `[DONE]` as the final
|
||||
* value and returns; throws `LlmError('STREAM_CLOSED')` when the stream ends
|
||||
* without it (truncated response — the model call cannot be trusted).
|
||||
* @param stream - raw SSE bytes; reads may split anywhere, including mid-UTF-8 sequence.
|
||||
* @returns each event's data payload in arrival order, the `[DONE]` sentinel last.
|
||||
*/
|
||||
export async function* parseSse(stream: AsyncIterable<Uint8Array>): AsyncGenerator<string> {
|
||||
const decoder = new TextDecoder()
|
||||
let buffer = ''
|
||||
|
||||
for await (const bytes of stream) {
|
||||
buffer += decoder.decode(bytes, { stream: true })
|
||||
// Events are separated by a blank line (\n\n; tolerate \r\n\r\n via the
|
||||
// per-line \r strip in eventData and a normalized split here).
|
||||
let boundary: number
|
||||
while ((boundary = buffer.search(/\r?\n\r?\n/)) !== -1) {
|
||||
const matched = /\r?\n\r?\n/.exec(buffer.slice(boundary))
|
||||
const block = buffer.slice(0, boundary)
|
||||
// matched cannot be null: search() just found the same pattern at 0.
|
||||
buffer = buffer.slice(boundary + (matched as RegExpExecArray)[0].length)
|
||||
const data = eventData(block)
|
||||
if (data === undefined) continue
|
||||
yield data
|
||||
if (data === DONE) return
|
||||
}
|
||||
}
|
||||
|
||||
// Flush any final un-terminated event (servers usually end with \n\n, but
|
||||
// a trailing block without one is still parseable).
|
||||
buffer += decoder.decode()
|
||||
const data = eventData(buffer)
|
||||
if (data !== undefined) {
|
||||
export async function* parseSse(stream: ReadableStream<BufferSource>): AsyncGenerator<string> {
|
||||
const events = stream
|
||||
.pipeThrough(new TextDecoderStream())
|
||||
.pipeThrough(new EventSourceParserStream())
|
||||
for await (const { data } of events) {
|
||||
yield data
|
||||
if (data === DONE) return
|
||||
}
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import type { Config } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { assemble, type AssembledResult } from './assemble.ts'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the hand-rolled adapter: V4 Flash + V4 Pro across
|
||||
* Real-API e2e for the direct-fetch adapter: V4 Flash + V4 Pro across
|
||||
* thinking modes and both official effort levels. Key-gated — skips
|
||||
* entirely without $DEEPSEEK_API_KEY (see vitest.e2e.config.ts).
|
||||
*/
|
||||
@@ -50,41 +50,40 @@ const weatherTool: ToolSchema = {
|
||||
}
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () => {
|
||||
it('flash + thinking disabled: plain text generation', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'disabled' })
|
||||
const result = await assemble(ctx,{
|
||||
it('flash dynamically switches from off to high', async () => {
|
||||
const ctx = await harness(FLASH, { reasoningEffort: 'off' })
|
||||
const withoutThinking = await assemble(ctx,{
|
||||
model: FLASH,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(textOf(result).toLowerCase()).toContain('pong')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
expect(result.usage?.inputTokens).toBeGreaterThan(0)
|
||||
expect(result.usage?.outputTokens).toBeGreaterThan(0)
|
||||
})
|
||||
expect(withoutThinking.finish.kind).toBe('stop')
|
||||
expect(textOf(withoutThinking).toLowerCase()).toContain('pong')
|
||||
expect(withoutThinking.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
expect(withoutThinking.usage?.inputTokens).toBeGreaterThan(0)
|
||||
expect(withoutThinking.usage?.outputTokens).toBeGreaterThan(0)
|
||||
|
||||
it('flash + thinking enabled (effort high): reasoning blocks + reasoning tokens', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'enabled', reasoningEffort: 'high' })
|
||||
const result = await assemble(ctx,{
|
||||
const withThinking = await assemble(ctx,{
|
||||
model: FLASH,
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(true)
|
||||
expect(textOf(result)).toContain('9.8')
|
||||
expect(result.usage?.reasoningTokens).toBeGreaterThan(0)
|
||||
expect(withThinking.finish.kind).toBe('stop')
|
||||
expect(withThinking.message.content.some(block => block.type === 'reasoning')).toBe(true)
|
||||
expect(textOf(withThinking)).toContain('9.8')
|
||||
expect(withThinking.usage?.reasoningTokens).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it.each(['high', 'max'] as const)(
|
||||
'pro + thinking enabled (effort %s): tool-call round trip with reasoning passback',
|
||||
async (effort) => {
|
||||
const ctx = await harness(PRO, { thinking: 'enabled', reasoningEffort: effort })
|
||||
const ctx = await harness(PRO, { thinking: 'enabled' })
|
||||
|
||||
// Turn 1: the model must call the tool (and think before it).
|
||||
const first = await assemble(ctx,{
|
||||
model: PRO,
|
||||
reasoningEffort: ReasoningEffortId(effort),
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
@@ -99,6 +98,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
|
||||
// block in history (the official thinking+tools passback rule).
|
||||
const second = await assemble(ctx,{
|
||||
model: PRO,
|
||||
reasoningEffort: ReasoningEffortId(effort),
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
{ role: 'assistant', content: first.message.content },
|
||||
|
||||
@@ -8,6 +8,7 @@ import LlmService, {
|
||||
LlmError,
|
||||
ProviderRequestId,
|
||||
QUOTA_EXCEEDED_CODE,
|
||||
ReasoningEffortId,
|
||||
userAgent,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
@@ -120,6 +121,7 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
// The wire request carried the auth header contents we configured.
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoning_effort: 'high',
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
})
|
||||
@@ -173,9 +175,45 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
expect(server.headers[0]?.['x-deepseek-harness-compact']).toBe('1')
|
||||
})
|
||||
|
||||
it('forwards thinking config onto the wire', async () => {
|
||||
it('switches dynamically from the configured high default through off to max', async () => {
|
||||
const server = await mockServer([
|
||||
{ kind: 'sse', events: textEvents },
|
||||
{ kind: 'sse', events: textEvents },
|
||||
{ kind: 'sse', events: textEvents },
|
||||
])
|
||||
const ctx = await harness(server.url, { thinking: 'enabled', reasoningEffort: 'high' })
|
||||
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('off'),
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi again' }] }],
|
||||
})
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'one more time' }] }],
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'enabled' },
|
||||
reasoning_effort: 'high',
|
||||
})
|
||||
expect(server.requests[1]).toMatchObject({
|
||||
thinking: { type: 'disabled' },
|
||||
})
|
||||
expect(server.requests[1]).not.toHaveProperty('reasoning_effort')
|
||||
expect(server.requests[2]).toMatchObject({
|
||||
thinking: { type: 'enabled' },
|
||||
reasoning_effort: 'max',
|
||||
})
|
||||
})
|
||||
|
||||
it('publishes only off and omits the wire effort when thinking is disabled', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url, { thinking: 'disabled', reasoningEffort: 'high' })
|
||||
const ctx = await harness(server.url, { thinking: 'disabled' })
|
||||
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
@@ -183,10 +221,52 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'disabled' },
|
||||
reasoning_effort: 'high',
|
||||
})
|
||||
expect(server.requests[0]).not.toHaveProperty('reasoning_effort')
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
|
||||
defaultEffort: ReasoningEffortId('off'),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects a per-request effort before I/O when thinking is disabled', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = await harness(server.url, { thinking: 'disabled' })
|
||||
|
||||
await expect(assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
expect(server.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it.each(['high', 'max'])(
|
||||
'rejects direct adapter effort %s before I/O when thinking is disabled',
|
||||
async (effort) => {
|
||||
const server = await mockServer([])
|
||||
const adapter = new DeepSeekAdapter({
|
||||
apiKey: 'test-key',
|
||||
baseURL: server.url,
|
||||
defaults: { thinking: 'disabled' },
|
||||
})
|
||||
|
||||
const stream = adapter.stream({
|
||||
provider: 'deepseek',
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId(effort),
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
await expect(async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
}).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
expect(server.requests).toHaveLength(0)
|
||||
},
|
||||
)
|
||||
|
||||
it.each([
|
||||
[401, 'AUTH'],
|
||||
[403, 'AUTH'],
|
||||
@@ -522,17 +602,129 @@ describe('plugin registration and config', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('registers retryPolicy from the provider config', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
retryPolicy: {
|
||||
mode: 'always',
|
||||
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
|
||||
},
|
||||
})
|
||||
|
||||
expect(ctx.llm.providerRetryPolicy('deepseek')).toEqual({
|
||||
mode: 'always',
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0.2,
|
||||
})
|
||||
})
|
||||
|
||||
it('owns the deepseek provider and advertises the default models', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'deepseek', name: 'DeepSeek' }])
|
||||
await expect(ctx.llm.listModels('deepseek')).resolves.toEqual([
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'deepseek-v4-flash', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'deepseek-v4-pro', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
])
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toEqual({ contextWindow: 128_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toMatchObject({
|
||||
provider: 'deepseek',
|
||||
id: 'deepseek-v4-flash',
|
||||
name: 'DeepSeek-V4-Flash',
|
||||
context: { contextWindow: 256_000 },
|
||||
reasoning: {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('off'), name: 'Off' },
|
||||
{ id: ReasoningEffortId('high'), name: 'High' },
|
||||
{ id: ReasoningEffortId('max'), name: 'Max' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it.each(['off', 'max'] as const)('uses the configured %s reasoning default', async (effort) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
reasoningEffort: effort,
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted-pass-through'))
|
||||
.resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('off'), name: 'Off' },
|
||||
{ id: ReasoningEffortId('high'), name: 'High' },
|
||||
{ id: ReasoningEffortId('max'), name: 'Max' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId(effort),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('accepts off as the default when thinking is deployment-disabled', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
thinking: 'disabled',
|
||||
reasoningEffort: 'off',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted-pass-through'))
|
||||
.resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
|
||||
defaultEffort: ReasoningEffortId('off'),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it.each(['high', 'max'] as const)(
|
||||
'rejects configured reasoning effort %s when thinking is disabled',
|
||||
async (reasoningEffort) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
thinking: 'disabled',
|
||||
reasoningEffort,
|
||||
})).rejects.toThrow(/only reasoningEffort "off"/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
},
|
||||
)
|
||||
|
||||
it.each(['high', 'max'] as const)(
|
||||
'rejects disabled-thinking effort %s at the direct constructor boundary',
|
||||
(reasoningEffort) => {
|
||||
expect(() => new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
defaults: { thinking: 'disabled', reasoningEffort },
|
||||
})).toThrow(/only reasoningEffort "off"/)
|
||||
},
|
||||
)
|
||||
|
||||
it('accepts disabled thinking with off at the direct constructor boundary', async () => {
|
||||
const adapter = new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
defaults: { thinking: 'disabled', reasoningEffort: 'off' },
|
||||
})
|
||||
await expect(adapter.resolveModel('deepseek', 'pass-through')).resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
|
||||
defaultEffort: ReasoningEffortId('off'),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('uses the default model catalog when apply is called directly', async () => {
|
||||
@@ -540,8 +732,8 @@ describe('plugin registration and config', () => {
|
||||
await ctx.plugin(LlmService)
|
||||
LlmDeepSeek.apply(ctx, { apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
await expect(ctx.llm.listModels('deepseek')).resolves.toEqual([
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'deepseek-v4-flash', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'deepseek-v4-pro', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'DeepSeek-V4-Pro', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
])
|
||||
})
|
||||
|
||||
@@ -565,10 +757,15 @@ describe('plugin registration and config', () => {
|
||||
{ provider: 'deepseek', id: 'private-fast', name: 'private-fast', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
{ provider: 'deepseek', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget', inputModalities: ['text'], outputModalities: ['text'] },
|
||||
])
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'private-fast'))
|
||||
.resolves.toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'arbitrary-unlisted'))
|
||||
.resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'private-fast'))
|
||||
.resolves.toMatchObject({ context: { contextWindow: 32_000 } })
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'private-reasoner'))
|
||||
.resolves.toMatchObject({
|
||||
name: 'Private Reasoner',
|
||||
description: 'Higher reasoning budget',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'arbitrary-unlisted'))
|
||||
.resolves.not.toHaveProperty('context')
|
||||
})
|
||||
|
||||
it('uses exact model capacity before the adapter-wide default', async () => {
|
||||
@@ -584,12 +781,12 @@ describe('plugin registration and config', () => {
|
||||
],
|
||||
})
|
||||
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'inherits-default'))
|
||||
.resolves.toEqual({ contextWindow: 256_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'exact-override'))
|
||||
.resolves.toEqual({ contextWindow: 64_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'unlisted-pass-through'))
|
||||
.resolves.toEqual({ contextWindow: 256_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'inherits-default'))
|
||||
.resolves.toMatchObject({ context: { contextWindow: 256_000 } })
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'exact-override'))
|
||||
.resolves.toMatchObject({ context: { contextWindow: 64_000 } })
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted-pass-through'))
|
||||
.resolves.toMatchObject({ context: { contextWindow: 256_000 } })
|
||||
})
|
||||
|
||||
it('allows an explicit empty model catalog', async () => {
|
||||
@@ -731,4 +928,16 @@ describe('plugin registration and config', () => {
|
||||
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
|
||||
})).rejects.toThrow(/streamIdleTimeoutMs/)
|
||||
})
|
||||
|
||||
it('rejects invalid nested retryPolicy before registering the provider', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
retryPolicy: { mode: 'normal', maxRetries: -1 },
|
||||
})).rejects.toThrow(/retryPolicy/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { serializeMessages, serializeRequest } from '../src/serialize.ts'
|
||||
@@ -188,15 +188,47 @@ describe('serializeRequest', () => {
|
||||
expect(wire.tools).toBeUndefined()
|
||||
})
|
||||
|
||||
it('applies adapter defaults for thinking and effort', () => {
|
||||
const wire = serializeRequest(request({ messages: history }), { thinking: 'enabled', reasoningEffort: 'max' })
|
||||
it('maps adapter-default thinking and the request reasoning effort', () => {
|
||||
const wire = serializeRequest(
|
||||
request({ messages: history, reasoningEffort: ReasoningEffortId('max') }),
|
||||
{ thinking: 'enabled', reasoningEffort: 'high' },
|
||||
)
|
||||
expect(wire.thinking).toEqual({ type: 'enabled' })
|
||||
expect(wire.reasoning_effort).toBe('max')
|
||||
})
|
||||
|
||||
it('maps off to disabled thinking without a wire reasoning effort', () => {
|
||||
const wire = serializeRequest(
|
||||
request({ messages: history, reasoningEffort: ReasoningEffortId('off') }),
|
||||
{ thinking: 'enabled', reasoningEffort: 'max' },
|
||||
)
|
||||
expect(wire.thinking).toEqual({ type: 'disabled' })
|
||||
expect(wire.reasoning_effort).toBeUndefined()
|
||||
})
|
||||
|
||||
it('re-enables thinking when max overrides an off default', () => {
|
||||
const wire = serializeRequest(
|
||||
request({ messages: history, reasoningEffort: ReasoningEffortId('max') }),
|
||||
{ reasoningEffort: 'off' },
|
||||
)
|
||||
expect(wire.thinking).toEqual({ type: 'enabled' })
|
||||
expect(wire.reasoning_effort).toBe('max')
|
||||
})
|
||||
|
||||
it('rejects enabling thinking when the deployment is locked to disabled', () => {
|
||||
expect(() => serializeRequest(
|
||||
request({ messages: history, reasoningEffort: ReasoningEffortId('high') }),
|
||||
{ thinking: 'disabled' },
|
||||
)).toThrow(expect.objectContaining({ code: 'UNSUPPORTED_REASONING_EFFORT' }))
|
||||
})
|
||||
|
||||
it('disables thinking for session-title requests without changing adapter defaults', () => {
|
||||
const wire = serializeRequest(
|
||||
request({ messages: history, purpose: 'session-title' }),
|
||||
request({
|
||||
messages: history,
|
||||
purpose: 'session-title',
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
}),
|
||||
{ thinking: 'enabled', reasoningEffort: 'max' },
|
||||
)
|
||||
expect(wire.thinking).toEqual({ type: 'disabled' })
|
||||
@@ -208,6 +240,19 @@ describe('serializeRequest', () => {
|
||||
expect(wire.thinking).toBeUndefined()
|
||||
expect(wire.reasoning_effort).toBeUndefined()
|
||||
})
|
||||
|
||||
it('preserves an explicit enabled default without inventing a wire effort', () => {
|
||||
const wire = serializeRequest(request({ messages: history }), { thinking: 'enabled' })
|
||||
expect(wire.thinking).toEqual({ type: 'enabled' })
|
||||
expect(wire.reasoning_effort).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects an effort outside the DeepSeek capability', () => {
|
||||
expect(() => serializeRequest(request({
|
||||
messages: history,
|
||||
reasoningEffort: ReasoningEffortId('medium'),
|
||||
}))).toThrow(expect.objectContaining({ code: 'UNSUPPORTED_REASONING_EFFORT' }))
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes: assistant content shapes', () => {
|
||||
|
||||
@@ -2,12 +2,21 @@ import { describe, expect, it } from 'vitest'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE, parseSse } from '../src/sse.ts'
|
||||
|
||||
/** Build a byte stream from string fragments (fragments = network reads). */
|
||||
async function* bytes(...fragments: (string | Uint8Array)[]): AsyncGenerator<Uint8Array> {
|
||||
/**
|
||||
* DeepSeek protocol contract only: the [DONE] sentinel and STREAM_CLOSED on
|
||||
* EOF without it. SSE framing (chunk splits, CRLF, multi-data joins, comments)
|
||||
* is eventsource-parser's contract, not re-proven here.
|
||||
*/
|
||||
|
||||
/** Build an SSE byte stream from string fragments (fragments = network reads). */
|
||||
function bytes(...fragments: string[]): ReadableStream<Uint8Array<ArrayBuffer>> {
|
||||
const encoder = new TextEncoder()
|
||||
for (const fragment of fragments) {
|
||||
yield typeof fragment === 'string' ? encoder.encode(fragment) : fragment
|
||||
}
|
||||
return new ReadableStream({
|
||||
start(controller) {
|
||||
for (const fragment of fragments) controller.enqueue(encoder.encode(fragment))
|
||||
controller.close()
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async function collect(stream: AsyncIterable<string>): Promise<string[]> {
|
||||
@@ -17,57 +26,14 @@ async function collect(stream: AsyncIterable<string>): Promise<string[]> {
|
||||
}
|
||||
|
||||
describe('parseSse', () => {
|
||||
it('parses simple events and the DONE sentinel', async () => {
|
||||
it('yields event payloads and the DONE sentinel', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('handles events split across reads at arbitrary positions', async () => {
|
||||
const events = await collect(parseSse(bytes('da', 'ta: {"a"', ':1}\n', '\ndata: [DO', 'NE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('handles multi-byte UTF-8 split across reads', async () => {
|
||||
const encoded = new TextEncoder().encode('data: {"text":"日本語"}\n\ndata: [DONE]\n\n')
|
||||
// Split inside the 3-byte sequence for 日.
|
||||
const splitAt = 16
|
||||
const events = await collect(parseSse(bytes(encoded.slice(0, splitAt), encoded.slice(splitAt))))
|
||||
expect(events).toEqual(['{"text":"日本語"}', DONE])
|
||||
})
|
||||
|
||||
it('tolerates CRLF line endings', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\r\n\r\ndata: [DONE]\r\n\r\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('joins multi-data events with newlines (SSE spec)', async () => {
|
||||
const events = await collect(parseSse(bytes('data: line1\ndata: line2\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['line1\nline2', DONE])
|
||||
})
|
||||
|
||||
it('ignores comments and non-data fields', async () => {
|
||||
const events = await collect(parseSse(bytes(': keepalive\nevent: chunk\nid: 7\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('skips blocks without data fields', async () => {
|
||||
const events = await collect(parseSse(bytes(': ping\n\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('preserves data lines without the optional space', async () => {
|
||||
const events = await collect(parseSse(bytes('data:{"a":1}\n\ndata:[DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('parses several events from one read', async () => {
|
||||
const events = await collect(parseSse(bytes('data: 1\n\ndata: 2\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['1', '2', DONE])
|
||||
})
|
||||
|
||||
it('flushes a final un-terminated DONE at stream end', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\n\ndata: [DONE]')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
it('stops yielding after DONE even when more data follows', async () => {
|
||||
const events = await collect(parseSse(bytes('data: [DONE]\n\ndata: {"late":1}\n\n')))
|
||||
expect(events).toEqual([DONE])
|
||||
})
|
||||
|
||||
it('throws STREAM_CLOSED when the stream ends without DONE', async () => {
|
||||
@@ -83,26 +49,10 @@ describe('parseSse', () => {
|
||||
await expect(collect(parseSse(bytes('data: {"a"')))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('stops yielding after DONE even when more data follows', async () => {
|
||||
const events = await collect(parseSse(bytes('data: [DONE]\n\ndata: {"late":1}\n\n')))
|
||||
expect(events).toEqual([DONE])
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseSse edge branches', () => {
|
||||
it('handles a lone CR-terminated data line', async () => {
|
||||
// Exercises the \r-strip branch on a line that is ONLY "data:…\r".
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\r\n\r\ndata:[DONE]\r\n\r\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('strips CR from non-data field lines too', async () => {
|
||||
const events = await collect(parseSse(bytes('event: chunk\r\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('treats bare "data:" lines as empty payload entries', async () => {
|
||||
const events = await collect(parseSse(bytes('data:\ndata: x\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['\nx', DONE])
|
||||
it('treats a final DONE missing its blank-line terminator as truncation', async () => {
|
||||
// Spec-strict framing: an event dispatches only on its blank-line
|
||||
// terminator, so an unterminated tail at EOF is STREAM_CLOSED — real
|
||||
// providers always terminate events, so a missing terminator is truncation.
|
||||
await expect(collect(parseSse(bytes('data: {"a":1}\n\ndata: [DONE]')))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 3952b5f802e4459ec31c819d280e6b8486fe2160
|
||||
README.zh.md: a57885dc2e2f3b4595d1552d44227fe6d36a5274
|
||||
README.md: 3cd64b8170ac0f6b6c4316f19bb816c65c223935
|
||||
README.zh.md: cd6a689d1a099002febb2e5bd003b4cf619ba565
|
||||
|
||||
@@ -19,6 +19,13 @@ Configure credentials and deployment-specific transport settings per provider. O
|
||||
apiKey: !!js process.env.OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
reasoning: high
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 3
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: anthropic
|
||||
apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
@@ -30,9 +37,11 @@ Configure credentials and deployment-specific transport settings per provider. O
|
||||
|
||||
Each provider name must exist in pi-ai's installed catalog and may appear only once in this plugin instance. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
|
||||
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelContext(provider, model)` performs the same exact descriptor lookup and returns its context window, keeping capacity metadata on the route-owning adapter rather than a consuming plugin.
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers.
|
||||
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
The `reasoning.efforts` list is pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model therefore exposes pi-ai's `off` choice. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -51,6 +60,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
|
||||
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. A terminal `stop` whose message carries no content blocks maps to a `finish {kind:'error'}` with code `EMPTY_RESPONSE` (retried by default policy) instead of a successful empty message.
|
||||
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
|
||||
- pi-ai's `off` thinking level crosses the Harness capability seam unchanged and becomes an omitted pi-ai common `reasoning` option at dispatch.
|
||||
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
|
||||
|
||||
## App attribution
|
||||
@@ -101,4 +111,4 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes.
|
||||
- **Retry policy is not an adapter option** — SDK retries are disabled so durable agent steps and `llm/retry` events own every visible attempt; direct `ctx.llm.stream()` calls remain single-attempt.
|
||||
- **Retry policy is provider-owned, not an SDK retry** — each provider profile may configure nested `retryPolicy`, which `dsh-llm-retry` executes at the agent failed-step seam; pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt.
|
||||
|
||||
@@ -19,6 +19,13 @@
|
||||
apiKey: !!js process.env.OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
reasoning: high
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 3
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: anthropic
|
||||
apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
@@ -30,9 +37,11 @@
|
||||
|
||||
每个提供方名称必须存在于 pi-ai 已安装 catalog 中，且在此插件实例中最多出现一次。向 `ctx.llm` 注册具有原子性：如果与另一适配器已拥有的任何提供方路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；未知模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
|
||||
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每个已配置提供方已安装的 pi-ai 模型。这是从 `getModels(provider)` 派生的提供方无关 selector 元数据；请求时解析仍会执行权威 catalog 查找，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelContext(provider, model)` 执行相同的精确 descriptor 查找并返回其上下文窗口，让容量元数据保留在拥有路由的适配器上，而非消费插件上。
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每个已配置提供方已安装的 pi-ai 模型。这是从 `getModels(provider)` 派生的提供方无关 selector 元数据；请求时解析仍会执行权威 catalog 查找，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找，并返回其身份、上下文窗口和可选思考级别，让权威元数据保留在拥有路由的适配器上，而非消费方。
|
||||
|
||||
受支持的 profile 字段是 `provider`、`apiKey`、`baseURL`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs` 和 `streamIdleTimeoutMs`。流 idle 间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。Harness 应用归因会胜过名称冲突的已配置标头。
|
||||
`reasoning.efforts` 列表是 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果，不经筛选或规范化，其中包括 `off`，以及模型对 `xhigh` 或 `max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID；提供方／模型协议拼写仍保留在 pi-ai 的 `thinkingLevelMap` 中。因此，不具备推理（reasoning）能力的模型也会公开 pi-ai 的 `off` 选项。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；任何未出现在确切模型能力中的显式值都会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `provider`、`apiKey`、`baseURL`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的 normal 默认值。流 idle 间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。Harness 应用归因会胜过名称冲突的已配置标头。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent 级重试预算。Idle 过期会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -51,6 +60,7 @@
|
||||
- pi-ai 工具调用参数是已解析对象；harness 存储原始 JSON 字符串。适配器会解析输入，并将输出重新字符串化。
|
||||
- pi-ai 将失败报告为流内错误事件；它们会映射到 `finish {kind:'error'|'aborted', failure}` chunk。提供方特定错误文本会区分终端 `QUOTA` 与短暂 `RATE_LIMIT`，针对已解析模型上下文窗口评估的文本与 usage 信号则将溢出规范化为 `CONTEXT_WINDOW_EXCEEDED`。携带零个内容块消息的终止 `stop` 会映射为 `finish {kind:'error'}`，code 为 `EMPTY_RESPONSE`（默认策略会重试），而非成功空消息。
|
||||
- pi-ai 将 reasoning token 折叠到输出 usage 中；没有可映射的独立 reasoning 计数。
|
||||
- pi-ai 的 `off` thinking 级别会原样穿过 Harness 能力 seam，并在分派时变为被省略的 pi-ai 通用 `reasoning` 选项。
|
||||
- `GenerateOptions.stop` 会以 `UNSUPPORTED_OPTION` 被拒绝，因为 pi-ai 的通用流式输出表层无法保证所有提供方都支持它。
|
||||
|
||||
## 应用归因
|
||||
@@ -101,4 +111,4 @@ pi-ai 事件会变为 harness reasoning、文本、工具调用、usage 与 fini
|
||||
- **不支持 `GenerateOptions.stop`**：pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence，因此适配器会拒绝该字段。
|
||||
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**：提供方特定位置由 pi-ai 决定，而非由 harness 拥有的协议覆盖决定。
|
||||
- **无法获取提供方 HTTP 状态**：pi-ai 错误事件不会在所有提供方上公开稳定 HTTP 状态；失败只公开稳定 harness 错误 code。
|
||||
- **重试策略不是适配器选项**：SDK 重试已禁用，因此持久 agent 步骤与 `llm/retry` 事件拥有每次可见尝试；直接 `ctx.llm.stream()` 调用仍只尝试一次。
|
||||
- **重试策略由提供方持有，而不是 SDK 重试**：每个提供方 profile 都可以配置嵌套的 `retryPolicy`，由 `dsh-llm-retry` 在 agent 的失败步骤 seam 上执行；pi-ai SDK 重试仍保持禁用，因此持久 agent 步骤与 `llm/retry` 事件拥有每次可见尝试，直接 `ctx.llm.stream()` 调用仍只尝试一次。
|
||||
|
||||
@@ -34,7 +34,7 @@
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"@earendil-works/pi-ai": "^0.81.1",
|
||||
"@earendil-works/pi-ai": "^0.82.1",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -7,14 +7,29 @@
|
||||
import { streamSimple } from '@earendil-works/pi-ai/compat'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all'
|
||||
import { getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import type {
|
||||
Api,
|
||||
Model,
|
||||
ModelThinkingLevel,
|
||||
SimpleStreamOptions,
|
||||
ThinkingLevel,
|
||||
} from '@earendil-works/pi-ai'
|
||||
import type { AttachmentStore } from '@deepseek-ai/dsh-attachment'
|
||||
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelContext, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
attributionHeaders,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
ReasoningEffortId,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmModelInfo,
|
||||
LlmResolvedModelInfo,
|
||||
ReasoningEffortId as ReasoningEffortIdType,
|
||||
ResolvedRetryPolicy,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveProfiles } from './config.ts'
|
||||
import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
@@ -33,7 +48,10 @@ export interface PiAiAdapterOptions {
|
||||
* Resolve a catalog model dynamically and apply only the configured endpoint
|
||||
* override, preserving the catalog's API/capability/compatibility metadata.
|
||||
*/
|
||||
function resolveModel(profile: PiAiProviderProfile, modelId: string): Model<Api> {
|
||||
function resolvePiModel(
|
||||
profile: Omit<PiAiProviderProfile, 'retryPolicy'>,
|
||||
modelId: string,
|
||||
): Model<Api> {
|
||||
const model = getBuiltinModels(profile.provider as BuiltinProvider).find(candidate => candidate.id === modelId) as Model<Api> | undefined
|
||||
if (model === undefined) {
|
||||
throw new LlmError(`pi-ai provider "${profile.provider}" has no catalog model "${modelId}"`, 'UNKNOWN_MODEL')
|
||||
@@ -42,10 +60,14 @@ function resolveModel(profile: PiAiProviderProfile, modelId: string): Model<Api>
|
||||
}
|
||||
|
||||
/** Copy profile stream knobs into pi-ai's common option vocabulary. */
|
||||
function profileOptions(profile: PiAiProviderProfile): SimpleStreamOptions {
|
||||
function profileOptions(
|
||||
profile: Omit<PiAiProviderProfile, 'retryPolicy'>,
|
||||
reasoning: ModelThinkingLevel | undefined,
|
||||
): SimpleStreamOptions {
|
||||
const enabledReasoning: ThinkingLevel | undefined = reasoning === 'off' ? undefined : reasoning
|
||||
return {
|
||||
...profile.apiKey === undefined ? {} : { apiKey: profile.apiKey },
|
||||
...profile.reasoning === undefined ? {} : { reasoning: profile.reasoning },
|
||||
...enabledReasoning === undefined ? {} : { reasoning: enabledReasoning },
|
||||
...profile.thinkingBudgets === undefined ? {} : { thinkingBudgets: profile.thinkingBudgets },
|
||||
...profile.cacheRetention === undefined ? {} : { cacheRetention: profile.cacheRetention },
|
||||
...profile.transport === undefined ? {} : { transport: profile.transport },
|
||||
@@ -56,6 +78,20 @@ function profileOptions(profile: PiAiProviderProfile): SimpleStreamOptions {
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate an explicit Harness/profile effort without invoking pi-ai's clamp. */
|
||||
function resolveReasoningLevel(
|
||||
model: Model<Api>,
|
||||
effort: ReasoningEffortIdType | ModelThinkingLevel | undefined,
|
||||
): ModelThinkingLevel | undefined {
|
||||
if (effort === undefined) return undefined
|
||||
const supported = getSupportedThinkingLevels(model)
|
||||
if (supported.some(level => level === effort)) return effort as ModelThinkingLevel
|
||||
throw new LlmError(
|
||||
`pi-ai provider "${model.provider}" model "${model.id}" does not support reasoning effort "${effort}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
|
||||
/** Merge deployment headers while removing case-insensitive attribution collisions. */
|
||||
function requestHeaders(headers: Readonly<Record<string, string>> | undefined): Record<string, string> {
|
||||
const attribution = attributionHeaders()
|
||||
@@ -80,6 +116,10 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
this.resolveAttachments = options.resolveAttachments ?? (() => undefined)
|
||||
}
|
||||
|
||||
override providerRetryPolicy(provider: string): ResolvedRetryPolicy | undefined {
|
||||
return this.profiles.get(provider)?.retryPolicy
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
const profile = this.profiles.get(provider)
|
||||
if (profile === undefined) {
|
||||
@@ -94,10 +134,11 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
})))
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const profile = this.profiles.get(provider)
|
||||
if (profile === undefined) {
|
||||
return Promise.reject(new LlmError(
|
||||
@@ -105,9 +146,28 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
'NO_ADAPTER',
|
||||
))
|
||||
}
|
||||
return Promise.resolve().then(() => ({
|
||||
contextWindow: resolveModel(profile, model).contextWindow,
|
||||
}))
|
||||
return Promise.resolve().then(() => {
|
||||
const resolvedModel = resolvePiModel(profile, model)
|
||||
const levels = getSupportedThinkingLevels(resolvedModel)
|
||||
const defaultLevel = resolveReasoningLevel(resolvedModel, profile.reasoning)
|
||||
return {
|
||||
provider,
|
||||
id: model,
|
||||
name: resolvedModel.name,
|
||||
inputModalities: [...resolvedModel.input],
|
||||
outputModalities: ['text'],
|
||||
context: { contextWindow: resolvedModel.contextWindow },
|
||||
reasoning: {
|
||||
efforts: levels.map(level => ({
|
||||
id: ReasoningEffortId(level),
|
||||
name: `${level.charAt(0).toUpperCase()}${level.slice(1)}`,
|
||||
})),
|
||||
...defaultLevel === undefined
|
||||
? {}
|
||||
: { defaultEffort: ReasoningEffortId(defaultLevel) },
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
@@ -118,7 +178,11 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
if (profile === undefined) {
|
||||
throw new LlmError(`pi-ai adapter does not own provider "${options.provider}"`, 'NO_ADAPTER')
|
||||
}
|
||||
const model = resolveModel(profile, options.model)
|
||||
const model = resolvePiModel(profile, options.model)
|
||||
const reasoning = resolveReasoningLevel(
|
||||
model,
|
||||
options.reasoningEffort ?? profile.reasoning,
|
||||
)
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
? consumer.signal
|
||||
@@ -144,7 +208,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
? toPiContext(options)
|
||||
: await toPiContext(options, attachments)
|
||||
const events = streamSimple(model, context, {
|
||||
...profileOptions(profile),
|
||||
...profileOptions(profile, reasoning),
|
||||
...options.temperature === undefined ? {} : { temperature: options.temperature },
|
||||
...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },
|
||||
...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) },
|
||||
|
||||
@@ -5,9 +5,11 @@
|
||||
*/
|
||||
|
||||
import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { CacheRetention, ThinkingBudgets, ThinkingLevel, Transport } from '@earendil-works/pi-ai'
|
||||
import type { CacheRetention, ModelThinkingLevel, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import z from 'schemastery'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
@@ -23,7 +25,7 @@ export interface PiAiProviderProfile {
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
reasoning?: ThinkingLevel
|
||||
reasoning?: ModelThinkingLevel
|
||||
/** Token budgets used by reasoning providers that support them. */
|
||||
thinkingBudgets?: ThinkingBudgets
|
||||
/** Prompt-cache retention preference. */
|
||||
@@ -36,12 +38,16 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** Validated profile with every adapter-owned default resolved. */
|
||||
export interface ResolvedPiAiProviderProfile extends PiAiProviderProfile {
|
||||
export interface ResolvedPiAiProviderProfile extends Omit<PiAiProviderProfile, 'retryPolicy'> {
|
||||
/** Positive finite provider-idle interval after defaulting. */
|
||||
streamIdleTimeoutMs: number
|
||||
/** Immutable retry policy captured with this provider route. */
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
/** Plugin configuration: the non-empty provider profiles this instance owns. */
|
||||
@@ -62,13 +68,14 @@ const profile = z.object({
|
||||
apiKey: z.string(),
|
||||
baseURL: z.string(),
|
||||
headers: z.dict(z.string()),
|
||||
reasoning: z.union(['minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
|
||||
reasoning: z.union(['off', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
|
||||
thinkingBudgets,
|
||||
cacheRetention: z.union(['none', 'short', 'long']),
|
||||
transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']),
|
||||
timeoutMs: z.natural(),
|
||||
websocketConnectTimeoutMs: z.natural(),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
retryPolicy: RetryPolicySchema,
|
||||
})
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
@@ -115,6 +122,10 @@ export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): Resol
|
||||
return {
|
||||
...source,
|
||||
streamIdleTimeoutMs,
|
||||
retryPolicy: resolveRetryPolicy(
|
||||
source.retryPolicy,
|
||||
`llm-pi-ai: provider "${source.provider}" retryPolicy`,
|
||||
),
|
||||
...source.headers === undefined ? {} : { headers: { ...source.headers } },
|
||||
...source.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...source.thinkingBudgets } },
|
||||
}
|
||||
|
||||
@@ -10,6 +10,9 @@
|
||||
* providers:
|
||||
* - provider: openai
|
||||
* apiKey: !!js process.env.OPENAI_API_KEY
|
||||
* retryPolicy:
|
||||
* mode: normal
|
||||
* maxRetries: 2
|
||||
* - provider: anthropic
|
||||
* apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
* - provider: openrouter
|
||||
@@ -37,7 +40,7 @@ export const inject = ['llm']
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const profiles = resolveProfiles(config.providers)
|
||||
const adapter = new PiAiAdapter({
|
||||
profiles,
|
||||
profiles: config.providers,
|
||||
resolveAttachments: () => ctx.get('attachments'),
|
||||
})
|
||||
ctx.llm.registerAdapter(profiles.map(entry => entry.provider), adapter)
|
||||
|
||||
@@ -28,6 +28,14 @@ export function mapUsage(usage: PiUsage): TokenUsage {
|
||||
}
|
||||
}
|
||||
|
||||
// XXX(pi-ai upstream): pi-ai flattens the caught error to `error.message`
|
||||
// (api/anthropic-messages.js: `errorMessage = error instanceof Error ?
|
||||
// error.message : JSON.stringify(error)`), discarding the original Error and its
|
||||
// `cause` chain before it reaches us. undici carries the actionable transport
|
||||
// detail on `cause` (e.g. `SocketError: other side closed`) but hands the fetch
|
||||
// wrapper a bare `terminated`, so we are left pattern-matching terse words here.
|
||||
// If pi-ai ever forwards the original Error (or a fetch/dispatcher hook that lets
|
||||
// us capture the cause ourselves), classify on `code`/`cause` instead of text.
|
||||
function classifyPiAiError(message: string): string {
|
||||
if (/\b(?:401|403)\b/.test(message)) return 'AUTH'
|
||||
if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE
|
||||
@@ -35,8 +43,19 @@ function classifyPiAiError(message: string): string {
|
||||
if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST'
|
||||
if (/\b5\d\d\b/.test(message)) return 'SERVER'
|
||||
if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT'
|
||||
// A stream truncated before the provider's terminal event: each pi-ai provider
|
||||
// throws its own wording when the wire closes mid-response without a terminal
|
||||
// event (`… stream ended before message_stop`, `… before a terminal response
|
||||
// event`, `… ended without a terminal event`, `Stream ended without
|
||||
// finish_reason`). The connection dropped mid-response, so this is a transport
|
||||
// truncation, not a model-level error.
|
||||
if (/stream ended (?:before|without)\b/i.test(message)) return 'TRANSPORT'
|
||||
if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)
|
||||
|| /\b(?:other side closed|HTTP2 request did not get a response|WebSocket closed unexpectedly)\b/i.test(message)) {
|
||||
|| /\b(?:other side closed|HTTP2 request did not get a response|WebSocket closed unexpectedly)\b/i.test(message)
|
||||
// undici renders a mid-stream socket drop as a bare `terminated` (its
|
||||
// `cause` — the real SocketError — was flattened away upstream); Node's
|
||||
// stream layer says `Premature close`.
|
||||
|| /\bterminated\b|premature close/i.test(message)) {
|
||||
return 'TRANSPORT'
|
||||
}
|
||||
return 'PI_AI_ERROR'
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import type { PiAiProviderProfile } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
@@ -9,7 +9,7 @@ import { assemble, type AssembledResult } from './assemble.ts'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the pi-ai-backed adapter: V4 Flash + V4 Pro with provider
|
||||
* defaults and representative high/xhigh reasoning. Mirrors the native
|
||||
* defaults and representative off/high/max reasoning. Mirrors the native
|
||||
* adapter's StreamChunk contract and exercises a replayed tool follow-up.
|
||||
* Key-gated.
|
||||
*/
|
||||
@@ -74,10 +74,24 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
expect(textOf(result).toLowerCase()).toContain('pong')
|
||||
})
|
||||
|
||||
it('flash + reasoning off: plain text without reasoning blocks', async () => {
|
||||
const ctx = await harness(FLASH)
|
||||
const result = await assemble(ctx,{
|
||||
model: FLASH,
|
||||
reasoningEffort: ReasoningEffortId('off'),
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
expect(textOf(result).toLowerCase()).toContain('pong')
|
||||
})
|
||||
|
||||
it.each([FLASH, PRO])('%s + reasoning high: reasoning blocks present', async (model) => {
|
||||
const ctx = await harness(model, { reasoning: 'high' })
|
||||
const ctx = await harness(model)
|
||||
const result = await assemble(ctx,{
|
||||
model,
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
})
|
||||
@@ -86,11 +100,12 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
expect(textOf(result)).toContain('9.8')
|
||||
})
|
||||
|
||||
it('pro + reasoning xhigh (wire max): tool-call round trip', async () => {
|
||||
const ctx = await harness(PRO, { reasoning: 'xhigh' })
|
||||
it('pro + reasoning max: tool-call round trip', async () => {
|
||||
const ctx = await harness(PRO)
|
||||
|
||||
const first = await assemble(ctx,{
|
||||
model: PRO,
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
@@ -103,6 +118,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
|
||||
const second = await assemble(ctx,{
|
||||
model: PRO,
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
first.message,
|
||||
|
||||
@@ -9,7 +9,7 @@ import type {
|
||||
SaveImageAttachment,
|
||||
StoredImageAttachment,
|
||||
} from '@deepseek-ai/dsh-attachment'
|
||||
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, ReasoningEffortId, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
@@ -139,7 +139,7 @@ describe('PiAiAdapter provider routing', () => {
|
||||
it('forwards common stream options and profile reasoning', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, {
|
||||
reasoning: 'xhigh',
|
||||
reasoning: 'max',
|
||||
cacheRetention: 'none',
|
||||
transport: 'sse',
|
||||
timeoutMs: 5000,
|
||||
@@ -163,6 +163,33 @@ describe('PiAiAdapter provider routing', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('uses a dynamic request effort and rejects unsupported efforts before network I/O', async () => {
|
||||
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'max' })
|
||||
|
||||
await assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: [],
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({ reasoning_effort: 'high' })
|
||||
|
||||
await assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('off'),
|
||||
messages: [],
|
||||
})
|
||||
expect(server.requests[1]).toMatchObject({ thinking: { type: 'disabled' } })
|
||||
expect(server.requests[1]).not.toHaveProperty('reasoning_effort')
|
||||
|
||||
await expect(assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
reasoningEffort: ReasoningEffortId('xhigh'),
|
||||
messages: [],
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
expect(server.requests).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('preserves omitted profile options when constructing the adapter directly', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = new Context()
|
||||
@@ -371,12 +398,31 @@ describe('provider profile lifecycle', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'openai' }, { provider: 'anthropic' }],
|
||||
providers: [
|
||||
{
|
||||
provider: 'openai',
|
||||
retryPolicy: {
|
||||
mode: 'always',
|
||||
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
|
||||
},
|
||||
},
|
||||
{ provider: 'anthropic' },
|
||||
],
|
||||
})
|
||||
expect(ctx.llm.listProviders()).toEqual([
|
||||
{ id: 'openai', name: 'openai' },
|
||||
{ id: 'anthropic', name: 'anthropic' },
|
||||
])
|
||||
expect(ctx.llm.providerRetryPolicy('openai')).toEqual({
|
||||
mode: 'always',
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0.2,
|
||||
})
|
||||
expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchObject({
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
})
|
||||
await fiber.dispose()
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
@@ -391,9 +437,68 @@ describe('provider profile lifecycle', () => {
|
||||
inputModalities: ['text', 'image'], outputModalities: ['text'],
|
||||
})
|
||||
expect(models.every(model => model.provider === 'openai')).toBe(true)
|
||||
const context = await ctx.llm.resolveModelContext('openai', 'gpt-4.1')
|
||||
expect(context).toBeDefined()
|
||||
expect(typeof context?.contextWindow).toBe('number')
|
||||
const info = await ctx.llm.resolveModelInfo('openai', 'gpt-4.1')
|
||||
expect(typeof info.context?.contextWindow).toBe('number')
|
||||
})
|
||||
|
||||
it('exposes pi-ai model thinking levels verbatim without inventing a provider default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'deepseek' }, { provider: 'openai' }],
|
||||
})
|
||||
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('off'), name: 'Off' },
|
||||
{ id: ReasoningEffortId('high'), name: 'High' },
|
||||
{ id: ReasoningEffortId('max'), name: 'Max' },
|
||||
],
|
||||
},
|
||||
})
|
||||
const extended = await ctx.llm.resolveModelInfo('openai', 'gpt-5.6-sol')
|
||||
expect(extended.reasoning?.efforts.map(effort => effort.id)).toEqual([
|
||||
ReasoningEffortId('off'),
|
||||
ReasoningEffortId('low'),
|
||||
ReasoningEffortId('medium'),
|
||||
ReasoningEffortId('high'),
|
||||
ReasoningEffortId('xhigh'),
|
||||
ReasoningEffortId('max'),
|
||||
])
|
||||
await expect(ctx.llm.resolveModelInfo('openai', 'gpt-4.1'))
|
||||
.resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('uses a supported profile reasoning value as the model default and rejects an unsupported one', async () => {
|
||||
const supported = new Context()
|
||||
await supported.plugin(LlmService)
|
||||
await supported.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'deepseek', reasoning: 'max' }],
|
||||
})
|
||||
await expect(supported.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toMatchObject({ reasoning: { defaultEffort: ReasoningEffortId('max') } })
|
||||
|
||||
const unsupported = new Context()
|
||||
await unsupported.plugin(LlmService)
|
||||
await unsupported.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'deepseek', reasoning: 'medium' }],
|
||||
})
|
||||
await expect(unsupported.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
|
||||
const disabled = new Context()
|
||||
await disabled.plugin(LlmService)
|
||||
await disabled.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'deepseek', reasoning: 'off' }],
|
||||
})
|
||||
await expect(disabled.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toMatchObject({ reasoning: { defaultEffort: ReasoningEffortId('off') } })
|
||||
})
|
||||
|
||||
it('accepts absent credentials for pi-ai ambient authentication', async () => {
|
||||
@@ -442,12 +547,29 @@ describe('provider profile lifecycle', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects invalid nested retryPolicy at the provider-profile boundary', async () => {
|
||||
expect(() => resolveProfiles([{
|
||||
provider: 'openai',
|
||||
retryPolicy: { mode: 'always', backoff: { jitterRatio: -1 } },
|
||||
}])).toThrow(/retryPolicy\.backoff\.jitterRatio/)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmPiAi, {
|
||||
providers: [{
|
||||
provider: 'openai',
|
||||
retryPolicy: { mode: 'normal', maxRetries: -1 },
|
||||
}],
|
||||
})).rejects.toThrow(/retryPolicy/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('constructs the adapter directly and rejects routes it does not own', async () => {
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai' }] })
|
||||
await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(adapter.resolveModelContext('anthropic', 'claude-sonnet-4'))
|
||||
await expect(adapter.resolveModel('anthropic', 'claude-sonnet-4'))
|
||||
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(adapter.resolveModelContext('openai', 'not-a-catalog-model'))
|
||||
await expect(adapter.resolveModel('openai', 'not-a-catalog-model'))
|
||||
.rejects.toMatchObject({ code: 'UNKNOWN_MODEL' })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of adapter.stream({ provider: 'anthropic', model: 'claude-sonnet-4', messages: [] })) { /* drain */ }
|
||||
|
||||
@@ -622,6 +622,15 @@ describe('mapStopReason / mapUsage', () => {
|
||||
'other side closed',
|
||||
'HTTP2 request did not get a response',
|
||||
'WebSocket closed unexpectedly',
|
||||
// undici flattens a mid-stream socket drop to this bare word (its SocketError
|
||||
// cause is discarded upstream before it reaches us).
|
||||
'terminated',
|
||||
'Premature close',
|
||||
// pi-ai's per-provider throws when the wire closes before the terminal event.
|
||||
'Anthropic stream ended before message_stop',
|
||||
'OpenAI Responses stream ended before a terminal response event',
|
||||
'openrouter stream ended without a terminal event',
|
||||
'Stream ended without finish_reason',
|
||||
])('maps pi-ai transport wording %j', (errorMessage) => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } })
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 96dc2314bac59b36a97627e038ac614f3db5f9b3
|
||||
README.zh.md: cbee3291d688dfda1c4109fb87630c030bf4b45c
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md
|
||||
README.md: 7a86652a794e70c4dfd00ab7427730387e3ec949
|
||||
README.zh.md: 6255cca8c3b669ddec401496acb1e79ad54a8b3e
|
||||
|
||||
@@ -2,42 +2,52 @@
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step.
|
||||
Function plugin that applies exact-provider retry policy through the agent loop's closed-step `agent/request-error` waterfall. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered turn.
|
||||
|
||||
The default policy permits two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion (a terminal stop with zero content blocks); the attempt produced nothing durable, so repeating it is safe. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
|
||||
Each provider adapter owns an optional nested `retryPolicy`, captured when its route registers on `ctx.llm` and carried with each call that reaches that registration's final adapter boundary. An in-flight failure retains that serving policy if the route is later disposed or replaced; a failure before any final adapter is selected has no provider policy and delegates. Omission uses normal mode: two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion that produced no durable content, so repeating it is safe. A normal policy can change its finite budget, eligible codes, and backoff. Always mode asks downstream recovery first, then retries every model-request failure without an attempt limit; success, cancellation, or plugin disposal stops it after active delegated recovery reaches quiescence.
|
||||
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward.
|
||||
Both modes use bounded exponential backoff with symmetric jitter. A valid `providerRetryAfterMs` at or below `maxDelayMs` replaces local backoff without jitter. An over-cap provider delay makes normal mode delegate, while always mode uses its configured local backoff so it cannot terminate on that instruction.
|
||||
|
||||
The separately published `./invariant` companion checks that every retry record names the current open turn and its latest closed step, has a unique step record and increasing retry number, and carries a positive bounded retry budget and non-negative bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the provider, mode, canonical resolved-policy key, failure, and scheduled delay. The key includes every behavior-affecting field and sorts normal-mode codes because eligibility uses set membership. Retry numbers continue only across events with the same provider and complete policy key, so a route replacement with different limits, code membership, or backoff starts its own history. Normal events include the finite maximum; always events omit it, and UIs render `∞`. After the wait, the listener returns `{ kind: 'retry' }`, and the loop closes the failed turn and opens a retry turn over the same durable history. Cancellation and plugin disposal abort active backoff, drain active delegated recovery before applying the abort, and make a callback captured before disposal fail closed.
|
||||
|
||||
The separately published `./invariant` companion checks that every retry record names the current open turn and latest closed step, matches the failed request's durable provider, carries non-empty provider and policy identities, has mode-specific bounds, a unique step record, the correct provider-policy retry number, and a bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
- name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
The executor has no policy config. Multi-provider adapters such as `dsh-llm-pi-ai` place `retryPolicy` inside each provider profile, avoiding a second provider-name list.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Transient request recovery
|
||||
### Model-request recovery
|
||||
|
||||
#### What the model sees
|
||||
|
||||
No retry event, delay, or failure prose is model-visible. After a retry, the next numbered step reconstructs the same explicit provider/model request from durable session history; failed chunks never enter derived messages.
|
||||
No retry event, delay, provider error, or failed partial output is model-visible. The retry turn reconstructs the same explicit provider/model request from durable surface history unless a downstream recovery policy deliberately changes that surface; failed chunks never enter derived messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Each retry is a new provider request and may repeat input-token billing. The finite budget caps attempts; `llm/retry` itself contributes no tokens.
|
||||
Each retry is a new provider request and may repeat input-token billing. Normal mode has a finite budget; always mode can consume unbounded requests until success or cancellation. `llm/retry` itself contributes no tokens.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface status event does not change cache identity.
|
||||
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface retry event does not change cache identity.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Agent steps are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
|
||||
- **Finite plugin budgets add** — this policy counts only configured transient codes; context-overflow compaction counts only its own code. A future policy with overlapping codes must document and test registration-order behavior.
|
||||
- **Agent turns are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
|
||||
- **Always mode retries permanent failures** — authentication, quota, invalid-request, protocol, and unrecoverable context errors continue until success, cancellation, or disposal; deployments own provider-specific cost and latency controls.
|
||||
- **Finite plugin budgets add** — normal mode counts only its configured codes and exact provider policy, while context-overflow compaction owns a separate budget. A future overlapping policy must document and test registration-order behavior.
|
||||
- **Recovery policies compose by waterfall order** — always mode accepts a downstream retry before applying its fallback. A later policy that ignores cancellation and never settles also prevents fallback, turn quiescence, and plugin disposal from completing.
|
||||
- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation.
|
||||
|
||||
@@ -2,42 +2,52 @@
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
一个函数插件，在 agent loop 的已关闭步骤恢复 seam 上重试特定的短暂模型请求失败。它不包装 `ctx.llm.stream()`：每次适配器调用仍是一次提供方尝试，每次重试都会开启新的编号步骤。
|
||||
一个函数插件，通过 agent loop（智能体循环）在已关闭步骤上触发的 `agent/request-error` waterfall（瀑布式事件）应用确切提供方重试策略。它不包装 `ctx.llm.stream()`：每次适配器调用仍是一次提供方尝试，每次重试都会开启新的编号轮次。
|
||||
|
||||
默认策略允许为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试两次，使用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对退化提供方完成的分类（携带零个内容块的终止 stop）；该尝试未产生持久内容，因此可安全重复。延迟边界必须适合 Node 支持的定时器范围。有效 `providerRetryAfterMs` 在已配置上限内时替换本地退避；超出上限的指令会委托给下一项恢复策略。
|
||||
每个提供方适配器都拥有可选的嵌套 `retryPolicy`；路由在 `ctx.llm` 上注册时会捕获该策略，任何到达该注册最终适配器边界的调用都会携带它。如果之后释放或替换路由，进行中的失败仍会保留为其提供服务的策略；在选中任何最终适配器前发生的失败没有提供方策略，会继续委托。省略策略时使用 normal mode：为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试两次，并采用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对未产生任何持久内容的退化提供方完成所作的分类，因此可安全重复。normal 策略可以更改其有限预算、合格 code 和退避配置。always mode 会先请求下游恢复，再无次数上限地重试每个模型请求失败；成功、取消或插件 dispose（资源释放）会在活跃的委托恢复完全停稳后终止它。
|
||||
|
||||
等待之前，插件会追加一个非表层 `llm/retry` 事件，携带失败与计划延迟。取消与插件 dispose 会中止等待；dispose 会排空插件的活跃退避，dispose 前捕获的 callback 如果在之后调用，将快速失败。
|
||||
两种 mode 都使用带对称 jitter 的有界指数退避。有效 `providerRetryAfterMs` 不超过 `maxDelayMs` 时会替换本地退避，并且不加 jitter。超出上限的提供方延迟会使 normal mode 继续委托；always mode 则改用已配置的本地退避，避免该指令终止重试。
|
||||
|
||||
单独发布的 `./invariant` 配套模块会检查每个重试记录是否指向当前开启轮次及其最新已关闭步骤，是否拥有唯一步骤记录与递增重试编号，以及是否携带正数有界重试预算和非负有界定时器延迟。完整 jitter 可以在下界调度为零毫秒。
|
||||
等待前，插件会追加一条不进入表层的 `llm/retry` 事件，其中包含提供方、mode、规范的解析策略 key、失败和计划延迟。该 key 包含所有影响行为的字段，并对 normal mode 的 code 排序，因为合格性采用集合成员关系判断。只有提供方与完整策略 key 都相同的事件才会延续重试编号；因此，用限制、code 成员关系或退避不同的路由替换后，会开始自己的历史。normal 事件包含有限上限；always 事件省略该上限，UI 会渲染 `∞`。等待结束后，监听器返回 `{ kind: 'retry' }`，循环关闭失败轮次，并在同一持久历史上开启重试轮次。取消与插件 dispose 会中止活跃退避，在应用中止前排空活跃的委托恢复，并使 dispose 前捕获的 callback 只能以失败结束。
|
||||
|
||||
单独发布的 `./invariant` 配套模块会检查每个重试记录是否指向当前开启轮次及其最新已关闭步骤，是否与失败请求的持久提供方匹配，是否携带非空的提供方与策略身份，是否满足 mode 特定边界，是否拥有唯一步骤记录和正确的提供方策略重试编号，以及是否携带有界定时器延迟。完整 jitter 可以在下界调度为零毫秒。
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
- name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
执行器没有策略配置。`dsh-llm-pi-ai` 等多提供方适配器会把 `retryPolicy` 放在每个提供方 profile 内，避免维护第二份提供方名称列表。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### 短暂请求恢复
|
||||
### 模型请求恢复
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
模型不会看到重试事件、延迟或失败文本。重试后，下一个编号步骤会从持久会话历史中重建相同的显式提供方／模型请求；失败 chunk 绝不会进入派生消息。
|
||||
模型不会看到重试事件、延迟、提供方错误或失败的部分输出。重试轮次会从持久表层历史中重建相同的显式提供方／模型请求，除非下游恢复策略有意更改该表层；失败分片绝不会进入派生消息。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
每次重试都是新的提供方请求，可能重复计费输入 token。有限预算会限制尝试次数；`llm/retry` 自身不产生 token。
|
||||
每次重试都是新的提供方请求，可能重复计费输入 token。normal mode 具有有限预算；always mode 可以在成功或取消前消耗无界数量的请求。`llm/retry` 自身不产生 token。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
重建请求保留之前的前缀，并可根据该提供方的规则复用 cache。非表层状态事件不会改变 cache 身份。
|
||||
重建请求保留之前的前缀，并可根据该提供方的规则复用 cache。非表层重试事件不会改变 cache 身份。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **Agent 步骤是唯一重试边界**：直接 `ctx.llm.stream()` 消费方仍只尝试一次，因为原始流无法将已发出 chunk 持久分隔为不同尝试。
|
||||
- **有限插件预算可叠加**：该策略只统计已配置短暂 code；上下文溢出压缩只统计自身 code。未来如有 code 重叠的策略，必须记录并测试注册顺序行为。
|
||||
- **Agent 轮次是唯一重试边界**：直接 `ctx.llm.stream()` 消费方仍只尝试一次，因为原始流无法将已发出 chunk 持久分隔为不同尝试。
|
||||
- **always mode 会重试永久性失败**：身份验证、配额、无效请求、协议和无法恢复的上下文错误都会继续重试，直至成功、取消或 dispose；部署负责提供方特定的成本与延迟控制。
|
||||
- **有限插件预算可叠加**：normal mode 只统计已配置 code 和确切提供方策略，上下文溢出压缩则拥有独立预算。未来如有重叠策略，必须记录并测试注册顺序行为。
|
||||
- **恢复策略按 waterfall 顺序组合**：always mode 会先接受下游重试，再应用自己的回退。后续策略如果忽略取消且永不结算，也会阻止回退、轮次完全停稳和插件 dispose 完成。
|
||||
- **`llm/retry` 记录调度，不是完成**：后续步骤与轮次事件用于确立成功、耗尽或取消。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-retry",
|
||||
"description": "Bounded transient LLM request retry policy for the DeepSeek Harness",
|
||||
"description": "Provider-routed LLM request retry policy for the DeepSeek Harness",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
|
||||
32
packages/llm/llm-retry/src/history.ts
Normal file
32
packages/llm/llm-retry/src/history.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
/** Durable request-route lookup for one closed model step. @module @deepseek-ai/dsh-llm-retry/history */
|
||||
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* Find the provider in force when one step closed, excluding later recovery mutations.
|
||||
* Request headers remain effective across turn boundaries until a newer full
|
||||
* snapshot changes them; every provider change requires a newer full snapshot.
|
||||
* @param events - session events containing the closed step.
|
||||
* @param turn - turn that owns the failed step.
|
||||
* @param step - failed step whose provider is required.
|
||||
* @returns the provider from the request header in force at that step boundary.
|
||||
*/
|
||||
export function providerForClosedStep(
|
||||
events: readonly SessionEvent[],
|
||||
turn: number,
|
||||
step: number,
|
||||
): string | undefined {
|
||||
const stepEndIndex = events.findLastIndex(event =>
|
||||
event.type === 'step/end'
|
||||
&& event.data.turn === turn
|
||||
&& event.data.step === step,
|
||||
)
|
||||
if (stepEndIndex < 0) return undefined
|
||||
for (let index = stepEndIndex; index >= 0; index -= 1) {
|
||||
// The loop bounds prove this indexed read exists.
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
const event = events[index]!
|
||||
if (event.type === 'request/header') return event.data.header.config.provider
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Bounded transient model-request retry policy on the agent loop's closed-step
|
||||
* Provider-routed model-request retry policy on the agent loop's closed-step
|
||||
* recovery seam. Each scheduled retry is durable before its cancellable wait.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-retry
|
||||
@@ -7,21 +7,33 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { Agent, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import type {} from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { Agent, RequestError, RequestErrorAction } from '@deepseek-ai/dsh-agent'
|
||||
import type { LlmFailure, ResolvedRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { providerForClosedStep } from './history.ts'
|
||||
|
||||
declare module '@deepseek-ai/dsh-session' {
|
||||
interface SessionEventMap {
|
||||
/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
|
||||
/** Durable, non-surface record of one provider-routed retry scheduled after a closed failed step. */
|
||||
'llm/retry': {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'normal'
|
||||
policyKey: string
|
||||
retry: number
|
||||
maxRetries: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
} | {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'always'
|
||||
policyKey: string
|
||||
retry: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -29,82 +41,19 @@ declare module '@deepseek-ai/dsh-session' {
|
||||
export const name = 'llm-retry'
|
||||
export const inject = ['agents']
|
||||
|
||||
const DEFAULT_MAX_TRANSIENT_RETRIES = 2
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
const DEFAULT_MAX_DELAY_MS = 10_000
|
||||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze(['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
|
||||
|
||||
/** Deployment-owned limits and classification for transient request recovery. */
|
||||
export interface Config {
|
||||
/** Maximum transient retries after the first request (default 2). */
|
||||
maxTransientRetries?: number
|
||||
/** Initial local exponential-backoff delay in milliseconds (default 500). */
|
||||
initialDelayMs?: number
|
||||
/** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
|
||||
maxDelayMs?: number
|
||||
/** Symmetric random multiplier range around one (default 0.1). */
|
||||
jitterRatio?: number
|
||||
/** Stable failure codes eligible for this policy. */
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
/** This policy executor has no config; providers own `retryPolicy`. */
|
||||
export type Config = Readonly<Record<string, never>>
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
export const Config: z<Config> = z.object({
|
||||
maxTransientRetries: z.number().step(1).min(0).default(DEFAULT_MAX_TRANSIENT_RETRIES),
|
||||
initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS),
|
||||
maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS),
|
||||
jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO),
|
||||
retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]),
|
||||
})
|
||||
export const Config = z.object({}) as unknown as z<Config>
|
||||
|
||||
interface ResolvedConfig {
|
||||
readonly maxTransientRetries: number
|
||||
readonly initialDelayMs: number
|
||||
readonly maxDelayMs: number
|
||||
readonly jitterRatio: number
|
||||
readonly retryableCodes: ReadonlySet<string>
|
||||
}
|
||||
|
||||
function resolveConfig(config: Config): ResolvedConfig {
|
||||
const maxTransientRetries = config.maxTransientRetries ?? DEFAULT_MAX_TRANSIENT_RETRIES
|
||||
const initialDelayMs = config.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS
|
||||
const maxDelayMs = config.maxDelayMs ?? DEFAULT_MAX_DELAY_MS
|
||||
const jitterRatio = config.jitterRatio ?? DEFAULT_JITTER_RATIO
|
||||
const codes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES]
|
||||
|
||||
if (!Number.isInteger(maxTransientRetries) || maxTransientRetries < 0) {
|
||||
throw new Error('llm-retry: maxTransientRetries must be a non-negative integer')
|
||||
function validateConfig(config: Config): void {
|
||||
const [key] = Object.keys(config)
|
||||
if (key === undefined) return
|
||||
if (key === 'retryPolicy') {
|
||||
throw new Error('llm-retry: retryPolicy belongs under each provider configuration')
|
||||
}
|
||||
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (initialDelayMs > maxDelayMs) {
|
||||
throw new Error('llm-retry: initialDelayMs must be less than or equal to maxDelayMs')
|
||||
}
|
||||
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) {
|
||||
throw new Error('llm-retry: jitterRatio must be between 0 and 1')
|
||||
}
|
||||
if (codes.length === 0) {
|
||||
throw new Error('llm-retry: retryableCodes must not be empty')
|
||||
}
|
||||
if (codes.some(code => code.length === 0)) {
|
||||
throw new Error('llm-retry: retryableCodes must contain only non-empty strings')
|
||||
}
|
||||
if (new Set(codes).size !== codes.length) {
|
||||
throw new Error('llm-retry: retryableCodes must not contain duplicates')
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
maxTransientRetries,
|
||||
initialDelayMs,
|
||||
maxDelayMs,
|
||||
jitterRatio,
|
||||
retryableCodes: new Set(codes),
|
||||
})
|
||||
throw new Error(`llm-retry: unknown key "${key}"`)
|
||||
}
|
||||
|
||||
/** Non-serializable seams used to make timing policy deterministic in tests. */
|
||||
@@ -113,13 +62,40 @@ export interface RetryInternals {
|
||||
random?: () => number
|
||||
}
|
||||
|
||||
function localDelay(config: ResolvedConfig, retry: number, random: () => number): number {
|
||||
type DownstreamOutcome =
|
||||
| { readonly type: 'decision'; readonly decision: RequestErrorAction }
|
||||
| { readonly type: 'error'; readonly error: unknown }
|
||||
|
||||
async function settleDownstream(
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
): Promise<DownstreamOutcome> {
|
||||
try {
|
||||
return { type: 'decision', decision: await next() }
|
||||
} catch (error: unknown) {
|
||||
return { type: 'error', error }
|
||||
}
|
||||
}
|
||||
|
||||
function localDelay(config: ResolvedRetryPolicy, retry: number, random: () => number): number {
|
||||
const exponent = Math.min(retry - 1, 1024)
|
||||
const exponential = Math.min(config.initialDelayMs * 2 ** exponent, config.maxDelayMs)
|
||||
const jitter = 1 - config.jitterRatio + 2 * config.jitterRatio * random()
|
||||
return Math.min(exponential * jitter, config.maxDelayMs)
|
||||
}
|
||||
|
||||
function retryPolicyKey(policy: ResolvedRetryPolicy): string {
|
||||
return policy.mode === 'always'
|
||||
? JSON.stringify([policy.mode, policy.initialDelayMs, policy.maxDelayMs, policy.jitterRatio])
|
||||
: JSON.stringify([
|
||||
policy.mode,
|
||||
policy.maxRetries,
|
||||
[...policy.retryableCodes].sort(),
|
||||
policy.initialDelayMs,
|
||||
policy.maxDelayMs,
|
||||
policy.jitterRatio,
|
||||
])
|
||||
}
|
||||
|
||||
function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean> {
|
||||
if (signal.aborted) return Promise.resolve(false)
|
||||
return new Promise((resolve) => {
|
||||
@@ -136,78 +112,154 @@ function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Install bounded transient request recovery.
|
||||
* Install provider-routed normal or unbounded request recovery.
|
||||
* @param ctx - plugin context that owns the listener and active waits.
|
||||
* @param config - retry budget, delay bounds, jitter, and eligible codes.
|
||||
* @param config - empty executor config; provider registrations own policy.
|
||||
* @param internals - non-serializable deterministic seams for tests.
|
||||
*/
|
||||
export function apply(ctx: Context, config: Config = {}, internals: RetryInternals = {}): void {
|
||||
const resolved = resolveConfig(config)
|
||||
validateConfig(config)
|
||||
const random = internals.random ?? Math.random
|
||||
const lifetime = new AbortController()
|
||||
const active = new Set<Promise<RequestErrorDecision>>()
|
||||
const active = new Set<Promise<RequestErrorAction>>()
|
||||
|
||||
function track(operation: Promise<RequestErrorAction>): Promise<RequestErrorAction> {
|
||||
const tracked = operation.finally(() => active.delete(tracked))
|
||||
active.add(tracked)
|
||||
return tracked
|
||||
}
|
||||
|
||||
async function backoff(
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
failure: LlmFailure,
|
||||
provider: string,
|
||||
policy: ResolvedRetryPolicy,
|
||||
policyKey: string,
|
||||
retry: number,
|
||||
delayMs: number,
|
||||
signal: AbortSignal,
|
||||
): Promise<RequestErrorDecision> {
|
||||
): Promise<RequestErrorAction> {
|
||||
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
|
||||
if (fusedSignal.aborted) return { action: 'fail' }
|
||||
agent.session.append('llm/retry', {
|
||||
turn,
|
||||
step,
|
||||
retry,
|
||||
maxRetries: resolved.maxTransientRetries,
|
||||
delayMs,
|
||||
failure,
|
||||
})
|
||||
if (!await cancellableDelay(delayMs, fusedSignal)) return { action: 'fail' }
|
||||
return { action: 'retry' }
|
||||
if (fusedSignal.aborted) return
|
||||
const eventData = policy.mode === 'normal'
|
||||
? {
|
||||
turn,
|
||||
step,
|
||||
provider,
|
||||
mode: policy.mode,
|
||||
policyKey,
|
||||
retry,
|
||||
maxRetries: policy.maxRetries,
|
||||
delayMs,
|
||||
failure,
|
||||
}
|
||||
: {
|
||||
turn,
|
||||
step,
|
||||
provider,
|
||||
mode: policy.mode,
|
||||
policyKey,
|
||||
retry,
|
||||
delayMs,
|
||||
failure,
|
||||
}
|
||||
agent.session.append('llm/retry', eventData)
|
||||
if (!await cancellableDelay(delayMs, fusedSignal)) return
|
||||
return { kind: 'retry' }
|
||||
}
|
||||
|
||||
const disposeListener = ctx.on('agent/request-error', (
|
||||
async function recover(
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
_error: RequestError,
|
||||
failure: LlmFailure,
|
||||
priorFailures: readonly LlmFailure[],
|
||||
policy: ResolvedRetryPolicy | undefined,
|
||||
signal: AbortSignal,
|
||||
next: () => Promise<RequestErrorDecision>,
|
||||
) => {
|
||||
// A waterfall may have captured this callback before its registration was
|
||||
// removed. Lifetime cancellation must prevent that stale callback from
|
||||
// entering a downstream policy after disposal.
|
||||
if (lifetime.signal.aborted) return Promise.resolve<RequestErrorDecision>({ action: 'fail' })
|
||||
if (!resolved.retryableCodes.has(failure.code)) return next()
|
||||
const priorTransientFailures = priorFailures.filter(item => resolved.retryableCodes.has(item.code)).length
|
||||
if (priorTransientFailures >= resolved.maxTransientRetries) return next()
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
): Promise<RequestErrorAction> {
|
||||
if (policy === undefined) return next()
|
||||
// The call-local policy belongs to the registration that served this
|
||||
// failure. Recover only the durable provider identity from the header;
|
||||
// downstream recovery may append later state before an always fallback.
|
||||
const provider = providerForClosedStep(agent.session.events, turn, step)
|
||||
/* v8 ignore next 3 -- agent-loop closes only steps whose request header was recorded */
|
||||
if (provider === undefined) {
|
||||
throw new Error(`llm-retry: no request provider for closed turn ${turn}/step ${step}`)
|
||||
}
|
||||
if (policy.mode === 'always') {
|
||||
if (signal.aborted || lifetime.signal.aborted) return
|
||||
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
|
||||
// The loop and plugin lifetime stay open until delegated recovery settles.
|
||||
// An abort then wins before the decision or fallback can mutate later state.
|
||||
const downstream = await settleDownstream(next)
|
||||
if (fusedSignal.aborted) return
|
||||
if (downstream.type === 'error') {
|
||||
ctx.logger.warn(
|
||||
`llm-retry: provider "${provider}" always policy ignored a downstream recovery failure: %o`,
|
||||
downstream.error,
|
||||
)
|
||||
}
|
||||
if (downstream.type === 'decision' && downstream.decision?.kind === 'retry') {
|
||||
return downstream.decision
|
||||
}
|
||||
} else if (!policy.retryableCodes.includes(failure.code)) {
|
||||
return next()
|
||||
}
|
||||
|
||||
const retry = priorTransientFailures + 1
|
||||
const policyKey = retryPolicyKey(policy)
|
||||
const firstPriorTurn = turn - priorFailures.length
|
||||
const priorPolicyRetry = agent.session.events.findLast((event): event is SessionEvent<'llm/retry'> =>
|
||||
event.type === 'llm/retry'
|
||||
&& event.data.turn >= firstPriorTurn
|
||||
&& event.data.turn < turn
|
||||
&& event.data.provider === provider
|
||||
&& event.data.policyKey === policyKey,
|
||||
)
|
||||
const previousRetry = priorPolicyRetry?.data.retry ?? 0
|
||||
if (policy.mode === 'normal' && previousRetry >= policy.maxRetries) return next()
|
||||
const retry = previousRetry + 1
|
||||
let delayMs: number
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& Number.isFinite(failure.providerRetryAfterMs)
|
||||
&& failure.providerRetryAfterMs > 0) {
|
||||
if (failure.providerRetryAfterMs > resolved.maxDelayMs) return next()
|
||||
delayMs = failure.providerRetryAfterMs
|
||||
if (failure.providerRetryAfterMs > policy.maxDelayMs) {
|
||||
if (policy.mode === 'normal') return next()
|
||||
delayMs = localDelay(policy, retry, random)
|
||||
} else {
|
||||
delayMs = failure.providerRetryAfterMs
|
||||
}
|
||||
} else {
|
||||
delayMs = localDelay(resolved, retry, random)
|
||||
delayMs = localDelay(policy, retry, random)
|
||||
}
|
||||
|
||||
const tracked = backoff(agent, turn, step, failure, retry, delayMs, signal)
|
||||
.finally(() => active.delete(tracked))
|
||||
active.add(tracked)
|
||||
return tracked
|
||||
return backoff(agent, turn, step, failure, provider, policy, policyKey, retry, delayMs, signal)
|
||||
}
|
||||
|
||||
const disposeListener = ctx.on('agent/request-error', (
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
error: RequestError,
|
||||
failure: LlmFailure,
|
||||
priorFailures: readonly LlmFailure[],
|
||||
policy: ResolvedRetryPolicy | undefined,
|
||||
signal: AbortSignal,
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
) => {
|
||||
// A waterfall may have captured this callback before its registration was
|
||||
// removed. Lifetime cancellation must prevent that stale callback from
|
||||
// entering a downstream policy after disposal.
|
||||
if (lifetime.signal.aborted) return Promise.resolve<RequestErrorAction>(undefined)
|
||||
return track(recover(agent, turn, step, error, failure, priorFailures, policy, signal, next))
|
||||
})
|
||||
|
||||
ctx.effect(() => async () => {
|
||||
disposeListener()
|
||||
lifetime.abort(new Error('llm-retry plugin disposed'))
|
||||
await Promise.allSettled([...active])
|
||||
}, 'llm-retry: abort and drain backoffs')
|
||||
}, 'llm-retry: abort and drain active recovery')
|
||||
}
|
||||
|
||||
@@ -2,8 +2,10 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
import { providerForClosedStep } from './history.ts'
|
||||
import type {} from './index.ts'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm-retry'
|
||||
@@ -13,21 +15,95 @@ export const name = 'llm-retry-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/** Validate the complete provider-neutral failure payload at the durable boundary. */
|
||||
function validateFailure(value: unknown, fail: InvariantFailure): asserts value is LlmFailure {
|
||||
if (typeof value !== 'object' || value === null) {
|
||||
fail('llm/retry failure must be an object')
|
||||
}
|
||||
const failure = value as Partial<LlmFailure>
|
||||
if (typeof failure.message !== 'string' || failure.message.length === 0) {
|
||||
fail('llm/retry failure.message must be a non-empty string')
|
||||
}
|
||||
if (typeof failure.code !== 'string' || failure.code.length === 0) {
|
||||
fail('llm/retry failure.code must be a non-empty string')
|
||||
}
|
||||
if (failure.status !== undefined
|
||||
&& (!Number.isInteger(failure.status) || failure.status < 100 || failure.status > 599)) {
|
||||
fail('llm/retry failure.status must be an integer from 100 through 599 when present')
|
||||
}
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& (!Number.isFinite(failure.providerRetryAfterMs) || failure.providerRetryAfterMs <= 0)) {
|
||||
fail('llm/retry failure.providerRetryAfterMs must be a positive finite number when present')
|
||||
}
|
||||
if (failure.requestId !== undefined
|
||||
&& (typeof failure.requestId !== 'string' || failure.requestId.length === 0)) {
|
||||
fail('llm/retry failure.requestId must be a non-empty string when present')
|
||||
}
|
||||
}
|
||||
|
||||
/** Find the first turn in the structured-failure retry chain containing `turn`. */
|
||||
function retryChainStart(history: readonly SessionEvent[], turn: number): number {
|
||||
let startIndex = history.findLastIndex(
|
||||
event => event.type === 'turn/start' && event.data.turn === turn,
|
||||
)
|
||||
while (startIndex >= 0) {
|
||||
const start = history[startIndex]
|
||||
if (start?.type !== 'turn/start' || start.data.trigger.kind !== 'retry') break
|
||||
|
||||
let endIndex = startIndex - 1
|
||||
while (endIndex >= 0 && history[endIndex]?.type !== 'turn/end') endIndex -= 1
|
||||
const end = history[endIndex]
|
||||
if (end?.type !== 'turn/end'
|
||||
|| end.data.reason.kind !== 'error'
|
||||
|| end.data.reason.failure === undefined) break
|
||||
|
||||
const previousStart = history.findLastIndex(
|
||||
(event, index) =>
|
||||
index < endIndex
|
||||
&& event.type === 'turn/start'
|
||||
&& event.data.turn === end.data.turn,
|
||||
)
|
||||
if (previousStart < 0) break
|
||||
startIndex = previousStart
|
||||
}
|
||||
return startIndex
|
||||
}
|
||||
|
||||
/** Validate one retry record against the open turn and most recently closed step. */
|
||||
function validateRetry(
|
||||
history: readonly SessionEvent[],
|
||||
event: SessionEvent<'llm/retry'>,
|
||||
fail: InvariantFailure,
|
||||
): void {
|
||||
const { turn, step, retry, maxRetries, delayMs } = event.data
|
||||
const { turn, step, provider, mode, policyKey, retry, delayMs } = event.data
|
||||
const failure: unknown = event.data.failure
|
||||
validateFailure(failure, fail)
|
||||
if (!Number.isSafeInteger(retry) || retry < 1) {
|
||||
fail('llm/retry retry must be a positive safe integer')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 1 || retry > maxRetries) {
|
||||
fail(`llm/retry retry ${retry} must not exceed a positive safe maxRetries ${maxRetries}`)
|
||||
if (typeof provider !== 'string' || provider.length === 0) {
|
||||
fail('llm/retry provider must be a non-empty string')
|
||||
}
|
||||
if (!(delayMs >= 0 && delayMs <= MAX_TIMER_DELAY_MS)) {
|
||||
fail(`llm/retry delayMs must be within 0..${MAX_TIMER_DELAY_MS}`)
|
||||
if (typeof policyKey !== 'string' || policyKey.length === 0) {
|
||||
fail('llm/retry policyKey must be a non-empty string')
|
||||
}
|
||||
switch (mode) {
|
||||
case 'normal': {
|
||||
const { maxRetries } = event.data
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 1 || retry > maxRetries) {
|
||||
fail(`llm/retry retry ${retry} must not exceed a positive safe maxRetries ${maxRetries}`)
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'always':
|
||||
if ('maxRetries' in event.data) fail('llm/retry always mode must omit maxRetries')
|
||||
break
|
||||
default:
|
||||
fail(`llm/retry mode must be normal or always, got ${String(mode)}`)
|
||||
}
|
||||
if (typeof delayMs !== 'number' || !Number.isFinite(delayMs)
|
||||
|| delayMs < 0 || delayMs > MAX_TIMER_DELAY_MS) {
|
||||
fail(`llm/retry delayMs must be a finite number within 0..${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
|
||||
const currentTurnEvents: SessionEvent[] = []
|
||||
@@ -58,15 +134,24 @@ function validateRetry(
|
||||
if (closedStep === undefined || step !== closedStep) {
|
||||
fail(`llm/retry names step ${step}, but the latest closed step is ${String(closedStep)}`)
|
||||
}
|
||||
const routedProvider = providerForClosedStep(history, turn, step)
|
||||
if (routedProvider !== provider) {
|
||||
fail(`llm/retry provider ${provider} does not match the failed request provider ${String(routedProvider)}`)
|
||||
}
|
||||
|
||||
const priorRetries = currentTurnEvents
|
||||
const chainStart = retryChainStart(history, turn)
|
||||
const chain = history.slice(Math.max(chainStart, 0))
|
||||
const lastSuccess = chain.findLastIndex(prior => prior.type === 'assistant/message')
|
||||
const chainRetries = chain.slice(lastSuccess + 1)
|
||||
.filter((prior): prior is SessionEvent<'llm/retry'> => prior.type === 'llm/retry')
|
||||
if (priorRetries.some(prior => prior.data.step === step)) {
|
||||
if (chainRetries.some(prior => prior.data.turn === turn && prior.data.step === step)) {
|
||||
fail(`llm/retry duplicates the retry record for turn ${turn}/step ${step}`)
|
||||
}
|
||||
const priorRetry = priorRetries[0]
|
||||
if (priorRetry !== undefined && retry <= priorRetry.data.retry) {
|
||||
fail(`llm/retry retry ${retry} must increase after retry ${priorRetry.data.retry}`)
|
||||
const priorPolicyRetry = chainRetries.findLast(prior =>
|
||||
prior.data.provider === provider && prior.data.policyKey === policyKey)
|
||||
const expectedRetry = (priorPolicyRetry?.data.retry ?? 0) + 1
|
||||
if (retry !== expectedRetry) {
|
||||
fail(`llm/retry retry ${retry} must equal provider policy retry ${expectedRetry}`)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SessionStore, { SessionId, type Session } from '@deepseek-ai/dsh-session'
|
||||
import { ProviderRequestId } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import InvariantService from '@deepseek-ai/dsh-invariants'
|
||||
import * as RetryInvariant from '@deepseek-ai/dsh-llm-retry/invariant'
|
||||
import { providerForClosedStep } from '../src/history.ts'
|
||||
|
||||
async function setup(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
@@ -17,121 +19,259 @@ function closeStep(ctx: Context, id: string, turn = 1, step = 1) {
|
||||
const session = ctx.sessions.create(SessionId(id))
|
||||
session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn, step })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn, step })
|
||||
return session
|
||||
}
|
||||
|
||||
function appendRetryTurn(session: Session, turn: number) {
|
||||
session.append('turn/start', { turn, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn, step: 1 })
|
||||
session.append('llm/retry', { turn, step: 1, ...normal })
|
||||
}
|
||||
|
||||
const failure = { message: 'provider busy', code: 'RATE_LIMIT', status: 429 }
|
||||
const normal = {
|
||||
provider: 'mock',
|
||||
mode: 'normal' as const,
|
||||
policyKey: 'normal-policy',
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 1,
|
||||
failure,
|
||||
}
|
||||
const always = {
|
||||
provider: 'mock',
|
||||
mode: 'always' as const,
|
||||
policyKey: 'always-policy',
|
||||
retry: 1,
|
||||
delayMs: 1,
|
||||
failure,
|
||||
}
|
||||
|
||||
describe('llm-retry invariants', () => {
|
||||
it('accepts increasing retry records for successive closed steps and ignores unrelated events', async () => {
|
||||
it('has no provider without the requested closed step or a route marker', () => {
|
||||
expect(providerForClosedStep([], 1, 1)).toBeUndefined()
|
||||
expect(providerForClosedStep([{
|
||||
type: 'step/end',
|
||||
data: { turn: 1, step: 1 },
|
||||
}] as never, 1, 1)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('accepts bounded and unbounded records after successive closed steps', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-valid')
|
||||
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn: 2, step: 1 })
|
||||
session.append('step/end', { turn: 2, step: 1 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 500, failure,
|
||||
})
|
||||
session.append('step/start', { turn: 1, step: 2 })
|
||||
session.append('step/end', { turn: 1, step: 2 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 2, maxRetries: 2, delayMs: 1_000, failure,
|
||||
})
|
||||
const zeroDelay = closeStep(ctx, 'retry-invariant-zero-delay')
|
||||
zeroDelay.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 1, delayMs: 0, failure,
|
||||
turn: 2, step: 1, ...normal, retry: 2, delayMs: 0,
|
||||
})
|
||||
const unbounded = closeStep(ctx, 'retry-invariant-always')
|
||||
unbounded.append('llm/retry', { turn: 1, step: 1, ...always })
|
||||
}).not.toThrow()
|
||||
expect(() => { ctx.emit('tools/change') }).not.toThrow()
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ retry: 0, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1.5, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1, maxRetries: 0, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 1, maxRetries: 1.5, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 3, maxRetries: 2, delayMs: 1 }, /must not exceed/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: -1 }, /delayMs/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: MAX_TIMER_DELAY_MS + 1 }, /delayMs/],
|
||||
])('rejects invalid retry bounds %#', async (data, message) => {
|
||||
it('validates the complete durable failure payload', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, `retry-invariant-bounds-${data.retry}-${data.maxRetries}-${data.delayMs}`)
|
||||
const complete = closeStep(ctx, 'retry-invariant-complete-failure')
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...data, failure })
|
||||
complete.append('llm/retry', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
...always,
|
||||
failure: {
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 25,
|
||||
requestId: ProviderRequestId('request-1'),
|
||||
},
|
||||
})
|
||||
}).not.toThrow()
|
||||
|
||||
const invalidFailures: readonly [string, unknown, RegExp][] = [
|
||||
['null', null, /failure must be an object/],
|
||||
['message-type', { message: 1, code: 'RATE_LIMIT' }, /failure\.message/],
|
||||
['message-empty', { message: '', code: 'RATE_LIMIT' }, /failure\.message/],
|
||||
['code-type', { message: 'failed', code: 1 }, /failure\.code/],
|
||||
['code-empty', { message: 'failed', code: '' }, /failure\.code/],
|
||||
['status-type', { message: 'failed', code: 'RATE_LIMIT', status: 429.5 }, /failure\.status/],
|
||||
['status-low', { message: 'failed', code: 'RATE_LIMIT', status: 99 }, /failure\.status/],
|
||||
['status-high', { message: 'failed', code: 'RATE_LIMIT', status: 600 }, /failure\.status/],
|
||||
[
|
||||
'retry-after-type',
|
||||
{ message: 'failed', code: 'RATE_LIMIT', providerRetryAfterMs: '25' },
|
||||
/failure\.providerRetryAfterMs/,
|
||||
],
|
||||
[
|
||||
'retry-after-zero',
|
||||
{ message: 'failed', code: 'RATE_LIMIT', providerRetryAfterMs: 0 },
|
||||
/failure\.providerRetryAfterMs/,
|
||||
],
|
||||
['request-id-type', { message: 'failed', code: 'RATE_LIMIT', requestId: 1 }, /failure\.requestId/],
|
||||
['request-id-empty', { message: 'failed', code: 'RATE_LIMIT', requestId: '' }, /failure\.requestId/],
|
||||
]
|
||||
for (const [name, invalidFailure, message] of invalidFailures) {
|
||||
const session = closeStep(ctx, `retry-invariant-failure-${name}`)
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, ...always, failure: invalidFailure,
|
||||
} as never)
|
||||
}).toThrow(message)
|
||||
}
|
||||
})
|
||||
|
||||
it.each([
|
||||
['retry-zero', { ...normal, retry: 0 }, /positive safe integer/],
|
||||
['retry-fraction', { ...normal, retry: 1.5 }, /positive safe integer/],
|
||||
['max-zero', { ...normal, maxRetries: 0 }, /positive safe maxRetries/],
|
||||
['max-fraction', { ...normal, maxRetries: 1.5 }, /positive safe maxRetries/],
|
||||
['over-budget', { ...normal, retry: 3 }, /must not exceed/],
|
||||
['always-maximum', { ...always, maxRetries: 2 }, /always mode must omit maxRetries/],
|
||||
['unknown-mode', { ...always, mode: 'sometimes' }, /mode must be normal or always/],
|
||||
['empty-provider', { ...always, provider: '' }, /provider must be a non-empty string/],
|
||||
['empty-policy-key', { ...always, policyKey: '' }, /policyKey must be a non-empty string/],
|
||||
['delay-negative', { ...normal, delayMs: -1 }, /delayMs/],
|
||||
['delay-overflow', { ...normal, delayMs: MAX_TIMER_DELAY_MS + 1 }, /delayMs/],
|
||||
['delay-type', { ...normal, delayMs: '1' }, /delayMs/],
|
||||
])('rejects invalid retry data: %s', async (name, data, message) => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, `retry-invariant-${name}`)
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...data } as never)
|
||||
}).toThrow(message)
|
||||
})
|
||||
|
||||
it('rejects retry records outside the matching closed-step boundary', async () => {
|
||||
it('rejects records outside the latest closed step of an open turn', async () => {
|
||||
const ctx = await setup()
|
||||
const absent = ctx.sessions.create(SessionId('retry-invariant-no-turn'))
|
||||
expect(() => {
|
||||
absent.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
absent.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/inside an open turn/)
|
||||
|
||||
const wrongTurn = closeStep(ctx, 'retry-invariant-wrong-turn')
|
||||
expect(() => {
|
||||
wrongTurn.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
wrongTurn.append('llm/retry', { turn: 2, step: 1, ...normal })
|
||||
}).toThrow(/open turn is 1/)
|
||||
|
||||
const openStep = ctx.sessions.create(SessionId('retry-invariant-open-step'))
|
||||
openStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
openStep.append('step/start', { turn: 1, step: 1 })
|
||||
expect(() => {
|
||||
openStep.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
openStep.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/step 1 is still open/)
|
||||
|
||||
const noStep = ctx.sessions.create(SessionId('retry-invariant-no-step'))
|
||||
noStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
expect(() => {
|
||||
noStep.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
noStep.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/latest closed step is undefined/)
|
||||
|
||||
const wrongStep = closeStep(ctx, 'retry-invariant-wrong-step')
|
||||
expect(() => {
|
||||
wrongStep.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
wrongStep.append('llm/retry', { turn: 1, step: 2, ...normal })
|
||||
}).toThrow(/latest closed step is 1/)
|
||||
|
||||
const closedTurn = closeStep(ctx, 'retry-invariant-closed-turn')
|
||||
closedTurn.append('turn/end', { turn: 1, reason: { kind: 'aborted' } })
|
||||
expect(() => {
|
||||
closedTurn.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
closedTurn.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
it('rejects duplicate and non-increasing retry records', async () => {
|
||||
it('rejects a second retry record for the same step', async () => {
|
||||
const ctx = await setup()
|
||||
const duplicate = closeStep(ctx, 'retry-invariant-duplicate')
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
expect(() => {
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 2, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/duplicates the retry record/)
|
||||
const session = closeStep(ctx, 'retry-invariant-duplicate')
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
|
||||
const nonIncreasing = closeStep(ctx, 'retry-invariant-non-increasing')
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
nonIncreasing.append('step/start', { turn: 1, step: 2 })
|
||||
nonIncreasing.append('step/end', { turn: 1, step: 2 })
|
||||
expect(() => {
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/must increase/)
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal, retry: 2 })
|
||||
}).toThrow(/duplicates the retry record/)
|
||||
})
|
||||
|
||||
it('binds retry numbering to the provider policy and resets it after success', async () => {
|
||||
const ctx = await setup()
|
||||
const mismatch = closeStep(ctx, 'retry-invariant-numbering')
|
||||
mismatch.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
mismatch.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
mismatch.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
mismatch.append('step/start', { turn: 2, step: 1 })
|
||||
mismatch.append('step/end', { turn: 2, step: 1 })
|
||||
expect(() => {
|
||||
mismatch.append('llm/retry', { turn: 2, step: 1, ...normal, retry: 1 })
|
||||
}).toThrow(/must equal provider policy retry 2/)
|
||||
|
||||
const reset = closeStep(ctx, 'retry-invariant-reset')
|
||||
reset.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
reset.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
reset.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
reset.append('step/start', { turn: 2, step: 1 })
|
||||
reset.append('assistant/message', {
|
||||
turn: 2,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: 'success' }],
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
}, { surfaceOp: 'append' })
|
||||
reset.append('step/end', { turn: 2, step: 1 })
|
||||
reset.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
reset.append('turn/start', { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reset.append('step/start', { turn: 3, step: 1 })
|
||||
reset.append('step/end', { turn: 3, step: 1 })
|
||||
expect(() => {
|
||||
reset.append('llm/retry', { turn: 3, step: 1, ...normal })
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('starts a fresh retry chain after incomplete predecessor boundaries', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
|
||||
const missingEnd = ctx.sessions.create(SessionId('retry-invariant-missing-end'))
|
||||
missingEnd.append('user/message', {
|
||||
content: [{ type: 'text', text: 'idle context' }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
appendRetryTurn(missingEnd, 2)
|
||||
|
||||
const nonFailureEnd = ctx.sessions.create(SessionId('retry-invariant-non-failure-end'))
|
||||
nonFailureEnd.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
nonFailureEnd.append('user/message', {
|
||||
content: [{ type: 'text', text: 'idle context' }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
appendRetryTurn(nonFailureEnd, 2)
|
||||
|
||||
const missingStart = ctx.sessions.create(SessionId('retry-invariant-missing-start'))
|
||||
missingStart.append('turn/end', {
|
||||
turn: 1,
|
||||
reason: { kind: 'error', step: 1, failure },
|
||||
})
|
||||
appendRetryTurn(missingStart, 2)
|
||||
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).resolves.toBeDefined()
|
||||
})
|
||||
|
||||
it('rejects a provider that does not match the failed request route', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-provider')
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...always, provider: 'other' })
|
||||
}).toThrow(/does not match the failed request provider mock/)
|
||||
})
|
||||
|
||||
it('validates existing histories on late registration', async () => {
|
||||
@@ -139,9 +279,7 @@ describe('llm-retry invariants', () => {
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = ctx.sessions.create(SessionId('retry-invariant-late'))
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).rejects.toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
@@ -7,10 +7,9 @@ import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import Include from '@cordisjs/plugin-include'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import LlmService, { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { LlmAdapter, LlmError, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, ResolvedRetryPolicy, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
@@ -21,6 +20,16 @@ let context: Context | undefined
|
||||
|
||||
class TransientOnceAdapter extends LlmAdapter {
|
||||
requests = 0
|
||||
private readonly retryPolicy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
maxRetries: 1,
|
||||
retryableCodes: ['RATE_LIMIT', 'SERVER'],
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'loader test provider retryPolicy')
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests += 1
|
||||
@@ -32,17 +41,6 @@ class TransientOnceAdapter extends LlmAdapter {
|
||||
}
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await context?.fiber.dispose()
|
||||
context = undefined
|
||||
@@ -87,7 +85,7 @@ describe('real Loader composition', () => {
|
||||
// Real-Loader composition resolves workspace packages through tsx at test
|
||||
// time; first resolution after the host/client program split is slow enough
|
||||
// to trip the default 5s budget on cold caches.
|
||||
it('loads the flat policy and records recovery through the shipping loop', { timeout: 60_000 }, async () => {
|
||||
it('loads provider-supplied policy and records recovery through the shipping loop', { timeout: 60_000 }, async () => {
|
||||
const loaded = await loadYaml([
|
||||
"- name: '@deepseek-ai/dsh-llm'",
|
||||
"- name: '@deepseek-ai/dsh-session'",
|
||||
@@ -95,12 +93,6 @@ describe('real Loader composition', () => {
|
||||
"- name: '@deepseek-ai/dsh-tools'",
|
||||
"- name: '@deepseek-ai/dsh-agent'",
|
||||
"- name: '@deepseek-ai/dsh-llm-retry'",
|
||||
' config:',
|
||||
' maxTransientRetries: 1',
|
||||
' initialDelayMs: 1',
|
||||
' maxDelayMs: 1',
|
||||
' jitterRatio: 0',
|
||||
' retryableCodes: [RATE_LIMIT, SERVER]',
|
||||
"- name: '@deepseek-ai/dsh-agent-loop'",
|
||||
])
|
||||
|
||||
@@ -113,9 +105,8 @@ describe('real Loader composition', () => {
|
||||
const adapter = new TransientOnceAdapter()
|
||||
loaded.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = loaded.agentLoop.create(SessionId('loader-retry'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(loaded, agent)
|
||||
agent.followup([{ type: 'text', text: 'recover' }])
|
||||
await idle
|
||||
agent.followup({ content: [{ type: 'text', text: 'recover' }], source: { kind: 'user' } })
|
||||
await agent.whenIdle()
|
||||
|
||||
expect(adapter.requests).toBe(2)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1)
|
||||
|
||||
@@ -34,16 +34,29 @@ describe.each(['jsonl', 'sqlite'] as const)('%s retry-event persistence', (kind)
|
||||
const session = ctx.sessions.create(SessionId(`retry-${kind}`))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const event = session.append('llm/retry', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
provider: 'mock',
|
||||
mode: 'always',
|
||||
policyKey: '["always",500,10000,0.1]',
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 750,
|
||||
failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 },
|
||||
})
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'aborted' } })
|
||||
session.append('turn/end', {
|
||||
turn: 1,
|
||||
reason: {
|
||||
kind: 'error',
|
||||
step: 1,
|
||||
failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 },
|
||||
},
|
||||
})
|
||||
|
||||
expect(session.deriveMessages()).toEqual([])
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
@@ -1,22 +1,30 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { afterEach, describe, expect, expectTypeOf, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Fiber } from 'cordis'
|
||||
import LlmService, { CallId, EMPTY_RESPONSE_CODE, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, EMPTY_RESPONSE_CODE, LlmAdapter, LlmError, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
AlwaysRetryPolicyConfig,
|
||||
BackoffConfig,
|
||||
GenerateOptions,
|
||||
NormalRetryPolicyConfig,
|
||||
ResolvedRetryPolicy,
|
||||
RetryPolicyConfig,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent, RequestErrorAction } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import * as retry from '../src/index.ts'
|
||||
|
||||
type ScriptEntry = Error | Iterable<StreamChunk> | AsyncIterable<StreamChunk>
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
readonly requests: GenerateOptions[] = []
|
||||
private retryPolicies: Readonly<Record<string, ResolvedRetryPolicy | undefined>> = {}
|
||||
|
||||
constructor(private readonly entries: ScriptEntry[]) {
|
||||
super()
|
||||
@@ -29,6 +37,21 @@ class ScriptedAdapter extends LlmAdapter {
|
||||
if (entry instanceof Error) throw entry
|
||||
yield* entry
|
||||
}
|
||||
|
||||
configureRetryPolicies(
|
||||
policies: Readonly<Record<string, RetryPolicyConfig | undefined>>,
|
||||
): void {
|
||||
this.retryPolicies = Object.fromEntries(Object.entries(policies).map(([provider, policy]) => [
|
||||
provider,
|
||||
policy === undefined
|
||||
? undefined
|
||||
: resolveRetryPolicy(policy, `retry test provider "${provider}" retryPolicy`),
|
||||
]))
|
||||
}
|
||||
|
||||
override providerRetryPolicy(provider: string): ResolvedRetryPolicy | undefined {
|
||||
return this.retryPolicies[provider]
|
||||
}
|
||||
}
|
||||
|
||||
async function* partialToolFailure(error: Error): AsyncGenerator<StreamChunk> {
|
||||
@@ -71,11 +94,11 @@ function emptyCompletion(): StreamChunk[] {
|
||||
}
|
||||
|
||||
async function harness(
|
||||
adapter: LlmAdapter,
|
||||
config: retry.Config = {},
|
||||
adapter: ScriptedAdapter,
|
||||
policies: Readonly<Record<string, RetryPolicyConfig | undefined>> = { mock: normalConfig() },
|
||||
beforeRetry?: (ctx: Context) => void,
|
||||
internals: retry.RetryInternals = {},
|
||||
): Promise<{ ctx: Context; retryFiber: Fiber }> {
|
||||
): Promise<{ ctx: Context; retryFiber: Fiber; disposeAdapter: () => void }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
@@ -83,18 +106,42 @@ async function harness(
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
beforeRetry?.(ctx)
|
||||
const resolvedConfig = Object.assign({
|
||||
maxTransientRetries: 2,
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0,
|
||||
}, config)
|
||||
adapter.configureRetryPolicies(policies)
|
||||
const retryFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
retry.apply(inner, resolvedConfig, internals)
|
||||
retry.apply(inner, {}, internals)
|
||||
}, { inject: retry.inject }))
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
return { ctx, retryFiber }
|
||||
const disposeAdapter = ctx.llm.registerAdapter(['mock', 'other'], adapter)
|
||||
return { ctx, retryFiber, disposeAdapter }
|
||||
}
|
||||
|
||||
function normalConfig(
|
||||
overrides: Partial<Omit<NormalRetryPolicyConfig, 'mode'>> = {},
|
||||
): NormalRetryPolicyConfig {
|
||||
const { backoff, ...policy } = overrides
|
||||
return {
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
...policy,
|
||||
backoff: {
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0,
|
||||
...backoff,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function alwaysConfig(backoff: BackoffConfig = {}): AlwaysRetryPolicyConfig {
|
||||
return {
|
||||
mode: 'always',
|
||||
backoff: {
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0,
|
||||
...backoff,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
@@ -127,33 +174,31 @@ afterEach(async () => {
|
||||
context = undefined
|
||||
})
|
||||
|
||||
describe('bounded transient retry policy', () => {
|
||||
describe('provider-routed retry policy', () => {
|
||||
it('records the scheduled delay before opening a fresh request attempt', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('busy', 'RATE_LIMIT', { status: 429 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
mock: normalConfig({ retryableCodes: ['SERVER', 'RATE_LIMIT'] }),
|
||||
}, undefined, { random: () => 0.5 }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-success'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = new Promise<Extract<(typeof agent.session.events)[number], { type: 'llm/retry' }>>((resolve) => {
|
||||
const dispose = context?.on('session/event', (session, event) => {
|
||||
if (session === agent.session && event.type === 'llm/retry') {
|
||||
dispose?.()
|
||||
resolve(event)
|
||||
}
|
||||
})
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
const event = await scheduled
|
||||
|
||||
expect(event.data).toEqual({
|
||||
turn: 1,
|
||||
step: 1,
|
||||
provider: 'mock',
|
||||
mode: 'normal',
|
||||
policyKey: '["normal",2,["RATE_LIMIT","SERVER"],500,10000,0]',
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 500,
|
||||
@@ -168,8 +213,8 @@ describe('bounded transient retry policy', () => {
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(agent.session.events.filter(item => item.type === 'step/start').map(item => item.data.step))
|
||||
.toEqual([1, 2])
|
||||
expect(agent.session.events.filter(item => item.type === 'step/start').map(item => item.data))
|
||||
.toEqual([{ turn: 1, step: 1 }, { turn: 2, step: 1 }])
|
||||
expect(agent.session.deriveMessages().at(-1)).toEqual({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'done' }],
|
||||
@@ -190,7 +235,7 @@ describe('bounded transient retry policy', () => {
|
||||
const agent = context.agentLoop.create(SessionId('retry-empty-response'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
const event = await scheduled
|
||||
expect(event.data.failure).toEqual({
|
||||
message: 'model returned a completed response with no content',
|
||||
@@ -202,8 +247,10 @@ describe('bounded transient retry policy', () => {
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => ({
|
||||
turn: event.data.turn,
|
||||
step: event.data.step,
|
||||
}))).toEqual([{ turn: 2, step: 1 }])
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'recovered' }],
|
||||
@@ -230,18 +277,20 @@ describe('bounded transient retry policy', () => {
|
||||
const agent = context.agentLoop.create(SessionId('retry-partial'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(500)
|
||||
await idle
|
||||
|
||||
const failedChunks = agent.session.events.filter(event =>
|
||||
event.type === 'assistant/chunk' && event.data.step === 1,
|
||||
event.type === 'assistant/chunk' && event.data.turn === 1 && event.data.step === 1,
|
||||
)
|
||||
expect(failedChunks).toHaveLength(6)
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => ({
|
||||
turn: event.data.turn,
|
||||
step: event.data.step,
|
||||
}))).toEqual([{ turn: 2, step: 1 }])
|
||||
expect(agent.session.events.some(event => event.type === 'tool/call')).toBe(false)
|
||||
expect(toolExecutions).toBe(0)
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
@@ -259,13 +308,15 @@ describe('bounded transient retry policy', () => {
|
||||
new LlmError('busy two', 'SERVER'),
|
||||
new LlmError('busy three', 'SERVER'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { jitterRatio: 0.1 }, undefined, {
|
||||
;({ ctx: context } = await harness(adapter, { mock: normalConfig({
|
||||
backoff: { jitterRatio: 0.1 },
|
||||
}) }, undefined, {
|
||||
random: () => samples.shift() ?? 0.5,
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-exhausted'), { provider: 'mock', model: 'mock' })
|
||||
const first = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
expect((await first).data.delayMs).toBe(450)
|
||||
|
||||
const second = waitForRetry(context, agent, 2)
|
||||
@@ -290,15 +341,13 @@ describe('bounded transient retry policy', () => {
|
||||
new LlmError('busy', 'SERVER'),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
jitterRatio: 1,
|
||||
}, undefined, { random: () => 0 }))
|
||||
;({ ctx: context } = await harness(adapter, { mock: normalConfig({
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 1 },
|
||||
}) }, undefined, { random: () => 0 }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-zero-delay'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
expect((await scheduled).data.delayMs).toBe(0)
|
||||
|
||||
const idle = waitForIdle(context, agent)
|
||||
@@ -313,10 +362,12 @@ describe('bounded transient retry policy', () => {
|
||||
new LlmError('wait', 'RATE_LIMIT', { providerRetryAfterMs: 2_000 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(accepted, { jitterRatio: 1 }))
|
||||
;({ ctx: context } = await harness(accepted, { mock: normalConfig({
|
||||
backoff: { jitterRatio: 1 },
|
||||
}) }))
|
||||
const acceptedAgent = context.agentLoop.create(SessionId('retry-after-accepted'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, acceptedAgent, 1)
|
||||
acceptedAgent.followup([{ type: 'text', text: 'go' }])
|
||||
acceptedAgent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
expect((await scheduled).data.delayMs).toBe(2_000)
|
||||
const acceptedIdle = waitForIdle(context, acceptedAgent)
|
||||
await vi.advanceTimersByTimeAsync(2_000)
|
||||
@@ -330,36 +381,397 @@ describe('bounded transient retry policy', () => {
|
||||
;({ ctx: context } = await harness(rejected))
|
||||
const rejectedAgent = context.agentLoop.create(SessionId('retry-after-rejected'), { provider: 'mock', model: 'mock' })
|
||||
const rejectedIdle = waitForIdle(context, rejectedAgent)
|
||||
rejectedAgent.followup([{ type: 'text', text: 'go' }])
|
||||
rejectedAgent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await rejectedIdle
|
||||
expect(rejected.requests).toHaveLength(1)
|
||||
expect(rejectedAgent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
})
|
||||
|
||||
it('uses local jittered backoff when always mode receives an over-cap Retry-After', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('wait too long', 'AUTH', { providerRetryAfterMs: 10 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig({
|
||||
initialDelayMs: 2,
|
||||
maxDelayMs: 4,
|
||||
jitterRatio: 0.5,
|
||||
}) }, undefined, { random: () => 1 }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-always-over-cap'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
expect((await scheduled).data.delayMs).toBe(3)
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(3)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('delegates non-transient failures without scheduling a timer', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-auth'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await idle
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('delegates when no final adapter served the failed request', async () => {
|
||||
const adapter = new ScriptedAdapter([textResponse('must not run')])
|
||||
const mounted = await harness(adapter, { mock: alwaysConfig() })
|
||||
context = mounted.ctx
|
||||
mounted.disposeAdapter()
|
||||
const agent = context.agentLoop.create(SessionId('retry-no-serving-policy'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'missing route' }], source: { kind: 'user' } })
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(0)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'error', failure: { code: 'NO_ADAPTER' } } },
|
||||
})
|
||||
})
|
||||
|
||||
it('selects policy by the failed request provider', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('mock auth failed', 'AUTH'),
|
||||
new LlmError('other auth failed', 'AUTH'),
|
||||
textResponse('other recovered'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
other: alwaysConfig({ initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 }),
|
||||
}))
|
||||
|
||||
const normalAgent = context.agentLoop.create(SessionId('retry-provider-normal'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const normalIdle = waitForIdle(context, normalAgent)
|
||||
normalAgent.followup({ content: [{ type: 'text', text: 'normal' }], source: { kind: 'user' } })
|
||||
await normalIdle
|
||||
expect(normalAgent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
|
||||
const alwaysAgent = context.agentLoop.create(SessionId('retry-provider-always'), {
|
||||
provider: 'other',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, alwaysAgent, 1)
|
||||
alwaysAgent.followup({ content: [{ type: 'text', text: 'always' }], source: { kind: 'user' } })
|
||||
expect((await scheduled).data).toMatchObject({
|
||||
provider: 'other',
|
||||
mode: 'always',
|
||||
retry: 1,
|
||||
delayMs: 1,
|
||||
})
|
||||
const alwaysIdle = waitForIdle(context, alwaysAgent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
await alwaysIdle
|
||||
|
||||
expect(adapter.requests.map(request => request.provider)).toEqual(['mock', 'other', 'other'])
|
||||
})
|
||||
|
||||
it('selects an always policy from the provider chosen by agent/request', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('rerouted auth failed', 'AUTH'),
|
||||
textResponse('rerouted recovery'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
other: alwaysConfig({ initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 }),
|
||||
}, (ctx) => {
|
||||
ctx.on('agent/request', async (_agent, _turn, _step, _signal, next) => ({
|
||||
...await next(),
|
||||
provider: 'other',
|
||||
}))
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-provider-rerouted'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'reroute' }], source: { kind: 'user' } })
|
||||
expect((await scheduled).data).toMatchObject({ provider: 'other', mode: 'always' })
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests.map(request => request.provider)).toEqual(['other', 'other'])
|
||||
})
|
||||
|
||||
it('keeps finite retry budgets scoped to the failed provider', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('mock failed', 'SERVER'),
|
||||
new LlmError('other failed', 'SERVER'),
|
||||
textResponse('other recovered'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
mock: normalConfig({
|
||||
maxRetries: 1,
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1 },
|
||||
}),
|
||||
other: normalConfig({
|
||||
maxRetries: 1,
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1 },
|
||||
}),
|
||||
}, (ctx) => {
|
||||
ctx.on('agent/request', async (_agent, turn, _step, _signal, next) => ({
|
||||
...await next(),
|
||||
provider: turn === 1 ? 'mock' : 'other',
|
||||
}))
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-provider-budgets'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup({
|
||||
content: [{ type: 'text', text: 'switch provider after failure' }],
|
||||
source: { kind: 'user' },
|
||||
})
|
||||
await vi.runAllTimersAsync()
|
||||
await idle
|
||||
|
||||
expect(adapter.requests.map(request => request.provider)).toEqual(['mock', 'other', 'other'])
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => ({
|
||||
provider: event.data.provider,
|
||||
retry: event.data.retry,
|
||||
}))).toEqual([
|
||||
{ provider: 'mock', retry: 1 },
|
||||
{ provider: 'other', retry: 1 },
|
||||
])
|
||||
})
|
||||
|
||||
it.each(['thrown', 'in-band'] as const)(
|
||||
'uses the serving registration policy and resets changed-policy history after a %s failure',
|
||||
async (failureKind) => {
|
||||
vi.useFakeTimers()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
const release = Promise.withResolvers<undefined>()
|
||||
const oldAdapter = new ScriptedAdapter([(async function * (): AsyncGenerator<StreamChunk> {
|
||||
entered.resolve(undefined)
|
||||
await release.promise
|
||||
if (failureKind === 'thrown') {
|
||||
throw new LlmError('old route auth failed', 'AUTH')
|
||||
}
|
||||
yield {
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'old route auth failed', code: 'AUTH' },
|
||||
},
|
||||
}
|
||||
})()])
|
||||
const mounted = await harness(oldAdapter, { mock: alwaysConfig({
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
}) })
|
||||
context = mounted.ctx
|
||||
const agent = context.agentLoop.create(SessionId('retry-serving-registration'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
agent.followup({
|
||||
content: [{ type: 'text', text: 'replace while in flight' }],
|
||||
source: { kind: 'user' },
|
||||
})
|
||||
await entered.promise
|
||||
|
||||
mounted.disposeAdapter()
|
||||
const replacement = new ScriptedAdapter([
|
||||
new LlmError('replacement failed', 'AUTH'),
|
||||
textResponse('replacement recovered'),
|
||||
])
|
||||
replacement.configureRetryPolicies({ mock: alwaysConfig({
|
||||
initialDelayMs: 3,
|
||||
maxDelayMs: 3,
|
||||
}) })
|
||||
context.llm.registerAdapter(['mock'], replacement)
|
||||
release.resolve(undefined)
|
||||
|
||||
const firstEvent = await scheduled
|
||||
expect(firstEvent.data).toMatchObject({
|
||||
provider: 'mock',
|
||||
mode: 'always',
|
||||
retry: 1,
|
||||
delayMs: 1,
|
||||
})
|
||||
const replacementScheduled = waitForRetry(context, agent, 1)
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
const replacementEvent = await replacementScheduled
|
||||
expect(replacementEvent.data).toMatchObject({
|
||||
provider: 'mock',
|
||||
mode: 'always',
|
||||
retry: 1,
|
||||
delayMs: 3,
|
||||
})
|
||||
expect(replacementEvent.data.policyKey).not.toBe(firstEvent.data.policyKey)
|
||||
await vi.advanceTimersByTimeAsync(3)
|
||||
await idle
|
||||
|
||||
expect(oldAdapter.requests).toHaveLength(1)
|
||||
expect(replacement.requests).toHaveLength(2)
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'replacement recovered' }],
|
||||
})
|
||||
},
|
||||
)
|
||||
|
||||
it('keeps always mode unbounded while preserving cancellable jittered backoff', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('auth one', 'AUTH'),
|
||||
new LlmError('auth two', 'AUTH'),
|
||||
new LlmError('auth three', 'AUTH'),
|
||||
new LlmError('auth four', 'AUTH'),
|
||||
textResponse('eventually recovered'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig({
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 4,
|
||||
jitterRatio: 0.1,
|
||||
}) }, undefined, { random: () => 1 }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-always-unbounded'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'keep trying' }], source: { kind: 'user' } })
|
||||
await vi.runAllTimersAsync()
|
||||
await idle
|
||||
|
||||
const events = agent.session.events.filter(event => event.type === 'llm/retry')
|
||||
expect(adapter.requests).toHaveLength(5)
|
||||
expect(events.map(event => ({
|
||||
provider: event.data.provider,
|
||||
mode: event.data.mode,
|
||||
retry: event.data.retry,
|
||||
delayMs: event.data.delayMs,
|
||||
hasMax: 'maxRetries' in event.data,
|
||||
}))).toEqual([
|
||||
{ provider: 'mock', mode: 'always', retry: 1, delayMs: 1.1, hasMax: false },
|
||||
{ provider: 'mock', mode: 'always', retry: 2, delayMs: 2.2, hasMax: false },
|
||||
{ provider: 'mock', mode: 'always', retry: 3, delayMs: 4, hasMax: false },
|
||||
{ provider: 'mock', mode: 'always', retry: 4, delayMs: 4, hasMax: false },
|
||||
])
|
||||
})
|
||||
|
||||
it('keeps failed error text and partial output out of every retried model context', async () => {
|
||||
vi.useFakeTimers()
|
||||
const diagnostic = 'private provider diagnostic must not enter context'
|
||||
const adapter = new ScriptedAdapter([
|
||||
partialToolFailure(new LlmError(diagnostic, 'AUTH')),
|
||||
textResponse('recovered without leaked context'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig({
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
}) }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-always-context-isolation'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'safe input' }], source: { kind: 'user' } })
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(adapter.requests[1]?.messages).toEqual(adapter.requests[0]?.messages)
|
||||
const retriedContext = JSON.stringify(adapter.requests[1]?.messages)
|
||||
expect(retriedContext).not.toContain(diagnostic)
|
||||
expect(retriedContext).not.toContain('discarded partial output')
|
||||
expect(agent.session.events.some(event =>
|
||||
event.type === 'llm/retry' && event.data.failure.message === diagnostic,
|
||||
)).toBe(true)
|
||||
})
|
||||
|
||||
it('lets downstream specialized recovery run before always fallback', async () => {
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('requires specialized recovery', 'AUTH'),
|
||||
textResponse('specialized recovery won'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig() }))
|
||||
context.on('agent/request-error', async () => ({ kind: 'retry' }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-always-composition'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'recover' }], source: { kind: 'user' } })
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['synchronously', () => { throw new Error('downstream recovery failed') }],
|
||||
['asynchronously', async () => { throw new Error('downstream recovery failed') }],
|
||||
])('falls back to always retry when downstream recovery throws %s', async (_kind, failDownstream) => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('requires fallback', 'AUTH'),
|
||||
textResponse('always recovered'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig({
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
}) }))
|
||||
context.on('agent/request-error', failDownstream)
|
||||
const agent = context.agentLoop.create(SessionId('retry-always-downstream-error'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'recover' }], source: { kind: 'user' } })
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('aborts and drains a captured backoff before plugin disposal completes', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'TRANSPORT'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
const mounted = await harness(adapter)
|
||||
const mounted = await harness(adapter, { mock: alwaysConfig() })
|
||||
context = mounted.ctx
|
||||
const agent = context.agentLoop.create(SessionId('retry-hmr'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
@@ -372,37 +784,122 @@ describe('bounded transient retry policy', () => {
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('does not make plugin disposal wait for a delegated recovery policy', async () => {
|
||||
it('drains delegated recovery before completing plugin disposal', async () => {
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
const mounted = await harness(adapter)
|
||||
const mounted = await harness(adapter, { mock: alwaysConfig() })
|
||||
context = mounted.ctx
|
||||
const downstream = Promise.withResolvers<RequestErrorDecision>()
|
||||
const release = Promise.withResolvers<undefined>()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
context.on('agent/request-error', () => {
|
||||
const order: string[] = []
|
||||
context.on('agent/request-error', async () => {
|
||||
entered.resolve(undefined)
|
||||
return downstream.promise
|
||||
await release.promise
|
||||
order.push('downstream')
|
||||
return { kind: 'retry' }
|
||||
})
|
||||
const agent = context.agentLoop.create(SessionId('retry-delegated-disposal'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
const idle = waitForIdle(context, agent).then(() => { order.push('idle') })
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await entered.promise
|
||||
|
||||
const disposing = mounted.retryFiber.dispose()
|
||||
const disposing = mounted.retryFiber.dispose().then(() => { order.push('disposed') })
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
const outcome = await Promise.race([
|
||||
disposing.then(() => 'disposed' as const),
|
||||
new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }),
|
||||
])
|
||||
if (timer !== undefined) clearTimeout(timer)
|
||||
downstream.resolve({ action: 'fail' })
|
||||
expect(outcome).toBe('blocked')
|
||||
|
||||
release.resolve(undefined)
|
||||
await disposing
|
||||
await idle
|
||||
|
||||
expect(outcome).toBe('disposed')
|
||||
expect(order[0]).toBe('downstream')
|
||||
expect(order).toEqual(expect.arrayContaining(['disposed', 'idle']))
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
})
|
||||
|
||||
it('drains delegated recovery before turn cancellation reaches idle', async () => {
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
const mounted = await harness(adapter, { mock: alwaysConfig() })
|
||||
context = mounted.ctx
|
||||
const downstream = Promise.withResolvers<RequestErrorAction>()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
const order: string[] = []
|
||||
context.on('agent/request-error', async () => {
|
||||
entered.resolve(undefined)
|
||||
const decision = await downstream.promise
|
||||
order.push('downstream')
|
||||
return decision
|
||||
})
|
||||
const agent = context.agentLoop.create(SessionId('retry-delegated-cancel'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent).then(() => { order.push('idle') })
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await entered.promise
|
||||
|
||||
agent.cancel({ kind: 'user' })
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
const outcome = await Promise.race([
|
||||
idle.then(() => 'idle' as const),
|
||||
new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }),
|
||||
])
|
||||
if (timer !== undefined) clearTimeout(timer)
|
||||
expect(outcome).toBe('blocked')
|
||||
|
||||
downstream.resolve({ kind: 'retry' })
|
||||
await idle
|
||||
|
||||
expect(order).toEqual(['downstream', 'idle'])
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'aborted' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('handles synchronous cancellation while entering delegated recovery', async () => {
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
const mounted = await harness(adapter, { mock: alwaysConfig() })
|
||||
context = mounted.ctx
|
||||
const downstream = Promise.withResolvers<RequestErrorAction>()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
context.on('agent/request-error', (agent) => {
|
||||
agent.cancel({ kind: 'user' })
|
||||
entered.resolve(undefined)
|
||||
return downstream.promise
|
||||
})
|
||||
const agent = context.agentLoop.create(SessionId('retry-delegated-sync-cancel'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await entered.promise
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
const outcome = await Promise.race([
|
||||
idle.then(() => 'idle' as const),
|
||||
new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }),
|
||||
])
|
||||
if (timer !== undefined) clearTimeout(timer)
|
||||
expect(outcome).toBe('blocked')
|
||||
|
||||
downstream.resolve({ kind: 'retry' })
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'aborted' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('fails a captured callback after disposal without entering downstream policy', async () => {
|
||||
@@ -410,8 +907,10 @@ describe('bounded transient retry policy', () => {
|
||||
const captured = Promise.withResolvers<undefined>()
|
||||
let invokeCaptured: (() => Promise<void>) | undefined
|
||||
const mounted = await harness(adapter, {}, (ctx) => {
|
||||
ctx.on('agent/request-error', (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
return new Promise<RequestErrorDecision>((resolve) => {
|
||||
ctx.on('agent/request-error', (
|
||||
_agent, _turn, _step, _error, _failure, _history, _retryPolicy, _signal, next,
|
||||
) => {
|
||||
return new Promise<RequestErrorAction>((resolve) => {
|
||||
invokeCaptured = async () => { resolve(await next()) }
|
||||
captured.resolve(undefined)
|
||||
})
|
||||
@@ -419,7 +918,9 @@ describe('bounded transient retry policy', () => {
|
||||
})
|
||||
context = mounted.ctx
|
||||
let downstreamCalls = 0
|
||||
context.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
context.on('agent/request-error', async (
|
||||
_agent, _turn, _step, _error, _failure, _history, _retryPolicy, _signal, next,
|
||||
) => {
|
||||
downstreamCalls += 1
|
||||
return next()
|
||||
})
|
||||
@@ -428,7 +929,7 @@ describe('bounded transient retry policy', () => {
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await captured.promise
|
||||
|
||||
await mounted.retryFiber.dispose()
|
||||
@@ -443,13 +944,13 @@ describe('bounded transient retry policy', () => {
|
||||
it('lets turn cancellation win during backoff without opening another step', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'TIMEOUT'),
|
||||
new LlmError('permanent', 'AUTH'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
;({ ctx: context } = await harness(adapter, { mock: alwaysConfig() }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-cancel'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.cancel({ kind: 'user' })
|
||||
@@ -463,14 +964,19 @@ describe('bounded transient retry policy', () => {
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('lets an earlier recovery listener cancel before retry policy runs', async () => {
|
||||
it.each([
|
||||
['normal', normalConfig()],
|
||||
['always', alwaysConfig()],
|
||||
])('lets an earlier recovery listener cancel before %s retry policy runs', async (_mode, policy) => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'SERVER'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {}, (ctx) => {
|
||||
ctx.on('agent/request-error', async (agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
;({ ctx: context } = await harness(adapter, { mock: policy }, (ctx) => {
|
||||
ctx.on('agent/request-error', async (
|
||||
agent, _turn, _step, _error, _failure, _history, _retryPolicy, _signal, next,
|
||||
) => {
|
||||
agent.cancel({ kind: 'user' })
|
||||
return next()
|
||||
})
|
||||
@@ -478,7 +984,7 @@ describe('bounded transient retry policy', () => {
|
||||
const agent = context.agentLoop.create(SessionId('retry-pre-cancel'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
@@ -502,7 +1008,7 @@ describe('bounded transient retry policy', () => {
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.followup([{ type: 'text', text: 'go' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
@@ -510,19 +1016,17 @@ describe('bounded transient retry policy', () => {
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ maxTransientRetries: -1 }, /maxTransientRetries/],
|
||||
[{ maxTransientRetries: 1.5 }, /maxTransientRetries/],
|
||||
[{ initialDelayMs: 0 }, /initialDelayMs/],
|
||||
[{ maxDelayMs: Number.POSITIVE_INFINITY }, /maxDelayMs/],
|
||||
[{ initialDelayMs: MAX_TIMER_DELAY_MS + 1 }, /initialDelayMs/],
|
||||
[{ maxDelayMs: MAX_TIMER_DELAY_MS + 1 }, /maxDelayMs/],
|
||||
[{ initialDelayMs: 20, maxDelayMs: 10 }, /less than or equal/],
|
||||
[{ jitterRatio: 1.1 }, /jitterRatio/],
|
||||
[{ retryableCodes: [] }, /must not be empty/],
|
||||
[{ retryableCodes: ['SERVER', 'SERVER'] }, /duplicates/],
|
||||
[{ retryableCodes: [''] }, /non-empty strings/],
|
||||
] as const)('fails direct composition for invalid config %#', (config, message) => {
|
||||
expect(() => { retry.apply(new Context(), config as retry.Config) }).toThrow(message)
|
||||
it('rejects retry policy configured on the executor instead of a provider', () => {
|
||||
expectTypeOf<{}>().toExtend<retry.Config>()
|
||||
expectTypeOf<{ retryPolicy: { mode: 'always' } }>().not.toExtend<retry.Config>()
|
||||
expect(() => {
|
||||
retry.apply(new Context(), { retryPolicy: { mode: 'always' } } as unknown as retry.Config)
|
||||
}).toThrow(/retryPolicy belongs under each provider/)
|
||||
})
|
||||
|
||||
it('rejects unknown executor config', () => {
|
||||
expect(() => {
|
||||
retry.apply(new Context(), { retryPolciy: {} } as unknown as retry.Config)
|
||||
}).toThrow(/unknown key "retryPolciy"/)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -39,13 +39,17 @@ async function harness(
|
||||
apiKey: 'mock-key',
|
||||
baseURL,
|
||||
streamIdleTimeoutMs: options.streamIdleTimeoutMs ?? 1_000,
|
||||
retryPolicy: {
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
backoff: {
|
||||
initialDelayMs: options.initialDelayMs ?? 10,
|
||||
maxDelayMs: options.initialDelayMs ?? 10,
|
||||
jitterRatio: 0,
|
||||
},
|
||||
},
|
||||
})
|
||||
await ctx.plugin(Retry, {
|
||||
maxTransientRetries: 2,
|
||||
initialDelayMs: options.initialDelayMs ?? 10,
|
||||
maxDelayMs: options.initialDelayMs ?? 10,
|
||||
jitterRatio: 0,
|
||||
})
|
||||
await ctx.plugin(Retry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
return ctx
|
||||
}
|
||||
@@ -62,7 +66,7 @@ function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
|
||||
function sendAndWait(ctx: Context, agent: Agent): Promise<void> {
|
||||
const idle = waitForIdle(ctx, agent)
|
||||
agent.followup([{ type: 'text', text: 'recover through the provider boundary' }])
|
||||
agent.followup({ content: [{ type: 'text', text: 'recover through the provider boundary' }], source: { kind: 'user' } })
|
||||
return idle
|
||||
}
|
||||
|
||||
@@ -102,8 +106,9 @@ describe('bounded retry through the real DeepSeek HTTP/SSE adapter', () => {
|
||||
|
||||
expect(server).toBeDefined()
|
||||
expect(server?.requests).toHaveLength(1)
|
||||
expect(agent.session.events.filter(event => event.type === 'step/start').map(event => event.data.step))
|
||||
.toEqual([1, 2])
|
||||
expect(agent.session.events.filter(event => event.type === 'step/start')
|
||||
.map(event => [event.data.turn, event.data.step]))
|
||||
.toEqual([[1, 1], [2, 1]])
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => event.data.failure.code))
|
||||
.toEqual(['TRANSPORT'])
|
||||
expect(finalAssistantText(agent)).toBe('connected after retry')
|
||||
@@ -131,10 +136,11 @@ describe('bounded retry through the real DeepSeek HTTP/SSE adapter', () => {
|
||||
expect(server.requests).toHaveLength(2)
|
||||
expect(server.requests[0]?.body).toEqual(server.requests[1]?.body)
|
||||
expect(agent.session.events.filter(event =>
|
||||
event.type === 'assistant/chunk' && event.data.step === 1,
|
||||
event.type === 'assistant/chunk' && event.data.turn === 1,
|
||||
)).toHaveLength(failedChunkCount)
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message')
|
||||
.map(event => [event.data.turn, event.data.step]))
|
||||
.toEqual([[2, 1]])
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => event.data.failure.code))
|
||||
.toEqual(['TRANSPORT'])
|
||||
expect(finalAssistantText(agent)).toBe('recovered response')
|
||||
@@ -157,8 +163,9 @@ describe('bounded retry through the real DeepSeek HTTP/SSE adapter', () => {
|
||||
expect(server.requests[0]?.body).toEqual(server.requests[1]?.body)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => event.data.failure.code))
|
||||
.toEqual(['EMPTY_RESPONSE'])
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message')
|
||||
.map(event => [event.data.turn, event.data.step]))
|
||||
.toEqual([[2, 1]])
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'completed' } },
|
||||
@@ -182,7 +189,7 @@ describe('bounded retry through the real DeepSeek HTTP/SSE adapter', () => {
|
||||
|
||||
expect(server.requests).toHaveLength(1)
|
||||
expect(agent.session.events.filter(event =>
|
||||
event.type === 'assistant/chunk' && event.data.step === 1,
|
||||
event.type === 'assistant/chunk' && event.data.turn === 1,
|
||||
)).toHaveLength(2)
|
||||
expect(agent.session.events.some(event => event.type === 'assistant/message')).toBe(false)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
|
||||
25
packages/llm/llm-retry/tsdown.config.ts
Normal file
25
packages/llm/llm-retry/tsdown.config.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/** Build the package root and invariant companion as independent bundles. */
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/invariant.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 981f7d58802d2ff18633b09b5a4ec8a7b1bf3383
|
||||
README.zh.md: 0a6535f41adb8ec90d02dbb56aec257853a19083
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 2328188e420df6de60f024982a31d37a858a303e
|
||||
README.zh.md: 586767a9e790fa68d27673836700fd789ce6a180
|
||||
|
||||
@@ -12,15 +12,20 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route is later disposed or replaced. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -30,7 +35,7 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use bounded normal retry policy, use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
@@ -41,7 +46,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates and defaults it under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
@@ -60,11 +65,11 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
|
||||
### Real adapters
|
||||
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses direct fetch with `eventsource-parser` SSE framing for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
|
||||
## Model Experience
|
||||
|
||||
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
|
||||
None, as the service adds no model-bound text, schema, or message; it only materializes and logs an adapter-configured reasoning effort.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
@@ -72,7 +77,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine.
|
||||
- **No retry execution, caching, or rate limiting ships in this service** — provider registration stores retry policy, but `llm/stream` remains a single-attempt call-wrapper seam. The agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure; `@deepseek-ai/dsh-llm-retry` is the optional executor loaded by the shared example spine.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
|
||||
@@ -12,15 +12,20 @@
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功，要么全部不生效，并且会随调用 fiber dispose。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略，并解析 normal 默认值。
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` 从拥有精确路由的适配器解析权威上下文容量。
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` 从拥有精确路由的适配器解析经校验的确切模型身份、可用上下文和推理（reasoning）元数据；异步适配器可选地支持取消。
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` 校验显式推理强度，并填入适配器配置的默认值，但不自动调整。
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始 chunk（token 级 delta）。消费方使用 `BlockAssembler` 将 chunk 组装为块／消息。
|
||||
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误，并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误；`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误，并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误；`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`；`llmRetryPolicyOf(stream)` 返回在该边界选中的确切注册所对应的不可变策略，即使之后释放或替换路由也不变。未到达最终适配器的调用没有服务策略。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
|
||||
提供方与模型元数据是发现表层，不是路由白名单。`registerAdapter()` 仍拥有提供方排他性，适配器则可以接受 `listModels()` 中不存在的模型 id；消费方禁止因模型未列出而拒绝请求。返回的元数据与输入脱离，无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
提供方与模型元数据是发现表层，不是路由白名单。`registerAdapter()` 仍拥有提供方排他性，并为每条路由捕获适配器的重试策略；适配器则可以接受 `listModels()` 中不存在的模型 id，消费方禁止因模型未列出而拒绝请求。返回的 selector 元数据与输入脱离，无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
|
||||
上下文容量是独立的正确性查询，不是 catalog 装饰或全局 LLM 设置。`resolveModelContext()` 会询问拥有精确提供方／模型路由的适配器；适配器可以描述未列出的动态模型，`undefined` 只表示容量不可用。无效的返回容量以 `INVALID_MODEL_CONTEXT` 失败。
|
||||
确切模型元数据是独立的正确性查询，不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方／模型路由的适配器查询一次；适配器可以描述未列出的动态模型，缺少 `context` 或 `reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
推理标识符是由适配器持有的不透明字符串，而非核心枚举。适配器会公布有序可选列表；模型能力 API 提供 `off` id 时，列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值，在存在 `defaultEffort` 时填入它，否则保留提供方默认值。异步模型解析器会接收调用方的 signal，并且必须在取消后迅速完成结算。`prepareCall()` 还会让精确适配器注册跨越请求头记录和最终分派，因此 HMR（热模块替换）不会将一个适配器的能力结果与另一个适配器的请求混用；复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
### 事件
|
||||
|
||||
@@ -30,7 +35,7 @@
|
||||
|
||||
### 扩展点
|
||||
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`，添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器；`GenerateOptions.model` 属于适配器，可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据，在已知精确容量时覆盖 `resolveModelContext()`；默认实现将路由 id 用作名称，不公布模型，也不返回容量。
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`，添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器；`GenerateOptions.model` 属于适配器，可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方持有的恢复配置，覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据；精确身份、容量或可选推理强度可用时，实现 `resolveModel()`；异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略，将路由和模型 id 用作名称，不公布模型，也不返回容量或推理元数据。
|
||||
- 包装 `llm/stream` 时，通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界；因此已发布 agent 重试策略改用 `agent/request-error`。
|
||||
|
||||
### 内容块词汇（`types.ts`）
|
||||
@@ -41,7 +46,7 @@
|
||||
|
||||
### 调用配置（`call-config.ts`）
|
||||
|
||||
`LlmCallConfig` 是一个会话请求的提供方 + 模型 + 采样标量（`provider`、`model`、`temperature`、`maxTokens`、`stop`，每个都与同名 `GenerateOptions` 字段 1:1 映射）。它是作为请求标头一部分记录在会话日志中的每会话状态（见 dsh-session `request/header` 事件），绝不是可静默调整的每次调用旋钮：`agent/request` waterfall 会提议替换，loop 则记录真实变更。`callConfigEquals(a, b)` 是逐字段真实变更检测器；`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper（`llm/stream` listener 与适配器只读，绝不改写）。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源，`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类，让适配器可以应用目的特定传输策略，而不改变普通会话请求。
|
||||
`LlmCallConfig` 是一个会话请求的提供方、模型、可选的适配器持有推理强度和采样标量（`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`，每个都与同名 `GenerateOptions` 字段 1:1 映射）。它是作为请求标头一部分记录在会话日志中的每会话状态（见 dsh-session `request/header` 事件），绝不是可静默调整的每次调用旋钮：`agent/request` waterfall 会提议替换，`prepareCall()` 在轮次 signal 控制下校验并填入默认值，loop 随后记录生效值，再使用准备完成调用的注册绑定流。`callConfigEquals(a, b)` 是逐字段真实变更检测器；`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper（`llm/stream` listener 与适配器只读，绝不改写）。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源，`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类，让适配器可以应用目的特定传输策略，而不改变普通会话请求。
|
||||
|
||||
### 应用归因（`attribution.ts`）
|
||||
|
||||
@@ -60,11 +65,11 @@
|
||||
|
||||
### 真实适配器
|
||||
|
||||
两个适配器使用不同内部机制实现 `LlmAdapter`：[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用手写 fetch/SSE，[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方／模型对。两者都遵循 `StreamChunk` 约定，定义见 `types.ts`：usage 先于 finish，工具参数保持原始字符串，错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
|
||||
两个适配器使用不同内部机制实现 `LlmAdapter`：[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用直接 fetch 加 `eventsource-parser` SSE 分帧，[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方／模型对。两者都遵循 `StreamChunk` 约定，定义见 `types.ts`：usage 先于 finish，工具参数保持原始字符串，错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
|
||||
|
||||
## 模型体验
|
||||
|
||||
无。该适配器注册表转发已组装的请求，不添加或更改任何模型边界文本、schema 或消息。
|
||||
无。服务不添加或更改任何模型边界文本、schema 或消息；它只会填入并记录适配器配置的推理强度。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
@@ -72,7 +77,7 @@
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **本服务不内置默认重试／缓存／速率限制策略**：`llm/stream` 仍是单次尝试调用包装 seam；agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`，其默认行为是保留原始失败。`@deepseek-ai/dsh-llm-retry` 是共享示例 spine 加载的可选策略插件。
|
||||
- **本服务不执行重试、缓存或速率限制**：提供方注册会存储重试策略，但 `llm/stream` 仍是单次尝试调用包装 seam。agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`，其默认行为是保留原始失败；`@deepseek-ai/dsh-llm-retry` 是共享示例 spine 加载的可选执行器。
|
||||
- **`GenerateOptions` 采样只包含 `temperature`／`maxTokens`／`stop`**：没有 `tool_choice`、`top_p` 或 penalty 字段；有产生方落地时词汇才会增长（见 [已删除惰性旋钮](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)）。
|
||||
- **由产生方调节的变体在实际产生前保持在外**：`prefill`、每工具 `strict`、块 `cache` 提示与 `agent` 消息源变体因没有产生方而被剪除（见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)）。
|
||||
- **`BlockAssembler` 只处理核心块 kind**：如果插件添加块类型的流从未由 `block-end` 关闭，`blocks()` 会抛出异常。
|
||||
|
||||
@@ -39,12 +39,17 @@
|
||||
"@deepseek-ai/dsh-attachment": "^0.0.1",
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-attachment": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,9 +6,15 @@
|
||||
|
||||
import { HarnessError } from './error.ts'
|
||||
import type { LlmFailure, StreamChunk } from './types.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
|
||||
/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakMap<Error, LlmFailure>
|
||||
/** Call-local facts captured when one model call enters its final adapter boundary. */
|
||||
export interface AdapterFailureScope {
|
||||
/** Errors and normalized facts proven to originate in this call's final adapter boundary. */
|
||||
readonly failures: WeakMap<Error, LlmFailure>
|
||||
/** Immutable policy of the exact adapter registration selected for this call. */
|
||||
retryPolicy?: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
|
||||
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
|
||||
@@ -54,7 +60,7 @@ export function markLlmAdapterFailure(
|
||||
message: errorMessage(error),
|
||||
code: harnessErrorCode(error),
|
||||
})
|
||||
failures.set(error, failure)
|
||||
failures.failures.set(error, failure)
|
||||
return error
|
||||
}
|
||||
|
||||
@@ -136,7 +142,7 @@ export function isLlmAdapterFailure(
|
||||
value: unknown,
|
||||
): value is Error & { code?: string } {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error && failures !== undefined && failures.has(value)
|
||||
return value instanceof Error && failures !== undefined && failures.failures.has(value)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -151,5 +157,18 @@ export function llmFailureOf(
|
||||
value: unknown,
|
||||
): LlmFailure | undefined {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error ? failures?.get(value) : undefined
|
||||
return value instanceof Error ? failures?.failures.get(value) : undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the retry policy of the exact registration selected at this call's
|
||||
* final adapter boundary. The policy remains available after that registration
|
||||
* is disposed or replaced; absence means no final adapter served the call.
|
||||
* @param stream - the exact stream returned by the model call.
|
||||
* @returns the immutable serving-registration policy, or `undefined`.
|
||||
*/
|
||||
export function llmRetryPolicyOf(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
): ResolvedRetryPolicy | undefined {
|
||||
return adapterFailureScopes.get(stream)?.retryPolicy
|
||||
}
|
||||
|
||||
@@ -38,3 +38,15 @@ export type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
export function ProviderRequestId(id: string): ProviderRequestId {
|
||||
return id as ProviderRequestId
|
||||
}
|
||||
|
||||
/** Adapter-owned identifier for one model's selectable reasoning effort. */
|
||||
export type ReasoningEffortId = Branded<'ReasoningEffortId'>
|
||||
|
||||
/**
|
||||
* Brand an adapter-owned reasoning-effort identifier.
|
||||
* @param id - the opaque identifier exposed by one model capability.
|
||||
* @returns the same string, branded; no validation is performed.
|
||||
*/
|
||||
export function ReasoningEffortId(id: string): ReasoningEffortId {
|
||||
return id as ReasoningEffortId
|
||||
}
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
/**
|
||||
* Conversation call configuration and freeze utilities. Provider routing,
|
||||
* model, and sampling values are request-header state that can affect cache
|
||||
* reuse; request waterfalls replace them and the loop logs changed snapshots
|
||||
* instead of allowing silent per-call drift.
|
||||
* model, reasoning effort, and sampling values are request-header state that
|
||||
* can affect cache reuse; request waterfalls replace them and the loop logs
|
||||
* changed snapshots instead of allowing silent per-call drift.
|
||||
* @module dsh-llm/call-config
|
||||
*/
|
||||
|
||||
import type { GenerateOptions } from './types.ts'
|
||||
import type { ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Process-local identities of request objects assembled by dsh-agent-loop. */
|
||||
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
|
||||
|
||||
/**
|
||||
* Provider + model + sampling scalars of one conversation's requests. Every field maps
|
||||
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
|
||||
* from the logged header rather than accepting these per call.
|
||||
* Provider, model, reasoning effort, and sampling scalars of one conversation's
|
||||
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
|
||||
* the loop builds requests from the logged header rather than accepting these
|
||||
* per call.
|
||||
*/
|
||||
export interface LlmCallConfig {
|
||||
provider: string
|
||||
model: string
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
stop?: string[]
|
||||
@@ -33,7 +36,13 @@ export interface LlmCallConfig {
|
||||
* @returns whether every field (including the `stop` list, element-wise) matches.
|
||||
*/
|
||||
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
|
||||
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
|
||||
if (
|
||||
a.provider !== b.provider
|
||||
|| a.model !== b.model
|
||||
|| a.reasoningEffort !== b.reasoningEffort
|
||||
|| a.temperature !== b.temperature
|
||||
|| a.maxTokens !== b.maxTokens
|
||||
) return false
|
||||
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
|
||||
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
|
||||
}
|
||||
|
||||
@@ -10,14 +10,17 @@ import { Context, Service } from 'cordis'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmResolvedModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
} from './types.ts'
|
||||
import { resolveRetryPolicy } from './retry-policy.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
import type { LlmCallConfig } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
import type { AdapterFailureScope } from './adapter-failure.ts'
|
||||
@@ -27,10 +30,11 @@ export * from './brand.ts'
|
||||
export * from './never.ts'
|
||||
export * from './error.ts'
|
||||
export * from './types.ts'
|
||||
export * from './retry-policy.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf, llmRetryPolicyOf } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -103,11 +107,25 @@ export class LlmError extends HarnessError {
|
||||
}
|
||||
}
|
||||
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
export interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
|
||||
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The hand-rolled
|
||||
* DeepSeek and pi-ai adapters intentionally exercise this contract through different internals.
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The direct-fetch
|
||||
* DeepSeek and library-backed pi-ai adapters intentionally exercise this contract through different internals.
|
||||
*/
|
||||
export abstract class LlmAdapter {
|
||||
/**
|
||||
@@ -119,6 +137,15 @@ export abstract class LlmAdapter {
|
||||
return { id: provider, name: provider }
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the provider-owned retry policy captured with this route.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
@@ -131,17 +158,20 @@ export abstract class LlmAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity for one model accepted by this adapter. Absence
|
||||
* means the adapter does not know the capacity, not that routing is invalid.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns provider-owned context metadata, or `undefined` when unavailable.
|
||||
* Resolve all metadata available for one exact model. This query is
|
||||
* independent of the advisory catalog and does not validate request routing.
|
||||
* @param provider - one provider route owned by this adapter.
|
||||
* @param model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; asynchronous
|
||||
* implementations must settle promptly after it aborts.
|
||||
* @returns provider/model identity plus any context and reasoning metadata.
|
||||
*/
|
||||
resolveModelContext(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({ provider, id: model, name: model })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -157,7 +187,7 @@ export abstract class LlmAdapter {
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
*/
|
||||
export class LlmService extends Service {
|
||||
private adapters = new Map<string, { adapter: LlmAdapter; provider: LlmProviderInfo }>()
|
||||
private adapters = new Map<string, AdapterRegistration>()
|
||||
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'llm')
|
||||
@@ -175,7 +205,7 @@ export class LlmService extends Service {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: { adapter: LlmAdapter; provider: LlmProviderInfo }[] = []
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
@@ -186,7 +216,13 @@ export class LlmService extends Service {
|
||||
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
|
||||
}
|
||||
unique.add(provider)
|
||||
registrations.push({ adapter, provider: { id: info.id, name: info.name } })
|
||||
const retryPolicy = adapter.providerRetryPolicy(provider)
|
||||
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
|
||||
registrations.push({
|
||||
adapter,
|
||||
provider: { id: info.id, name: info.name },
|
||||
retryPolicy,
|
||||
})
|
||||
}
|
||||
for (const registration of registrations) this.adapters.set(registration.provider.id, registration)
|
||||
yield () => {
|
||||
@@ -206,6 +242,15 @@ export class LlmService extends Service {
|
||||
return [...this.adapters.values()].map(({ provider }) => ({ ...provider }))
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the retry policy captured when one provider route was registered.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @returns the provider-owned policy, with normal defaults already resolved.
|
||||
*/
|
||||
providerRetryPolicy(provider: string): ResolvedRetryPolicy {
|
||||
return this.registration(provider).retryPolicy
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
@@ -242,29 +287,170 @@ export class LlmService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity from the adapter that owns one exact route.
|
||||
* This query is independent of the advisory model catalog: an unlisted model
|
||||
* may return metadata, while `undefined` never rejects later routing.
|
||||
* Resolve and validate all metadata from the adapter that owns one exact
|
||||
* route. The result is detached from adapter-owned objects; catalog
|
||||
* membership remains advisory and does not control request routing.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @returns detached context metadata, or `undefined` when the adapter has none.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns exact model identity plus available context and reasoning metadata.
|
||||
*/
|
||||
async resolveModelContext(
|
||||
async resolveModelInfo(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
|
||||
if (context === undefined) return undefined
|
||||
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return this.resolveModelInfoFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelInfoFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const provider = registration.provider.id
|
||||
const resolved = await registration.adapter.resolveModel(provider, model, signal)
|
||||
if (
|
||||
typeof resolved.provider !== 'string'
|
||||
|| resolved.provider !== provider
|
||||
|| typeof resolved.id !== 'string'
|
||||
|| resolved.id !== model
|
||||
|| typeof resolved.name !== 'string'
|
||||
|| resolved.name.length === 0
|
||||
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_INFO',
|
||||
)
|
||||
}
|
||||
const context = resolved.context
|
||||
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_CONTEXT',
|
||||
)
|
||||
}
|
||||
return { contextWindow: context.contextWindow }
|
||||
const info: LlmResolvedModelInfo = {
|
||||
provider,
|
||||
id: model,
|
||||
name: resolved.name,
|
||||
...resolved.description === undefined ? {} : { description: resolved.description },
|
||||
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
|
||||
}
|
||||
const reasoning = resolved.reasoning
|
||||
if (reasoning === undefined) return info
|
||||
if (reasoning.efforts.length === 0) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
const seen = new Set<string>()
|
||||
const efforts = reasoning.efforts.map((effort) => {
|
||||
if (
|
||||
typeof effort.id !== 'string'
|
||||
|| effort.id.length === 0
|
||||
|| typeof effort.name !== 'string'
|
||||
|| effort.name.length === 0
|
||||
|| (effort.description !== undefined && typeof effort.description !== 'string')
|
||||
|| seen.has(effort.id)
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid or duplicate reasoning effort metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
seen.add(effort.id)
|
||||
return {
|
||||
id: effort.id,
|
||||
name: effort.name,
|
||||
...effort.description === undefined ? {} : { description: effort.description },
|
||||
}
|
||||
})
|
||||
if (reasoning.defaultEffort !== undefined && !seen.has(reasoning.defaultEffort)) {
|
||||
throw new LlmError(
|
||||
`adapter returned an unknown default reasoning effort for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
return {
|
||||
...info,
|
||||
reasoning: {
|
||||
efforts,
|
||||
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
|
||||
/**
|
||||
* Validate a conversation call config against its exact model capability and
|
||||
* materialize an adapter-configured default. Unsupported explicit efforts
|
||||
* reject before provider I/O; no clamping or aliasing is performed. This
|
||||
* standalone query does not bind a later dispatch; use {@link prepareCall}
|
||||
* when logging and streaming must share one adapter registration.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a detached config only when a default must be materialized.
|
||||
*/
|
||||
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig> {
|
||||
return this.resolveCallConfigFor(this.registration(config.provider), config, signal)
|
||||
}
|
||||
|
||||
private async resolveCallConfigFor(
|
||||
registration: AdapterRegistration,
|
||||
config: LlmCallConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmCallConfig> {
|
||||
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
|
||||
const requested = config.reasoningEffort
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${requested}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return config
|
||||
}
|
||||
const effective = requested ?? reasoning.defaultEffort
|
||||
if (effective === undefined) return config
|
||||
if (!reasoning.efforts.some(effort => effort.id === effective)) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return requested === effective ? config : { ...config, reasoningEffort: effective }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one call under its current adapter registration. The returned
|
||||
* one-shot handle keeps that registration across header logging and dispatch,
|
||||
* so HMR cannot combine one adapter's capability result with another adapter.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a prepared config and its registration-bound stream entry point.
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
|
||||
const registration = this.registration(config.provider)
|
||||
const resolvedConfig = deepFreeze(structuredClone(
|
||||
await this.resolveCallConfigFor(registration, config, signal),
|
||||
))
|
||||
let dispatched = false
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
}
|
||||
dispatched = true
|
||||
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private registration(provider: string): AdapterRegistration {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
return registration
|
||||
@@ -297,11 +483,28 @@ export class LlmService extends Service {
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const adapter = this.registration(options.provider).adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
failures.retryPolicy = registration.retryPolicy
|
||||
const resolvedConfig = prepared === undefined
|
||||
? await this.resolveCallConfigFor(registration, options, options.signal)
|
||||
: prepared.config
|
||||
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
|
||||
throw new LlmError(
|
||||
'prepared LLM call config changed before adapter dispatch',
|
||||
'INVALID_PREPARED_CALL',
|
||||
)
|
||||
}
|
||||
const resolvedOptions = prepared !== undefined || callConfigEquals(options, resolvedConfig)
|
||||
? options
|
||||
: Object.isFrozen(options)
|
||||
? deepFreeze({ ...options, ...resolvedConfig })
|
||||
: { ...options, ...resolvedConfig }
|
||||
const adapter = registration.adapter
|
||||
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
@@ -341,18 +544,37 @@ export class LlmService extends Service {
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* adapter selection remains fixed through asynchronous exact-model resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
return this.streamWithRegistration(options)
|
||||
}
|
||||
|
||||
private streamWithRegistration(
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = { failures: new WeakMap<Error, LlmFailure>() }
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
options,
|
||||
() => this.adapterStream(options, failures, prepared),
|
||||
)
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
interface AdapterRegistration {
|
||||
readonly adapter: LlmAdapter
|
||||
readonly provider: LlmProviderInfo
|
||||
readonly retryPolicy: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
|
||||
191
packages/llm/llm/src/retry-policy.ts
Normal file
191
packages/llm/llm/src/retry-policy.ts
Normal file
@@ -0,0 +1,191 @@
|
||||
/**
|
||||
* Provider-owned request-retry policy configuration and resolution.
|
||||
*
|
||||
* Adapters expose one resolved policy per registered provider route; the
|
||||
* optional dsh-llm-retry plugin executes it on the agent's failed-step seam.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm/retry-policy
|
||||
*/
|
||||
|
||||
import z from 'schemastery'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { EMPTY_RESPONSE_CODE } from './error.ts'
|
||||
|
||||
const DEFAULT_MAX_RETRIES = 2
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
const DEFAULT_MAX_DELAY_MS = 10_000
|
||||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze([
|
||||
EMPTY_RESPONSE_CODE,
|
||||
'RATE_LIMIT',
|
||||
'SERVER',
|
||||
'TIMEOUT',
|
||||
'TRANSPORT',
|
||||
])
|
||||
|
||||
/** Bounded exponential backoff with symmetric jitter around each local delay. */
|
||||
export interface BackoffConfig {
|
||||
/** Initial local exponential-backoff delay in milliseconds (default 500). */
|
||||
initialDelayMs?: number
|
||||
/** Maximum locally scheduled or accepted provider delay in milliseconds (default 10000). */
|
||||
maxDelayMs?: number
|
||||
/** Symmetric random multiplier range around one (default 0.1). */
|
||||
jitterRatio?: number
|
||||
}
|
||||
|
||||
/** Current bounded transient retry behavior for one provider route. */
|
||||
export interface NormalRetryPolicyConfig {
|
||||
/** Retry only configured transient failure codes. */
|
||||
mode: 'normal'
|
||||
/** Maximum eligible retries after the first request (default 2). */
|
||||
maxRetries?: number
|
||||
/** Stable failure codes eligible for this policy. */
|
||||
retryableCodes?: string[]
|
||||
/** Local exponential-backoff and jitter configuration. */
|
||||
backoff?: BackoffConfig
|
||||
}
|
||||
|
||||
/** Unbounded retry behavior for every model-request failure on one provider route. */
|
||||
export interface AlwaysRetryPolicyConfig {
|
||||
/** Retry every model-request failure until success, cancellation, or disposal. */
|
||||
mode: 'always'
|
||||
/** Local exponential-backoff and jitter configuration. */
|
||||
backoff?: BackoffConfig
|
||||
}
|
||||
|
||||
/** Provider-owned model-request retry policy configuration. */
|
||||
export type RetryPolicyConfig = NormalRetryPolicyConfig | AlwaysRetryPolicyConfig
|
||||
|
||||
/** Fully resolved backoff shared by both retry modes. */
|
||||
export interface ResolvedRetryBackoff {
|
||||
readonly initialDelayMs: number
|
||||
readonly maxDelayMs: number
|
||||
readonly jitterRatio: number
|
||||
}
|
||||
|
||||
/** Fully resolved bounded transient retry policy. */
|
||||
export interface ResolvedNormalRetryPolicy extends ResolvedRetryBackoff {
|
||||
readonly mode: 'normal'
|
||||
readonly maxRetries: number
|
||||
readonly retryableCodes: readonly string[]
|
||||
}
|
||||
|
||||
/** Fully resolved unbounded retry policy. */
|
||||
export interface ResolvedAlwaysRetryPolicy extends ResolvedRetryBackoff {
|
||||
readonly mode: 'always'
|
||||
}
|
||||
|
||||
/** Immutable provider policy captured when its adapter route is registered. */
|
||||
export type ResolvedRetryPolicy = ResolvedNormalRetryPolicy | ResolvedAlwaysRetryPolicy
|
||||
|
||||
const backoffSchema: z<BackoffConfig> = z.object({
|
||||
initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS),
|
||||
maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS),
|
||||
jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO),
|
||||
})
|
||||
|
||||
const normalPolicySchema: z<NormalRetryPolicyConfig> = z.object({
|
||||
mode: z.const('normal').required(),
|
||||
maxRetries: z.number().step(1).min(0).max(Number.MAX_SAFE_INTEGER).default(DEFAULT_MAX_RETRIES),
|
||||
retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]),
|
||||
backoff: backoffSchema,
|
||||
})
|
||||
|
||||
const alwaysPolicySchema: z<AlwaysRetryPolicyConfig> = z.object({
|
||||
mode: z.const('always').required(),
|
||||
backoff: backoffSchema,
|
||||
})
|
||||
|
||||
/** Cordis schema embedded by each concrete provider configuration. */
|
||||
export const RetryPolicySchema: z<RetryPolicyConfig> = z.union([
|
||||
normalPolicySchema,
|
||||
alwaysPolicySchema,
|
||||
])
|
||||
|
||||
const NORMAL_POLICY_KEYS: ReadonlySet<string> = new Set([
|
||||
'mode', 'maxRetries', 'retryableCodes', 'backoff',
|
||||
])
|
||||
const ALWAYS_POLICY_KEYS: ReadonlySet<string> = new Set(['mode', 'backoff'])
|
||||
const BACKOFF_KEYS: ReadonlySet<string> = new Set(['initialDelayMs', 'maxDelayMs', 'jitterRatio'])
|
||||
|
||||
function validateKeys(value: object, allowed: ReadonlySet<string>, path: string): void {
|
||||
for (const key of Object.keys(value)) {
|
||||
if (!allowed.has(key)) throw new Error(`${path}: unknown key "${key}"`)
|
||||
}
|
||||
}
|
||||
|
||||
function resolveBackoff(config: BackoffConfig | undefined, path: string): ResolvedRetryBackoff {
|
||||
if (config !== undefined) validateKeys(config, BACKOFF_KEYS, path)
|
||||
const initialDelayMs = config?.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS
|
||||
const maxDelayMs = config?.maxDelayMs ?? DEFAULT_MAX_DELAY_MS
|
||||
const jitterRatio = config?.jitterRatio ?? DEFAULT_JITTER_RATIO
|
||||
|
||||
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`${path}.initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`${path}.maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (initialDelayMs > maxDelayMs) {
|
||||
throw new Error(`${path}.initialDelayMs must be less than or equal to maxDelayMs`)
|
||||
}
|
||||
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) {
|
||||
throw new Error(`${path}.jitterRatio must be between 0 and 1`)
|
||||
}
|
||||
|
||||
return Object.freeze({ initialDelayMs, maxDelayMs, jitterRatio })
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate, default, and detach one provider-owned retry policy.
|
||||
* @param config - optional provider configuration; omission selects normal defaults.
|
||||
* @param path - diagnostic path naming the provider config that owns the value.
|
||||
* @returns an immutable policy safe to capture in provider registration state.
|
||||
*/
|
||||
export function resolveRetryPolicy(
|
||||
config: RetryPolicyConfig | undefined,
|
||||
path: string,
|
||||
): ResolvedRetryPolicy {
|
||||
if (config === undefined) {
|
||||
return Object.freeze({
|
||||
mode: 'normal',
|
||||
maxRetries: DEFAULT_MAX_RETRIES,
|
||||
retryableCodes: DEFAULT_RETRYABLE_CODES,
|
||||
...resolveBackoff(undefined, `${path}.backoff`),
|
||||
})
|
||||
}
|
||||
|
||||
switch (config.mode) {
|
||||
case 'normal': {
|
||||
validateKeys(config, NORMAL_POLICY_KEYS, path)
|
||||
const maxRetries = config.maxRetries ?? DEFAULT_MAX_RETRIES
|
||||
const retryableCodes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES]
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 0) {
|
||||
throw new Error(`${path}.maxRetries must be a non-negative safe integer`)
|
||||
}
|
||||
if (retryableCodes.length === 0) {
|
||||
throw new Error(`${path}.retryableCodes must not be empty`)
|
||||
}
|
||||
if (retryableCodes.some(code => typeof code !== 'string' || code.length === 0)) {
|
||||
throw new Error(`${path}.retryableCodes must contain only non-empty strings`)
|
||||
}
|
||||
if (new Set(retryableCodes).size !== retryableCodes.length) {
|
||||
throw new Error(`${path}.retryableCodes must not contain duplicates`)
|
||||
}
|
||||
return Object.freeze({
|
||||
mode: 'normal',
|
||||
maxRetries,
|
||||
retryableCodes: Object.freeze([...retryableCodes]),
|
||||
...resolveBackoff(config.backoff, `${path}.backoff`),
|
||||
})
|
||||
}
|
||||
case 'always':
|
||||
validateKeys(config, ALWAYS_POLICY_KEYS, path)
|
||||
return Object.freeze({
|
||||
mode: 'always',
|
||||
...resolveBackoff(config.backoff, `${path}.backoff`),
|
||||
})
|
||||
default:
|
||||
throw new Error(`${path}.mode must be "normal" or "always"`)
|
||||
}
|
||||
}
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
|
||||
import type { CallId, ProviderRequestId } from './brand.ts'
|
||||
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
|
||||
export interface LlmFailure {
|
||||
@@ -185,6 +185,35 @@ export interface LlmModelContext {
|
||||
contextWindow: number
|
||||
}
|
||||
|
||||
/** Display metadata for one adapter-owned reasoning effort. */
|
||||
export interface LlmReasoningEffortInfo {
|
||||
/** Opaque stable value accepted by {@link GenerateOptions.reasoningEffort}. */
|
||||
id: ReasoningEffortId
|
||||
/** Human-readable effort name for selectors and diagnostics. */
|
||||
name: string
|
||||
/** Optional user-facing distinction from otherwise similar efforts. */
|
||||
description?: string
|
||||
}
|
||||
|
||||
/** Selectable reasoning efforts for one exact provider/model route. */
|
||||
export interface LlmModelReasoningInfo {
|
||||
/** Supported efforts in adapter-preferred display order. */
|
||||
efforts: readonly LlmReasoningEffortInfo[]
|
||||
/**
|
||||
* Adapter-configured default materialized into requests when callers omit
|
||||
* an effort. Absence preserves the provider's own default.
|
||||
*/
|
||||
defaultEffort?: ReasoningEffortId
|
||||
}
|
||||
|
||||
/** Exact-route model metadata resolved by its owning adapter. */
|
||||
export interface LlmResolvedModelInfo extends LlmModelInfo {
|
||||
/** Provider-owned context capacity when known. */
|
||||
context?: LlmModelContext
|
||||
/** Adapter-owned selectable reasoning levels when exposed. */
|
||||
reasoning?: LlmModelReasoningInfo
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
@@ -225,11 +254,12 @@ export interface GenerateOptions {
|
||||
/** Registered provider route selecting the adapter instance. */
|
||||
provider: string
|
||||
model: string
|
||||
/** Adapter-owned reasoning effort selected for this exact model. */
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
/**
|
||||
* Ordered conversation messages, exactly as the provider sees them (after
|
||||
* the `system` slot). A loop-built request assembles them as
|
||||
* `EpochHeader.messagePrefix` + the derived history (dsh-agent-loop); a
|
||||
* hand-built one-shot passes any list.
|
||||
* the derived history (dsh-agent-loop); a hand-built one-shot passes any list.
|
||||
*/
|
||||
messages: Message[]
|
||||
/** System prompt text (adapters map to the provider's system slot). */
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from '../src/call-config.ts'
|
||||
import { ReasoningEffortId } from '../src/brand.ts'
|
||||
import type { GenerateOptions } from '../src/types.ts'
|
||||
|
||||
describe('callConfigEquals', () => {
|
||||
@@ -14,6 +15,11 @@ describe('callConfigEquals', () => {
|
||||
expect(callConfigEquals(base, base)).toBe(true)
|
||||
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, reasoningEffort: ReasoningEffortId('high') }, base)).toBe(false)
|
||||
expect(callConfigEquals(
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
)).toBe(true)
|
||||
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
|
||||
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)
|
||||
|
||||
85
packages/llm/llm/tests/retry-policy.spec.ts
Normal file
85
packages/llm/llm/tests/retry-policy.spec.ts
Normal file
@@ -0,0 +1,85 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
resolveRetryPolicy,
|
||||
RetryPolicySchema,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
|
||||
describe('provider retry policy', () => {
|
||||
it('resolves immutable normal defaults', () => {
|
||||
const policy = resolveRetryPolicy(undefined, 'provider.retryPolicy')
|
||||
|
||||
expect(policy).toEqual({
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
retryableCodes: ['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'],
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0.1,
|
||||
})
|
||||
expect(Object.isFrozen(policy)).toBe(true)
|
||||
if (policy.mode !== 'normal') throw new Error('expected normal policy')
|
||||
expect(Object.isFrozen(policy.retryableCodes)).toBe(true)
|
||||
})
|
||||
|
||||
it('resolves and detaches a configured normal policy', () => {
|
||||
const retryableCodes = ['BUSY']
|
||||
const config: RetryPolicyConfig = {
|
||||
mode: 'normal',
|
||||
maxRetries: 4,
|
||||
retryableCodes,
|
||||
backoff: {
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0,
|
||||
},
|
||||
}
|
||||
|
||||
const policy = resolveRetryPolicy(config, 'provider.retryPolicy')
|
||||
retryableCodes.push('LATE')
|
||||
|
||||
expect(policy).toEqual({
|
||||
mode: 'normal',
|
||||
maxRetries: 4,
|
||||
retryableCodes: ['BUSY'],
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0,
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves always mode with default backoff', () => {
|
||||
expect(resolveRetryPolicy({ mode: 'always' }, 'provider.retryPolicy')).toEqual({
|
||||
mode: 'always',
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0.1,
|
||||
})
|
||||
expect(RetryPolicySchema).toBeDefined()
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ mode: 'normal', maxRetries: -1 }, /maxRetries/],
|
||||
[{ mode: 'normal', maxRetries: 1.5 }, /maxRetries/],
|
||||
[{ mode: 'normal', maxRetries: Number.MAX_SAFE_INTEGER + 1 }, /maxRetries/],
|
||||
[{ mode: 'always', backoff: { initialDelayMs: 0 } }, /initialDelayMs/],
|
||||
[{ mode: 'normal', backoff: { maxDelayMs: Number.POSITIVE_INFINITY } }, /maxDelayMs/],
|
||||
[{ mode: 'normal', backoff: { initialDelayMs: MAX_TIMER_DELAY_MS + 1 } }, /initialDelayMs/],
|
||||
[{ mode: 'always', backoff: { maxDelayMs: MAX_TIMER_DELAY_MS + 1 } }, /maxDelayMs/],
|
||||
[{ mode: 'normal', backoff: { initialDelayMs: 20, maxDelayMs: 10 } }, /less than or equal/],
|
||||
[{ mode: 'always', backoff: { jitterRatio: 1.1 } }, /jitterRatio/],
|
||||
[{ mode: 'normal', retryableCodes: [] }, /must not be empty/],
|
||||
[{ mode: 'normal', retryableCodes: ['SERVER', 'SERVER'] }, /duplicates/],
|
||||
[{ mode: 'normal', retryableCodes: [''] }, /non-empty strings/],
|
||||
[{ mode: 'normal', retryableCodes: [429] }, /non-empty strings/],
|
||||
[{ mode: 'normal', maxRetires: 1 }, /unknown key "maxRetires"/],
|
||||
[{ mode: 'always', maxRetries: 1 }, /unknown key "maxRetries"/],
|
||||
[{ mode: 'always', backoff: { initialDelay: 1 } }, /unknown key "initialDelay"/],
|
||||
[{ mode: 'sometimes' }, /mode must be "normal" or "always"/],
|
||||
] as const)('rejects invalid policy %#', (config, message) => {
|
||||
expect(() => {
|
||||
resolveRetryPolicy(config as unknown as RetryPolicyConfig, 'provider.retryPolicy')
|
||||
}).toThrow(message)
|
||||
})
|
||||
})
|
||||
@@ -10,10 +10,19 @@ import LlmService, {
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
llmFailureOf,
|
||||
llmRetryPolicyOf,
|
||||
ProviderRequestId,
|
||||
ReasoningEffortId,
|
||||
resolveRetryPolicy,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmModelReasoningInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
constructor(private script: StreamChunk[]) {
|
||||
@@ -49,6 +58,7 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
private readonly provider: LlmProviderInfo,
|
||||
private readonly models: readonly LlmModelInfo[],
|
||||
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
|
||||
private readonly reasoning: Readonly<Record<string, LlmModelReasoningInfo>> = {},
|
||||
) {
|
||||
super(SCRIPT)
|
||||
}
|
||||
@@ -61,11 +71,17 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(this.contexts[model])
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
|
||||
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -159,6 +175,72 @@ describe('LlmService', () => {
|
||||
expect(chunks).toEqual(SCRIPT)
|
||||
})
|
||||
|
||||
it('captures provider-owned retry policy at registration and defaults omission', async () => {
|
||||
const configured = resolveRetryPolicy({ mode: 'always' }, 'test retryPolicy')
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override providerRetryPolicy(provider: string) {
|
||||
return provider === 'configured' ? configured : undefined
|
||||
}
|
||||
}(SCRIPT)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['configured', 'defaulted'], adapter)
|
||||
|
||||
expect(ctx.llm.providerRetryPolicy('configured')).toBe(configured)
|
||||
expect(ctx.llm.providerRetryPolicy('defaulted')).toMatchObject({
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
})
|
||||
expect(() => ctx.llm.providerRetryPolicy('missing')).toThrow(
|
||||
expect.objectContaining({ code: 'NO_ADAPTER' }),
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps the serving registration policy on an in-flight call after route replacement', async () => {
|
||||
const oldPolicy = resolveRetryPolicy({ mode: 'always' }, 'old retryPolicy')
|
||||
const newPolicy = resolveRetryPolicy({ mode: 'normal', maxRetries: 0 }, 'new retryPolicy')
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
const release = Promise.withResolvers<undefined>()
|
||||
const failure = new LlmError('old route failed', 'AUTH')
|
||||
const oldAdapter = new class extends LlmAdapter {
|
||||
override providerRetryPolicy(): typeof oldPolicy {
|
||||
return oldPolicy
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
entered.resolve(undefined)
|
||||
await release.promise
|
||||
throw failure
|
||||
}
|
||||
}()
|
||||
const newAdapter = new class extends ScriptedAdapter {
|
||||
override providerRetryPolicy(): typeof newPolicy {
|
||||
return newPolicy
|
||||
}
|
||||
}(SCRIPT)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const disposeOld = ctx.llm.registerAdapter(['route'], oldAdapter)
|
||||
const stream = ctx.llm.stream({ provider: 'route', model: 'model', messages: [] })
|
||||
const outcome = (async (): Promise<unknown> => {
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
return error
|
||||
}
|
||||
return undefined
|
||||
})()
|
||||
await entered.promise
|
||||
|
||||
disposeOld()
|
||||
ctx.llm.registerAdapter(['route'], newAdapter)
|
||||
release.resolve(undefined)
|
||||
|
||||
expect(await outcome).toBe(failure)
|
||||
expect(llmRetryPolicyOf(stream)).toBe(oldPolicy)
|
||||
expect(ctx.llm.providerRetryPolicy('route')).toBe(newPolicy)
|
||||
})
|
||||
|
||||
it('throws NO_ADAPTER for unregistered providers', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -173,6 +255,7 @@ describe('LlmService', () => {
|
||||
expect((caught as LlmError).code).toBe('NO_ADAPTER')
|
||||
expect((caught as LlmError).message).toContain('no adapter registered')
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
expect(llmRetryPolicyOf(stream)).toBeUndefined()
|
||||
})
|
||||
|
||||
it.each(['done', 'value'] as const)('tags a throwing IteratorResult.%s getter without replacing its Error', async (field) => {
|
||||
@@ -739,8 +822,32 @@ describe('LlmService', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
|
||||
provider: 'plain', id: 'unlisted', name: 'unlisted',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
|
||||
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
|
||||
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
|
||||
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
|
||||
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
|
||||
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
|
||||
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
|
||||
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
|
||||
})
|
||||
|
||||
it('resolves detached model context independently of advisory catalog membership', async () => {
|
||||
@@ -753,11 +860,241 @@ describe('LlmService', () => {
|
||||
{ unlisted: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
source.contextWindow = 64_000
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
|
||||
provider: 'route', id: 'other', name: 'other',
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const source = {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('standard'), name: 'Standard' },
|
||||
{ id: ReasoningEffortId('ultra'), name: 'Ultra', description: 'Largest budget' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
|
||||
expect(resolved.reasoning).toEqual(source)
|
||||
source.efforts[0]!.name = 'mutated'
|
||||
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
|
||||
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})
|
||||
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
|
||||
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ efforts: [] }, 'empty effort list'],
|
||||
[{ efforts: [{ id: '', name: 'Empty' }] }, 'empty id'],
|
||||
[{ efforts: [{ id: 'valid', name: '' }] }, 'empty name'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid', description: 1 }] }, 'non-string description'],
|
||||
[{ efforts: [{ id: 'same', name: 'One' }, { id: 'same', name: 'Two' }] }, 'duplicate id'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid' }], defaultEffort: 'other' }, 'unknown default'],
|
||||
] as const)('rejects invalid model reasoning metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: metadata as unknown as LlmModelReasoningInfo },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
|
||||
})
|
||||
|
||||
it('rejects unsupported reasoning efforts without clamping', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: { efforts: [{ id: ReasoningEffortId('ultra'), name: 'Ultra' }] } },
|
||||
))
|
||||
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'plain',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
})
|
||||
|
||||
it('resolves reasoning defaults at the final adapter boundary after routing middleware', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends RecordingAdapter {
|
||||
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
|
||||
const reasoning: LlmModelReasoningInfo = {
|
||||
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning,
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['routed'], adapter)
|
||||
const disposeRouting = ctx.on('llm/stream', (options, next) => {
|
||||
options.provider = 'routed'
|
||||
return next()
|
||||
})
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'initial',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
disposeRouting()
|
||||
|
||||
const frozenRequest: GenerateOptions = Object.freeze({
|
||||
provider: 'routed',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})
|
||||
for await (const _chunk of ctx.llm.stream(frozenRequest)) { /* drain */ }
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
|
||||
})
|
||||
|
||||
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends RecordingAdapter {
|
||||
override async resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return {
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning: await reasoning.promise,
|
||||
}
|
||||
}
|
||||
}(SCRIPT)
|
||||
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
|
||||
const draining = (async () => {
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
})()
|
||||
|
||||
await started.promise
|
||||
disposeFirst()
|
||||
const second = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], second)
|
||||
reasoning.resolve({
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
})
|
||||
await draining
|
||||
|
||||
expect(first.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('high'))
|
||||
expect(second.lastOptions).toBeUndefined()
|
||||
})
|
||||
|
||||
it('prepares a one-shot registration-bound call and rejects config drift', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{
|
||||
model: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
},
|
||||
)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(Object.isFrozen(prepared.config)).toBe(true)
|
||||
const stream = prepared.stream({
|
||||
...prepared.config,
|
||||
model: 'other',
|
||||
messages: [],
|
||||
})
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toMatchObject({ code: 'INVALID_PREPARED_CALL' })
|
||||
expect(() => prepared.stream({
|
||||
...prepared.config,
|
||||
messages: [],
|
||||
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
|
||||
})
|
||||
|
||||
it('passes cancellation through exact-model resolution', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
reject(new Error('missing reasoning signal'))
|
||||
return
|
||||
}
|
||||
if (signal.aborted) {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
return
|
||||
}
|
||||
signal.addEventListener('abort', () => {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
}, { once: true })
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const controller = new AbortController()
|
||||
const resolving = ctx.llm.resolveCallConfig(
|
||||
{ provider: 'route', model: 'model' },
|
||||
controller.signal,
|
||||
)
|
||||
|
||||
await started.promise
|
||||
const reason = new Error('cancel reasoning')
|
||||
controller.abort(reason)
|
||||
await expect(resolving).rejects.toBe(reason)
|
||||
})
|
||||
|
||||
it.each([0, -1, 1.5, Number.NaN])(
|
||||
@@ -770,7 +1107,7 @@ describe('LlmService', () => {
|
||||
[],
|
||||
{ model: { contextWindow } },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelContext('route', 'model'))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
|
||||
},
|
||||
)
|
||||
|
||||
@@ -22,6 +22,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: ccb18d725feaa397520f5ee17e2900355e7d08c2
|
||||
README.zh.md: 6ab48b0f5a704fa85e4bceffd886490462287f6a
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: 578728ded9cf51a12abcd70d541404e995028f26
|
||||
README.zh.md: 51518e98c43e740822970c1a39154f550ea962dc
|
||||
|
||||
@@ -6,7 +6,7 @@ Replay-aware token measurement through the singleton `ctx.tokenMeter` service. I
|
||||
|
||||
## Configuration
|
||||
|
||||
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelContext()`.
|
||||
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelInfo().context`.
|
||||
|
||||
## Measurement contract
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
## 配置
|
||||
|
||||
估算器没有设置。它有意使用一项固定启发式规则：每个 token 按四个字符估算，再加上角色、块与请求 envelope 字段的结构开销。任何 key 都会被拒绝，包括已废弃的全局 `contextWindow`；模型容量属于拥有精确提供方／模型路由的适配器，可通过 `ctx.llm.resolveModelContext()` 获取。
|
||||
估算器没有设置。它有意使用一项固定启发式规则：每个 token 按四个字符估算，再加上角色、块与请求 envelope 字段的结构开销。任何 key 都会被拒绝，包括已废弃的全局 `contextWindow`；模型容量属于拥有精确提供方／模型路由的适配器，可通过 `ctx.llm.resolveModelInfo().context` 获取。
|
||||
|
||||
## 测量契约
|
||||
|
||||
|
||||
@@ -389,7 +389,6 @@ export class TokenMeterService extends Service {
|
||||
private _estimateHeader(header: EpochHeader | undefined): number {
|
||||
if (header === undefined) return 0
|
||||
let tokens = 0
|
||||
for (const message of header.messagePrefix ?? []) tokens += this.estimateMessage(message)
|
||||
if (header.system !== undefined) {
|
||||
tokens += Math.ceil(header.system.length / CHARS_PER_TOKEN) + ROLE_OVERHEAD
|
||||
}
|
||||
|
||||
@@ -194,7 +194,7 @@ describe('TokenMeterService pricing', () => {
|
||||
expect(snapshot.nodes).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('prices header, prefix, tools, and surface when no reusable usage exists', () => {
|
||||
it('prices header, tools, and surface when no reusable usage exists', () => {
|
||||
const service = meter()
|
||||
const session = new Session(SessionId('heuristic'))
|
||||
session.append('user/message', {
|
||||
@@ -203,7 +203,6 @@ describe('TokenMeterService pricing', () => {
|
||||
}, { surfaceOp: 'append' })
|
||||
appendHeader(session, header('deepseek-v4-flash', {
|
||||
system: 'system',
|
||||
messagePrefix: [textMessage('prefix')],
|
||||
tools: [{ name: 'read', description: 'read', parameters: { type: 'object' } }],
|
||||
}))
|
||||
const result = service.measure(session)
|
||||
@@ -365,10 +364,6 @@ describe('replay anchors and surface folds', () => {
|
||||
...anchoredHeader,
|
||||
config: { ...anchoredHeader.config, temperature: 0.2 },
|
||||
}).baseline.kind).toBe('estimated')
|
||||
expect(service.measure(session, {
|
||||
...anchoredHeader,
|
||||
messagePrefix: [textMessage('prefix')],
|
||||
}).baseline.kind).toBe('estimated')
|
||||
expect(service.measure(session, {
|
||||
...anchoredHeader,
|
||||
tools: [{ name: 'read', description: 'read', parameters: { type: 'object' } }],
|
||||
|
||||
Reference in New Issue
Block a user