Merge remote-tracking branch 'origin/master' into worktree/pr628-merge-20260727
# Conflicts: # .agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md # docs/architecture.i18n.yaml # docs/config-catalog.md # docs/cordis-catalog/events.md # docs/cordis-catalog/services.md # docs/core-data-structures/llm-streaming.i18n.yaml # docs/core-data-structures/llm-streaming.md # docs/core-data-structures/llm-streaming.zh.md # docs/event-producer-consumer.md # docs/module-graph.md # examples/headless-agent/tests/headless.snapshot.ts # packages/compact/compact-basic/tests/compact-loop-repro.spec.ts # packages/cordis/tool-cordis/src/api-catalog.ts # packages/core/agent-loop/README.md # packages/core/agent-loop/src/loop.ts # packages/examples/agent-spine-demo/README.md # packages/llm/README.md # packages/llm/llm-deepseek/src/adapter.ts # packages/llm/llm-pi-ai/src/adapter.ts # packages/llm/llm-retry/README.md # packages/llm/llm/README.md # packages/llm/llm/src/index.ts # packages/llm/llm/tests/service.spec.ts # packages/support/llm-replay/src/index.ts # packages/support/llm-replay/tests/llm-replay.spec.ts
This commit is contained in:
6
packages/llm/llm/README.i18n.yaml
Normal file
6
packages/llm/llm/README.i18n.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 2328188e420df6de60f024982a31d37a858a303e
|
||||
README.zh.md: 586767a9e790fa68d27673836700fd789ce6a180
|
||||
@@ -1,5 +1,7 @@
|
||||
# dsh-llm
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin.
|
||||
|
||||
## Service: `LlmService` (ctx key: `llm`)
|
||||
@@ -12,14 +14,18 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route is later disposed or replaced. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -29,7 +35,7 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use bounded normal retry policy, use the route id as its name, advertise no models, and return no capacity.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use bounded normal retry policy, use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
@@ -40,7 +46,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates and defaults it under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
@@ -59,11 +65,11 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
|
||||
### Real adapters
|
||||
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses direct fetch with `eventsource-parser` SSE framing for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
|
||||
## Model Experience
|
||||
|
||||
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
|
||||
None, as the service adds no model-bound text, schema, or message; it only materializes and logs an adapter-configured reasoning effort.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
@@ -72,8 +78,8 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No retry execution, caching, or rate limiting ships in this service** — provider registration stores retry policy, but `llm/stream` remains a single-attempt call-wrapper seam. The agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure; `@deepseek-ai/dsh-llm-retry` is the optional executor loaded by the shared example spine.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
- **`APP_IDENTITY.url` names a repository that does not exist yet** — `FIXME`: creating the public `deepseek-ai/deepseek-harness-sdk` repo gates the first release.
|
||||
- **`GenerateOptions.sessionId` is a locally-declared brand** — importing dsh-session's `SessionId` would cycle; a future ids-owning package would dissolve the workaround.
|
||||
|
||||
85
packages/llm/llm/README.zh.md
Normal file
85
packages/llm/llm/README.zh.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# dsh-llm
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
提供方无关的 LLM 词汇与抽象服务。本包定义 agent loop、会话日志和每个插件使用的规范语言。
|
||||
|
||||
## 服务:`LlmService`(ctx key:`llm`)
|
||||
|
||||
一个适配器注册表加单一流式调用表层,可通过 waterfall 事件拦截。
|
||||
|
||||
### 公开 API
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber dispose。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略,并解析 normal 默认值。
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` 从拥有精确路由的适配器解析经校验的确切模型身份、可用上下文和推理(reasoning)元数据;异步适配器可选地支持取消。
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` 校验显式推理强度,并填入适配器配置的默认值,但不自动调整。
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始 chunk(token 级 delta)。消费方使用 `BlockAssembler` 将 chunk 组装为块/消息。
|
||||
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误,并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误;`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`;`llmRetryPolicyOf(stream)` 返回在该边界选中的确切注册所对应的不可变策略,即使之后释放或替换路由也不变。未到达最终适配器的调用没有服务策略。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
|
||||
提供方与模型元数据是发现表层,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,并为每条路由捕获适配器的重试策略;适配器则可以接受 `listModels()` 中不存在的模型 id,消费方禁止因模型未列出而拒绝请求。返回的 selector 元数据与输入脱离,无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
|
||||
确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型,缺少 `context` 或 `reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
推理标识符是由适配器持有的不透明字符串,而非核心枚举。适配器会公布有序可选列表;模型能力 API 提供 `off` id 时,列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值,在存在 `defaultEffort` 时填入它,否则保留提供方默认值。异步模型解析器会接收调用方的 signal,并且必须在取消后迅速完成结算。`prepareCall()` 还会让精确适配器注册跨越请求头记录和最终分派,因此 HMR(热模块替换)不会将一个适配器的能力结果与另一个适配器的请求混用;复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
### 事件
|
||||
|
||||
| 事件 | 模式 | 用途 |
|
||||
|---|---|---|
|
||||
| `llm/stream` | waterfall | 拦截/包装每次流式模型调用,用于缓存、日志或路由 |
|
||||
|
||||
### 扩展点
|
||||
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方持有的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据。
|
||||
- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`。
|
||||
|
||||
### 内容块词汇(`types.ts`)
|
||||
|
||||
消息是类型化内容块数组:`text`、`reasoning`、`tool-call`、`tool-result`。联合从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加块类型。loop 产生的 assistant 消息还会携带提供方/模型溯源与可选适配器私有回放状态。dispatch 前,`LlmService` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型/提供方间恢复或转换该状态。核心块集只包含每条已发布路径都支持的块。多模态内容(图像、音频等)没有核心块类型;需要它的功能会通过 map 添加,并一并添加支持它的适配器/UI/压缩实现。
|
||||
|
||||
流式输出是原始 chunk 协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)。`BlockAssembler` 是将 chunk 组装为块/消息的唯一共享实现。
|
||||
|
||||
### 调用配置(`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` 是一个会话请求的提供方、模型、可选的适配器持有推理强度和采样标量(`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,`prepareCall()` 在轮次 signal 控制下校验并填入默认值,loop 随后记录生效值,再使用准备完成调用的注册绑定流。`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper(`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。
|
||||
|
||||
### 应用归因(`attribution.ts`)
|
||||
|
||||
每个产品适配器都会在提供方 HTTP 请求上发送应用身份。`attributionHeaders(identity?)` 构建标准 `User-Agent`,默认为公开 `APP_IDENTITY`;白标部署可以替换它,但不能抑制它。适配器会直接验证 wire 标头,或通过自身库 hook 验证。详见 [归因 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md)。
|
||||
|
||||
### 类
|
||||
|
||||
- `LlmAdapter`:提供方适配器的抽象基类。唯一必需方法是 `stream()`。
|
||||
- `BlockAssembler`:将原始 chunk 逐步组装为完整内容块与 assistant 消息。agent loop 向它提供原始 chunk(同时记录以供回放),并读取已组装块/消息以构建历史。
|
||||
- `HarnessError`:harness 错误分类体系的基类,包含稳定 `code` 字符串(与面向人的 `message` 不同)加 `cause` 链接。它位于所有其他包都导入的叶子包中,因此可以共享单一基类,无需新的依赖边。每包错误(`LlmError`、`ToolArgsError`、`InvariantError` 等)都会扩展它。`isHarnessError(value)` 在 seam 处收窄类型。
|
||||
- `LlmError`:扩展 `HarnessError`;其稳定 `code` 字符串(`NO_ADAPTER`、`DUPLICATE_ADAPTER` 与 `AUTH`/`RATE_LIMIT` 等适配器 code)与冻结可序列化 `failure.code` 匹配。Payload 还可以保留已验证状态、`Retry-After` 和品牌化提供方请求 id 事实;策略位于错误之外。
|
||||
- `errorChain(value)`:渲染抛出值的完整 `cause` 链与 AggregateError 成员,供诊断表层使用,包括 UI 通知、logger 行和持久 `turn/end` 消息。因此 undici 的 `TypeError: fetch failed` 等传输包装层会显示底层 `ECONNREFUSED`/DNS/TLS 详细信息,而不是将其遮蔽。该函数只负责渲染:请按 `code` 路由,绝不解析结果。
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE`:当请求超过模型上下文窗口时,无论通过抛出 HTTP 还是带内 finish 交付,两个 DeepSeek 适配器都使用的提供方无关 code。`isContextWindowExceededError(detail)` 是它们针对 OpenAI 兼容提供方详细信息的共享保守分类器。
|
||||
- `QUOTA_EXCEEDED_CODE`:帐户配额、余额、点数、预算或用量限制耗尽时使用的非短暂提供方无关 code。`isQuotaExceededError(detail)` 使这些失败与请求速率限制保持区分。
|
||||
- `EMPTY_RESPONSE_CODE`:对退化提供方完成使用的提供方无关 code,两个适配器均使用:一个不携带任何内容块的终止 `stop`。它会被分类为错误 finish(而非成功空消息),因为尝试未产生持久内容;`dsh-llm-retry` 默认重试它。
|
||||
|
||||
### 真实适配器
|
||||
|
||||
两个适配器使用不同内部机制实现 `LlmAdapter`:[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用直接 fetch 加 `eventsource-parser` SSE 分帧,[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `StreamChunk` 约定,定义见 `types.ts`:usage 先于 finish,工具参数保持原始字符串,错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
|
||||
|
||||
## 模型体验
|
||||
|
||||
无。服务不添加或更改任何模型边界文本、schema 或消息;它只会填入并记录适配器配置的推理强度。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
透传;注册表保留已组装请求前缀,cache 复用与路由边界属于所选适配器和提供方。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **本服务不执行重试、缓存或速率限制**:提供方注册会存储重试策略,但 `llm/stream` 仍是单次尝试调用包装 seam。agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`,其默认行为是保留原始失败;`@deepseek-ai/dsh-llm-retry` 是共享示例 spine 加载的可选执行器。
|
||||
- **`GenerateOptions` 采样只包含 `temperature`/`maxTokens`/`stop`**:没有 `tool_choice`、`top_p` 或 penalty 字段;有产生方落地时词汇才会增长(见 [已删除惰性旋钮](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md))。
|
||||
- **由产生方调节的变体在实际产生前保持在外**:`prefill`、每工具 `strict`、块 `cache` 提示与 `agent` 消息源变体因没有产生方而被剪除(见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md))。
|
||||
- **`BlockAssembler` 只处理核心块 kind**:如果插件添加块类型的流从未由 `block-end` 关闭,`blocks()` 会抛出异常。
|
||||
- **`APP_IDENTITY.url` 指向一个尚不存在的仓库**:`FIXME`:创建公开 `deepseek-ai/deepseek-harness-sdk` 仓库是首次发布的前置条件。
|
||||
- **`GenerateOptions.sessionId` 是本地声明的品牌类型**:导入 dsh-session 的 `SessionId` 会产生循环;未来拥有 id 的包可以消除该权宜之计。
|
||||
@@ -53,8 +53,10 @@ export function markLlmAdapterFailure(
|
||||
const error = value instanceof Error
|
||||
? value as Error & { code?: string }
|
||||
: new HarnessError(String(value), 'UNKNOWN', { cause: value })
|
||||
const carried = error instanceof HarnessError ? ownFailureSnapshot(error) : undefined
|
||||
const failure = carried !== undefined && carried.code === error.code ? carried : Object.freeze({
|
||||
// Cross-package copies preserve own data but not class identity. Trust the
|
||||
// carried facts only when both own properties agree after validation.
|
||||
const carried = ownFailureSnapshot(error)
|
||||
const failure = carried !== undefined && carried.code === ownErrorCode(error) ? carried : Object.freeze({
|
||||
message: errorMessage(error),
|
||||
code: harnessErrorCode(error),
|
||||
})
|
||||
@@ -62,6 +64,16 @@ export function markLlmAdapterFailure(
|
||||
return error
|
||||
}
|
||||
|
||||
/** Read a foreign error's own data-backed `code` without invoking accessors. */
|
||||
function ownErrorCode(error: Error): unknown {
|
||||
try {
|
||||
const descriptor = Object.getOwnPropertyDescriptor(error, 'code')
|
||||
return descriptor !== undefined && 'value' in descriptor ? descriptor.value : undefined
|
||||
} catch (_sdkPropertyTrap) {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Snapshot an own data property without invoking an SDK-defined accessor. */
|
||||
function ownFailureSnapshot(error: Error): LlmFailure | undefined {
|
||||
try {
|
||||
|
||||
@@ -38,3 +38,15 @@ export type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
export function ProviderRequestId(id: string): ProviderRequestId {
|
||||
return id as ProviderRequestId
|
||||
}
|
||||
|
||||
/** Adapter-owned identifier for one model's selectable reasoning effort. */
|
||||
export type ReasoningEffortId = Branded<'ReasoningEffortId'>
|
||||
|
||||
/**
|
||||
* Brand an adapter-owned reasoning-effort identifier.
|
||||
* @param id - the opaque identifier exposed by one model capability.
|
||||
* @returns the same string, branded; no validation is performed.
|
||||
*/
|
||||
export function ReasoningEffortId(id: string): ReasoningEffortId {
|
||||
return id as ReasoningEffortId
|
||||
}
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
/**
|
||||
* Conversation call configuration and freeze utilities. Provider routing,
|
||||
* model, and sampling values are request-header state that can affect cache
|
||||
* reuse; request waterfalls replace them and the loop logs changed snapshots
|
||||
* instead of allowing silent per-call drift.
|
||||
* model, reasoning effort, and sampling values are request-header state that
|
||||
* can affect cache reuse; request waterfalls replace them and the loop logs
|
||||
* changed snapshots instead of allowing silent per-call drift.
|
||||
* @module dsh-llm/call-config
|
||||
*/
|
||||
|
||||
import type { GenerateOptions } from './types.ts'
|
||||
import type { ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Process-local identities of request objects assembled by dsh-agent-loop. */
|
||||
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
|
||||
|
||||
/**
|
||||
* Provider + model + sampling scalars of one conversation's requests. Every field maps
|
||||
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
|
||||
* from the logged header rather than accepting these per call.
|
||||
* Provider, model, reasoning effort, and sampling scalars of one conversation's
|
||||
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
|
||||
* the loop builds requests from the logged header rather than accepting these
|
||||
* per call.
|
||||
*/
|
||||
export interface LlmCallConfig {
|
||||
provider: string
|
||||
model: string
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
stop?: string[]
|
||||
@@ -33,7 +36,13 @@ export interface LlmCallConfig {
|
||||
* @returns whether every field (including the `stop` list, element-wise) matches.
|
||||
*/
|
||||
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
|
||||
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
|
||||
if (
|
||||
a.provider !== b.provider
|
||||
|| a.model !== b.model
|
||||
|| a.reasoningEffort !== b.reasoningEffort
|
||||
|| a.temperature !== b.temperature
|
||||
|| a.maxTokens !== b.maxTokens
|
||||
) return false
|
||||
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
|
||||
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
|
||||
}
|
||||
|
||||
@@ -10,8 +10,8 @@ import { Context, Service } from 'cordis'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmResolvedModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
@@ -19,7 +19,8 @@ import type {
|
||||
import { resolveRetryPolicy } from './retry-policy.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
import type { LlmCallConfig } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
import type { AdapterFailureScope } from './adapter-failure.ts'
|
||||
@@ -106,11 +107,25 @@ export class LlmError extends HarnessError {
|
||||
}
|
||||
}
|
||||
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
export interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
|
||||
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The hand-rolled
|
||||
* DeepSeek and pi-ai adapters intentionally exercise this contract through different internals.
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The direct-fetch
|
||||
* DeepSeek and library-backed pi-ai adapters intentionally exercise this contract through different internals.
|
||||
*/
|
||||
export abstract class LlmAdapter {
|
||||
/**
|
||||
@@ -143,17 +158,20 @@ export abstract class LlmAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity for one model accepted by this adapter. Absence
|
||||
* means the adapter does not know the capacity, not that routing is invalid.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns provider-owned context metadata, or `undefined` when unavailable.
|
||||
* Resolve all metadata available for one exact model. This query is
|
||||
* independent of the advisory catalog and does not validate request routing.
|
||||
* @param provider - one provider route owned by this adapter.
|
||||
* @param model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; asynchronous
|
||||
* implementations must settle promptly after it aborts.
|
||||
* @returns provider/model identity plus any context and reasoning metadata.
|
||||
*/
|
||||
resolveModelContext(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({ provider, id: model, name: model })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -169,11 +187,7 @@ export abstract class LlmAdapter {
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
*/
|
||||
export class LlmService extends Service {
|
||||
private adapters = new Map<string, {
|
||||
adapter: LlmAdapter
|
||||
provider: LlmProviderInfo
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
}>()
|
||||
private adapters = new Map<string, AdapterRegistration>()
|
||||
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'llm')
|
||||
@@ -191,11 +205,7 @@ export class LlmService extends Service {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: {
|
||||
adapter: LlmAdapter
|
||||
provider: LlmProviderInfo
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
}[] = []
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
@@ -275,33 +285,170 @@ export class LlmService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity from the adapter that owns one exact route.
|
||||
* This query is independent of the advisory model catalog: an unlisted model
|
||||
* may return metadata, while `undefined` never rejects later routing.
|
||||
* Resolve and validate all metadata from the adapter that owns one exact
|
||||
* route. The result is detached from adapter-owned objects; catalog
|
||||
* membership remains advisory and does not control request routing.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @returns detached context metadata, or `undefined` when the adapter has none.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns exact model identity plus available context and reasoning metadata.
|
||||
*/
|
||||
async resolveModelContext(
|
||||
async resolveModelInfo(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
|
||||
if (context === undefined) return undefined
|
||||
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return this.resolveModelInfoFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelInfoFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const provider = registration.provider.id
|
||||
const resolved = await registration.adapter.resolveModel(provider, model, signal)
|
||||
if (
|
||||
typeof resolved.provider !== 'string'
|
||||
|| resolved.provider !== provider
|
||||
|| typeof resolved.id !== 'string'
|
||||
|| resolved.id !== model
|
||||
|| typeof resolved.name !== 'string'
|
||||
|| resolved.name.length === 0
|
||||
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_INFO',
|
||||
)
|
||||
}
|
||||
const context = resolved.context
|
||||
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_CONTEXT',
|
||||
)
|
||||
}
|
||||
return { contextWindow: context.contextWindow }
|
||||
const info: LlmResolvedModelInfo = {
|
||||
provider,
|
||||
id: model,
|
||||
name: resolved.name,
|
||||
...resolved.description === undefined ? {} : { description: resolved.description },
|
||||
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
|
||||
}
|
||||
const reasoning = resolved.reasoning
|
||||
if (reasoning === undefined) return info
|
||||
if (reasoning.efforts.length === 0) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
const seen = new Set<string>()
|
||||
const efforts = reasoning.efforts.map((effort) => {
|
||||
if (
|
||||
typeof effort.id !== 'string'
|
||||
|| effort.id.length === 0
|
||||
|| typeof effort.name !== 'string'
|
||||
|| effort.name.length === 0
|
||||
|| (effort.description !== undefined && typeof effort.description !== 'string')
|
||||
|| seen.has(effort.id)
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid or duplicate reasoning effort metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
seen.add(effort.id)
|
||||
return {
|
||||
id: effort.id,
|
||||
name: effort.name,
|
||||
...effort.description === undefined ? {} : { description: effort.description },
|
||||
}
|
||||
})
|
||||
if (reasoning.defaultEffort !== undefined && !seen.has(reasoning.defaultEffort)) {
|
||||
throw new LlmError(
|
||||
`adapter returned an unknown default reasoning effort for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
return {
|
||||
...info,
|
||||
reasoning: {
|
||||
efforts,
|
||||
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
private registration(provider: string): {
|
||||
adapter: LlmAdapter
|
||||
provider: LlmProviderInfo
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
} {
|
||||
/**
|
||||
* Validate a conversation call config against its exact model capability and
|
||||
* materialize an adapter-configured default. Unsupported explicit efforts
|
||||
* reject before provider I/O; no clamping or aliasing is performed. This
|
||||
* standalone query does not bind a later dispatch; use {@link prepareCall}
|
||||
* when logging and streaming must share one adapter registration.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a detached config only when a default must be materialized.
|
||||
*/
|
||||
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig> {
|
||||
return this.resolveCallConfigFor(this.registration(config.provider), config, signal)
|
||||
}
|
||||
|
||||
private async resolveCallConfigFor(
|
||||
registration: AdapterRegistration,
|
||||
config: LlmCallConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmCallConfig> {
|
||||
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
|
||||
const requested = config.reasoningEffort
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${requested}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return config
|
||||
}
|
||||
const effective = requested ?? reasoning.defaultEffort
|
||||
if (effective === undefined) return config
|
||||
if (!reasoning.efforts.some(effort => effort.id === effective)) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return requested === effective ? config : { ...config, reasoningEffort: effective }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one call under its current adapter registration. The returned
|
||||
* one-shot handle keeps that registration across header logging and dispatch,
|
||||
* so HMR cannot combine one adapter's capability result with another adapter.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a prepared config and its registration-bound stream entry point.
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
|
||||
const registration = this.registration(config.provider)
|
||||
const resolvedConfig = deepFreeze(structuredClone(
|
||||
await this.resolveCallConfigFor(registration, config, signal),
|
||||
))
|
||||
let dispatched = false
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
}
|
||||
dispatched = true
|
||||
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private registration(provider: string): AdapterRegistration {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
return registration
|
||||
@@ -334,13 +481,28 @@ export class LlmService extends Service {
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const registration = this.registration(options.provider)
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
failures.retryPolicy = registration.retryPolicy
|
||||
const resolvedConfig = prepared === undefined
|
||||
? await this.resolveCallConfigFor(registration, options, options.signal)
|
||||
: prepared.config
|
||||
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
|
||||
throw new LlmError(
|
||||
'prepared LLM call config changed before adapter dispatch',
|
||||
'INVALID_PREPARED_CALL',
|
||||
)
|
||||
}
|
||||
const resolvedOptions = prepared !== undefined || callConfigEquals(options, resolvedConfig)
|
||||
? options
|
||||
: Object.isFrozen(options)
|
||||
? deepFreeze({ ...options, ...resolvedConfig })
|
||||
: { ...options, ...resolvedConfig }
|
||||
const adapter = registration.adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
@@ -380,18 +542,37 @@ export class LlmService extends Service {
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* adapter selection remains fixed through asynchronous exact-model resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return this.streamWithRegistration(options)
|
||||
}
|
||||
|
||||
private streamWithRegistration(
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = { failures: new WeakMap<Error, LlmFailure>() }
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
options,
|
||||
() => this.adapterStream(options, failures, prepared),
|
||||
)
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
interface AdapterRegistration {
|
||||
readonly adapter: LlmAdapter
|
||||
readonly provider: LlmProviderInfo
|
||||
readonly retryPolicy: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
*/
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { CallId, ProviderRequestId } from './brand.ts'
|
||||
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
|
||||
export interface LlmFailure {
|
||||
@@ -161,6 +161,35 @@ export interface LlmModelContext {
|
||||
contextWindow: number
|
||||
}
|
||||
|
||||
/** Display metadata for one adapter-owned reasoning effort. */
|
||||
export interface LlmReasoningEffortInfo {
|
||||
/** Opaque stable value accepted by {@link GenerateOptions.reasoningEffort}. */
|
||||
id: ReasoningEffortId
|
||||
/** Human-readable effort name for selectors and diagnostics. */
|
||||
name: string
|
||||
/** Optional user-facing distinction from otherwise similar efforts. */
|
||||
description?: string
|
||||
}
|
||||
|
||||
/** Selectable reasoning efforts for one exact provider/model route. */
|
||||
export interface LlmModelReasoningInfo {
|
||||
/** Supported efforts in adapter-preferred display order. */
|
||||
efforts: readonly LlmReasoningEffortInfo[]
|
||||
/**
|
||||
* Adapter-configured default materialized into requests when callers omit
|
||||
* an effort. Absence preserves the provider's own default.
|
||||
*/
|
||||
defaultEffort?: ReasoningEffortId
|
||||
}
|
||||
|
||||
/** Exact-route model metadata resolved by its owning adapter. */
|
||||
export interface LlmResolvedModelInfo extends LlmModelInfo {
|
||||
/** Provider-owned context capacity when known. */
|
||||
context?: LlmModelContext
|
||||
/** Adapter-owned selectable reasoning levels when exposed. */
|
||||
reasoning?: LlmModelReasoningInfo
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
@@ -201,6 +230,8 @@ export interface GenerateOptions {
|
||||
/** Registered provider route selecting the adapter instance. */
|
||||
provider: string
|
||||
model: string
|
||||
/** Adapter-owned reasoning effort selected for this exact model. */
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
/**
|
||||
* Ordered conversation messages, exactly as the provider sees them (after
|
||||
* the `system` slot). A loop-built request assembles them as
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from '../src/call-config.ts'
|
||||
import { ReasoningEffortId } from '../src/brand.ts'
|
||||
import type { GenerateOptions } from '../src/types.ts'
|
||||
|
||||
describe('callConfigEquals', () => {
|
||||
@@ -14,6 +15,11 @@ describe('callConfigEquals', () => {
|
||||
expect(callConfigEquals(base, base)).toBe(true)
|
||||
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, reasoningEffort: ReasoningEffortId('high') }, base)).toBe(false)
|
||||
expect(callConfigEquals(
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
)).toBe(true)
|
||||
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
|
||||
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)
|
||||
|
||||
@@ -12,10 +12,17 @@ import LlmService, {
|
||||
llmFailureOf,
|
||||
llmRetryPolicyOf,
|
||||
ProviderRequestId,
|
||||
ReasoningEffortId,
|
||||
resolveRetryPolicy,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmModelReasoningInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
constructor(private script: StreamChunk[]) {
|
||||
@@ -51,6 +58,7 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
private readonly provider: LlmProviderInfo,
|
||||
private readonly models: readonly LlmModelInfo[],
|
||||
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
|
||||
private readonly reasoning: Readonly<Record<string, LlmModelReasoningInfo>> = {},
|
||||
) {
|
||||
super(SCRIPT)
|
||||
}
|
||||
@@ -63,11 +71,17 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(this.contexts[model])
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
|
||||
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -360,6 +374,34 @@ describe('LlmService', () => {
|
||||
expect(facts).not.toBe(carried)
|
||||
})
|
||||
|
||||
it('keeps validated failure facts across package copies with matching own codes', async () => {
|
||||
const original = Object.assign(new Error('provider busy'), {
|
||||
code: 'RATE_LIMIT',
|
||||
failure: {
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 1_500,
|
||||
requestId: 'req-cross-copy',
|
||||
},
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 1_500,
|
||||
requestId: 'req-cross-copy',
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps an unknown SDK Error exact without trusting its private code or accessors', async () => {
|
||||
const original = Object.assign(new Error('socket closed'), { code: 'ECONNRESET' })
|
||||
Object.defineProperty(original, 'failure', {
|
||||
@@ -393,6 +435,64 @@ describe('LlmService', () => {
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'LLM adapter failed', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('keeps an SDK Error exact without trusting accessor-backed carried facts', async () => {
|
||||
const original = Object.assign(new Error('busy'), {
|
||||
failure: { message: 'busy', code: 'SERVER', status: 503 },
|
||||
})
|
||||
Object.defineProperty(original, 'code', {
|
||||
get() { throw new Error('SDK code accessor must not escape') },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'busy', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('does not trust carried facts matched only by an inherited code', async () => {
|
||||
class InheritedCodeError extends Error {
|
||||
get code(): string { return 'SERVER' }
|
||||
}
|
||||
const original = Object.assign(new InheritedCodeError('busy'), {
|
||||
failure: { message: 'busy', code: 'SERVER', status: 503 },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'busy', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('keeps an SDK Error exact when code descriptor inspection is trapped', async () => {
|
||||
const target = Object.assign(new Error('busy'), {
|
||||
code: 'SERVER',
|
||||
failure: { message: 'busy', code: 'SERVER', status: 503 },
|
||||
})
|
||||
const original = new Proxy(target, {
|
||||
getOwnPropertyDescriptor(value, property) {
|
||||
if (property === 'code') throw new Error('SDK code descriptor trap')
|
||||
return Reflect.getOwnPropertyDescriptor(value, property)
|
||||
},
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'busy', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('falls back safely when SDK objects trap failure inspection or expose malformed facts', async () => {
|
||||
const propertyTrap = new Proxy(new HarnessError('descriptor trapped', 'SERVER'), {
|
||||
getOwnPropertyDescriptor(target, property) {
|
||||
@@ -722,8 +822,32 @@ describe('LlmService', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
|
||||
provider: 'plain', id: 'unlisted', name: 'unlisted',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
|
||||
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
|
||||
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
|
||||
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
|
||||
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
|
||||
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
|
||||
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
|
||||
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
|
||||
})
|
||||
|
||||
it('resolves detached model context independently of advisory catalog membership', async () => {
|
||||
@@ -736,11 +860,241 @@ describe('LlmService', () => {
|
||||
{ unlisted: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
source.contextWindow = 64_000
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
|
||||
provider: 'route', id: 'other', name: 'other',
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const source = {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('standard'), name: 'Standard' },
|
||||
{ id: ReasoningEffortId('ultra'), name: 'Ultra', description: 'Largest budget' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
|
||||
expect(resolved.reasoning).toEqual(source)
|
||||
source.efforts[0]!.name = 'mutated'
|
||||
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
|
||||
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})
|
||||
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
|
||||
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ efforts: [] }, 'empty effort list'],
|
||||
[{ efforts: [{ id: '', name: 'Empty' }] }, 'empty id'],
|
||||
[{ efforts: [{ id: 'valid', name: '' }] }, 'empty name'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid', description: 1 }] }, 'non-string description'],
|
||||
[{ efforts: [{ id: 'same', name: 'One' }, { id: 'same', name: 'Two' }] }, 'duplicate id'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid' }], defaultEffort: 'other' }, 'unknown default'],
|
||||
] as const)('rejects invalid model reasoning metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: metadata as unknown as LlmModelReasoningInfo },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
|
||||
})
|
||||
|
||||
it('rejects unsupported reasoning efforts without clamping', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: { efforts: [{ id: ReasoningEffortId('ultra'), name: 'Ultra' }] } },
|
||||
))
|
||||
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'plain',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
})
|
||||
|
||||
it('resolves reasoning defaults at the final adapter boundary after routing middleware', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends RecordingAdapter {
|
||||
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
|
||||
const reasoning: LlmModelReasoningInfo = {
|
||||
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning,
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['routed'], adapter)
|
||||
const disposeRouting = ctx.on('llm/stream', (options, next) => {
|
||||
options.provider = 'routed'
|
||||
return next()
|
||||
})
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'initial',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
disposeRouting()
|
||||
|
||||
const frozenRequest: GenerateOptions = Object.freeze({
|
||||
provider: 'routed',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})
|
||||
for await (const _chunk of ctx.llm.stream(frozenRequest)) { /* drain */ }
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
|
||||
})
|
||||
|
||||
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends RecordingAdapter {
|
||||
override async resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return {
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning: await reasoning.promise,
|
||||
}
|
||||
}
|
||||
}(SCRIPT)
|
||||
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
|
||||
const draining = (async () => {
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
})()
|
||||
|
||||
await started.promise
|
||||
disposeFirst()
|
||||
const second = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], second)
|
||||
reasoning.resolve({
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
})
|
||||
await draining
|
||||
|
||||
expect(first.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('high'))
|
||||
expect(second.lastOptions).toBeUndefined()
|
||||
})
|
||||
|
||||
it('prepares a one-shot registration-bound call and rejects config drift', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{
|
||||
model: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
},
|
||||
)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(Object.isFrozen(prepared.config)).toBe(true)
|
||||
const stream = prepared.stream({
|
||||
...prepared.config,
|
||||
model: 'other',
|
||||
messages: [],
|
||||
})
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toMatchObject({ code: 'INVALID_PREPARED_CALL' })
|
||||
expect(() => prepared.stream({
|
||||
...prepared.config,
|
||||
messages: [],
|
||||
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
|
||||
})
|
||||
|
||||
it('passes cancellation through exact-model resolution', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
reject(new Error('missing reasoning signal'))
|
||||
return
|
||||
}
|
||||
if (signal.aborted) {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
return
|
||||
}
|
||||
signal.addEventListener('abort', () => {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
}, { once: true })
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const controller = new AbortController()
|
||||
const resolving = ctx.llm.resolveCallConfig(
|
||||
{ provider: 'route', model: 'model' },
|
||||
controller.signal,
|
||||
)
|
||||
|
||||
await started.promise
|
||||
const reason = new Error('cancel reasoning')
|
||||
controller.abort(reason)
|
||||
await expect(resolving).rejects.toBe(reason)
|
||||
})
|
||||
|
||||
it.each([0, -1, 1.5, Number.NaN])(
|
||||
@@ -753,7 +1107,7 @@ describe('LlmService', () => {
|
||||
[],
|
||||
{ model: { contextWindow } },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelContext('route', 'model'))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
|
||||
},
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user