Merge remote-tracking branch 'origin/master' into xtr/agent-loop-message-machine
# Conflicts: # docs/architecture.i18n.yaml # docs/architecture.md # docs/architecture.zh.md # docs/cordis-catalog/services.md # docs/core-data-structures/core.i18n.yaml # docs/core-data-structures/core.md # docs/core-data-structures/core.zh.md # docs/core-data-structures/llm-streaming.i18n.yaml # docs/core-data-structures/session.i18n.yaml # docs/event-producer-consumer.md # examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl # examples/headless-agent/tests/snapshots/advanced-toolchain/session.1.jsonl # examples/headless-agent/tests/snapshots/advanced-toolchain/session.2.jsonl # packages/core/agent-loop/README.i18n.yaml # packages/core/agent-loop/README.md # packages/core/agent-loop/README.zh.md # packages/core/agent-loop/src/loop.ts # packages/core/agent/README.i18n.yaml # packages/core/agent/tests/llm-target.spec.ts # packages/core/session/tests/request-header.spec.ts
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 981f7d58802d2ff18633b09b5a4ec8a7b1bf3383
|
||||
README.zh.md: 0a6535f41adb8ec90d02dbb56aec257853a19083
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 3efb3ece3caadeaceaa3c504ba4b10ddb951127a
|
||||
README.zh.md: 4af8b8d08cc96ff0e36b10b15e1d86afd43004c9
|
||||
|
||||
@@ -13,14 +13,18 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -30,7 +34,7 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
@@ -41,7 +45,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates and defaults it under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
@@ -60,11 +64,11 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
|
||||
### Real adapters
|
||||
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses direct fetch with `eventsource-parser` SSE framing for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
|
||||
## Model Experience
|
||||
|
||||
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
|
||||
None, as the service adds no model-bound text, schema, or message; it only materializes and logs an adapter-configured reasoning effort.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
|
||||
@@ -13,14 +13,18 @@
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber dispose。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` 从拥有精确路由的适配器解析权威上下文容量。
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` 从拥有精确路由的适配器解析经校验的确切模型身份、可用上下文和推理(reasoning)元数据;异步适配器可选地支持取消。
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` 校验显式推理强度,并填入适配器配置的默认值,但不自动调整。
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始 chunk(token 级 delta)。消费方使用 `BlockAssembler` 将 chunk 组装为块/消息。
|
||||
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误,并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误;`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
|
||||
提供方与模型元数据是发现表层,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,适配器则可以接受 `listModels()` 中不存在的模型 id;消费方禁止因模型未列出而拒绝请求。返回的元数据与输入脱离,无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
|
||||
上下文容量是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelContext()` 会询问拥有精确提供方/模型路由的适配器;适配器可以描述未列出的动态模型,`undefined` 只表示容量不可用。无效的返回容量以 `INVALID_MODEL_CONTEXT` 失败。
|
||||
确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型,缺少 `context` 或 `reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
推理标识符是由适配器持有的不透明字符串,而非核心枚举。适配器会公布有序可选列表;模型能力 API 提供 `off` id 时,列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值,在存在 `defaultEffort` 时填入它,否则保留提供方默认值。异步模型解析器会接收调用方的 signal,并且必须在取消后迅速完成结算。`prepareCall()` 还会让精确适配器注册跨越请求头记录和最终分派,因此 HMR(热模块替换)不会将一个适配器的能力结果与另一个适配器的请求混用;复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
### 事件
|
||||
|
||||
@@ -30,7 +34,7 @@
|
||||
|
||||
### 扩展点
|
||||
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据,在已知精确容量时覆盖 `resolveModelContext()`;默认实现将路由 id 用作名称,不公布模型,也不返回容量。
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据。
|
||||
- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`。
|
||||
|
||||
### 内容块词汇(`types.ts`)
|
||||
@@ -41,7 +45,7 @@
|
||||
|
||||
### 调用配置(`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` 是一个会话请求的提供方 + 模型 + 采样标量(`provider`、`model`、`temperature`、`maxTokens`、`stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,loop 则记录真实变更。`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper(`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。
|
||||
`LlmCallConfig` 是一个会话请求的提供方、模型、可选的适配器持有推理强度和采样标量(`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,`prepareCall()` 在轮次 signal 控制下校验并填入默认值,loop 随后记录生效值,再使用准备完成调用的注册绑定流。`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper(`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。
|
||||
|
||||
### 应用归因(`attribution.ts`)
|
||||
|
||||
@@ -60,11 +64,11 @@
|
||||
|
||||
### 真实适配器
|
||||
|
||||
两个适配器使用不同内部机制实现 `LlmAdapter`:[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用手写 fetch/SSE,[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `StreamChunk` 约定,定义见 `types.ts`:usage 先于 finish,工具参数保持原始字符串,错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
|
||||
两个适配器使用不同内部机制实现 `LlmAdapter`:[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用直接 fetch 加 `eventsource-parser` SSE 分帧,[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `StreamChunk` 约定,定义见 `types.ts`:usage 先于 finish,工具参数保持原始字符串,错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
|
||||
|
||||
## 模型体验
|
||||
|
||||
无。该适配器注册表转发已组装的请求,不添加或更改任何模型边界文本、schema 或消息。
|
||||
无。服务不添加或更改任何模型边界文本、schema 或消息;它只会填入并记录适配器配置的推理强度。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -38,3 +38,15 @@ export type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
export function ProviderRequestId(id: string): ProviderRequestId {
|
||||
return id as ProviderRequestId
|
||||
}
|
||||
|
||||
/** Adapter-owned identifier for one model's selectable reasoning effort. */
|
||||
export type ReasoningEffortId = Branded<'ReasoningEffortId'>
|
||||
|
||||
/**
|
||||
* Brand an adapter-owned reasoning-effort identifier.
|
||||
* @param id - the opaque identifier exposed by one model capability.
|
||||
* @returns the same string, branded; no validation is performed.
|
||||
*/
|
||||
export function ReasoningEffortId(id: string): ReasoningEffortId {
|
||||
return id as ReasoningEffortId
|
||||
}
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
/**
|
||||
* Conversation call configuration and freeze utilities. Provider routing,
|
||||
* model, and sampling values are request-header state that can affect cache
|
||||
* reuse; request waterfalls replace them and the loop logs changed snapshots
|
||||
* instead of allowing silent per-call drift.
|
||||
* model, reasoning effort, and sampling values are request-header state that
|
||||
* can affect cache reuse; request waterfalls replace them and the loop logs
|
||||
* changed snapshots instead of allowing silent per-call drift.
|
||||
* @module dsh-llm/call-config
|
||||
*/
|
||||
|
||||
import type { GenerateOptions } from './types.ts'
|
||||
import type { ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Process-local identities of request objects assembled by dsh-agent-loop. */
|
||||
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
|
||||
|
||||
/**
|
||||
* Provider + model + sampling scalars of one conversation's requests. Every field maps
|
||||
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
|
||||
* from the logged header rather than accepting these per call.
|
||||
* Provider, model, reasoning effort, and sampling scalars of one conversation's
|
||||
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
|
||||
* the loop builds requests from the logged header rather than accepting these
|
||||
* per call.
|
||||
*/
|
||||
export interface LlmCallConfig {
|
||||
provider: string
|
||||
model: string
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
stop?: string[]
|
||||
@@ -33,7 +36,13 @@ export interface LlmCallConfig {
|
||||
* @returns whether every field (including the `stop` list, element-wise) matches.
|
||||
*/
|
||||
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
|
||||
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
|
||||
if (
|
||||
a.provider !== b.provider
|
||||
|| a.model !== b.model
|
||||
|| a.reasoningEffort !== b.reasoningEffort
|
||||
|| a.temperature !== b.temperature
|
||||
|| a.maxTokens !== b.maxTokens
|
||||
) return false
|
||||
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
|
||||
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
|
||||
}
|
||||
|
||||
@@ -10,14 +10,15 @@ import { Context, Service } from 'cordis'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmResolvedModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
} from './types.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
import type { LlmCallConfig } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
import type { AdapterFailureScope } from './adapter-failure.ts'
|
||||
@@ -103,11 +104,25 @@ export class LlmError extends HarnessError {
|
||||
}
|
||||
}
|
||||
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
export interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
|
||||
* @param options - fully assembled request carrying the prepared config.
|
||||
* @returns the chunk stream, including the `llm/stream` waterfall.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
|
||||
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The hand-rolled
|
||||
* DeepSeek and pi-ai adapters intentionally exercise this contract through different internals.
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The direct-fetch
|
||||
* DeepSeek and library-backed pi-ai adapters intentionally exercise this contract through different internals.
|
||||
*/
|
||||
export abstract class LlmAdapter {
|
||||
/**
|
||||
@@ -131,17 +146,20 @@ export abstract class LlmAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity for one model accepted by this adapter. Absence
|
||||
* means the adapter does not know the capacity, not that routing is invalid.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns provider-owned context metadata, or `undefined` when unavailable.
|
||||
* Resolve all metadata available for one exact model. This query is
|
||||
* independent of the advisory catalog and does not validate request routing.
|
||||
* @param provider - one provider route owned by this adapter.
|
||||
* @param model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; asynchronous
|
||||
* implementations must settle promptly after it aborts.
|
||||
* @returns provider/model identity plus any context and reasoning metadata.
|
||||
*/
|
||||
resolveModelContext(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({ provider, id: model, name: model })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -157,7 +175,7 @@ export abstract class LlmAdapter {
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
*/
|
||||
export class LlmService extends Service {
|
||||
private adapters = new Map<string, { adapter: LlmAdapter; provider: LlmProviderInfo }>()
|
||||
private adapters = new Map<string, AdapterRegistration>()
|
||||
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'llm')
|
||||
@@ -175,7 +193,7 @@ export class LlmService extends Service {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: { adapter: LlmAdapter; provider: LlmProviderInfo }[] = []
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
@@ -240,29 +258,170 @@ export class LlmService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity from the adapter that owns one exact route.
|
||||
* This query is independent of the advisory model catalog: an unlisted model
|
||||
* may return metadata, while `undefined` never rejects later routing.
|
||||
* Resolve and validate all metadata from the adapter that owns one exact
|
||||
* route. The result is detached from adapter-owned objects; catalog
|
||||
* membership remains advisory and does not control request routing.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @returns detached context metadata, or `undefined` when the adapter has none.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns exact model identity plus available context and reasoning metadata.
|
||||
*/
|
||||
async resolveModelContext(
|
||||
async resolveModelInfo(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
|
||||
if (context === undefined) return undefined
|
||||
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return this.resolveModelInfoFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelInfoFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const provider = registration.provider.id
|
||||
const resolved = await registration.adapter.resolveModel(provider, model, signal)
|
||||
if (
|
||||
typeof resolved.provider !== 'string'
|
||||
|| resolved.provider !== provider
|
||||
|| typeof resolved.id !== 'string'
|
||||
|| resolved.id !== model
|
||||
|| typeof resolved.name !== 'string'
|
||||
|| resolved.name.length === 0
|
||||
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_INFO',
|
||||
)
|
||||
}
|
||||
const context = resolved.context
|
||||
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_CONTEXT',
|
||||
)
|
||||
}
|
||||
return { contextWindow: context.contextWindow }
|
||||
const info: LlmResolvedModelInfo = {
|
||||
provider,
|
||||
id: model,
|
||||
name: resolved.name,
|
||||
...resolved.description === undefined ? {} : { description: resolved.description },
|
||||
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
|
||||
}
|
||||
const reasoning = resolved.reasoning
|
||||
if (reasoning === undefined) return info
|
||||
if (reasoning.efforts.length === 0) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
const seen = new Set<string>()
|
||||
const efforts = reasoning.efforts.map((effort) => {
|
||||
if (
|
||||
typeof effort.id !== 'string'
|
||||
|| effort.id.length === 0
|
||||
|| typeof effort.name !== 'string'
|
||||
|| effort.name.length === 0
|
||||
|| (effort.description !== undefined && typeof effort.description !== 'string')
|
||||
|| seen.has(effort.id)
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid or duplicate reasoning effort metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
seen.add(effort.id)
|
||||
return {
|
||||
id: effort.id,
|
||||
name: effort.name,
|
||||
...effort.description === undefined ? {} : { description: effort.description },
|
||||
}
|
||||
})
|
||||
if (reasoning.defaultEffort !== undefined && !seen.has(reasoning.defaultEffort)) {
|
||||
throw new LlmError(
|
||||
`adapter returned an unknown default reasoning effort for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_REASONING',
|
||||
)
|
||||
}
|
||||
return {
|
||||
...info,
|
||||
reasoning: {
|
||||
efforts,
|
||||
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
|
||||
/**
|
||||
* Validate a conversation call config against its exact model capability and
|
||||
* materialize an adapter-configured default. Unsupported explicit efforts
|
||||
* reject before provider I/O; no clamping or aliasing is performed. This
|
||||
* standalone query does not bind a later dispatch; use {@link prepareCall}
|
||||
* when logging and streaming must share one adapter registration.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a detached config only when a default must be materialized.
|
||||
*/
|
||||
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig> {
|
||||
return this.resolveCallConfigFor(this.registration(config.provider), config, signal)
|
||||
}
|
||||
|
||||
private async resolveCallConfigFor(
|
||||
registration: AdapterRegistration,
|
||||
config: LlmCallConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmCallConfig> {
|
||||
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
|
||||
const requested = config.reasoningEffort
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${requested}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return config
|
||||
}
|
||||
const effective = requested ?? reasoning.defaultEffort
|
||||
if (effective === undefined) return config
|
||||
if (!reasoning.efforts.some(effort => effort.id === effective)) {
|
||||
throw new LlmError(
|
||||
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
|
||||
'UNSUPPORTED_REASONING_EFFORT',
|
||||
)
|
||||
}
|
||||
return requested === effective ? config : { ...config, reasoningEffort: effective }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one call under its current adapter registration. The returned
|
||||
* one-shot handle keeps that registration across header logging and dispatch,
|
||||
* so HMR cannot combine one adapter's capability result with another adapter.
|
||||
* @param config - provider/model route and optional request controls.
|
||||
* @param signal - optional cancellation for adapter-owned capability lookup.
|
||||
* @returns a prepared config and its registration-bound stream entry point.
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
|
||||
const registration = this.registration(config.provider)
|
||||
const resolvedConfig = deepFreeze(structuredClone(
|
||||
await this.resolveCallConfigFor(registration, config, signal),
|
||||
))
|
||||
let dispatched = false
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
}
|
||||
dispatched = true
|
||||
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
private registration(provider: string): AdapterRegistration {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
return registration
|
||||
@@ -295,11 +454,27 @@ export class LlmService extends Service {
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const adapter = this.registration(options.provider).adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
const resolvedConfig = prepared === undefined
|
||||
? await this.resolveCallConfigFor(registration, options, options.signal)
|
||||
: prepared.config
|
||||
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
|
||||
throw new LlmError(
|
||||
'prepared LLM call config changed before adapter dispatch',
|
||||
'INVALID_PREPARED_CALL',
|
||||
)
|
||||
}
|
||||
const resolvedOptions = prepared !== undefined || callConfigEquals(options, resolvedConfig)
|
||||
? options
|
||||
: Object.isFrozen(options)
|
||||
? deepFreeze({ ...options, ...resolvedConfig })
|
||||
: { ...options, ...resolvedConfig }
|
||||
const adapter = registration.adapter
|
||||
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
@@ -339,18 +514,36 @@ export class LlmService extends Service {
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* adapter selection remains fixed through asynchronous exact-model resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
* untagged for the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return this.streamWithRegistration(options)
|
||||
}
|
||||
|
||||
private streamWithRegistration(
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
options,
|
||||
() => this.adapterStream(options, failures, prepared),
|
||||
)
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
interface AdapterRegistration {
|
||||
readonly adapter: LlmAdapter
|
||||
readonly provider: LlmProviderInfo
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
*/
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { CallId, ProviderRequestId } from './brand.ts'
|
||||
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
|
||||
|
||||
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
|
||||
export interface LlmFailure {
|
||||
@@ -161,6 +161,35 @@ export interface LlmModelContext {
|
||||
contextWindow: number
|
||||
}
|
||||
|
||||
/** Display metadata for one adapter-owned reasoning effort. */
|
||||
export interface LlmReasoningEffortInfo {
|
||||
/** Opaque stable value accepted by {@link GenerateOptions.reasoningEffort}. */
|
||||
id: ReasoningEffortId
|
||||
/** Human-readable effort name for selectors and diagnostics. */
|
||||
name: string
|
||||
/** Optional user-facing distinction from otherwise similar efforts. */
|
||||
description?: string
|
||||
}
|
||||
|
||||
/** Selectable reasoning efforts for one exact provider/model route. */
|
||||
export interface LlmModelReasoningInfo {
|
||||
/** Supported efforts in adapter-preferred display order. */
|
||||
efforts: readonly LlmReasoningEffortInfo[]
|
||||
/**
|
||||
* Adapter-configured default materialized into requests when callers omit
|
||||
* an effort. Absence preserves the provider's own default.
|
||||
*/
|
||||
defaultEffort?: ReasoningEffortId
|
||||
}
|
||||
|
||||
/** Exact-route model metadata resolved by its owning adapter. */
|
||||
export interface LlmResolvedModelInfo extends LlmModelInfo {
|
||||
/** Provider-owned context capacity when known. */
|
||||
context?: LlmModelContext
|
||||
/** Adapter-owned selectable reasoning levels when exposed. */
|
||||
reasoning?: LlmModelReasoningInfo
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
@@ -201,6 +230,8 @@ export interface GenerateOptions {
|
||||
/** Registered provider route selecting the adapter instance. */
|
||||
provider: string
|
||||
model: string
|
||||
/** Adapter-owned reasoning effort selected for this exact model. */
|
||||
reasoningEffort?: ReasoningEffortId
|
||||
/**
|
||||
* Ordered conversation messages, exactly as the provider sees them (after
|
||||
* the `system` slot). A loop-built request assembles them as
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from '../src/call-config.ts'
|
||||
import { ReasoningEffortId } from '../src/brand.ts'
|
||||
import type { GenerateOptions } from '../src/types.ts'
|
||||
|
||||
describe('callConfigEquals', () => {
|
||||
@@ -14,6 +15,11 @@ describe('callConfigEquals', () => {
|
||||
expect(callConfigEquals(base, base)).toBe(true)
|
||||
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, reasoningEffort: ReasoningEffortId('high') }, base)).toBe(false)
|
||||
expect(callConfigEquals(
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
{ ...base, reasoningEffort: ReasoningEffortId('high') },
|
||||
)).toBe(true)
|
||||
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
|
||||
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)
|
||||
|
||||
@@ -11,9 +11,16 @@ import LlmService, {
|
||||
LlmError,
|
||||
llmFailureOf,
|
||||
ProviderRequestId,
|
||||
ReasoningEffortId,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmModelReasoningInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
constructor(private script: StreamChunk[]) {
|
||||
@@ -49,6 +56,7 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
private readonly provider: LlmProviderInfo,
|
||||
private readonly models: readonly LlmModelInfo[],
|
||||
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
|
||||
private readonly reasoning: Readonly<Record<string, LlmModelReasoningInfo>> = {},
|
||||
) {
|
||||
super(SCRIPT)
|
||||
}
|
||||
@@ -61,11 +69,17 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(this.contexts[model])
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
|
||||
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -739,8 +753,32 @@ describe('LlmService', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
|
||||
provider: 'plain', id: 'unlisted', name: 'unlisted',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
|
||||
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
|
||||
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
|
||||
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
|
||||
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
|
||||
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
|
||||
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
|
||||
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
|
||||
})
|
||||
|
||||
it('resolves detached model context independently of advisory catalog membership', async () => {
|
||||
@@ -753,11 +791,241 @@ describe('LlmService', () => {
|
||||
{ unlisted: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
source.contextWindow = 64_000
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
|
||||
provider: 'route', id: 'other', name: 'other',
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const source = {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('standard'), name: 'Standard' },
|
||||
{ id: ReasoningEffortId('ultra'), name: 'Ultra', description: 'Largest budget' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
|
||||
expect(resolved.reasoning).toEqual(source)
|
||||
source.efforts[0]!.name = 'mutated'
|
||||
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
|
||||
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})
|
||||
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
|
||||
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ efforts: [] }, 'empty effort list'],
|
||||
[{ efforts: [{ id: '', name: 'Empty' }] }, 'empty id'],
|
||||
[{ efforts: [{ id: 'valid', name: '' }] }, 'empty name'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid', description: 1 }] }, 'non-string description'],
|
||||
[{ efforts: [{ id: 'same', name: 'One' }, { id: 'same', name: 'Two' }] }, 'duplicate id'],
|
||||
[{ efforts: [{ id: 'valid', name: 'Valid' }], defaultEffort: 'other' }, 'unknown default'],
|
||||
] as const)('rejects invalid model reasoning metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: metadata as unknown as LlmModelReasoningInfo },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
|
||||
})
|
||||
|
||||
it('rejects unsupported reasoning efforts without clamping', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: { efforts: [{ id: ReasoningEffortId('ultra'), name: 'Ultra' }] } },
|
||||
))
|
||||
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
await expect(ctx.llm.resolveCallConfig({
|
||||
provider: 'route',
|
||||
model: 'plain',
|
||||
reasoningEffort: ReasoningEffortId('standard'),
|
||||
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
|
||||
})
|
||||
|
||||
it('resolves reasoning defaults at the final adapter boundary after routing middleware', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends RecordingAdapter {
|
||||
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
|
||||
const reasoning: LlmModelReasoningInfo = {
|
||||
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning,
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['routed'], adapter)
|
||||
const disposeRouting = ctx.on('llm/stream', (options, next) => {
|
||||
options.provider = 'routed'
|
||||
return next()
|
||||
})
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'initial',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
disposeRouting()
|
||||
|
||||
const frozenRequest: GenerateOptions = Object.freeze({
|
||||
provider: 'routed',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})
|
||||
for await (const _chunk of ctx.llm.stream(frozenRequest)) { /* drain */ }
|
||||
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
|
||||
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
|
||||
})
|
||||
|
||||
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends RecordingAdapter {
|
||||
override async resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return {
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning: await reasoning.promise,
|
||||
}
|
||||
}
|
||||
}(SCRIPT)
|
||||
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
|
||||
const draining = (async () => {
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
})()
|
||||
|
||||
await started.promise
|
||||
disposeFirst()
|
||||
const second = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], second)
|
||||
reasoning.resolve({
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
})
|
||||
await draining
|
||||
|
||||
expect(first.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('high'))
|
||||
expect(second.lastOptions).toBeUndefined()
|
||||
})
|
||||
|
||||
it('prepares a one-shot registration-bound call and rejects config drift', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{
|
||||
model: {
|
||||
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
},
|
||||
)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(Object.isFrozen(prepared.config)).toBe(true)
|
||||
const stream = prepared.stream({
|
||||
...prepared.config,
|
||||
model: 'other',
|
||||
messages: [],
|
||||
})
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toMatchObject({ code: 'INVALID_PREPARED_CALL' })
|
||||
expect(() => prepared.stream({
|
||||
...prepared.config,
|
||||
messages: [],
|
||||
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
|
||||
})
|
||||
|
||||
it('passes cancellation through exact-model resolution', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
reject(new Error('missing reasoning signal'))
|
||||
return
|
||||
}
|
||||
if (signal.aborted) {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
return
|
||||
}
|
||||
signal.addEventListener('abort', () => {
|
||||
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
|
||||
}, { once: true })
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const controller = new AbortController()
|
||||
const resolving = ctx.llm.resolveCallConfig(
|
||||
{ provider: 'route', model: 'model' },
|
||||
controller.signal,
|
||||
)
|
||||
|
||||
await started.promise
|
||||
const reason = new Error('cancel reasoning')
|
||||
controller.abort(reason)
|
||||
await expect(resolving).rejects.toBe(reason)
|
||||
})
|
||||
|
||||
it.each([0, -1, 1.5, Number.NaN])(
|
||||
@@ -770,7 +1038,7 @@ describe('LlmService', () => {
|
||||
[],
|
||||
{ model: { contextWindow } },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelContext('route', 'model'))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
|
||||
},
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user