Merge remote-tracking branch 'origin/master' into xtr/agent-loop-message-machine

# Conflicts:
#	docs/architecture.i18n.yaml
#	docs/architecture.md
#	docs/architecture.zh.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/core.i18n.yaml
#	docs/core-data-structures/core.md
#	docs/core-data-structures/core.zh.md
#	docs/core-data-structures/llm-streaming.i18n.yaml
#	docs/core-data-structures/session.i18n.yaml
#	docs/event-producer-consumer.md
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl
#	examples/headless-agent/tests/snapshots/advanced-toolchain/session.1.jsonl
#	examples/headless-agent/tests/snapshots/advanced-toolchain/session.2.jsonl
#	packages/core/agent-loop/README.i18n.yaml
#	packages/core/agent-loop/README.md
#	packages/core/agent-loop/README.zh.md
#	packages/core/agent-loop/src/loop.ts
#	packages/core/agent/README.i18n.yaml
#	packages/core/agent/tests/llm-target.spec.ts
#	packages/core/session/tests/request-header.spec.ts
This commit is contained in:
_Kerman
2026-07-27 16:44:06 +08:00
220 changed files with 4592 additions and 969 deletions

View File

@@ -1,6 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
README.md: 981f7d58802d2ff18633b09b5a4ec8a7b1bf3383
README.zh.md: 0a6535f41adb8ec90d02dbb56aec257853a19083
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
README.md: 3efb3ece3caadeaceaa3c504ba4b10ddb951127a
README.zh.md: 4af8b8d08cc96ff0e36b10b15e1d86afd43004c9

View File

@@ -13,14 +13,18 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
### Events
@@ -30,7 +34,7 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
### Extension points
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)
@@ -41,7 +45,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
### Call configuration (`call-config.ts`)
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates and defaults it under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
### App attribution (`attribution.ts`)
@@ -60,11 +64,11 @@ Every product adapter sends application identity on provider HTTP requests. `att
### Real adapters
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses direct fetch with `eventsource-parser` SSE framing for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
## Model Experience
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
None, as the service adds no model-bound text, schema, or message; it only materializes and logs an adapter-configured reasoning effort.
#### KV Cache effect

View File

@@ -13,14 +13,18 @@
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber dispose。
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` 从拥有精确路由的适配器解析权威上下文容量
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` 从拥有精确路由的适配器解析经校验的确切模型身份、可用上下文和推理reasoning元数据异步适配器可选地支持取消
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` 校验显式推理强度,并填入适配器配置的默认值,但不自动调整。
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始 chunktoken 级 delta。消费方使用 `BlockAssembler` 将 chunk 组装为块/消息。
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误,并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误;`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`
提供方与模型元数据是发现表层,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,适配器则可以接受 `listModels()` 中不存在的模型 id消费方禁止因模型未列出而拒绝请求。返回的元数据与输入脱离无效或重复适配器配置项会以 `INVALID_ADAPTER``INVALID_CATALOG` 失败。
上下文容量是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelContext()`询问拥有精确提供方/模型路由的适配器;适配器可以描述未列出的动态模型,`undefined` 只表示容量不可用。无效的返回容量`INVALID_MODEL_CONTEXT` 失败。
确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()`拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型,缺少 `context``reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会`INVALID_MODEL_INFO``INVALID_MODEL_CONTEXT``INVALID_MODEL_REASONING` 失败。
推理标识符是由适配器持有的不透明字符串,而非核心枚举。适配器会公布有序可选列表;模型能力 API 提供 `off` id 时,列表也会包含它。`resolveCallConfig()` 只接受与已公布标识符完全一致的值,在存在 `defaultEffort` 时填入它,否则保留提供方默认值。异步模型解析器会接收调用方的 signal并且必须在取消后迅速完成结算。`prepareCall()` 还会让精确适配器注册跨越请求头记录和最终分派,因此 HMR热模块替换不会将一个适配器的能力结果与另一个适配器的请求混用复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
### 事件
@@ -30,7 +34,7 @@
### 扩展点
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据,在已知精确容量时覆盖 `resolveModelContext()`默认实现将路由 id 用作名称,不公布模型,也不返回容量。
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据
- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`
### 内容块词汇(`types.ts`
@@ -41,7 +45,7 @@
### 调用配置(`call-config.ts`
`LlmCallConfig` 是一个会话请求的提供方 + 模型 + 采样标量(`provider``model``temperature``maxTokens``stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,loop 则记录真实变更`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。
`LlmCallConfig` 是一个会话请求的提供方、模型、可选的适配器持有推理强度和采样标量(`provider``model``reasoningEffort``temperature``maxTokens``stop`,每个都与同名 `GenerateOptions` 字段 1:1 映射)。它是作为请求标头一部分记录在会话日志中的每会话状态(见 dsh-session `request/header` 事件),绝不是可静默调整的每次调用旋钮:`agent/request` waterfall 会提议替换,`prepareCall()` 在轮次 signal 控制下校验并填入默认值loop 随后记录生效值,再使用准备完成调用的注册绑定流`callConfigEquals(a, b)` 是逐字段真实变更检测器;`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper`llm/stream` listener 与适配器只读,绝不改写)。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源,`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类,让适配器可以应用目的特定传输策略,而不改变普通会话请求。
### 应用归因(`attribution.ts`
@@ -60,11 +64,11 @@
### 真实适配器
两个适配器使用不同内部机制实现 `LlmAdapter`[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用手写 fetch/SSE[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `StreamChunk` 约定,定义见 `types.ts`usage 先于 finish工具参数保持原始字符串错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
两个适配器使用不同内部机制实现 `LlmAdapter`[`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) 针对 `deepseek` 路由使用直接 fetch`eventsource-parser` SSE 分帧[`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) 则通过 `@earendil-works/pi-ai` 动态解析已配置提供方/模型对。两者都遵循 `StreamChunk` 约定,定义见 `types.ts`usage 先于 finish工具参数保持原始字符串错误使用两种已批准路径之一。设计理由见 [双 LLM 适配器](../../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)。
## 模型体验
无。该适配器注册表转发已组装的请求,不添加或更改任何模型边界文本、schema 或消息。
无。服务不添加或更改任何模型边界文本、schema 或消息;它只会填入并记录适配器配置的推理强度
#### KV Cache 影响

View File

@@ -38,3 +38,15 @@ export type ProviderRequestId = Branded<'ProviderRequestId'>
export function ProviderRequestId(id: string): ProviderRequestId {
return id as ProviderRequestId
}
/** Adapter-owned identifier for one model's selectable reasoning effort. */
export type ReasoningEffortId = Branded<'ReasoningEffortId'>
/**
* Brand an adapter-owned reasoning-effort identifier.
* @param id - the opaque identifier exposed by one model capability.
* @returns the same string, branded; no validation is performed.
*/
export function ReasoningEffortId(id: string): ReasoningEffortId {
return id as ReasoningEffortId
}

View File

@@ -1,24 +1,27 @@
/**
* Conversation call configuration and freeze utilities. Provider routing,
* model, and sampling values are request-header state that can affect cache
* reuse; request waterfalls replace them and the loop logs changed snapshots
* instead of allowing silent per-call drift.
* model, reasoning effort, and sampling values are request-header state that
* can affect cache reuse; request waterfalls replace them and the loop logs
* changed snapshots instead of allowing silent per-call drift.
* @module dsh-llm/call-config
*/
import type { GenerateOptions } from './types.ts'
import type { ReasoningEffortId } from './brand.ts'
/** Process-local identities of request objects assembled by dsh-agent-loop. */
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
/**
* Provider + model + sampling scalars of one conversation's requests. Every field maps
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
* from the logged header rather than accepting these per call.
* Provider, model, reasoning effort, and sampling scalars of one conversation's
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
* the loop builds requests from the logged header rather than accepting these
* per call.
*/
export interface LlmCallConfig {
provider: string
model: string
reasoningEffort?: ReasoningEffortId
temperature?: number
maxTokens?: number
stop?: string[]
@@ -33,7 +36,13 @@ export interface LlmCallConfig {
* @returns whether every field (including the `stop` list, element-wise) matches.
*/
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
if (
a.provider !== b.provider
|| a.model !== b.model
|| a.reasoningEffort !== b.reasoningEffort
|| a.temperature !== b.temperature
|| a.maxTokens !== b.maxTokens
) return false
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
}

View File

@@ -10,14 +10,15 @@ import { Context, Service } from 'cordis'
import type {
GenerateOptions,
LlmFailure,
LlmModelContext,
LlmModelInfo,
LlmResolvedModelInfo,
LlmProviderInfo,
Message,
StreamChunk,
} from './types.ts'
import type { ProviderRequestId } from './brand.ts'
import { deepFreeze } from './call-config.ts'
import { callConfigEquals, deepFreeze } from './call-config.ts'
import type { LlmCallConfig } from './call-config.ts'
import { HarnessError } from './error.ts'
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
import type { AdapterFailureScope } from './adapter-failure.ts'
@@ -103,11 +104,25 @@ export class LlmError extends HarnessError {
}
}
/** One model call whose config and adapter registration were resolved together. */
export interface PreparedLlmCall {
/** Detached, deep-frozen config with any adapter-owned default materialized. */
readonly config: LlmCallConfig
/**
* Dispatch this call once through the registration captured during
* preparation. The request's call-config fields must match {@link config};
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
* @param options - fully assembled request carrying the prepared config.
* @returns the chunk stream, including the `llm/stream` waterfall.
*/
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
/**
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The hand-rolled
* DeepSeek and pi-ai adapters intentionally exercise this contract through different internals.
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The direct-fetch
* DeepSeek and library-backed pi-ai adapters intentionally exercise this contract through different internals.
*/
export abstract class LlmAdapter {
/**
@@ -131,17 +146,20 @@ export abstract class LlmAdapter {
}
/**
* Resolve context capacity for one model accepted by this adapter. Absence
* means the adapter does not know the capacity, not that routing is invalid.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns provider-owned context metadata, or `undefined` when unavailable.
* Resolve all metadata available for one exact model. This query is
* independent of the advisory catalog and does not validate request routing.
* @param provider - one provider route owned by this adapter.
* @param model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; asynchronous
* implementations must settle promptly after it aborts.
* @returns provider/model identity plus any context and reasoning metadata.
*/
resolveModelContext(
_provider: string,
_model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(undefined)
resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({ provider, id: model, name: model })
}
/**
@@ -157,7 +175,7 @@ export abstract class LlmAdapter {
* surface, interceptable via the `llm/stream` waterfall.
*/
export class LlmService extends Service {
private adapters = new Map<string, { adapter: LlmAdapter; provider: LlmProviderInfo }>()
private adapters = new Map<string, AdapterRegistration>()
constructor(ctx: Context) {
super(ctx, 'llm')
@@ -175,7 +193,7 @@ export class LlmService extends Service {
const dispose = this.ctx.effect(function* (this: LlmService) {
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
const unique = new Set<string>()
const registrations: { adapter: LlmAdapter; provider: LlmProviderInfo }[] = []
const registrations: AdapterRegistration[] = []
for (const provider of providers) {
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
if (unique.has(provider) || this.adapters.has(provider)) {
@@ -240,29 +258,170 @@ export class LlmService extends Service {
}
/**
* Resolve context capacity from the adapter that owns one exact route.
* This query is independent of the advisory model catalog: an unlisted model
* may return metadata, while `undefined` never rejects later routing.
* Resolve and validate all metadata from the adapter that owns one exact
* route. The result is detached from adapter-owned objects; catalog
* membership remains advisory and does not control request routing.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @returns detached context metadata, or `undefined` when the adapter has none.
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
* @returns exact model identity plus available context and reasoning metadata.
*/
async resolveModelContext(
async resolveModelInfo(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
if (context === undefined) return undefined
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
return this.resolveModelInfoFor(this.registration(provider), model, signal)
}
private async resolveModelInfoFor(
registration: AdapterRegistration,
model: string,
signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
const provider = registration.provider.id
const resolved = await registration.adapter.resolveModel(provider, model, signal)
if (
typeof resolved.provider !== 'string'
|| resolved.provider !== provider
|| typeof resolved.id !== 'string'
|| resolved.id !== model
|| typeof resolved.name !== 'string'
|| resolved.name.length === 0
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
) {
throw new LlmError(
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_INFO',
)
}
const context = resolved.context
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
throw new LlmError(
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_CONTEXT',
)
}
return { contextWindow: context.contextWindow }
const info: LlmResolvedModelInfo = {
provider,
id: model,
name: resolved.name,
...resolved.description === undefined ? {} : { description: resolved.description },
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
}
const reasoning = resolved.reasoning
if (reasoning === undefined) return info
if (reasoning.efforts.length === 0) {
throw new LlmError(
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
const seen = new Set<string>()
const efforts = reasoning.efforts.map((effort) => {
if (
typeof effort.id !== 'string'
|| effort.id.length === 0
|| typeof effort.name !== 'string'
|| effort.name.length === 0
|| (effort.description !== undefined && typeof effort.description !== 'string')
|| seen.has(effort.id)
) {
throw new LlmError(
`adapter returned invalid or duplicate reasoning effort metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
seen.add(effort.id)
return {
id: effort.id,
name: effort.name,
...effort.description === undefined ? {} : { description: effort.description },
}
})
if (reasoning.defaultEffort !== undefined && !seen.has(reasoning.defaultEffort)) {
throw new LlmError(
`adapter returned an unknown default reasoning effort for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
return {
...info,
reasoning: {
efforts,
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
},
}
}
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
/**
* Validate a conversation call config against its exact model capability and
* materialize an adapter-configured default. Unsupported explicit efforts
* reject before provider I/O; no clamping or aliasing is performed. This
* standalone query does not bind a later dispatch; use {@link prepareCall}
* when logging and streaming must share one adapter registration.
* @param config - provider/model route and optional request controls.
* @param signal - optional cancellation for adapter-owned capability lookup.
* @returns a detached config only when a default must be materialized.
*/
async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig> {
return this.resolveCallConfigFor(this.registration(config.provider), config, signal)
}
private async resolveCallConfigFor(
registration: AdapterRegistration,
config: LlmCallConfig,
signal?: AbortSignal,
): Promise<LlmCallConfig> {
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
const requested = config.reasoningEffort
if (reasoning === undefined) {
if (requested !== undefined) {
throw new LlmError(
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${requested}"`,
'UNSUPPORTED_REASONING_EFFORT',
)
}
return config
}
const effective = requested ?? reasoning.defaultEffort
if (effective === undefined) return config
if (!reasoning.efforts.some(effort => effort.id === effective)) {
throw new LlmError(
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
'UNSUPPORTED_REASONING_EFFORT',
)
}
return requested === effective ? config : { ...config, reasoningEffort: effective }
}
/**
* Resolve one call under its current adapter registration. The returned
* one-shot handle keeps that registration across header logging and dispatch,
* so HMR cannot combine one adapter's capability result with another adapter.
* @param config - provider/model route and optional request controls.
* @param signal - optional cancellation for adapter-owned capability lookup.
* @returns a prepared config and its registration-bound stream entry point.
*/
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
const registration = this.registration(config.provider)
const resolvedConfig = deepFreeze(structuredClone(
await this.resolveCallConfigFor(registration, config, signal),
))
let dispatched = false
return Object.freeze({
config: resolvedConfig,
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
if (dispatched) {
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
}
dispatched = true
return this.streamWithRegistration(options, { registration, config: resolvedConfig })
},
})
}
private registration(provider: string): AdapterRegistration {
const registration = this.adapters.get(provider)
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
return registration
@@ -295,11 +454,27 @@ export class LlmService extends Service {
private async * adapterStream(
options: GenerateOptions,
failures: AdapterFailureScope,
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
): AsyncGenerator<StreamChunk> {
let iterator: AsyncIterator<StreamChunk>
try {
const adapter = this.registration(options.provider).adapter
const stream = adapter.stream(this.forAdapter(options, adapter))
const registration = prepared?.registration ?? this.registration(options.provider)
const resolvedConfig = prepared === undefined
? await this.resolveCallConfigFor(registration, options, options.signal)
: prepared.config
if (prepared !== undefined && !callConfigEquals(options, resolvedConfig)) {
throw new LlmError(
'prepared LLM call config changed before adapter dispatch',
'INVALID_PREPARED_CALL',
)
}
const resolvedOptions = prepared !== undefined || callConfigEquals(options, resolvedConfig)
? options
: Object.isFrozen(options)
? deepFreeze({ ...options, ...resolvedConfig })
: { ...options, ...resolvedConfig }
const adapter = registration.adapter
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
iterator = stream[Symbol.asyncIterator]()
} catch (error: unknown) {
throw markLlmAdapterFailure(failures, error)
@@ -339,18 +514,36 @@ export class LlmService extends Service {
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
* `options.provider`. Replay state is retained only when the same adapter
* instance owns its historical provider and the target provider. Final
* adapter selection, dispatch, and iteration failures retain their original
* Error identity and are tagged in a call-local scope for narrow agent-loop
* request recovery; middleware and nested-call failures remain untagged for
* the outer call.
* adapter selection remains fixed through asynchronous exact-model resolution
* and dispatch. Selection, dispatch, and iteration failures retain their
* original Error identity and are tagged in a call-local scope for narrow
* agent-loop request recovery; middleware and nested-call failures remain
* untagged for the outer call.
* @param options - the full request; `options.provider` selects the adapter.
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
*/
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
return this.streamWithRegistration(options)
}
private streamWithRegistration(
options: GenerateOptions,
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
): AsyncIterable<StreamChunk> {
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
const stream = this.ctx.waterfall(
this,
'llm/stream',
options,
() => this.adapterStream(options, failures, prepared),
)
return bindAdapterFailureScope(stream, failures)
}
}
interface AdapterRegistration {
readonly adapter: LlmAdapter
readonly provider: LlmProviderInfo
}
export default LlmService

View File

@@ -5,7 +5,7 @@
*/
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId, ProviderRequestId } from './brand.ts'
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
export interface LlmFailure {
@@ -161,6 +161,35 @@ export interface LlmModelContext {
contextWindow: number
}
/** Display metadata for one adapter-owned reasoning effort. */
export interface LlmReasoningEffortInfo {
/** Opaque stable value accepted by {@link GenerateOptions.reasoningEffort}. */
id: ReasoningEffortId
/** Human-readable effort name for selectors and diagnostics. */
name: string
/** Optional user-facing distinction from otherwise similar efforts. */
description?: string
}
/** Selectable reasoning efforts for one exact provider/model route. */
export interface LlmModelReasoningInfo {
/** Supported efforts in adapter-preferred display order. */
efforts: readonly LlmReasoningEffortInfo[]
/**
* Adapter-configured default materialized into requests when callers omit
* an effort. Absence preserves the provider's own default.
*/
defaultEffort?: ReasoningEffortId
}
/** Exact-route model metadata resolved by its owning adapter. */
export interface LlmResolvedModelInfo extends LlmModelInfo {
/** Provider-owned context capacity when known. */
context?: LlmModelContext
/** Adapter-owned selectable reasoning levels when exposed. */
reasoning?: LlmModelReasoningInfo
}
/**
* Raw streaming protocol emitted by adapters.
* Block indexes correlate interleaved deltas, and `block-end` carries the
@@ -201,6 +230,8 @@ export interface GenerateOptions {
/** Registered provider route selecting the adapter instance. */
provider: string
model: string
/** Adapter-owned reasoning effort selected for this exact model. */
reasoningEffort?: ReasoningEffortId
/**
* Ordered conversation messages, exactly as the provider sees them (after
* the `system` slot). A loop-built request assembles them as

View File

@@ -6,6 +6,7 @@
import { describe, expect, it } from 'vitest'
import { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from '../src/call-config.ts'
import { ReasoningEffortId } from '../src/brand.ts'
import type { GenerateOptions } from '../src/types.ts'
describe('callConfigEquals', () => {
@@ -14,6 +15,11 @@ describe('callConfigEquals', () => {
expect(callConfigEquals(base, base)).toBe(true)
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
expect(callConfigEquals({ ...base, reasoningEffort: ReasoningEffortId('high') }, base)).toBe(false)
expect(callConfigEquals(
{ ...base, reasoningEffort: ReasoningEffortId('high') },
{ ...base, reasoningEffort: ReasoningEffortId('high') },
)).toBe(true)
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)

View File

@@ -11,9 +11,16 @@ import LlmService, {
LlmError,
llmFailureOf,
ProviderRequestId,
ReasoningEffortId,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
import type {
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
} from '@deepseek-ai/dsh-llm'
class ScriptedAdapter extends LlmAdapter {
constructor(private script: StreamChunk[]) {
@@ -49,6 +56,7 @@ class CatalogAdapter extends ScriptedAdapter {
private readonly provider: LlmProviderInfo,
private readonly models: readonly LlmModelInfo[],
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
private readonly reasoning: Readonly<Record<string, LlmModelReasoningInfo>> = {},
) {
super(SCRIPT)
}
@@ -61,11 +69,17 @@ class CatalogAdapter extends ScriptedAdapter {
return Promise.resolve(this.models)
}
override resolveModelContext(
_provider: string,
override resolveModel(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(this.contexts[model])
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
})
}
}
@@ -739,8 +753,32 @@ describe('LlmService', () => {
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
provider: 'plain', id: 'unlisted', name: 'unlisted',
})
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
})
it.each([
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new class extends ScriptedAdapter {
override resolveModel(): Promise<LlmResolvedModelInfo> {
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
}
}(SCRIPT)
ctx.llm.registerAdapter(['route'], adapter)
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
})
it('resolves detached model context independently of advisory catalog membership', async () => {
@@ -753,11 +791,241 @@ describe('LlmService', () => {
{ unlisted: source },
))
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
expect(resolved).toEqual({ contextWindow: 32_000 })
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
expect(resolved.context).toEqual({ contextWindow: 32_000 })
source.contextWindow = 64_000
expect(resolved).toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
expect(resolved.context).toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
provider: 'route', id: 'other', name: 'other',
})
})
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const source = {
efforts: [
{ id: ReasoningEffortId('standard'), name: 'Standard' },
{ id: ReasoningEffortId('ultra'), name: 'Ultra', description: 'Largest budget' },
],
defaultEffort: ReasoningEffortId('standard'),
}
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: source },
))
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
expect(resolved.reasoning).toEqual(source)
source.efforts[0]!.name = 'mutated'
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
provider: 'route',
model: 'model',
reasoningEffort: ReasoningEffortId('standard'),
})
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
})
it.each([
[{ efforts: [] }, 'empty effort list'],
[{ efforts: [{ id: '', name: 'Empty' }] }, 'empty id'],
[{ efforts: [{ id: 'valid', name: '' }] }, 'empty name'],
[{ efforts: [{ id: 'valid', name: 'Valid', description: 1 }] }, 'non-string description'],
[{ efforts: [{ id: 'same', name: 'One' }, { id: 'same', name: 'Two' }] }, 'duplicate id'],
[{ efforts: [{ id: 'valid', name: 'Valid' }], defaultEffort: 'other' }, 'unknown default'],
] as const)('rejects invalid model reasoning metadata (%s: %s)', async (metadata, _label) => {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: metadata as unknown as LlmModelReasoningInfo },
))
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
})
it('rejects unsupported reasoning efforts without clamping', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: { efforts: [{ id: ReasoningEffortId('ultra'), name: 'Ultra' }] } },
))
await expect(ctx.llm.resolveCallConfig({
provider: 'route',
model: 'model',
reasoningEffort: ReasoningEffortId('standard'),
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
await expect(ctx.llm.resolveCallConfig({
provider: 'route',
model: 'plain',
reasoningEffort: ReasoningEffortId('standard'),
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
})
it('resolves reasoning defaults at the final adapter boundary after routing middleware', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new class extends RecordingAdapter {
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
const reasoning: LlmModelReasoningInfo = {
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
defaultEffort: ReasoningEffortId('standard'),
}
return Promise.resolve({
provider,
id: model,
name: model,
reasoning,
})
}
}(SCRIPT)
ctx.llm.registerAdapter(['routed'], adapter)
const disposeRouting = ctx.on('llm/stream', (options, next) => {
options.provider = 'routed'
return next()
})
for await (const _chunk of ctx.llm.stream({
provider: 'initial',
model: 'model',
messages: [],
})) { /* drain */ }
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
disposeRouting()
const frozenRequest: GenerateOptions = Object.freeze({
provider: 'routed',
model: 'model',
messages: [],
})
for await (const _chunk of ctx.llm.stream(frozenRequest)) { /* drain */ }
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
})
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const started = Promise.withResolvers<undefined>()
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
const first = new class extends RecordingAdapter {
override async resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
started.resolve(undefined)
return {
provider,
id: model,
name: model,
reasoning: await reasoning.promise,
}
}
}(SCRIPT)
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
const draining = (async () => {
for await (const _chunk of ctx.llm.stream({
provider: 'route',
model: 'model',
messages: [],
})) { /* drain */ }
})()
await started.promise
disposeFirst()
const second = new RecordingAdapter(SCRIPT)
ctx.llm.registerAdapter(['route'], second)
reasoning.resolve({
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
defaultEffort: ReasoningEffortId('high'),
})
await draining
expect(first.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('high'))
expect(second.lastOptions).toBeUndefined()
})
it('prepares a one-shot registration-bound call and rejects config drift', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{
model: {
efforts: [{ id: ReasoningEffortId('high'), name: 'High' }],
defaultEffort: ReasoningEffortId('high'),
},
},
)
ctx.llm.registerAdapter(['route'], adapter)
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
expect(Object.isFrozen(prepared.config)).toBe(true)
const stream = prepared.stream({
...prepared.config,
model: 'other',
messages: [],
})
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toMatchObject({ code: 'INVALID_PREPARED_CALL' })
expect(() => prepared.stream({
...prepared.config,
messages: [],
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
})
it('passes cancellation through exact-model resolution', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const started = Promise.withResolvers<undefined>()
const adapter = new class extends ScriptedAdapter {
override resolveModel(
_provider: string,
_model: string,
signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
started.resolve(undefined)
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
if (signal === undefined) {
reject(new Error('missing reasoning signal'))
return
}
if (signal.aborted) {
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
return
}
signal.addEventListener('abort', () => {
reject(signal.reason instanceof Error ? signal.reason : new Error('reasoning aborted'))
}, { once: true })
})
}
}(SCRIPT)
ctx.llm.registerAdapter(['route'], adapter)
const controller = new AbortController()
const resolving = ctx.llm.resolveCallConfig(
{ provider: 'route', model: 'model' },
controller.signal,
)
await started.promise
const reason = new Error('cancel reasoning')
controller.abort(reason)
await expect(resolving).rejects.toBe(reason)
})
it.each([0, -1, 1.5, Number.NaN])(
@@ -770,7 +1038,7 @@ describe('LlmService', () => {
[],
{ model: { contextWindow } },
))
await expect(ctx.llm.resolveModelContext('route', 'model'))
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
},
)