fix(agent-loop): rematerialize adapter defaults
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 5a22689d0b15ae5de1b82e37cf8c2c283c14af97
|
||||
README.zh.md: 0fa8b16eee72eee741b07d27b1ed61297f5420c2
|
||||
README.md: 19bc84146c9b03a6ed039a7bbe9e60ebecf50838
|
||||
README.zh.md: 80772d4c06a426318fe5ddff5c997fc9f2e75129
|
||||
|
||||
@@ -32,14 +32,14 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
name: DeepSeek-V4-Flash
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
contextWindow: 512000
|
||||
```
|
||||
|
||||
The plugin registers the single provider route `deepseek` together with its resolved `retryPolicy`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` as `DeepSeek-V4-Flash` and `deepseek-v4-pro` as `DeepSeek-V4-Pro`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. The adapter default is 1,000,000; pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
`maxTokens` is the adapter-configured output cap for conversation requests and defaults to 256,000. Exact-model resolution exposes it as `defaultMaxTokens`; `LlmService` materializes that value into `GenerateOptions.maxTokens` before the agent loop writes `request/header`, so the wire request remains reconstructable. An explicit request or `AgentOptions.maxTokens` value wins and is serialized as `max_tokens`.
|
||||
`maxTokens` is the adapter-configured output cap for conversation requests and defaults to 256,000. Exact-model resolution exposes it as `defaultMaxTokens`; `LlmService` materializes that value into `GenerateOptions.maxTokens` before the agent loop writes `request/header`, so the wire request remains reconstructable. An explicit request or `AgentOptions.maxTokens` value wins and is serialized as `max_tokens`. The adapter does not clamp this request budget against `contextWindow`; deployments with a smaller context or provider output limit must configure a compatible `maxTokens`.
|
||||
|
||||
The same exact-model result exposes ordered `off`, `high`, and `max` efforts under `reasoning` for every pass-through model when deployment policy permits thinking. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header`. `high` and `max` enable thinking and serialize as the official top-level `reasoning_effort`; adapter-owned `off` instead serializes `thinking.type: disabled` and omits `reasoning_effort`. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O.
|
||||
|
||||
|
||||
@@ -32,14 +32,14 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
|
||||
name: DeepSeek-V4-Flash
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
contextWindow: 512000
|
||||
```
|
||||
|
||||
该插件注册唯一提供方路由 `deepseek`，同时注册解析后的 `retryPolicy`。请求使用 `provider: deepseek` 选择该路由；其 `model` 会作为协议 `model` 字符串原样传递，因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`（名称为 `DeepSeek-V4-Flash`）和 `deepseek-v4-pro`（名称为 `DeepSeek-V4-Pro`），两者的上下文窗口均为 1,000,000 token；显式列表会替换这些默认值，`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek')` 公开给 ACP（Agent Client Protocol）编辑器和 Web 选择器等客户端，但仍只提供建议：未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。
|
||||
|
||||
`contextWindow` 对每个已配置模型都可选，不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek', model).context` 先返回精确模型值，再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。适配器默认值为 1,000,000；因此，压力敏感插件可以获得由部署决定的容量，不会将模型 selector 视为权威。为 `deepseek` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
`maxTokens` 是适配器为对话请求配置的输出上限，默认值为 256,000。确切模型解析会将其公开为 `defaultMaxTokens`；`LlmService` 会在 agent loop（智能体循环）写入 `request/header` 前，将该值填入 `GenerateOptions.maxTokens`，从而仍可根据持久记录重建协议请求。显式的请求值或 `AgentOptions.maxTokens` 值优先，并会序列化为 `max_tokens`。
|
||||
`maxTokens` 是适配器为对话请求配置的输出上限，默认值为 256,000。确切模型解析会将其公开为 `defaultMaxTokens`；`LlmService` 会在 agent loop（智能体循环）写入 `request/header` 前，将该值填入 `GenerateOptions.maxTokens`，从而仍可根据持久记录重建协议请求。显式的请求值或 `AgentOptions.maxTokens` 值优先，并会序列化为 `max_tokens`。适配器不会根据 `contextWindow` 自动调低该请求预算；上下文或提供方输出上限较小的部署必须配置与其相容的 `maxTokens`。
|
||||
|
||||
同一确切模型结果会在部署策略允许思考时，为每个原样传递模型在 `reasoning` 下公开有序的 `off`、`high` 和 `max` 推理（reasoning）强度。`reasoningEffort` 选择部署默认值，省略时回退为 `high`。`agent/request` 可以在每个会话步骤替换它；解析后的值会记录在 `request/header`。`high` 和 `max` 会启用思考，并序列化为官方顶层 `reasoning_effort`；适配器持有的 `off` 则序列化为 `thinking.type: disabled`，且省略 `reasoning_effort`。不支持的值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
|
||||
@@ -244,7 +244,7 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
}
|
||||
|
||||
private async * request(options: GenerateOptions, signal: AbortSignal): AsyncIterable<StreamChunk> {
|
||||
const body = serializeRequest(options, this.options.defaults, this.maxTokens)
|
||||
const body = serializeRequest(options, this.options.defaults)
|
||||
// Prepared outside the try so the TRANSPORT label below covers exactly the
|
||||
// transport boundary, never a serialization failure.
|
||||
const payload = JSON.stringify(body)
|
||||
|
||||
@@ -137,13 +137,11 @@ export function serializeMessages(messages: Message[]): WireMessage[] {
|
||||
* provider defaults apply.
|
||||
* @param options - the harness request (model, history, system, tools, sampling).
|
||||
* @param defaults - adapter-level thinking defaults; undefined fields put nothing on the wire.
|
||||
* @param defaultMaxTokens - adapter output default used only when the request omits a cap.
|
||||
* @returns the chat-completions request body.
|
||||
*/
|
||||
export function serializeRequest(
|
||||
options: GenerateOptions,
|
||||
defaults: RequestDefaults = {},
|
||||
defaultMaxTokens?: number,
|
||||
): WireRequest {
|
||||
const messages: WireMessage[] = []
|
||||
if (options.system !== undefined) {
|
||||
@@ -162,7 +160,6 @@ export function serializeRequest(
|
||||
// A short title budget must produce visible text; conversation and
|
||||
// compaction calls continue to inherit the adapter's thinking defaults.
|
||||
const resolvedThinking = resolveThinking(options, defaults)
|
||||
const maxTokens = options.maxTokens ?? defaultMaxTokens
|
||||
|
||||
return {
|
||||
model: options.model,
|
||||
@@ -175,7 +172,7 @@ export function serializeRequest(
|
||||
: {},
|
||||
...tools !== undefined && tools.length > 0 ? { tools } : {},
|
||||
...options.temperature !== undefined ? { temperature: options.temperature } : {},
|
||||
...maxTokens === undefined ? {} : { max_tokens: maxTokens },
|
||||
...options.maxTokens === undefined ? {} : { max_tokens: options.maxTokens },
|
||||
...options.stop !== undefined ? { stop: options.stop } : {},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -170,13 +170,6 @@ describe('serializeRequest', () => {
|
||||
expect(wire.stop).toEqual(['END'])
|
||||
})
|
||||
|
||||
it('uses the adapter maxTokens default only when the request omits a cap', () => {
|
||||
expect(serializeRequest(request({ messages: history }), {}, 256_000).max_tokens)
|
||||
.toBe(256_000)
|
||||
expect(serializeRequest(request({ messages: history, maxTokens: 8_192 }), {}, 256_000).max_tokens)
|
||||
.toBe(8_192)
|
||||
})
|
||||
|
||||
it('maps tools to the wire function shape', () => {
|
||||
const wire = serializeRequest(request({
|
||||
messages: history,
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 3ffcfb59fa7d3e3077a59d0b0c8ebd9afa590e7e
|
||||
README.zh.md: df44b2bad1d5fa5c03dff86de584bfff63dbf297
|
||||
README.md: 2dbd530ca17ef34787cb4195c04ca85d768980b7
|
||||
README.zh.md: 9928113f5cbfc49887980fde57ad4ee9f37dbd22
|
||||
|
||||
@@ -25,7 +25,7 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
|
||||
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context`, `defaultMaxTokens`, or `reasoning` fields preserve unknown capacity, provider-owned output defaults, or unavailable reasoning capability. Invalid identity, context, output default, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, `INVALID_MODEL_MAX_TOKENS`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
`defaultMaxTokens` is an adapter-configured per-request output cap, not a model hard limit. `resolveCallConfig()` materializes it only when the request omits `maxTokens`; an explicit cap wins. Reasoning identifiers are opaque adapter-owned strings rather than a core enum: the same resolution accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
`defaultMaxTokens` is an adapter-configured per-request output cap, not a model hard limit. `resolveCallConfig()` materializes it only when the request omits `maxTokens`; an explicit cap wins. Reasoning identifiers are opaque adapter-owned strings rather than a core enum: the same resolution accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally reports which `maxTokens` and `reasoningEffort` fields it materialized in `adapterDefaults` and retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -48,7 +48,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates it and materializes adapter defaults under the turn signal, and the loop logs the effective value before using the prepared call's registration-bound stream. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `prepareCall()` validates it and materializes adapter defaults under the turn signal, and the loop logs the effective value plus adapter-default provenance before using the prepared call's registration-bound stream. The next proposal omits marked defaults so a changed route resolves its own values; unmarked explicit fields persist. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
|
||||
确切模型元数据是独立的正确性查询，不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方／模型路由的适配器查询一次；适配器可以描述未列出的动态模型，缺少 `context`、`defaultMaxTokens` 或 `reasoning` 字段会分别保留未知容量、提供方持有的输出默认值或不可用的推理能力。无效的身份、上下文、输出默认值或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT`、`INVALID_MODEL_MAX_TOKENS` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
`defaultMaxTokens` 是适配器配置的单次请求输出上限，不是模型硬上限。仅当请求省略 `maxTokens` 时，`resolveCallConfig()` 才会填入该值；显式上限优先。推理标识符是由适配器持有的不透明字符串，而非核心枚举：同一次解析只接受与已公布标识符完全一致的值，在存在 `defaultEffort` 时填入它，否则保留提供方默认值。异步模型解析器会接收调用方的 signal，并且必须在取消后迅速结束。`prepareCall()` 还会让精确适配器注册跨越请求头记录和最终分派，因此 HMR（热模块替换）不会将一个适配器的能力结果与另一个适配器的请求混用；复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
`defaultMaxTokens` 是适配器配置的单次请求输出上限，不是模型硬上限。仅当请求省略 `maxTokens` 时，`resolveCallConfig()` 才会填入该值；显式上限优先。推理标识符是由适配器持有的不透明字符串，而非核心枚举：同一次解析只接受与已公布标识符完全一致的值，在存在 `defaultEffort` 时填入它，否则保留提供方默认值。异步模型解析器会接收调用方的 signal，并且必须在取消后迅速结束。`prepareCall()` 还会通过 `adapterDefaults` 报告它填入了哪些 `maxTokens` 和 `reasoningEffort` 字段，并让精确适配器注册跨越请求头记录和最终分派，因此 HMR（热模块替换）不会将一个适配器的能力结果与另一个适配器的请求混用；复用其一次性句柄或更改调用配置字段会以 `INVALID_PREPARED_CALL` 失败。不支持的显式或配置推理强度会在提供方 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
### 事件
|
||||
|
||||
@@ -48,7 +48,7 @@
|
||||
|
||||
### 调用配置（`call-config.ts`）
|
||||
|
||||
`LlmCallConfig` 是一个会话中各次请求的提供方、模型、可选的适配器持有推理强度和采样标量（`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`，每个都与同名 `GenerateOptions` 字段 1:1 映射）。它是作为请求标头一部分记录在会话日志中的每会话状态（见 dsh-session `request/header` 事件），绝不是可静默调整的每次调用旋钮：`agent/request` waterfall 会提议替换，`prepareCall()` 在轮次 signal 控制下校验它并填入适配器默认值，loop 随后记录生效值，再使用已准备调用中与注册绑定的流。`callConfigEquals(a, b)` 是逐字段真实变更检测器；`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper（`llm/stream` listener 与适配器只读，绝不改写）。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源，`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类，让适配器可以应用目的特定传输策略，而不改变普通会话请求。
|
||||
`LlmCallConfig` 是一个会话中各次请求的提供方、模型、可选的适配器持有推理强度和采样标量（`provider`、`model`、`reasoningEffort`、`temperature`、`maxTokens`、`stop`，每个都与同名 `GenerateOptions` 字段 1:1 映射）。它是作为请求标头一部分记录在会话日志中的每会话状态（见 dsh-session `request/header` 事件），绝不是可静默调整的每次调用旋钮：`agent/request` waterfall 会提议替换，`prepareCall()` 在轮次 signal 控制下校验它并填入适配器默认值，loop 随后记录生效值及适配器默认值来源，再使用已准备调用中与注册绑定的流。下一次提议会省略带标记的默认值，使变更后的路由解析自身的值；未带标记的显式字段会保留。`callConfigEquals(a, b)` 是逐字段真实变更检测器；`deepFreeze(value)` 是 loop 在 dispatch 前对每个已构建请求应用的所有权 helper（`llm/stream` listener 与适配器只读，绝不改写）。`markAgentLoopRequest()` 为该精确对象添加进程本地 loop 溯源，`isAgentLoopRequest()` 让观测方可以将其与同样可能冻结并关联会话、但独立记录的辅助调用区分。`GenerateOptions.purpose` 对已记录辅助压缩与会话标题调用分类，让适配器可以应用目的特定传输策略，而不改变普通会话请求。
|
||||
|
||||
### 应用归因（`attribution.ts`）
|
||||
|
||||
|
||||
@@ -27,6 +27,15 @@ export interface LlmCallConfig {
|
||||
stop?: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Effective config fields supplied by exact-model adapter resolution rather
|
||||
* than by the caller's request proposal.
|
||||
*/
|
||||
export interface LlmCallConfigAdapterDefaults {
|
||||
reasoningEffort?: true
|
||||
maxTokens?: true
|
||||
}
|
||||
|
||||
/**
|
||||
* Field-wise equality over {@link LlmCallConfig} — the comparison a caller
|
||||
* runs to decide whether a proposed configuration is a real change (worth a
|
||||
|
||||
@@ -20,7 +20,7 @@ import { resolveRetryPolicy } from './retry-policy.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
import type { LlmCallConfig } from './call-config.ts'
|
||||
import type { LlmCallConfig, LlmCallConfigAdapterDefaults } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
import type { AdapterFailureScope } from './adapter-failure.ts'
|
||||
@@ -34,7 +34,7 @@ export * from './message.ts'
|
||||
export * from './retry-policy.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export type { LlmCallConfig, LlmCallConfigAdapterDefaults } from './call-config.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf, llmRetryPolicyOf } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
@@ -113,6 +113,8 @@ export class LlmError extends HarnessError {
|
||||
export interface PreparedLlmCall {
|
||||
/** Detached, deep-frozen config with any adapter-owned default materialized. */
|
||||
readonly config: LlmCallConfig
|
||||
/** Config fields materialized by the captured adapter rather than proposed by the caller. */
|
||||
readonly adapterDefaults: LlmCallConfigAdapterDefaults
|
||||
/**
|
||||
* Dispatch this call once through the registration captured during
|
||||
* preparation. The request's call-config fields must match {@link config};
|
||||
@@ -447,12 +449,20 @@ export class LlmService extends Service {
|
||||
*/
|
||||
async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall> {
|
||||
const registration = this.registration(config.provider)
|
||||
const resolvedConfig = deepFreeze(structuredClone(
|
||||
await this.resolveCallConfigFor(registration, config, signal),
|
||||
))
|
||||
const resolved = await this.resolveCallConfigFor(registration, config, signal)
|
||||
const resolvedConfig = deepFreeze(structuredClone(resolved))
|
||||
const adapterDefaults = deepFreeze<LlmCallConfigAdapterDefaults>({
|
||||
...config.reasoningEffort === undefined && resolved.reasoningEffort !== undefined
|
||||
? { reasoningEffort: true }
|
||||
: {},
|
||||
...config.maxTokens === undefined && resolved.maxTokens !== undefined
|
||||
? { maxTokens: true }
|
||||
: {},
|
||||
})
|
||||
let dispatched = false
|
||||
return Object.freeze({
|
||||
config: resolvedConfig,
|
||||
adapterDefaults,
|
||||
stream: (options: GenerateOptions): AsyncIterable<StreamChunk> => {
|
||||
if (dispatched) {
|
||||
throw new LlmError('a prepared LLM call can only be dispatched once', 'INVALID_PREPARED_CALL')
|
||||
|
||||
@@ -906,7 +906,12 @@ describe('LlmService', () => {
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{},
|
||||
{ model: source },
|
||||
{
|
||||
model: source,
|
||||
providerDefault: {
|
||||
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
|
||||
},
|
||||
},
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
|
||||
@@ -920,6 +925,8 @@ describe('LlmService', () => {
|
||||
})
|
||||
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
|
||||
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
|
||||
const providerDefault = { provider: 'route', model: 'providerDefault' }
|
||||
await expect(ctx.llm.resolveCallConfig(providerDefault)).resolves.toBe(providerDefault)
|
||||
})
|
||||
|
||||
it('materializes an adapter-owned maxTokens default while preserving an explicit cap', async () => {
|
||||
@@ -941,8 +948,12 @@ describe('LlmService', () => {
|
||||
model: 'model',
|
||||
maxTokens: 256_000,
|
||||
})
|
||||
const preparedDefault = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(preparedDefault.adapterDefaults).toEqual({ maxTokens: true })
|
||||
const explicit = { provider: 'route', model: 'model', maxTokens: 8_192 }
|
||||
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
|
||||
const preparedExplicit = await ctx.llm.prepareCall(explicit)
|
||||
expect(preparedExplicit.adapterDefaults).toEqual({})
|
||||
})
|
||||
|
||||
it.each([0, 1.5, Number.MAX_SAFE_INTEGER + 1])(
|
||||
@@ -1106,6 +1117,8 @@ describe('LlmService', () => {
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
const prepared = await ctx.llm.prepareCall({ provider: 'route', model: 'model' })
|
||||
expect(Object.isFrozen(prepared.config)).toBe(true)
|
||||
expect(Object.isFrozen(prepared.adapterDefaults)).toBe(true)
|
||||
expect(prepared.adapterDefaults).toEqual({ reasoningEffort: true })
|
||||
const stream = prepared.stream({
|
||||
...prepared.config,
|
||||
model: 'other',
|
||||
|
||||
Reference in New Issue
Block a user