diff --git a/packages/llm/llm-pi-ai/README.i18n.yaml b/packages/llm/llm-pi-ai/README.i18n.yaml index 69efba1977..c8ae1899bd 100644 --- a/packages/llm/llm-pi-ai/README.i18n.yaml +++ b/packages/llm/llm-pi-ai/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md -README.md: 894aecc720f0a7616c0127d439b41129d94ef667 -README.zh.md: 63464f80ee3036ddec3fb6828ecccc68c5524478 +README.md: f208f553ab3a1f80c5b71f4792e5fc80459f9fa5 +README.zh.md: 24ae4b0e2021eeea373eacb8cc1dfc39063fee8b diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 894aecc720..f208f553ab 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -35,6 +35,15 @@ Configure credentials, the model catalog, and deployment-specific transport sett models: - id: claude-sonnet-4-5 contextWindow: 200000 + # Catalog route with one model reshaped in place; the rest of the + # catalog keeps serving (a models list would replace it instead). + deepseek: + apiKeyEnv: DEEPSEEK_API_KEY + modelOverrides: + deepseek-v4-pro: + reasoningEfforts: + off: + high: high # Hand-declared route: pi-ai ships nothing under this key, so the profile # supplies the whole provider. acme-gateway: @@ -68,6 +77,8 @@ The dict shape makes duplicate routes unrepresentable, and the pre-release array A profile's `models` list *replaces* the route's installed catalog rather than extending it; omitting it (or leaving it empty) serves that catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a catalog route to two models, correcting one capacity, or adding a model newer than the installed catalog are all one-line edits — but declaring any `models` list means every model the route should keep serving must appear in it, an entry of nothing but `id` being enough. The configurable entry fields are `id`, `name`, `contextWindow`, `maxTokens`, `reasoningEfforts`, and `compat`. Pricing and input modalities have no harness consumer and ride the installed entry or are absent. +`modelOverrides` reshapes individual installed-catalog models without that cost: each key is a catalog model id, each value the same fields a `models` entry takes with the id living in the key, and the rest of the catalog keeps serving untouched — "correct one model, keep the other thirty-seven" as a three-line edit. An override becomes that catalog entry's configuration, so capacities, efforts, and compat resolve through the same path with the same diagnostics and the same request-default semantics as a `models` entry. Overrides are only meaningful on a catalog route serving its catalog: one set beside a `models` list (which already replaces the catalog), on a hand-declared route (whose models are fully spelled in `models`), or naming a model the catalog does not describe is refused rather than skipped, because a silently unchanged model is a typo someone would otherwise hunt for. + ### Per-model reasoning efforts `reasoningEfforts` declares a model's selectable thinking levels: each key is a level selectors offer, its value the spelling dispatch sends on the wire, so `high: high` passes the canonical name through while `max: ultra` renames it for a gateway with its own vocabulary. Keys come from pi-ai's level set (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`); a level not declared is not offered. Omitting the field keeps the installed catalog entry's capability (a hand-declared model has none and does not reason); `false` declares a non-reasoning model, which is how a profile strips reasoning from a catalog model its gateway cannot serve; an empty declaration is refused rather than guessing between those two meanings. @@ -98,7 +109,7 @@ A model that carries reasoning metadata — from the installed catalog or from i A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`. -Supported profile fields are `apiKey`, `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. +Supported profile fields are `apiKey`, `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`. diff --git a/packages/llm/llm-pi-ai/README.zh.md b/packages/llm/llm-pi-ai/README.zh.md index 63464f80ee..24ae4b0e20 100644 --- a/packages/llm/llm-pi-ai/README.zh.md +++ b/packages/llm/llm-pi-ai/README.zh.md @@ -35,6 +35,15 @@ models: - id: claude-sonnet-4-5 contextWindow: 200000 + # Catalog route with one model reshaped in place; the rest of the + # catalog keeps serving (a models list would replace it instead). + deepseek: + apiKeyEnv: DEEPSEEK_API_KEY + modelOverrides: + deepseek-v4-pro: + reasoningEfforts: + off: + high: high # Hand-declared route: pi-ai ships nothing under this key, so the profile # supplies the whole provider. acme-gateway: @@ -68,6 +77,8 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩充它；省略它（或留空）则原样服务该 catalog。每个条目都会从同 `id` 的已安装模型继承自身未设置的字段，因此把 catalog 路由收窄到两个模型、更正某个容量，或加入一个比已安装 catalog 更新的模型，都是一行编辑——但一旦声明了 `models` 列表，该路由要继续服务的每个模型就都必须出现在其中，条目哪怕只写一个 `id` 也足够。可配置的条目字段是 `id`、`name`、`contextWindow`、`maxTokens`、`reasoningEfforts` 与 `compat`。定价与输入模态没有 harness 消费方，因此沿用已安装条目或直接缺席。 +`modelOverrides` 无需这份代价就能就地重塑单个已安装 catalog 模型：每个键是一个 catalog 模型 id，每个值可写 `models` 条目接受的同一批字段，只是 id 落在键上，而 catalog 的其余部分原样继续服务——「改一个模型、其余三十七个原样保留」只是一次三行编辑。一条覆盖会成为该 catalog 条目的配置，因此容量、档位与 compat 沿与 `models` 条目相同的路径解析，携带相同的诊断与相同的请求默认值语义。覆盖只在正服务自身 catalog 的 catalog 路由上才有意义：与 `models` 列表并存的一份（该列表本就替换了 catalog）、落在手工声明路由上的一份（其模型已在 `models` 中完整写出），或点名了 catalog 未描述模型的一份，都会被拒绝而非跳过，因为一个静默保持原样的模型，就是一个否则要有人费力追查的笔误。 + ### 按模型的推理档位 `reasoningEfforts` 声明模型可选的思考级别：每个键是选择器提供的一个档位，其值是分派在协议中发送的拼写，因此 `high: high` 原样透传规范名称，而 `max: ultra` 则为使用自有词汇的网关改名。键取自 pi-ai 的档位集合（`off`、`minimal`、`low`、`medium`、`high`、`xhigh`、`max`）；未声明的档位不会被提供。省略该字段会保留已安装 catalog 条目的能力（手工声明的模型没有这份能力，也不推理）；`false` 声明一个不具备推理能力的模型，profile 正是以此从其网关无法服务的 catalog 模型上剥除推理；空声明会被拒绝，而不是在这两种含义之间去猜。 @@ -98,7 +109,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩 **没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型，以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档，但 `off` 会被翻译成*省略* reasoning 选项，而那与「不点名任何档位」产出的请求逐字节相同：选它关不掉任何东西，于是自身默认就在思考的提供方，会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用，界面就只剩提供方默认这一项，不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。**描述**一个模型则从不这样失败：同一提供方下各模型接受的档位并不一致，因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」，而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处，而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。 -受支持的 profile 字段是 `apiKey`、`apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。 +受支持的 profile 字段是 `apiKey`、`apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。 适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。 diff --git a/packages/llm/llm-pi-ai/src/catalog.ts b/packages/llm/llm-pi-ai/src/catalog.ts index e3c9207927..3285d1595a 100644 --- a/packages/llm/llm-pi-ai/src/catalog.ts +++ b/packages/llm/llm-pi-ai/src/catalog.ts @@ -184,6 +184,15 @@ export interface PiAiModelProfile { compat?: PiAiCompatProfile } +/** + * Customization of one installed catalog model, keyed by its id in the + * route's `modelOverrides` dict — the same fields a `models` entry may set, + * with the id living in the key. Unlike a `models` list, overrides leave the + * rest of the catalog serving untouched, which is what makes "correct one + * model, keep the other thirty-seven" a three-line edit. + */ +export type PiAiModelOverride = Omit + /** The route-level facts model materialization reads. */ export interface RouteCatalogRequest { /** Provider route key, stamped onto every materialized model. */ @@ -194,6 +203,8 @@ export interface RouteCatalogRequest { baseURL?: string /** Configured catalog; absent means the whole installed catalog for this route. */ models?: readonly PiAiModelProfile[] + /** Installed-catalog customizations by model id; only meaningful while `models` is absent. */ + modelOverrides?: Readonly> /** Reasoning-dispatch switches for every `openai-completions` model on the route; entries override per field. */ compat?: PiAiCompatProfile /** Context capacity for a model neither the entry nor the catalog sizes. */ @@ -381,9 +392,35 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog { // schema materializes `[]` for the absent case, and an empty catalog could // serve no request anyway, so both mean "serve the installed catalog". const configured = request.models ?? [] + const overrides = request.modelOverrides ?? {} + // Every miss is refused, never skipped: an override that lands nowhere is a + // typo someone would otherwise hunt for in a silently unchanged model. + for (const [id, override] of Object.entries(overrides)) { + if (id.length === 0) invalid(provider, 'has a modelOverrides entry with an empty model id') + if (defaults.size === 0) { + invalid(provider, `sets modelOverrides for "${id}", but the installed catalog does not describe this route;` + + ' a declared route spells every model out in its models list') + } + if (configured.length > 0) { + invalid(provider, `sets modelOverrides for "${id}" beside a models list; models already replaces the served` + + ' catalog, so declare the fields on its entries') + } + if (!defaults.has(id)) { + invalid(provider, `modelOverrides names "${id}", which the installed catalog does not describe`) + } + // The id lives in the dict key; a value carrying its own would quietly + // rename the model it meant to customize. The static shape already omits + // it — this guards the schema boundary, which passes unknown keys through. + if ('id' in override) { + invalid(provider, `modelOverrides entry "${id}" sets "id", which is the dict key`) + } + } + // An override becomes the catalog entry's configuration, so everything a + // models entry may declare — capacities, efforts, compat — resolves through + // the same path with the same diagnostics and request-default semantics. const entries: readonly PiAiModelProfile[] = configured.length > 0 ? configured - : [...defaults.values()].map(model => ({ id: model.id })) + : [...defaults.values()].map(model => ({ id: model.id, ...overrides[model.id] })) if (entries.length === 0) { invalid(provider, 'resolves no models; the installed catalog does not describe this route, so its models' + ' must be listed in configuration') diff --git a/packages/llm/llm-pi-ai/src/config.ts b/packages/llm/llm-pi-ai/src/config.ts index 9d4cca089c..d93f68bcd9 100644 --- a/packages/llm/llm-pi-ai/src/config.ts +++ b/packages/llm/llm-pi-ai/src/config.ts @@ -22,7 +22,7 @@ import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { normalizeApiKey, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm' import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm' import { resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts' -import type { PiAiCompatProfile, PiAiModelProfile, PiAiReasoningEfforts } from './catalog.ts' +import type { PiAiCompatProfile, PiAiModelOverride, PiAiModelProfile, PiAiReasoningEfforts } from './catalog.ts' import { buildProvider, supportedProtocols } from './provider.ts' /** Default maximum idle interval while an adapter stream read is outstanding. */ @@ -34,7 +34,13 @@ export const DEFAULT_CONTEXT_WINDOW = 262_144 /** Output capability assumed for a model neither configuration nor the catalog sizes. */ export const DEFAULT_MAX_TOKENS = 32_768 -export type { PiAiCompatProfile, PiAiModelProfile, PiAiReasoningEfforts, PiAiThinkingFormat } from './catalog.ts' +export type { + PiAiCompatProfile, + PiAiModelOverride, + PiAiModelProfile, + PiAiReasoningEfforts, + PiAiThinkingFormat, +} from './catalog.ts' /** Configuration for one pi-ai provider route; the `providers` dict key IS the route. */ export interface PiAiProviderProfile { @@ -62,6 +68,15 @@ export interface PiAiProviderProfile { * unset fields from the installed model of the same id. */ models?: PiAiModelProfile[] + /** + * Installed-catalog customizations by model id: each entry reshapes that + * one model with the same fields a {@link models} entry takes, while the + * rest of the catalog keeps serving untouched. Only meaningful on a catalog + * route with no `models` list — `models` already replaces the catalog, so + * an override beside it, on a route the catalog does not ship, or naming a + * model the catalog does not describe is refused rather than skipped. + */ + modelOverrides?: Record /** * Reasoning-dispatch switches for every `openai-completions` model on this * route; each model's own `compat` overrides per field. What neither sets @@ -176,6 +191,15 @@ const modelProfile: z = z.object({ compat: compatProfile, }) +/** A {@link modelProfile} whose id lives in the `modelOverrides` dict key. */ +const modelOverride: z = z.object({ + name: z.string(), + contextWindow: z.number().step(1).min(1), + maxTokens: z.number().step(1).min(1), + reasoningEfforts: z.union([z.const(false), reasoningEfforts]), + compat: compatProfile, +}) + const profile = z.object({ apiKey: z.string().role('secret'), apiKeyEnv: z.string().role('credential-ref'), @@ -183,6 +207,7 @@ const profile = z.object({ api: z.union(supportedProtocols()), baseURL: z.string(), models: z.array(modelProfile), + modelOverrides: z.dict(modelOverride), compat: compatProfile, defaultContextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW), defaultMaxTokens: z.number().step(1).min(1).default(DEFAULT_MAX_TOKENS), @@ -291,6 +316,7 @@ export function resolveProfiles( ...source.api === undefined ? {} : { api: source.api }, ...source.baseURL === undefined ? {} : { baseURL: source.baseURL }, ...source.models === undefined ? {} : { models: source.models }, + ...source.modelOverrides === undefined ? {} : { modelOverrides: source.modelOverrides }, ...source.compat === undefined ? {} : { compat: source.compat }, defaultContextWindow: source.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW, defaultMaxTokens: source.defaultMaxTokens ?? DEFAULT_MAX_TOKENS, diff --git a/packages/llm/llm-pi-ai/src/index.ts b/packages/llm/llm-pi-ai/src/index.ts index ea81f66fec..e00b9f3c2a 100644 --- a/packages/llm/llm-pi-ai/src/index.ts +++ b/packages/llm/llm-pi-ai/src/index.ts @@ -70,6 +70,7 @@ export type { PiAiAdapterOptions } from './adapter.ts' export { Config } from './config.ts' export type { PiAiCompatProfile, + PiAiModelOverride, PiAiModelProfile, PiAiProviderProfile, PiAiReasoningEfforts, diff --git a/packages/llm/llm-pi-ai/tests/catalog.spec.ts b/packages/llm/llm-pi-ai/tests/catalog.spec.ts index fbfcd653cf..de806558c5 100644 --- a/packages/llm/llm-pi-ai/tests/catalog.spec.ts +++ b/packages/llm/llm-pi-ai/tests/catalog.spec.ts @@ -573,6 +573,77 @@ describe('per-model reasoning efforts', () => { }) }) +describe('modelOverrides', () => { + const deepseekModel = (): Model => { + const [model] = getBuiltinModels('deepseek') + if (model === undefined) throw new Error('the installed catalog ships no deepseek model') + return model + } + + it('reshapes one catalog model while the rest of the catalog keeps serving', () => { + const catalogSize = getBuiltinModels('deepseek').length + const target = deepseekModel() + const resolved = resolveProfiles({ + deepseek: { + modelOverrides: { + [target.id]: { + name: 'DeepSeek (proxied)', + maxTokens: 4096, + reasoningEfforts: { off: null, high: 'high' }, + }, + }, + }, + }) + const models = resolved.get('deepseek')?.piProvider.getModels() ?? [] + const reshaped = models.find(model => model.id === target.id) + if (reshaped === undefined) throw new Error('the overridden model vanished from the route') + + // The whole catalog still serves — that is the difference from `models`, + // which replaces it. + expect(models).toHaveLength(catalogSize) + expect(reshaped.name).toBe('DeepSeek (proxied)') + expect(getSupportedThinkingLevels(reshaped)).toEqual(['off', 'high']) + // An override's cap is explicit configuration, so it becomes the request + // default exactly as a models entry's would. + expect(resolved.get('deepseek')?.configuredMaxTokens.get(target.id)).toBe(4096) + // A sibling the overrides do not name is byte-identical to the catalog. + const sibling = models.find(model => model.id !== target.id) + expect(sibling?.maxTokens).toBe(getBuiltinModels('deepseek').find(model => model.id === sibling?.id)?.maxTokens) + }) + + it('refuses every override that lands nowhere instead of skipping it', () => { + expect(() => resolveProfiles({ + deepseek: { modelOverrides: { 'no-such-model': { name: 'ghost' } } }, + })).toThrow(/which the installed catalog does not describe/) + expect(() => resolveProfiles({ + 'acme-gateway': { + api: 'openai-completions', + baseURL: 'https://acme.test', + models: [{ id: 'm' }], + modelOverrides: { m: { name: 'renamed' } }, + }, + })).toThrow(/a declared route spells every model out/) + const declaredOnly = deepseekModel() + expect(() => resolveProfiles({ + deepseek: { + models: [{ id: declaredOnly.id }], + modelOverrides: { [declaredOnly.id]: { name: 'renamed' } }, + }, + })).toThrow(/models already replaces the served catalog/) + expect(() => resolveProfiles({ + deepseek: { modelOverrides: { '': { name: 'nameless' } } }, + })).toThrow(/empty model id/) + // The dict key is the id; a value smuggling its own would quietly rename + // the model it meant to customize. The schema passes unknown keys + // through, so resolution is the boundary that refuses it — the variable + // indirection mirrors that boundary by sidestepping the literal check. + const smuggled = { name: 'x', id: 'other' } + expect(() => resolveProfiles({ + deepseek: { modelOverrides: { [deepseekModel().id]: smuggled } }, + })).toThrow(/sets "id", which is the dict key/) + }) +}) + describe('reasoning-dispatch compat switches', () => { /** The materialized models of one route, keyed by id. */ function modelsOf(providers: Record, route: string): Map> {