fix(llm-pi-ai): let a model declare the request modalities it accepts
A model the installed pi-ai catalog does not describe was reported as text-only with no way to say otherwise, so a vision model added through the custom-provider form was refused at every image admission point. The justification in the source described the DeepSeek chat-completions serializer, which does reject image blocks; the pi-ai request converter and every wire protocol it speaks carry images. Modalities now resolve entry `input` -> installed catalog entry -> route `defaultInput`, the chain the two capacity fallbacks already use, so the route value is a fallback and never narrows a catalog model. Its default stays `[text]`: nothing can interrogate a gateway for its modalities, and over-claiming admits an image the provider rejects mid-turn, after prompt admission has already committed the message.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: abf1924a1d1297a8fb7300710223886ccf8bc931
|
||||
README.zh.md: f4c5ddd6dbe05ae709145cfac341f17a716bac82
|
||||
README.md: b8276e3b9a77cfa23ad32733a8b3754d3bfd8a9a
|
||||
README.zh.md: 94387c879f04be9bfe09c1518b50540e4d79ba84
|
||||
|
||||
@@ -91,6 +91,10 @@ How a thinking level travels — `reasoning_effort` alone, DeepSeek's `thinking:
|
||||
|
||||
A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768), so a listing that discloses nothing but ids still yields a serviceable route. Both fallbacks are guesses by construction, which is why they are route fields a deployment whose gateway serves smaller models corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap.
|
||||
|
||||
Request modalities resolve entry `input` → installed catalog entry → route `defaultInput` (default `[text]`), the same order and the same fallback role the capacities above use. So a catalog model keeps the modalities the catalog records for it, and a narrower route default never strips them; a gateway whose *undescribed* models all take images declares `[text, image]` once at the route instead of on every entry. An entry's empty list means the same as an absent one — it describes a model accepting nothing, so it states no answer and resolution continues past it, which is what keeps a catalog model's own modalities when a `models` entry names it without declaring any. The route's may not be empty, since nothing sits below it to answer instead.
|
||||
|
||||
`[text]` is the absence of a declaration rather than a guess at the endpoint, which is why the fallback here is conservative where the capacity fallbacks are merely plausible. Nothing interrogates a gateway for what it accepts, and the two wrong answers do not cost the same: the harness refuses an image before it is attached when a model's modalities omit one, so under-claiming costs a refusal naming the model, while over-claiming admits an image the provider then rejects mid-turn — after the message is durable, which leaves the session repeating a request that cannot succeed.
|
||||
|
||||
Resolution still fails loud, naming the offending route and model, when a route cannot be served at all: a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list of uniquely-identified models. That resolution runs inside the section schema, so an unserviceable profile is refused **where it is written** — `settings.mutate` answers `settings-rejected` naming the route and model — rather than being stored and then quietly disabling every route in the namespace. The settings seam keeps a namespace's last good value for an already-stored section that fails, so this cannot strand a deployment. `api` accepts the protocols in `supportedProtocols()` and is only needed when the catalog cannot supply one: a model absent from the catalog inherits the protocol its shipped siblings agree on, so adding a model to a single-protocol catalog route restates nothing.
|
||||
|
||||
`baseURL` sets the endpoint of every model on the route, so private proxies such as `https://proxy.example.com:8443` remain supported; a catalog route that omits it keeps each catalog model's own endpoint. Naming `api` on a catalog route repoints the whole route at that protocol, which is how a deployment moves a provider between, say, Responses and Chat Completions.
|
||||
@@ -109,7 +113,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
|
||||
|
||||
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
|
||||
@@ -91,6 +91,10 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩
|
||||
|
||||
条目与已安装 catalog 都没有给出尺寸的模型,会采用该路由的 `defaultContextWindow`(262,144)与 `defaultMaxTokens`(32,768),因此一份只公布 id 的列表同样能产出可服务的路由。两个回退值本质上都是猜测,这正是它们作为路由字段、供网关服务更小模型的部署一次性更正的原因,而不是埋在适配器里的常量;回退值只用于给模型定尺寸,绝不会变成每请求上限。
|
||||
|
||||
请求模态的解析顺序是:条目的 `input` → 已安装 catalog 条目 → 路由的 `defaultInput`(默认 `[text]`),与上面两个容量字段的顺序和「回退值」定位完全一致。因此 catalog 模型保留 catalog 为它记录的模态,更窄的路由默认值也绝不会把它剥掉;而**未被 catalog 描述的**模型全都接受图片的网关,只需在路由上写一次 `[text, image]`,不必逐条目写。条目的空列表与缺省同义——它描述的是一个什么都不接受的模型,因此不作答,解析继续往下走——这正是当 `models` 条目点到某个 catalog 模型却不声明模态时,该模型仍保留 catalog 自有模态的原因。路由的那个则不得为空,因为它下面已经没有可以代为作答的层级。
|
||||
|
||||
`[text]` 是「尚未声明」,而不是对端点的猜测——这也是为什么这里的回退值取保守值,而两个容量回退值只是取一个说得过去的值。这里没有任何环节会去询问网关实际接受什么,而两种猜错的代价并不对等:模态中不含图片时,Harness 会在图片被附加之前就拒绝,因此少声明的代价是一次点名该模型的拒绝;而多声明会接纳一张图片、再由提供方在轮次中途拒绝——此时消息已经持久化,会话便会不断重复一个不可能成功的请求。
|
||||
|
||||
路由完全无法服务时解析仍会失败得响亮,并点名出问题的路由与模型:catalog 未提供的路由需要 `api`、`baseURL`,以及一个由唯一标识的模型组成的非空 `models` 列表。该解析在分节 schema 内部运行,因此无法服务的 profile 会在**写入之处**被拒绝——`settings.mutate` 以 `settings-rejected` 点名路由与模型——而不是先存下来、再悄悄让该 namespace 下每条路由失效。对于已经存下的、在此失败的分节,settings seam 会保留该 namespace 上一份可用值,因此这不会把部署卡死。`api` 接受 `supportedProtocols()` 中的协议,且仅在 catalog 无法提供协议时才需要:catalog 中不存在的模型会继承其同门模型一致同意的协议,因此向单协议 catalog 路由添加模型无需重述任何内容。
|
||||
|
||||
`baseURL` 设定该路由下每个模型的端点,因此仍支持 `https://proxy.example.com:8443` 等私有 proxy;省略它的 catalog 路由会保留每个 catalog 模型自己的端点。在 catalog 路由上点名 `api` 会把整条路由改指到该协议,这正是部署把某个提供方在 Responses 与 Chat Completions 之间迁移的方式。
|
||||
@@ -109,7 +113,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩
|
||||
|
||||
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获;省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。若已配置标头中有同名项,则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获;省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。若已配置标头中有同名项,则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
|
||||
@@ -31,12 +31,34 @@ import type {
|
||||
*/
|
||||
const NO_COST: ModelCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
export type PiAiModality = Model<Api>['input'][number]
|
||||
|
||||
/**
|
||||
* Input modalities for a model the installed catalog does not describe. The
|
||||
* request converter keeps only text blocks, so text is the adapter's actual
|
||||
* capability rather than a deployment choice.
|
||||
* Every pi-ai request modality. The `Record` key type is a drift gate: a pi-ai
|
||||
* upgrade that adds or removes a modality fails compilation here naming the
|
||||
* drifted key, instead of silently narrowing what a profile may declare.
|
||||
*/
|
||||
const TEXT_ONLY: Model<Api>['input'] = ['text']
|
||||
const MODALITY_GATE: Record<PiAiModality, true> = {
|
||||
text: true,
|
||||
image: true,
|
||||
}
|
||||
|
||||
/** Every request modality a profile may declare. */
|
||||
export const MODALITIES = Object.keys(MODALITY_GATE) as readonly PiAiModality[]
|
||||
|
||||
/**
|
||||
* One entry's modality list, or `undefined` when it states no answer. Absent
|
||||
* and empty mean the same thing — `[]` describes a model that accepts nothing
|
||||
* and could serve no request — which is what makes an entry naming a catalog
|
||||
* model without declaring modalities keep the catalog's, since the config
|
||||
* schema materializes `[]` for an absent array.
|
||||
* @param configured - the list a `models` or `modelOverrides` entry supplied.
|
||||
* @returns the declared modalities, or `undefined` to ask the next level.
|
||||
*/
|
||||
function declaredInput(configured: readonly PiAiModality[] | undefined): Model<Api>['input'] | undefined {
|
||||
return configured === undefined || configured.length === 0 ? undefined : [...configured]
|
||||
}
|
||||
|
||||
/**
|
||||
* Every pi-ai thinking level, in pi-ai's canonical escalation order. The
|
||||
@@ -171,6 +193,18 @@ export interface PiAiModelProfile {
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Request modalities this model accepts. Absent — or empty, which describes
|
||||
* a model that accepts nothing and so states no answer either — keeps the
|
||||
* installed catalog entry's modalities, then the route's `defaultInput`.
|
||||
* Declaring images is what makes a hand-declared vision model usable, and
|
||||
* declaring text alone corrects a catalog model whose gateway does not serve
|
||||
* what the catalog records. This is a claim about the endpoint, not a check
|
||||
* of it: nothing interrogates a gateway for what it accepts, so a model
|
||||
* claiming images its endpoint refuses is refused by the provider instead,
|
||||
* mid-turn.
|
||||
*/
|
||||
input?: PiAiModality[]
|
||||
/**
|
||||
* Selectable reasoning efforts. Absent inherits the installed catalog
|
||||
* entry's capability (a hand-declared model has none and does not reason);
|
||||
@@ -210,6 +244,8 @@ export interface RouteCatalogRequest {
|
||||
defaultContextWindow: number
|
||||
/** Output capability for a model neither the entry nor the catalog sizes. */
|
||||
defaultMaxTokens: number
|
||||
/** Modalities for a model neither the entry nor the catalog declares. */
|
||||
defaultInput: Model<Api>['input']
|
||||
}
|
||||
|
||||
/** Report a route the deployment cannot serve, naming the settings key at fault. */
|
||||
@@ -474,7 +510,7 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
api,
|
||||
provider,
|
||||
baseUrl,
|
||||
input: base?.input ?? TEXT_ONLY,
|
||||
input: declaredInput(entry.input) ?? base?.input ?? request.defaultInput,
|
||||
cost: base?.cost ?? NO_COST,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
|
||||
@@ -21,8 +21,14 @@ import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts'
|
||||
import type { PiAiCompatProfile, PiAiModelOverride, PiAiModelProfile, PiAiReasoningEfforts } from './catalog.ts'
|
||||
import { MODALITIES, resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts'
|
||||
import type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiReasoningEfforts,
|
||||
} from './catalog.ts'
|
||||
import { buildProvider, supportedProtocols } from './provider.ts'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
@@ -34,8 +40,21 @@ export const DEFAULT_CONTEXT_WINDOW = 262_144
|
||||
/** Output capability assumed for a model neither configuration nor the catalog sizes. */
|
||||
export const DEFAULT_MAX_TOKENS = 32_768
|
||||
|
||||
/**
|
||||
* Modalities assumed for a model neither configuration nor the catalog
|
||||
* declares. Text is the floor every supported protocol certainly carries, so
|
||||
* this is the absence of a declaration rather than a guess at the endpoint:
|
||||
* nothing can interrogate a gateway for its modalities, and the two wrong
|
||||
* answers do not cost the same. Under-claiming refuses the image before it is
|
||||
* attached, naming the model. Over-claiming admits one the provider then
|
||||
* rejects mid-turn, after the message is durable, leaving the session
|
||||
* repeating a request that cannot succeed.
|
||||
*/
|
||||
export const DEFAULT_INPUT: readonly PiAiModality[] = ['text']
|
||||
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiReasoningEfforts,
|
||||
@@ -90,6 +109,17 @@ export interface PiAiProviderProfile {
|
||||
* never becomes a per-request cap on its own.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Request modalities for a model this route lists that neither its entry's
|
||||
* {@link PiAiModelProfile.input} nor the installed catalog declares (default
|
||||
* `[text]`). A fallback like the capacities above, not an override: a
|
||||
* catalog model keeps the modalities the catalog records for it, and this
|
||||
* value never narrows one. A gateway serving vision models the catalog does
|
||||
* not describe declares `[text, image]` once here instead of on every entry.
|
||||
* Unlike an entry's list, this one may not be empty — nothing sits below it
|
||||
* to answer instead.
|
||||
*/
|
||||
defaultInput?: PiAiModality[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -180,6 +210,10 @@ const modelFields = {
|
||||
name: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
maxTokens: z.number().step(1).min(1),
|
||||
// No explicit default, unlike the route's `defaultInput`: schemastery
|
||||
// materializes `[]` for an absent array, and resolution reads that as "no
|
||||
// answer here" so the catalog entry below still applies.
|
||||
input: z.array(z.union(MODALITIES)),
|
||||
// The union, not a bare dict: schemastery materializes an absent dict as
|
||||
// `{}`, and absent must stay distinguishable — it means "inherit the
|
||||
// installed catalog's capability", while `false` disables reasoning.
|
||||
@@ -205,6 +239,7 @@ const profile = z.object({
|
||||
compat: compatProfile,
|
||||
defaultContextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW),
|
||||
defaultMaxTokens: z.number().step(1).min(1).default(DEFAULT_MAX_TOKENS),
|
||||
defaultInput: z.array(z.union(MODALITIES)).default([...DEFAULT_INPUT]),
|
||||
headers: z.dict(z.string()),
|
||||
reasoning: z.union(THINKING_LEVELS),
|
||||
thinkingBudgets,
|
||||
@@ -288,6 +323,15 @@ export function resolveProfiles(
|
||||
`llm-pi-ai: provider "${provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
// Detached from the configuration object because pi-ai types `Model.input`
|
||||
// mutable and every model falling through to it is handed this array. The
|
||||
// schema's explicit default covers an absent key, so an empty list here is
|
||||
// always one someone typed — and unlike an entry's, nothing below it can
|
||||
// answer instead — so it is refused rather than read as "no answer".
|
||||
const defaultInput = [...source.defaultInput ?? DEFAULT_INPUT]
|
||||
if (defaultInput.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" defaultInput must name at least one modality`)
|
||||
}
|
||||
// The route key, not the installed provider's own name: the directory has
|
||||
// always shown route keys, and a catalog route must not silently rename
|
||||
// itself on every configuration surface just because it gained a profile.
|
||||
@@ -299,6 +343,7 @@ export function resolveProfiles(
|
||||
...source.models === undefined ? {} : { models: source.models },
|
||||
...source.modelOverrides === undefined ? {} : { modelOverrides: source.modelOverrides },
|
||||
...source.compat === undefined ? {} : { compat: source.compat },
|
||||
defaultInput,
|
||||
defaultContextWindow: source.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW,
|
||||
defaultMaxTokens: source.defaultMaxTokens ?? DEFAULT_MAX_TOKENS,
|
||||
})
|
||||
|
||||
@@ -186,6 +186,75 @@ describe('hand-declared providers', () => {
|
||||
expect(resolved.get('acme-gateway')?.configuredMaxTokens.get('sized')).toBe(512)
|
||||
})
|
||||
|
||||
it('takes a model’s declared modalities, then the catalog’s, then the route’s', () => {
|
||||
const vision = getBuiltinModels('anthropic').find(model => model.input.includes('image'))
|
||||
if (vision === undefined) throw new Error('the installed catalog ships no anthropic vision model')
|
||||
const resolved = resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
// One route, two modality sets: the entry field is what says so.
|
||||
models: [{ id: 'bare' }, { id: 'seeing', input: ['text', 'image'] }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
'seeing-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://seeing.test',
|
||||
// A gateway whose undescribed models all take images says so once
|
||||
// rather than on every entry; an entry still outranks it.
|
||||
defaultInput: ['text', 'image'],
|
||||
models: [{ id: 'bare' }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
// The route value is a fallback, never an override: a catalog model
|
||||
// keeps what the catalog records even under a narrower route default,
|
||||
// exactly as it keeps its own contextWindow.
|
||||
'anthropic': { defaultInput: ['text'] },
|
||||
})
|
||||
const inputOf = (route: string, id: string): readonly string[] | undefined =>
|
||||
resolved.get(route)?.piProvider.getModels().find(model => model.id === id)?.input
|
||||
|
||||
// Nothing can interrogate the endpoint, so an undeclared model on a route
|
||||
// that names no default claims only the modality every supported protocol
|
||||
// carries. Claiming images instead would admit one the provider then
|
||||
// rejects mid-turn, after the durable message is already in the log.
|
||||
expect(inputOf('acme-gateway', 'bare')).toEqual(['text'])
|
||||
expect(inputOf('acme-gateway', 'seeing')).toEqual(['text', 'image'])
|
||||
expect(inputOf('acme-gateway', 'deaf')).toEqual(['text'])
|
||||
expect(inputOf('seeing-gateway', 'bare')).toEqual(['text', 'image'])
|
||||
expect(inputOf('seeing-gateway', 'deaf')).toEqual(['text'])
|
||||
expect(inputOf('anthropic', vision.id)).toEqual(vision.input)
|
||||
})
|
||||
|
||||
it('reads an entry’s empty modality list as no answer, and the route’s as unserviceable', () => {
|
||||
// Absent and empty are the same request on an entry, exactly as they are
|
||||
// for the route's `models` list — which matters because the config schema
|
||||
// materializes `[]` for an absent array, so an entry naming a catalog
|
||||
// model without declaring modalities must keep the catalog's rather than
|
||||
// describe a model that accepts nothing.
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
const resolved = resolveProfiles({
|
||||
'deepseek': { baseURL: 'https://catalog.test', models: [{ id: catalogModel.id, input: [] }] },
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'bare', input: [] }],
|
||||
},
|
||||
})
|
||||
expect(resolved.get('acme-gateway')?.piProvider.getModels()[0]?.input).toEqual(['text'])
|
||||
expect(resolved.get('deepseek')?.piProvider.getModels()[0]?.input).toEqual(catalogModel.input)
|
||||
|
||||
// Nothing sits below the route value, so its empty list states no answer
|
||||
// anything could take, and is refused where it is written.
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
defaultInput: [],
|
||||
models: [{ id: 'bare' }],
|
||||
},
|
||||
})).toThrow(/defaultInput must name at least one modality/)
|
||||
})
|
||||
|
||||
it('rejects a model the route cannot identify', () => {
|
||||
const declare = (model: LlmPiAi.PiAiModelProfile): (() => unknown) =>
|
||||
() => resolveProfiles({ 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models: [model] } })
|
||||
|
||||
Reference in New Issue
Block a user