review: sync the note this supersedes, gate the schema paths, export the modality type
The declared-provider-catalog note still recorded modalities as closed because "nothing reads them: context.ts keeps only text blocks" — the same reasoning this change disproves. Rewrite that fact in place, keep pricing closed for its own still-current reason, and cross-link both directions. The new resolver cases never crossed the config schema, so neither acceptance path this change adds had an executable gate: config.spec.ts now refuses an unknown modality at both levels, pins the empty route list to the namespace validator the settings seam actually runs, and pins the `[]` materialization the inheritance rule depends on. A catalog.spec.ts case re-asserts every rung through a written settings section and `ctx.llm`, which is what the note's Testing section claimed. Also export PiAiModality from the package entry — it types two public config fields and consumers could not name it — record the unverified declaration as a Known Limitation, and stop sharing one fallback array across a route's models.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: b8276e3b9a77cfa23ad32733a8b3754d3bfd8a9a
|
||||
README.zh.md: 94387c879f04be9bfe09c1518b50540e4d79ba84
|
||||
README.md: c68db532e94c1b0e956d933fbdcf51e5bb63dd29
|
||||
README.zh.md: f0f3f7f9f4abd3578cc6e6d1c496df23f5976a3d
|
||||
|
||||
@@ -194,6 +194,7 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
- **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work).
|
||||
- **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says, so a model list is only as current as its last edit. Nothing here queries a provider for the models it serves; a route gains a model when someone writes one.
|
||||
- **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround.
|
||||
- **A modality declaration is not verified, and over-claiming outlives the turn** — nothing interrogates an endpoint for what it accepts, so a model declaring `image` its gateway does not serve is refused by the provider mid-turn rather than here. Prompt admission commits the user message durably before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is another image-capable model, a fork before the image, or a new session; rolling an unconsumed image message back out of the log on a failed send is deferred.
|
||||
- **An unauthenticated route depends on its protocol** — naming no credential resolves the route as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder credential referenced by `apiKeyEnv` or an `Authorization` entry in `headers`.
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
|
||||
@@ -194,6 +194,7 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
|
||||
- **`headers` 可能承载一条脱敏器看不见的凭据**:profile 的 `headers` 是纯字符串字典,因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回,并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用;把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。
|
||||
- **路由的 catalog 不会自我刷新**:catalog 就是 `settings.yaml` 所写的内容,因此模型列表的新鲜度只到最近一次编辑为止。这里没有任何环节会去问提供方它服务哪些模型;路由要多一个模型,得有人写进去。
|
||||
- **每条路由只有一种协议格式**:`api` 作用于整条路由,因此混合协议的 catalog 路由(跨 Responses 与 Chat Completions 的 OpenAI 式 catalog)无法承载另一种协议的模型,向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。
|
||||
- **模态声明不经验证,且多声明的后果超出本轮**:没有任何环节会去询问端点接受什么,因此声明了网关并不提供的 `image` 的模型不会在这里被拦下,而是由提供方在轮次中途拒绝。prompt 准入在构造请求之前就把用户消息持久化提交,于是被拒绝的图片留在会话日志里:该模型会不断重发它,而模型选择拒绝切换到任何纯文本模型。恢复途径是换一个确实支持图片的模型、fork 到图片之前,或开启新会话;发送失败时把尚未消费的图片消息从日志中回滚出去这件事已暂缓。
|
||||
- **未认证路由取决于其协议**:不点名凭据会让路由解析为「已配置但无密钥」,但 pi-ai 的 OpenAI 兼容实现仍要求 API key 或 `Authorization` 标头,因此无鉴权的本地服务需要一个由 `apiKeyEnv` 引用的占位凭据,或在 `headers` 中给出 `Authorization` 条目。
|
||||
- **不支持 `GenerateOptions.stop`**:pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence,因此适配器会拒绝该字段。
|
||||
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**:提供方特定位置由 pi-ai 决定,而非由 harness 拥有的协议覆盖决定。
|
||||
|
||||
@@ -510,7 +510,7 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
api,
|
||||
provider,
|
||||
baseUrl,
|
||||
input: declaredInput(entry.input) ?? base?.input ?? request.defaultInput,
|
||||
input: declaredInput(entry.input) ?? base?.input ?? [...request.defaultInput],
|
||||
cost: base?.cost ?? NO_COST,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
|
||||
@@ -324,10 +324,10 @@ export function resolveProfiles(
|
||||
)
|
||||
}
|
||||
// Detached from the configuration object because pi-ai types `Model.input`
|
||||
// mutable and every model falling through to it is handed this array. The
|
||||
// schema's explicit default covers an absent key, so an empty list here is
|
||||
// always one someone typed — and unlike an entry's, nothing below it can
|
||||
// answer instead — so it is refused rather than read as "no answer".
|
||||
// mutable. The schema's explicit default covers an absent key, so an empty
|
||||
// list here is always one someone typed — and unlike an entry's, nothing
|
||||
// below it can answer instead — so it is refused rather than read as "no
|
||||
// answer".
|
||||
const defaultInput = [...source.defaultInput ?? DEFAULT_INPUT]
|
||||
if (defaultInput.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" defaultInput must name at least one modality`)
|
||||
|
||||
@@ -71,6 +71,7 @@ export type { PiAiAdapterOptions } from './adapter.ts'
|
||||
export { Config } from './config.ts'
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModality,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiProviderProfile,
|
||||
|
||||
@@ -212,10 +212,6 @@ describe('hand-declared providers', () => {
|
||||
const inputOf = (route: string, id: string): readonly string[] | undefined =>
|
||||
resolved.get(route)?.piProvider.getModels().find(model => model.id === id)?.input
|
||||
|
||||
// Nothing can interrogate the endpoint, so an undeclared model on a route
|
||||
// that names no default claims only the modality every supported protocol
|
||||
// carries. Claiming images instead would admit one the provider then
|
||||
// rejects mid-turn, after the durable message is already in the log.
|
||||
expect(inputOf('acme-gateway', 'bare')).toEqual(['text'])
|
||||
expect(inputOf('acme-gateway', 'seeing')).toEqual(['text', 'image'])
|
||||
expect(inputOf('acme-gateway', 'deaf')).toEqual(['text'])
|
||||
@@ -224,6 +220,43 @@ describe('hand-declared providers', () => {
|
||||
expect(inputOf('anthropic', vision.id)).toEqual(vision.input)
|
||||
})
|
||||
|
||||
it('carries a written modality declaration all the way to the seam’s model metadata', async () => {
|
||||
// The resolver-level cases above cannot see a break between the settings
|
||||
// document and `LlmModelInfo`, so each rung is asserted once more through
|
||||
// a written section, the plugin's own registration, and `ctx.llm`.
|
||||
const dir = await home()
|
||||
const ctx = await bootWithSettings(dir, {})
|
||||
await ctx.settings.update(settingsNamespace('llm-pi-ai'), {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test/v1',
|
||||
models: [{ id: 'bare' }, { id: 'seeing', input: ['text', 'image'] }],
|
||||
},
|
||||
'vision-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://vision.test/v1',
|
||||
defaultInput: ['text', 'image'],
|
||||
models: [{ id: 'bare' }, { id: 'deaf', input: ['text'] }],
|
||||
},
|
||||
'anthropic': { defaultInput: ['text'] },
|
||||
},
|
||||
})
|
||||
|
||||
const listed = async (provider: string): Promise<Record<string, readonly string[] | undefined>> =>
|
||||
Object.fromEntries((await ctx.llm.listModels(provider)).map(model => [model.id, model.inputModalities]))
|
||||
|
||||
expect(await listed('acme-gateway')).toEqual({ bare: ['text'], seeing: ['text', 'image'] })
|
||||
expect(await listed('vision-gateway')).toEqual({ bare: ['text', 'image'], deaf: ['text'] })
|
||||
expect((await ctx.llm.resolveModelInfo('acme-gateway', 'seeing')).inputModalities).toEqual(['text', 'image'])
|
||||
|
||||
// A catalog vision model keeps what the catalog records even under a
|
||||
// narrower route default: the route value is a fallback, not an override.
|
||||
const vision = getBuiltinModels('anthropic').find(model => model.input.includes('image'))
|
||||
if (vision === undefined) throw new Error('the installed catalog ships no anthropic vision model')
|
||||
expect((await ctx.llm.resolveModelInfo('anthropic', vision.id)).inputModalities).toEqual(vision.input)
|
||||
})
|
||||
|
||||
it('reads an entry’s empty modality list as no answer, and the route’s as unserviceable', () => {
|
||||
// Absent and empty are the same request on an entry, exactly as they are
|
||||
// for the route's `models` list — which matters because the config schema
|
||||
|
||||
@@ -1,18 +1,24 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Config } from '../src/config.ts'
|
||||
import { assertServiceable, Config } from '../src/config.ts'
|
||||
|
||||
/** Validate one hand-declared route, with the caller's fields layered onto it. */
|
||||
const routeWith = (profile: Record<string, unknown>): (() => unknown) =>
|
||||
() => Config({
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm' }],
|
||||
...profile,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
/** Validate that route with the caller's fields on its single model entry. */
|
||||
const configWith = (model: Record<string, unknown>): (() => unknown) =>
|
||||
routeWith({ models: [{ id: 'm', ...model }] })
|
||||
|
||||
describe('reasoning schema boundary', () => {
|
||||
const configWith = (model: Record<string, unknown>): (() => unknown) =>
|
||||
() => Config({
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm', ...model }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
it('rejects a level pi-ai does not know at the write that produced it', () => {
|
||||
expect(configWith({ reasoningEfforts: { ultra: 'x' } })).toThrow(/"off"/)
|
||||
expect(configWith({ reasoningEfforts: { high: 42 } })).toThrow()
|
||||
@@ -30,3 +36,31 @@ describe('reasoning schema boundary', () => {
|
||||
expect(configWith({ compat: { thinkingFormat: 'quantum' } })).toThrow(/expected/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('modality schema boundary', () => {
|
||||
it('rejects a modality pi-ai does not know, at either level', () => {
|
||||
expect(configWith({ input: ['audio'] })).toThrow(/expected/)
|
||||
expect(routeWith({ defaultInput: ['text', 'audio'] })).toThrow(/expected/)
|
||||
})
|
||||
|
||||
it('refuses a route whose models could accept nothing', () => {
|
||||
// The pair the settings seam runs: the schema accepts the empty list as
|
||||
// well-typed, and the namespace validator is what refuses it. Asserting
|
||||
// only the schema would report this route as writable.
|
||||
expect(routeWith({ defaultInput: [] })).not.toThrow()
|
||||
expect(() => { assertServiceable(routeWith({ defaultInput: [] })() as Config) })
|
||||
.toThrow(/defaultInput must name at least one modality/)
|
||||
})
|
||||
|
||||
type Materialized = {
|
||||
providers: Record<string, { defaultInput?: unknown; models?: { input?: unknown }[] }>
|
||||
}
|
||||
|
||||
it('materializes an absent entry list as empty and an absent route list as text', () => {
|
||||
// The empty-list inheritance rule exists because of exactly this: an entry
|
||||
// that declares nothing reaches resolution as `[]`, not as `undefined`.
|
||||
const absent = configWith({})() as Materialized
|
||||
expect(absent.providers['acme-gateway']?.models?.[0]?.input).toEqual([])
|
||||
expect(absent.providers['acme-gateway']?.defaultInput).toEqual(['text'])
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user