review: sync the note this supersedes, gate the schema paths, export the modality type

The declared-provider-catalog note still recorded modalities as closed
because "nothing reads them: context.ts keeps only text blocks" — the
same reasoning this change disproves. Rewrite that fact in place, keep
pricing closed for its own still-current reason, and cross-link both
directions.

The new resolver cases never crossed the config schema, so neither
acceptance path this change adds had an executable gate: config.spec.ts
now refuses an unknown modality at both levels, pins the empty route
list to the namespace validator the settings seam actually runs, and
pins the `[]` materialization the inheritance rule depends on. A
catalog.spec.ts case re-asserts every rung through a written settings
section and `ctx.llm`, which is what the note's Testing section claimed.

Also export PiAiModality from the package entry — it types two public
config fields and consumers could not name it — record the unverified
declaration as a Known Limitation, and stop sharing one fallback array
across a route's models.
This commit is contained in:
Yichen Jiang
2026-08-12 15:49:23 +08:00
parent 94289d057e
commit 50edee2dab
14 changed files with 109 additions and 35 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: b8276e3b9a77cfa23ad32733a8b3754d3bfd8a9a
README.zh.md: 94387c879f04be9bfe09c1518b50540e4d79ba84
README.md: c68db532e94c1b0e956d933fbdcf51e5bb63dd29
README.zh.md: f0f3f7f9f4abd3578cc6e6d1c496df23f5976a3d

View File

@@ -194,6 +194,7 @@ Recorded response content appends to the next request and does not invalidate it
- **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work).
- **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says, so a model list is only as current as its last edit. Nothing here queries a provider for the models it serves; a route gains a model when someone writes one.
- **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround.
- **A modality declaration is not verified, and over-claiming outlives the turn** — nothing interrogates an endpoint for what it accepts, so a model declaring `image` its gateway does not serve is refused by the provider mid-turn rather than here. Prompt admission commits the user message durably before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is another image-capable model, a fork before the image, or a new session; rolling an unconsumed image message back out of the log on a failed send is deferred.
- **An unauthenticated route depends on its protocol** — naming no credential resolves the route as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder credential referenced by `apiKeyEnv` or an `Authorization` entry in `headers`.
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.

View File

@@ -194,6 +194,7 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
- **`headers` 可能承载一条脱敏器看不见的凭据**profile 的 `headers` 是纯字符串字典,因此设在其中的 `Authorization``api-key` 会被脱敏后的 `describe()` 原样返回,并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用;把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。
- **路由的 catalog 不会自我刷新**catalog 就是 `settings.yaml` 所写的内容,因此模型列表的新鲜度只到最近一次编辑为止。这里没有任何环节会去问提供方它服务哪些模型;路由要多一个模型,得有人写进去。
- **每条路由只有一种协议格式**`api` 作用于整条路由,因此混合协议的 catalog 路由(跨 Responses 与 Chat Completions 的 OpenAI 式 catalog无法承载另一种协议的模型向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。
- **模态声明不经验证,且多声明的后果超出本轮**:没有任何环节会去询问端点接受什么,因此声明了网关并不提供的 `image` 的模型不会在这里被拦下而是由提供方在轮次中途拒绝。prompt 准入在构造请求之前就把用户消息持久化提交于是被拒绝的图片留在会话日志里该模型会不断重发它而模型选择拒绝切换到任何纯文本模型。恢复途径是换一个确实支持图片的模型、fork 到图片之前,或开启新会话;发送失败时把尚未消费的图片消息从日志中回滚出去这件事已暂缓。
- **未认证路由取决于其协议**:不点名凭据会让路由解析为「已配置但无密钥」,但 pi-ai 的 OpenAI 兼容实现仍要求 API key 或 `Authorization` 标头,因此无鉴权的本地服务需要一个由 `apiKeyEnv` 引用的占位凭据,或在 `headers` 中给出 `Authorization` 条目。
- **不支持 `GenerateOptions.stop`**pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence因此适配器会拒绝该字段。
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**:提供方特定位置由 pi-ai 决定,而非由 harness 拥有的协议覆盖决定。

View File

@@ -510,7 +510,7 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
api,
provider,
baseUrl,
input: declaredInput(entry.input) ?? base?.input ?? request.defaultInput,
input: declaredInput(entry.input) ?? base?.input ?? [...request.defaultInput],
cost: base?.cost ?? NO_COST,
contextWindow,
maxTokens,

View File

@@ -324,10 +324,10 @@ export function resolveProfiles(
)
}
// Detached from the configuration object because pi-ai types `Model.input`
// mutable and every model falling through to it is handed this array. The
// schema's explicit default covers an absent key, so an empty list here is
// always one someone typed — and unlike an entry's, nothing below it can
// answer instead — so it is refused rather than read as "no answer".
// mutable. The schema's explicit default covers an absent key, so an empty
// list here is always one someone typed — and unlike an entry's, nothing
// below it can answer instead — so it is refused rather than read as "no
// answer".
const defaultInput = [...source.defaultInput ?? DEFAULT_INPUT]
if (defaultInput.length === 0) {
throw new Error(`llm-pi-ai: provider "${provider}" defaultInput must name at least one modality`)

View File

@@ -71,6 +71,7 @@ export type { PiAiAdapterOptions } from './adapter.ts'
export { Config } from './config.ts'
export type {
PiAiCompatProfile,
PiAiModality,
PiAiModelOverride,
PiAiModelProfile,
PiAiProviderProfile,

View File

@@ -212,10 +212,6 @@ describe('hand-declared providers', () => {
const inputOf = (route: string, id: string): readonly string[] | undefined =>
resolved.get(route)?.piProvider.getModels().find(model => model.id === id)?.input
// Nothing can interrogate the endpoint, so an undeclared model on a route
// that names no default claims only the modality every supported protocol
// carries. Claiming images instead would admit one the provider then
// rejects mid-turn, after the durable message is already in the log.
expect(inputOf('acme-gateway', 'bare')).toEqual(['text'])
expect(inputOf('acme-gateway', 'seeing')).toEqual(['text', 'image'])
expect(inputOf('acme-gateway', 'deaf')).toEqual(['text'])
@@ -224,6 +220,43 @@ describe('hand-declared providers', () => {
expect(inputOf('anthropic', vision.id)).toEqual(vision.input)
})
it('carries a written modality declaration all the way to the seams model metadata', async () => {
// The resolver-level cases above cannot see a break between the settings
// document and `LlmModelInfo`, so each rung is asserted once more through
// a written section, the plugin's own registration, and `ctx.llm`.
const dir = await home()
const ctx = await bootWithSettings(dir, {})
await ctx.settings.update(settingsNamespace('llm-pi-ai'), {
providers: {
'acme-gateway': {
api: 'openai-completions',
baseURL: 'https://acme.test/v1',
models: [{ id: 'bare' }, { id: 'seeing', input: ['text', 'image'] }],
},
'vision-gateway': {
api: 'openai-completions',
baseURL: 'https://vision.test/v1',
defaultInput: ['text', 'image'],
models: [{ id: 'bare' }, { id: 'deaf', input: ['text'] }],
},
'anthropic': { defaultInput: ['text'] },
},
})
const listed = async (provider: string): Promise<Record<string, readonly string[] | undefined>> =>
Object.fromEntries((await ctx.llm.listModels(provider)).map(model => [model.id, model.inputModalities]))
expect(await listed('acme-gateway')).toEqual({ bare: ['text'], seeing: ['text', 'image'] })
expect(await listed('vision-gateway')).toEqual({ bare: ['text', 'image'], deaf: ['text'] })
expect((await ctx.llm.resolveModelInfo('acme-gateway', 'seeing')).inputModalities).toEqual(['text', 'image'])
// A catalog vision model keeps what the catalog records even under a
// narrower route default: the route value is a fallback, not an override.
const vision = getBuiltinModels('anthropic').find(model => model.input.includes('image'))
if (vision === undefined) throw new Error('the installed catalog ships no anthropic vision model')
expect((await ctx.llm.resolveModelInfo('anthropic', vision.id)).inputModalities).toEqual(vision.input)
})
it('reads an entrys empty modality list as no answer, and the routes as unserviceable', () => {
// Absent and empty are the same request on an entry, exactly as they are
// for the route's `models` list — which matters because the config schema

View File

@@ -1,18 +1,24 @@
import { describe, expect, it } from 'vitest'
import { Config } from '../src/config.ts'
import { assertServiceable, Config } from '../src/config.ts'
/** Validate one hand-declared route, with the caller's fields layered onto it. */
const routeWith = (profile: Record<string, unknown>): (() => unknown) =>
() => Config({
providers: {
'acme-gateway': {
api: 'openai-completions',
baseURL: 'https://acme.test',
models: [{ id: 'm' }],
...profile,
},
},
})
/** Validate that route with the caller's fields on its single model entry. */
const configWith = (model: Record<string, unknown>): (() => unknown) =>
routeWith({ models: [{ id: 'm', ...model }] })
describe('reasoning schema boundary', () => {
const configWith = (model: Record<string, unknown>): (() => unknown) =>
() => Config({
providers: {
'acme-gateway': {
api: 'openai-completions',
baseURL: 'https://acme.test',
models: [{ id: 'm', ...model }],
},
},
})
it('rejects a level pi-ai does not know at the write that produced it', () => {
expect(configWith({ reasoningEfforts: { ultra: 'x' } })).toThrow(/"off"/)
expect(configWith({ reasoningEfforts: { high: 42 } })).toThrow()
@@ -30,3 +36,31 @@ describe('reasoning schema boundary', () => {
expect(configWith({ compat: { thinkingFormat: 'quantum' } })).toThrow(/expected/)
})
})
describe('modality schema boundary', () => {
it('rejects a modality pi-ai does not know, at either level', () => {
expect(configWith({ input: ['audio'] })).toThrow(/expected/)
expect(routeWith({ defaultInput: ['text', 'audio'] })).toThrow(/expected/)
})
it('refuses a route whose models could accept nothing', () => {
// The pair the settings seam runs: the schema accepts the empty list as
// well-typed, and the namespace validator is what refuses it. Asserting
// only the schema would report this route as writable.
expect(routeWith({ defaultInput: [] })).not.toThrow()
expect(() => { assertServiceable(routeWith({ defaultInput: [] })() as Config) })
.toThrow(/defaultInput must name at least one modality/)
})
type Materialized = {
providers: Record<string, { defaultInput?: unknown; models?: { input?: unknown }[] }>
}
it('materializes an absent entry list as empty and an absent route list as text', () => {
// The empty-list inheritance rule exists because of exactly this: an entry
// that declares nothing reaches resolution as `[]`, not as `undefined`.
const absent = configWith({})() as Materialized
expect(absent.providers['acme-gateway']?.models?.[0]?.input).toEqual([])
expect(absent.providers['acme-gateway']?.defaultInput).toEqual(['text'])
})
})