fix(host): pin model discovery to loopback and drop its unread wire field
llm.discoverModels was reachable from any declared trusted host. The method takes a caller-supplied baseURL and makes the host issue a GET to it, then reports the status or the parsed body — so on a LAN deployment an anonymous caller had a probe for whatever the host can reach and the browser cannot, plus a path that carries a draft credential. The PRIVILEGED_METHODS doc already states the rule this broke: trustedHosts is a DNS-rebinding fence, not authentication, so the configuration plane stays loopback-same-origin. It is in that set now, asserted both against the hand-built fence and over real HTTP beside the catalog reads that deliberately stay reachable. supportsDiscovery and listModelDiscoveryNamespaces are gone. The field was required on the wire and read by nobody: its own contract said a surface should offer the action "instead of naming an adapter family it would have to hardcode", while the surface hardcodes llm-pi-ai in two places and gates the button on whether there is anything to probe. Its shape did not fit the second caller either — the create card has no row to read a per-row field from. Keeping a required field alive for a consumer that may never arrive costs every producer and fixture a value nobody consults, which is exactly how the fixtures drifted. The registry that fed it had no other production consumer, so registration and disposal are now observed through the offer itself. The Agent Note claimed the key is never logged, which the wire schema beside it already contradicts, and predated both the provider field and the catalog-answer path. The two new public types pointed at core.md without a type-equiv block or manifest entry, so the generated service catalog named documentation that did not exist.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 07de1c0aceeccff5f3f14a43c4888b4481c0293a
|
||||
README.zh.md: 0e8895a0192c7f18e2b6ee8869896080f7ff49e2
|
||||
README.md: 6f3fa0bb0eb0236ab885ef803ace8069b1d52302
|
||||
README.zh.md: c0897d92da83084c5eeac02d5fd24c34a71e1af2
|
||||
|
||||
@@ -79,6 +79,16 @@ Supported profile fields are `apiKey`, `apiKeyEnv`, `displayName`, `api`, `baseU
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
## Endpoint interrogation
|
||||
|
||||
The plugin offers `ctx.llm.registerModelDiscovery('llm-pi-ai', …)`, which answers "which models can this provider serve?" for a route a configuration surface is editing or drafting. It is deliberately *not* a catalog refresh: nothing is stored, and the reply is candidates the surface offers for adoption. `settings.yaml` remains the only thing that decides what a route serves.
|
||||
|
||||
A request naming a route the **installed catalog ships is answered from that catalog**, with no network call: pi-ai's registry is the authoritative list for its own providers, and it carries the context windows and output caps a listing endpoint would not disclose. Such a route needs no `baseURL` at all. Only a route the catalog does not describe — a gateway, a self-hosted server — is interrogated over the wire, and one that names no endpoint is told to set one or enter its models by hand.
|
||||
|
||||
Interrogation reads `openai-completions` and `openai-responses`, whose `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; every other protocol answers `DISCOVERY_UNSUPPORTED` so the surface falls back to hand-entry instead of an authentication failure being reported as a provider with no models. The `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments.
|
||||
|
||||
Most listings disclose an id and nothing else; `context_window`/`context_length` and `max_output_tokens`/`max_tokens` are read when a gateway supplies them, entries without a usable id are skipped rather than failing the whole listing, and everything else the adopting surface still owes. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared length is checked first but never trusted as the bound. An unreachable endpoint, a refused credential, a non-JSON body, and a body with no `data` array all fail with `DISCOVERY_FAILED` and a message naming the endpoint and, for a 401 or 403 alone, the credential. Cancellation during the body read surfaces as `ABORTED`, like a cancellation before the request went out.
|
||||
|
||||
## Provider/model routing and replay
|
||||
|
||||
Each resolution produces one **immutable** snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — and every operation captures a whole snapshot before its first `await`. A configuration change builds a *new* collection rather than mutating the one in use: `Models.streamSimple()` resolves its provider lazily, when the stream is first consumed, which is after the credential await, so a mutated collection would let a request that started under one configuration finish under another or fail on a provider that no longer exists. This is what makes the seam's per-step call freeze (`llm.prepareCall()`) hold end to end — switching models mid-reply takes effect on the next step, never inside the one in flight. Requests reach their provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use.
|
||||
|
||||
@@ -79,6 +79,16 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
## 端点询问
|
||||
|
||||
插件提供 `ctx.llm.registerModelDiscovery('llm-pi-ai', …)`,用来回答「这个提供方能服务哪些模型?」——针对配置界面正在编辑或起草的路由。它刻意**不是** catalog 刷新:什么都不存储,回复是界面供用户采纳的候选。`settings.yaml` 始终是唯一决定路由服务什么的东西。
|
||||
|
||||
点名了**已安装 catalog 所提供路由**的请求,直接由该 catalog 作答,完全不联网:pi-ai 的注册表才是它自家提供方的权威列表,且携带列表端点不会公布的上下文窗口与输出上限。这类路由根本不需要 `baseURL`。只有 catalog 未描述的路由——网关、自建服务——才会经协议层询问;若它也没给端点,则会被告知去设置一个或手工填写模型。
|
||||
|
||||
询问只读 `openai-completions` 与 `openai-responses`,它们「`GET /models` + bearer 认证」的形状是网关、自建服务与官方端点三方一致认可的那一种。Azure 尽管出身 OpenAI 也被排除——它用 `api-key` 标头认证并要求 `api-version` 查询参数——Codex 则走 OAuth;其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答,让界面回退到手工填写,而不是把认证失败报成一个没有模型的提供方。`baseURL` 按前缀而非待解析 URL 处理,因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。
|
||||
|
||||
多数列表只公布 id;`context_window`/`context_length` 与 `max_output_tokens`/`max_tokens` 在网关提供时会被读取,没有可用 id 的条目会被跳过而不是让整份列表失败,其余仍由采纳方补齐。回复在四兆字节上限下读取,且上限落在实际收到的字节上——端点是用户自己填的 URL,因此会先看声明长度,但绝不把它当作边界。端点不可达、凭据被拒、响应非 JSON、以及响应没有 `data` 数组,都会以 `DISCOVERY_FAILED` 失败,消息点名端点;仅当 401 或 403 时才点名凭据。读取响应体期间被取消会呈现为 `ABORTED`,与请求发出之前被取消一致。
|
||||
|
||||
## 提供方/模型路由与回放
|
||||
|
||||
每次解析产出一份**不可变**快照——profiles 加上一个持有各路由所建 `Provider` 的 `createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份快照。配置变化会构造**新**集合,而不是改动正在被使用的那个:`Models.streamSimple()` 是惰性的,它在流首次被消费时才解析 provider,而那已在 credential await 之后,因此改动共享集合会让一个在旧配置下开始的请求在新配置下结束,或者撞上一个已不存在的 provider。这正是 seam 的每步调用冻结(`llm.prepareCall()`)能贯通到底的原因——回复途中切换模型会在下一步生效,绝不会影响在途的那一步。请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方,只替换其模型列表,因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造,表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。
|
||||
|
||||
@@ -191,6 +191,12 @@ export async function discoverModels(
|
||||
'DISCOVERY_FAILED',
|
||||
)
|
||||
}
|
||||
// A draft that has not chosen a protocol yet is asked as OpenAI Chat
|
||||
// Completions: it is the shape a gateway is overwhelmingly likely to speak,
|
||||
// and the alternative — refusing until the field is filled — would withhold
|
||||
// the action from the case it exists for. The cost is a misdirected message
|
||||
// when the endpoint speaks something else (an Anthropic gateway answers 401,
|
||||
// which reads as a credential problem), and hand-entry remains the way out.
|
||||
const api = request.api ?? 'openai-completions'
|
||||
if (!LISTABLE_PROTOCOLS.has(api)) {
|
||||
throw new LlmError(
|
||||
|
||||
@@ -245,7 +245,7 @@ describe('draft-provider model discovery', () => {
|
||||
it('is offered for the namespace, and refuses one it does not serve', async () => {
|
||||
const ctx = await harness()
|
||||
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-pi-ai'])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' })).resolves.not.toHaveLength(0)
|
||||
await expect(ctx.llm.discoverModels('llm-deepseek', { baseURL: 'https://api.deepseek.com' }))
|
||||
.rejects.toMatchObject({ code: 'NO_DISCOVERY' })
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: '' }))
|
||||
@@ -256,10 +256,11 @@ describe('draft-provider model discovery', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmPiAi, {})
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-pi-ai'])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' })).resolves.not.toHaveLength(0)
|
||||
|
||||
await fiber.dispose()
|
||||
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual([])
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { provider: 'openai' }))
|
||||
.rejects.toMatchObject({ code: 'NO_DISCOVERY' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -491,15 +491,6 @@ export class LlmService extends Service {
|
||||
return () => void dispose()
|
||||
}
|
||||
|
||||
/**
|
||||
* List the settings namespaces that can interrogate a provider endpoint, so
|
||||
* a surface can offer the action only where it will work.
|
||||
* @returns the namespaces in registration order.
|
||||
*/
|
||||
listModelDiscoveryNamespaces(): string[] {
|
||||
return [...this.discoveries.keys()]
|
||||
}
|
||||
|
||||
/**
|
||||
* Interrogate one provider endpoint for the models it advertises. The
|
||||
* request describes a draft, not a stored route, so nothing here reads or
|
||||
|
||||
@@ -212,14 +212,15 @@ describe('model discovery registry', () => {
|
||||
const discover = vi.fn(() => Promise.resolve([{ id: 'from-endpoint' }]))
|
||||
|
||||
const dispose = ctx.llm.registerModelDiscovery('llm-example', discover)
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-example'])
|
||||
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.resolves.toEqual([{ id: 'from-endpoint' }])
|
||||
expect(discover).toHaveBeenCalledWith({ baseURL: 'https://gateway.example/v1' })
|
||||
|
||||
// Disposal is observed through the offer itself, which is the only thing
|
||||
// the registration ever produced.
|
||||
dispose()
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual([])
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.rejects.toThrow(/no model discovery is registered/)
|
||||
})
|
||||
|
||||
it('rejects an unnamed namespace and a second registration of the same one', async () => {
|
||||
@@ -229,7 +230,9 @@ describe('model discovery registry', () => {
|
||||
expect(() => ctx.llm.registerModelDiscovery('', discover)).toThrow(/non-empty settings namespace/)
|
||||
ctx.llm.registerModelDiscovery('llm-example', discover)
|
||||
expect(() => ctx.llm.registerModelDiscovery('llm-example', discover)).toThrow(/already registered/)
|
||||
expect(ctx.llm.listModelDiscoveryNamespaces()).toEqual(['llm-example'])
|
||||
// The refused second registration left the first one serving.
|
||||
await expect(ctx.llm.discoverModels('llm-example', { baseURL: 'https://gateway.example/v1' }))
|
||||
.resolves.toEqual([])
|
||||
})
|
||||
|
||||
it('normalizes what an interrogation returns without inventing capacities', async () => {
|
||||
|
||||
Reference in New Issue
Block a user