feat(llm): interrogate a draft provider endpoint for its models

Once a pi-ai route became a declaration rather than a catalog lookup,
adding an OpenAI-compatible gateway meant knowing its model ids up
front. Most such endpoints publish that list at `GET /models`, but no
seam operation could ask: every one is keyed by a registered provider
route, and the provider being added has no route, no stored profile,
and no stored credential — the endpoint and key are values in a form.

Interrogation is therefore keyed by settings namespace, which a
configuration surface already holds from the configurable-provider
directory. `registerModelDiscovery` offers it per namespace,
`discoverModels` asks, and the request carries the draft itself. The
reply is candidates, not a catalog: every field but the id is optional
because most listings disclose nothing else, and adopting one is a
settings write like any other. Nothing here reads or writes settings or
credentials, so `settings.yaml` still decides what a route serves.

`llm.discoverModels` carries the same draft over the wire. Its apiKey is
the third and last payload a secret may ride, and it is never stored,
logged, or echoed; every refusal folds into `model-discovery-failed`,
naming the endpoint asked but never the credential offered.

The pi-ai side is a plain GET for OpenAI-compatible protocols only —
their listing shape is the one gateways, self-hosted servers, and the
official endpoints agree on. Others say so, sending the user to
hand-entry rather than reporting a guessed shape as an empty provider.
The reply is read under a four-megabyte ceiling held on the bytes
actually received, because the endpoint is a URL the user typed.
This commit is contained in:
Yichen Jiang
2026-08-04 10:14:46 +08:00
parent 9948a37cbc
commit ecee93ec26
34 changed files with 985 additions and 18 deletions

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-04-draft-provider-endpoint-interrogation.md
2026-08-04-draft-provider-endpoint-interrogation.md: 86b3148626fea90f1d87b80084f7cd4bafeeb1f8
2026-08-04-draft-provider-endpoint-interrogation.zh.md: 0f6a63385dc628938c702aca1895b608e4eeaf9a

View File

@@ -0,0 +1,50 @@
# Agent Note: Interrogating a draft provider endpoint
Status: implemented
English | [中文](2026-08-04-draft-provider-endpoint-interrogation.zh.md)
## Problem
Once a pi-ai route became [a declaration rather than a catalog lookup](2026-08-03-pi-ai-declared-provider-catalog.md), a person adding an OpenAI-compatible gateway had to know its model ids before they could configure it. The adapter no longer constrains them to an installed catalog, which is the point, but it also means nothing tells the user what the endpoint actually serves — and most of these endpoints do publish that list at `GET /models`.
The obvious answer, a dynamic runtime catalog refreshed in the background, was rejected with the layer below it: it makes a route's model list external mutable state needing a cache, an invalidation story, and an offline path, while the product need is narrower. What is needed is a *question asked once*, whose answer the user adopts into `settings.yaml` — so `settings.yaml` remains the only thing deciding what a route serves.
The awkward part is that the question is about something that does not exist yet. The provider being added has no route, no stored profile, and no stored credential; the endpoint and key are values in a form the user is still typing. Every existing seam operation is keyed by a registered provider route, so none of them can carry this.
## Decision
Interrogation is keyed by **settings namespace**, not by provider route:
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` lets an adapter plugin offer to interrogate endpoints for the namespace it owns; `ctx.llm.listModelDiscoveryNamespaces()` lets a surface offer the action only where it works; `ctx.llm.discoverModels(settingsNs, request)` asks. The namespace is the right key because a configuration surface already holds it from the configurable-provider directory, and because a provider being added has no route to name.
- `LlmModelDiscoveryRequest` carries the draft — `baseURL`, an optional `api`, an optional `apiKey`, and a signal. Nothing in this path reads or writes settings or credentials; the caller owns both.
- `LlmDiscoveredModel` makes every field but `id` optional, because most listings disclose an id and nothing else. The reply is candidates, not a catalog: a surface adopting one still owes the capacities the adapter requires.
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored, logged, or echoed. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
`dsh-llm-pi-ai` implements it as a plain `GET {baseURL}/models` for OpenAI-compatible protocols only. Their listing shape is the one a gateway, a self-hosted server, and the official endpoints all agree on, which is the case this action exists for. Every other protocol answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting a guessed response shape as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage shape for its own caller-supplied URLs.
### Why not pi-ai's own refresh machinery
pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `ModelsStore`, and the layer below already builds pi-ai `Provider` objects. Routing interrogation through them would have meant constructing a throwaway provider and collection per question, with a store whose entire purpose — persisting a catalog across runs — contradicts the decision that `settings.yaml` owns the catalog. It would also have bought nothing: **no built-in pi-ai provider implements `fetchModels`**, so the HTTP call and its response parsing are this package's code either way. A direct fetch says what is actually happening.
## Alternatives considered
**Key interrogation by provider route.** Symmetric with every other seam operation, and it would let the request omit the endpoint. But the case that motivates the feature — adding a provider — has no route, so the operation would only work for providers already configured, which are the ones that need it least.
**Put the capability on `LlmAdapter`.** Adapters are reached through a route registration, so this has the same problem, plus it would make an adapter instance answer questions about endpoints it does not serve.
**Have the host read the stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. Accepting the draft keeps what the user sees and what is asked identical.
**Interrogate every pi-ai protocol.** Anthropic's listing happens to share OpenAI's envelope, and Google's does not. Supporting the ones that are easy would make coverage arbitrary and, worse, make a wrong guess at a response shape indistinguishable from a provider with no models. A protocol that says it cannot be interrogated sends the user to hand-entry, which is the documented fallback.
**Buffer the reply with `response.text()` and check its length.** Simpler, but the bound would arrive after the bytes did, and the endpoint is whatever URL the user typed.
## Consequences
A person adding a gateway can ask it what it serves instead of hunting through its documentation, and the answer arrives as candidates they choose from rather than as configuration written behind their back. The seam gained a registry that is deliberately small: one offer per namespace, no storage, no lifecycle beyond the fiber.
What it costs: the wire gained a third secret-carrying payload, so the configuration plane's write-only surface is now three methods rather than two. Discovery coverage is protocol-shaped rather than provider-shaped — an Anthropic-compatible gateway must be filled in by hand even though its listing would parse. And because nothing re-runs the question, a model list is still only as current as its last edit; that is the same trade the layer below made deliberately.
## Testing
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, and the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — a listing with and without disclosed capacities, a preserved deployment path, an absent credential, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/host/apiproxy/tests/api-proxy-config.spec.ts` covers the RPC over a real proxy: the draft reaching its namespace whole, absent fields staying absent, no namespace or credential being written, and a failure surfacing as `model-discovery-failed` with the credential absent from the serialized error.

View File

@@ -0,0 +1,50 @@
# Agent Note: 询问草稿中的提供方端点
Status: implemented
[English](2026-08-04-draft-provider-endpoint-interrogation.md) | 中文
## Problem
当 pi-ai 路由变成[一份声明而非 catalog 查表](2026-08-03-pi-ai-declared-provider-catalog.md)之后,要接入一个 OpenAI 兼容网关的人,必须先知道它的模型 id 才能完成配置。适配器不再把人限制在已安装 catalog 里——这正是那次改动的目的——但也意味着没有任何东西告诉用户该端点究竟服务什么,而这类端点大多在 `GET /models` 上公布了这份列表。
显而易见的答案——后台刷新的运行时动态 catalog——已随下层一并被拒绝它会把路由的模型列表变成需要缓存、失效语义与离线路径的外部可变状态而产品需求要窄得多。真正需要的是**只问一次**,其答案由用户采纳进 `settings.yaml`——从而让 `settings.yaml` 始终是唯一决定路由服务什么的东西。
麻烦之处在于,被问的对象还不存在。正在新增的提供方没有路由、没有已存 profile、也没有已存凭据端点与密钥都是用户尚在输入的表单值。而现有的每个 seam 操作都以已注册的提供方路由为键,因此没有一个能承载它。
## Decision
询问以 **settings namespace** 为键,而不是提供方路由:
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` 让适配器插件为自己拥有的 namespace 提供「询问端点」的能力;`ctx.llm.listModelDiscoveryNamespaces()` 让界面只在可用之处提供该动作;`ctx.llm.discoverModels(settingsNs, request)` 发起询问。以 namespace 为键是对的,因为配置界面已经从可配置提供方目录里拿到了它,也因为正在新增的提供方没有路由可点名。
- `LlmModelDiscoveryRequest` 携带草稿——`baseURL`、可选的 `api`、可选的 `apiKey`,以及一个 signal。这条路径既不读也不写 settings 与 credentials两者都归调用方所有。
- `LlmDiscoveredModel``id` 外每个字段都可选,因为大多数列表只公布 id。回复是候选而非 catalog采纳其中一条的界面仍要补上适配器所需的容量。
- `llm.discoverModels` 把同一份草稿送过协议层。它的 `apiKey` 是 secret 可以搭乘的第三个、也是最后一个载荷(另两个是 `settings.update`/`mutate``credentials.set`),且绝不被存储、记录或回显。每一种拒绝都折叠为 `model-discovery-failed`其消息是适配器自己的文本details 点名被询问的端点,绝不点名所提供的凭据。
`dsh-llm-pi-ai` 的实现只是一次朴素的 `GET {baseURL}/models`,且仅限 OpenAI 兼容协议。它们的列表形状是网关、自建服务与官方端点三方一致认可的那一种,而这正是该动作存在的场景。其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答,让界面回退到手工填写,而不是把猜错的响应形状报成一个空提供方。`baseURL` 按前缀而非待解析 URL 处理,因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。回复在四兆字节上限下读取,且上限落在实际收到的字节上——端点是用户自己填的 URL因此会先看声明的 `content-length` 作为善意提示,但绝不把它当作边界;这与 `dsh-web-fetch` 面对自己的调用方提供 URL 时所用的两段式形状一致。
### 为什么不用 pi-ai 自己的 refresh 机制
pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()``ModelsStore`,而下层本来就在构造 pi-ai `Provider` 对象。把询问接到它们上面,意味着每问一次就要构造一个用完即弃的 provider 与集合,而那个 store 的全部目的——跨运行持久化 catalog——恰恰与「`settings.yaml` 拥有 catalog」的决定相抵触。而且它什么也换不来**没有任何一个 pi-ai 内置 provider 实现了 `fetchModels`**,因此 HTTP 调用及其响应解析无论如何都是本包的代码。直接 fetch 才如实说出正在发生的事。
## Alternatives considered
**以提供方路由为键。** 与其他每个 seam 操作对称,也能让请求省去端点。但催生该功能的场景——新增提供方——没有路由,于是这个操作只对已配置好的提供方可用,而它们恰恰最不需要它。
**把能力挂在 `LlmAdapter` 上。** 适配器要经由路由注册才能抵达,因此问题相同;而且这会让一个适配器实例去回答它并不服务的端点的问题。
**让 host 读已存 profile而不是接受草稿。** 对已配置好的提供方来说,不会有 secret 跨越协议层。但这样一来新增提供方就必须先保存一份不可用的配置,而端点已改却尚未保存的表单会静默地去询问旧地址。接受草稿让用户看见的与被询问的保持一致。
**询问 pi-ai 的每一种协议。** Anthropic 的列表恰好与 OpenAI 共用同一层信封,而 Google 的不是。只支持容易的那几种会让覆盖范围变得任意;更糟的是,猜错的响应形状会与「该提供方没有模型」无法区分。一个明说自己无法被询问的协议,会把用户送去手工填写——那正是既定的回退路径。
**用 `response.text()` 缓冲整个回复再判断长度。** 更简单,但上限会在字节已经到达之后才生效,而端点是用户随手填的任意 URL。
## Consequences
接入网关的人可以直接问它服务什么而不必去翻它的文档答案以候选形式抵达由用户自己挑选而不是被背着写进配置。seam 因此多了一个刻意保持很小的注册表:每个 namespace 一份、不存储、除 fiber 外没有生命周期。
代价是:协议层多了第三个承载 secret 的载荷,配置面的只写接口从两个方法变成三个。发现能力按协议而非按提供方划分——一个 Anthropic 兼容网关即便其列表能被解析,也仍须手工填写。而且由于没有任何环节会重跑该询问,模型列表的新鲜度依旧只到最近一次编辑为止;这与下层刻意做出的取舍是同一个。
## Testing
`packages/llm/llm/tests/topology.spec.ts` 覆盖注册表:每个 namespace 一份、随 fiber dispose、丢弃重复与不可用 id 且不凭空补容量的归一化,以及 `NO_DISCOVERY`/`INVALID_DISCOVERY` 两种拒绝。`packages/llm/llm-pi-ai/tests/discovery.spec.ts` 针对本地 HTTP 服务器驱动探测——含与不含公布容量的列表、被保留的部署路径、无凭据、被丢弃的行、401/403 与服务器故障之别、非列表与非 JSON 响应、不可达端点、调用方取消、不支持的协议,以及尺寸上限的「声明长度」与「流式」两种形态。`packages/host/apiproxy/tests/api-proxy-config.spec.ts` 在真实 proxy 上覆盖该 RPC草稿完整抵达其 namespace、缺席字段保持缺席、没有 namespace 或凭据被写入,以及失败以 `model-discovery-failed` 呈现且序列化后的错误里不含凭据。