feat(llm): interrogate a draft provider endpoint for its models

Once a pi-ai route became a declaration rather than a catalog lookup,
adding an OpenAI-compatible gateway meant knowing its model ids up
front. Most such endpoints publish that list at `GET /models`, but no
seam operation could ask: every one is keyed by a registered provider
route, and the provider being added has no route, no stored profile,
and no stored credential — the endpoint and key are values in a form.

Interrogation is therefore keyed by settings namespace, which a
configuration surface already holds from the configurable-provider
directory. `registerModelDiscovery` offers it per namespace,
`discoverModels` asks, and the request carries the draft itself. The
reply is candidates, not a catalog: every field but the id is optional
because most listings disclose nothing else, and adopting one is a
settings write like any other. Nothing here reads or writes settings or
credentials, so `settings.yaml` still decides what a route serves.

`llm.discoverModels` carries the same draft over the wire. Its apiKey is
the third and last payload a secret may ride, and it is never stored,
logged, or echoed; every refusal folds into `model-discovery-failed`,
naming the endpoint asked but never the credential offered.

The pi-ai side is a plain GET for OpenAI-compatible protocols only —
their listing shape is the one gateways, self-hosted servers, and the
official endpoints agree on. Others say so, sending the user to
hand-entry rather than reporting a guessed shape as an empty provider.
The reply is read under a four-megabyte ceiling held on the bytes
actually received, because the endpoint is a URL the user typed.
This commit is contained in:
Yichen Jiang
2026-08-04 10:14:46 +08:00
parent 9948a37cbc
commit ecee93ec26
34 changed files with 985 additions and 18 deletions

View File

@@ -14,6 +14,9 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber. The handle also carries `replace(entries)`: the candidate set is validated in full before anything moves, so an entry another registration already declares leaves the current set intact, and an empty array is legal there. A plugin whose declared set follows its configuration must use `replace` rather than disposing and re-registering — the latter strands the directory empty whenever the new set is refused.
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` List the declared directory in declaration order; configuration surfaces merge it with `listProviders()` to mark each entry live or dormant.
- `ctx.llm.registerModelDiscovery(settingsNs: string, discover): () => void` Offer to interrogate provider endpoints for the settings namespace this plugin owns. One offer per namespace (`INVALID_DISCOVERY`/`DUPLICATE_DISCOVERY`), disposed with the calling fiber.
- `ctx.llm.listModelDiscoveryNamespaces(): string[]` List the namespaces that can interrogate an endpoint, so a surface offers the action only where it works.
- `ctx.llm.discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise<LlmDiscoveredModel[]>` Ask one endpoint which models it advertises.
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context, output-default, and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
@@ -23,6 +26,8 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route is later disposed or replaced. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
Interrogating an endpoint is configuration-time work over a *draft*, which is why it is keyed by settings namespace rather than by provider route: the provider a surface is adding does not exist yet, so there is no route to name. The request carries the endpoint, the protocol, and a credential the harness uses for that one interrogation and never stores — nothing here reads or writes settings or credentials, and the reply is candidate metadata a surface may offer for adoption, never a registered catalog. `LlmDiscoveredModel` makes every field but `id` optional because most provider listings disclose an id and nothing else; a surface adopting one still owes the capacities its adapter requires. Duplicate and unusable ids are dropped, an unserved namespace fails with `NO_DISCOVERY`, and an empty namespace or endpoint fails with `INVALID_DISCOVERY`.
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
Every topology commit point — adapter routes registering or disposing, directory entries appearing or withdrawing — emits the payload-free `llm/adapters-updated` event after the mutation, so consumers re-read `listProviders()`/`listModels()`/`listConfigurableProviders()` instead of polling. Observer failures are contained (logged, non-vetoing); only `INVARIANT`-coded failures rethrow after the fan-out.