fix(llm): capture an immutable snapshot per pi-ai operation

Review found four defects in the declared-provider work.

`PiAiAdapter` reused one `Models` collection and mutated it whenever the
configuration changed. `Models.streamSimple()` resolves its provider
lazily — when the stream is first consumed, which is after the adapter
awaits the route's credential — so a configuration change landing in
that window let an in-flight request finish under a configuration it
never resolved against, or fail on a provider that no longer existed.
Each resolution now produces an immutable snapshot and every operation
captures one before its first await, which is what makes the seam's
per-step freeze (`llm.prepareCall()`) hold end to end: switching models
mid-reply takes effect on the next step, never inside the one in flight.

`defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`.
The two answer different questions: pi-ai requires that field as the
model's output capability, while the seam's is a cap the deployment
chose to send on requests naming none, so every request had started
carrying a number nobody picked. Only an explicitly configured cap
reaches the seam now.

The configurable-provider directory was refreshed by disposing its
registration and making a new one. A candidate set the registry refuses
— a profile keyed `deepseek-official`, which llm-deepseek declares —
left the whole directory withdrawn and the Models page empty, silently,
because the settings callback contains the failure. The seam's
registration handle now carries `replace()` with the same
validate-first atomicity `registerAdapter` has.

The protocol table offered every pi-ai streaming API, including four
whose authentication a profile cannot express: Bedrock signs with SigV4
over AWS credentials and a region, Vertex needs a project, a location,
and ADC, Azure needs provider environment plus an api-version, and Codex
uses OAuth. Offering them handed back routes that cannot authenticate.
Catalog routes still reach them through their own provider.
This commit is contained in:
Yichen Jiang
2026-08-04 11:42:35 +08:00
parent d6126c25f2
commit 4c80cab108
21 changed files with 476 additions and 104 deletions

View File

@@ -421,8 +421,8 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
jsDoc: '/**\n * Describe provider routes with a registered adapter.\n * @returns detached provider metadata in registration order.\n */',
},
{
signature: 'registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void',
jsDoc: '/**\n * Declare provider routes an adapter plugin can activate through\n * configuration. Registration is all-or-nothing: an empty list, invalid\n * entry, or a provider already declared by any registration throws\n * `LlmError` without registering the rest. Disposed with the fiber.\n * @param entries - every configurable provider this plugin owns.\n * @returns the disposer that withdraws all of them.\n */',
signature: 'registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle',
jsDoc: '/**\n * Declare provider routes an adapter plugin can activate through\n * configuration. Registration is all-or-nothing: an empty list, invalid\n * entry, or a provider already declared by any registration throws\n * `LlmError` without registering the rest. Disposed with the fiber.\n * @param entries - every configurable provider this plugin owns.\n * @returns a handle that withdraws all of them, and can atomically replace them.\n */',
},
{
signature: 'listConfigurableProviders(): LlmConfigurableProvider[]',
@@ -1889,6 +1889,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'DirectoryPickerNativeCapability',
declaration: 'export interface DirectoryPickerNativeCapability {\n kind: \'native\';\n pick(signal: AbortSignal): Promise<string | null>;\n}',
},
{
name: 'DirectoryRegistrationHandle',
declaration: 'export interface DirectoryRegistrationHandle {\n (): void;\n replace(entries: readonly LlmConfigurableProvider[]): void;\n}',
},
{
name: 'Domain',
declaration: 'export interface Domain<S extends DomainSpec> {\n readonly name: string;\n readonly global: DomainGlobalHandleOf<S>;\n table<N extends keyof S[\'tables\'] & string>(name: N): KvTable<TableKeyOf<S, N>, TableValueOf<S, N>>;\n close(): Promise<void>;\n}',

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: e597eedeb4d6e0ebf402b5547f71c9aff370d3dd
README.zh.md: e28105f1253c138b9bb0baf5d00e0c7eba0d7b52
README.md: dde6ce989a0fd87bf2dc60ce0d85deb1856d92d5
README.zh.md: bb87ada7b5cf883b458c8cf9ace66bbd64fa154f

View File

@@ -55,17 +55,19 @@ The dict shape makes duplicate routes unrepresentable, and the pre-release array
A profile's `models` list *replaces* the route's installed catalog rather than extending it; omitting it (or leaving it empty) serves that catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a catalog route to two models, correcting one capacity, or adding a model newer than the installed catalog are all one-line edits. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, `maxTokens`, and `reasoning`; pricing and input modalities have no harness consumer and ride the installed entry or are absent, while reasoning-level spellings and OpenAI-compatibility quirks have no configuration surface at all because restating them cannot be validated.
Resolution fails loud, naming the offending route and model, when a route cannot be served: a model the installed catalog does not describe needs an explicit `contextWindow` and `maxTokens`, and a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list. `api` accepts the protocols in `supportedProtocols()` — pi-ai's own streaming API set — and is only needed when the catalog cannot supply one: a model absent from the catalog inherits the protocol its shipped siblings agree on, so adding a model to a single-protocol catalog route restates nothing.
Resolution fails loud, naming the offending route and model, when a route cannot be served: a model the installed catalog does not describe needs an explicit `contextWindow` and `maxTokens`, and a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list. `api` accepts the protocols in `supportedProtocols()` and is only needed when the catalog cannot supply one: a model absent from the catalog inherits the protocol its shipped siblings agree on, so adding a model to a single-protocol catalog route restates nothing.
`baseURL` sets the endpoint of every model on the route, so private proxies such as `https://proxy.example.com:8443` remain supported; a catalog route that omits it keeps each catalog model's own endpoint. Naming `api` on a catalog route repoints the whole route at that protocol, which is how a deployment moves a provider between, say, Responses and Chat Completions.
`supportedProtocols()` is deliberately narrower than pi-ai's full streaming API set: it holds only the protocols a profile can *completely* describe with a key, an endpoint, and headers. Bedrock signs with SigV4 over AWS credentials and a region, Vertex needs a project, a location, and application-default credentials, Azure needs provider environment plus an api-version, and Codex authenticates through OAuth — offering those would hand back a route that cannot authenticate. Catalog routes still reach them through their own provider; only an explicit override is refused.
## Dynamic configuration (settings + credentials)
The adapter reads its profiles through a thunk **once per operation** instead of freezing them at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`, and because `providers` is a dict, the base and the user's `llm-pi-ai:` settings section merge **per provider**: a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. Without a mounted settings service the entry config alone drives the adapter, unchanged.
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; exactly that variable without a mounted seam). A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin replaces its registration atomically (same adapter instance, candidate set validated first), so a route another adapter already owns leaves the previous routes serving and reverting to a working configuration re-applies. Provider key order never counts as a change. A live settings snapshot naming an unknown provider (or failing any other resolver bound) keeps the last good profiles and logs the failure; the entry config itself still fails plugin load.
The adapter exposes each configured route's models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata read from the same pi-ai `Models` collection the request path uses, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, configured output cap, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers. A model's `maxTokens` becomes the seam's `defaultMaxTokens`, so a request that names no output cap carries the configured one.
The adapter exposes each configured route's models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata read from the same pi-ai `Models` collection the request path uses, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, configured output cap, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers. A model's **configured** `maxTokens` becomes the seam's `defaultMaxTokens`, so a request that names no output cap carries the one the deployment chose; a value inherited from the installed catalog is the model's output *capability* and never becomes a request default on its own.
The `reasoning.efforts` list is pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model therefore exposes pi-ai's `off` choice. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. pi-ai's common stream options represent `off` by omitting `reasoning`.
@@ -75,7 +77,7 @@ The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes
## Provider/model routing and replay
Each resolved route contributes one pi-ai `Provider` to the adapter's `createModels()` collection, and requests reach the provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use.
Each resolution produces one **immutable** snapshot — the profiles plus a `createModels()` collection holding the `Provider` each route built — and every operation captures a whole snapshot before its first `await`. A configuration change builds a *new* collection rather than mutating the one in use: `Models.streamSimple()` resolves its provider lazily, when the stream is first consumed, which is after the credential await, so a mutated collection would let a request that started under one configuration finish under another or fail on a provider that no longer exists. This is what makes the seam's per-step call freeze (`llm.prepareCall()`) hold end to end — switching models mid-reply takes effect on the next step, never inside the one in flight. Requests reach their provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use.
Credentials never enter that collection. The harness resolves a route's key through its own seam before the request reaches pi-ai and passes it as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override; `Models` therefore holds no credential store, and the harness keeps its fail-loud reference semantics. A route naming no credential resolves as configured-but-keyless and leaves the requirement to the protocol, which is where it actually lives.

View File

@@ -55,17 +55,19 @@
profile 的 `models` 列表是*替换*该路由已安装 catalog而不是扩充它省略它或留空则原样服务该 catalog。每个条目都会从同 `id` 的已安装模型继承自身未设置的字段,因此把 catalog 路由收窄到两个模型、更正某个容量,或加入一个比已安装 catalog 更新的模型,都是一行编辑。只有 harness 会消费的字段可配置——`id``name``contextWindow``maxTokens``reasoning`;定价与输入模态没有 harness 消费方,因此沿用已安装条目或直接缺席,而思考级别的协议拼写与 OpenAI 兼容性怪癖则完全没有配置面,因为重述它们无法被校验。
解析会失败得响亮,并点名出问题的路由与模型:已安装 catalog 未描述的模型需要显式的 `contextWindow``maxTokens`catalog 未提供的路由则需要 `api``baseURL` 和非空的 `models` 列表。`api` 接受 `supportedProtocols()` 中的协议——即 pi-ai 自己的流式 API 集合——且仅在 catalog 无法提供协议时才需要catalog 中不存在的模型会继承其同门模型一致同意的协议,因此向单协议 catalog 路由添加模型无需重述任何内容。
解析会失败得响亮,并点名出问题的路由与模型:已安装 catalog 未描述的模型需要显式的 `contextWindow``maxTokens`catalog 未提供的路由则需要 `api``baseURL` 和非空的 `models` 列表。`api` 接受 `supportedProtocols()` 中的协议且仅在 catalog 无法提供协议时才需要catalog 中不存在的模型会继承其同门模型一致同意的协议,因此向单协议 catalog 路由添加模型无需重述任何内容。
`baseURL` 设定该路由下每个模型的端点,因此仍支持 `https://proxy.example.com:8443` 等私有 proxy省略它的 catalog 路由会保留每个 catalog 模型自己的端点。在 catalog 路由上点名 `api` 会把整条路由改指到该协议,这正是部署把某个提供方在 Responses 与 Chat Completions 之间迁移的方式。
`supportedProtocols()` 刻意窄于 pi-ai 的完整流式 API 集合:它只保留 profile 能用密钥、端点与标头**完整描述**的那些协议。Bedrock 要用 AWS 凭据与 region 做 SigV4 签名Vertex 需要 project、location 与应用默认凭据Azure 需要提供方环境外加 api-versionCodex 走 OAuth——提供它们只会交回一个无法完成认证的路由。catalog 路由仍可经自己的 provider 抵达这些协议;被拒绝的只有显式覆盖。
## 动态配置settings + credentials
适配器经由一个 thunk **每操作读取一次** profile而非在构造期冻结。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace并以其 `cordis.yml` 条目为组合 `base`;由于 `providers` 是字典base 与用户的 `llm-pi-ai:` settings 分节**按提供方**合并:用户可以新增路由、覆盖组合路由的单个字段,或把路由指向另一个 proxy全部在下一次请求生效无需重启。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。
凭据按每次 stream 调用解析:非空的字面 `apiKey` 优先,其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`(活跃环境之下的 `$DSH_HOME/.env`;未挂载 seam 时恰好读取该环境变量)。只有完全没有点名任何凭据的 profile——仅限这一种情况——才交给 pi-ai 的环境发现。路由集合与每条路由捕获的重试策略是注册级事实:两者任一变化时,插件都会原子地替换自己的注册(同一适配器实例,候选集合先经校验),因此某条路由若已被另一适配器占有,先前的路由会继续服务,而改回可用配置时注册会重新生效。提供方键的顺序绝不算作变化。存活 settings 快照若点名未知提供方(或违反任何其他 resolver 约束),则保留最后可用 profile 并记录失败entry 配置本身仍会使插件加载失败。
适配器通过 `ctx.llm.listModels(provider)` 公开每条已配置路由的模型。这是从请求路径所用的同一个 pi-ai `Models` 集合读取的提供方无关 selector 元数据,因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找,并返回其身份、上下文窗口、已配置输出上限和可选思考级别,让权威元数据保留在拥有路由的适配器上,而非消费方。模型的 `maxTokens` 会成为 seam 的 `defaultMaxTokens`,因此未点名输出上限的请求会携带已配置的那一个
适配器通过 `ctx.llm.listModels(provider)` 公开每条已配置路由的模型。这是从请求路径所用的同一个 pi-ai `Models` 集合读取的提供方无关 selector 元数据,因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找,并返回其身份、上下文窗口、已配置输出上限和可选思考级别,让权威元数据保留在拥有路由的适配器上,而非消费方。模型**已配置**`maxTokens` 会成为 seam 的 `defaultMaxTokens`,因此未点名输出上限的请求会携带部署选定的那一个;而从已安装 catalog 继承来的值是模型的输出**能力**,绝不会自行变成请求默认值
`reasoning.efforts` 列表是 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果,不经筛选或规范化,其中包括 `off`,以及模型对 `xhigh``max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID提供方模型在协议格式中的表示仍保留在 pi-ai 的 `thinkingLevelMap` 中。因此不具备推理reasoning能力的模型也会公开 pi-ai 的 `off` 选项。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;任何未出现在确切模型能力中的显式值都会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败而不会被自动调整。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`
@@ -75,7 +77,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog而不是扩
## 提供方/模型路由与回放
条已解析路由都会向适配器的 `createModels()` 集合贡献一个 pi-ai `Provider`请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方,只替换其模型列表,因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造,表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。
次解析产出一份**不可变**快照——profiles 加上一个持有各路由所建 `Provider``createModels()` 集合——每个操作都在自己第一个 `await` 之前整体捕获一份快照。配置变化会构造**新**集合,而不是改动正在被使用的那个:`Models.streamSimple()` 是惰性的,它在流首次被消费时才解析 provider而那已在 credential await 之后,因此改动共享集合会让一个在旧配置下开始的请求在新配置下结束,或者撞上一个已不存在的 provider。这正是 seam 的每步调用冻结(`llm.prepareCall()`)能贯通到底的原因——回复途中切换模型会在下一步生效,绝不会影响在途的那一步。请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方,只替换其模型列表,因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造,表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。
凭据绝不进入该集合。harness 在请求抵达 pi-ai 之前经自身 seam 解析路由密钥,并作为请求的 `apiKey` 选项传入,而 pi-ai 将其视为优先级最高的 auth 覆盖;因此 `Models` 不持有任何凭据存储harness 也保住了自己失败得响亮的引用语义。没有点名任何凭据的路由会解析为「已配置但无密钥」,把该要求留给协议——那才是它真正所在的位置。

View File

@@ -1,10 +1,17 @@
/**
* Generic pi-ai-backed implementation of the Harness LLM seam.
*
* The adapter owns one pi-ai `Models` collection and keeps it in step with the
* resolved profiles: each route contributes the `Provider` its resolution built,
* so model lookup, protocol dispatch, and request auth all reach pi-ai through
* its supported runtime rather than the deprecated global compatibility entry.
* Each resolution produces one **immutable** snapshot — the profiles plus a
* `Models` collection holding the `Provider` each route built — and an
* operation captures a whole snapshot before its first `await`. A
* configuration change builds a *new* collection rather than mutating the one
* in use, because `Models.streamSimple()` is lazy: it resolves the provider
* when the stream is first consumed, which is after the credential await, so a
* mutated collection would let a request that started under one configuration
* finish under another — or fail with a provider that no longer exists. This is
* what makes the seam's per-step call freeze (`llm.prepareCall()`) hold all the
* way down: switching models mid-reply takes effect on the next step, never
* inside the one in flight.
*
* Credentials stay outside that collection. The harness resolves a route's key
* through its own seam and passes it as the request's `apiKey` option, which
@@ -18,6 +25,7 @@ import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
import type {
Api,
Model,
Models,
ModelThinkingLevel,
MutableModels,
SimpleStreamOptions,
@@ -42,6 +50,14 @@ import type { ResolvedPiAiProviderProfile } from './config.ts'
import { toPiContext } from './context.ts'
import { toStreamChunks } from './stream.ts'
/** One resolution's frozen view: the profiles and the collection built from them. */
interface PiAiSnapshot {
/** The resolved profiles this collection was built from, used as its identity. */
profiles: ReadonlyMap<string, ResolvedPiAiProviderProfile>
/** Providers for exactly those profiles; never mutated once published. */
models: Models
}
/** Constructor options for {@link PiAiAdapter}: the two resolution seams the plugin owns. */
export interface PiAiAdapterOptions {
/** Current validated profiles by provider route; called once per operation. */
@@ -107,41 +123,40 @@ function requestHeaders(headers: Readonly<Record<string, string>> | undefined):
* restart; model descriptors come from the collection those profiles built.
*/
export class PiAiAdapter extends LlmAdapter {
private readonly models: MutableModels = createModels()
private registered: ReadonlyMap<string, ResolvedPiAiProviderProfile> | undefined
private snapshot: PiAiSnapshot | undefined
constructor(private readonly config: PiAiAdapterOptions) {
super()
}
/**
* The `Models` collection for the current profiles. Resolution memoizes its
* result, so an unchanged configuration is recognized by identity and the
* collection is rebuilt only when the route set or any profile actually
* changes.
* The snapshot for the current profiles. Resolution memoizes its result, so
* an unchanged configuration is recognized by identity; a changed one gets a
* brand-new collection, leaving any snapshot an operation already captured
* untouched for as long as that operation holds it.
*/
private collection(): MutableModels {
private current(): PiAiSnapshot {
const profiles = this.config.profiles()
if (profiles === this.registered) return this.models
this.models.clearProviders()
for (const profile of profiles.values()) this.models.setProvider(profile.piProvider)
this.registered = profiles
return this.models
if (this.snapshot?.profiles === profiles) return this.snapshot
const models: MutableModels = createModels()
for (const profile of profiles.values()) models.setProvider(profile.piProvider)
this.snapshot = { profiles, models }
return this.snapshot
}
/** The profile for one route, or the seam's own not-owned failure. */
private profileOf(provider: string): ResolvedPiAiProviderProfile {
const profile = this.config.profiles().get(provider)
/** The profile for one route within one snapshot, or the not-owned failure. */
private profileOf(snapshot: PiAiSnapshot, provider: string): ResolvedPiAiProviderProfile {
const profile = snapshot.profiles.get(provider)
if (profile === undefined) {
throw new LlmError(`pi-ai adapter does not own provider "${provider}"`, 'NO_ADAPTER')
}
return profile
}
/** The configured descriptor for one exact route/model pair. */
private modelOf(provider: string, model: string): Model<Api> {
this.profileOf(provider)
const resolved = this.collection().getModel(provider, model)
/** The configured descriptor for one exact route/model pair within one snapshot. */
private modelOf(snapshot: PiAiSnapshot, provider: string, model: string): Model<Api> {
this.profileOf(snapshot, provider)
const resolved = snapshot.models.getModel(provider, model)
if (resolved === undefined) {
throw new LlmError(`pi-ai provider "${provider}" has no configured model "${model}"`, 'UNKNOWN_MODEL')
}
@@ -149,13 +164,14 @@ export class PiAiAdapter extends LlmAdapter {
}
override providerRetryPolicy(provider: string): ResolvedRetryPolicy | undefined {
return this.config.profiles().get(provider)?.retryPolicy
return this.current().profiles.get(provider)?.retryPolicy
}
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
return Promise.resolve().then(() => {
this.profileOf(provider)
return this.collection().getModels(provider).map(model => ({
const snapshot = this.current()
this.profileOf(snapshot, provider)
return snapshot.models.getModels(provider).map(model => ({
provider,
id: model.id,
name: model.name,
@@ -169,16 +185,20 @@ export class PiAiAdapter extends LlmAdapter {
_signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
return Promise.resolve().then(() => {
const profile = this.profileOf(provider)
const resolvedModel = this.modelOf(provider, model)
const snapshot = this.current()
const profile = this.profileOf(snapshot, provider)
const resolvedModel = this.modelOf(snapshot, provider, model)
const levels = getSupportedThinkingLevels(resolvedModel)
const defaultLevel = resolveReasoningLevel(resolvedModel, profile.reasoning)
// Only a cap the deployment configured is a request default; the
// catalog's `maxTokens` sizes the model and stops there.
const configuredMaxTokens = profile.configuredMaxTokens.get(model)
return {
provider,
id: model,
name: resolvedModel.name,
context: { contextWindow: resolvedModel.contextWindow },
defaultMaxTokens: resolvedModel.maxTokens,
...configuredMaxTokens === undefined ? {} : { defaultMaxTokens: configuredMaxTokens },
reasoning: {
efforts: levels.map(level => ({
id: ReasoningEffortId(level),
@@ -196,13 +216,14 @@ export class PiAiAdapter extends LlmAdapter {
if (options.stop !== undefined) {
throw new LlmError('llm-pi-ai does not support GenerateOptions.stop', 'UNSUPPORTED_OPTION')
}
// One resolution per stream call: the profile snapshot, the model
// descriptor, and the credential freeze here and hold for this whole
// request, so an in-flight stream never observes a configuration change and
// the next call re-resolves.
const profile = this.profileOf(options.provider)
const collection = this.collection()
const model = this.modelOf(options.provider, options.model)
// One capture per stream call, taken before any await: the profile, the
// model descriptor, and the collection all come from the same immutable
// snapshot, and the credential freezes with them. A configuration change
// mid-request builds a separate snapshot, so this request finishes under
// the one it started with and the next call picks up the new one.
const snapshot = this.current()
const profile = this.profileOf(snapshot, options.provider)
const model = this.modelOf(snapshot, options.provider, options.model)
const reasoning = resolveReasoningLevel(
model,
options.reasoningEffort ?? profile.reasoning,
@@ -217,7 +238,7 @@ export class PiAiAdapter extends LlmAdapter {
using watchdog = idleWatchdog(upstream, streamIdleTimeoutMs, 'LLM_STREAM_IDLE_TIMEOUT')
try {
const events = collection.streamSimple(model, toPiContext(options), {
const events = snapshot.models.streamSimple(model, toPiContext(options), {
...profileOptions(profile, reasoning, apiKey),
...options.temperature === undefined ? {} : { temperature: options.temperature },
...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },

View File

@@ -79,7 +79,11 @@ export interface PiAiModelProfile {
name?: string
/** Maximum combined request and response context in tokens. */
contextWindow?: number
/** Per-request output cap materialized when a caller omits one. */
/**
* Maximum output tokens. Configuring one also makes it this model's
* per-request default; the value inherited from the installed catalog is the
* model's capability and never becomes a request default on its own.
*/
maxTokens?: number
/** Whether the model exposes reasoning; defaults to the catalog capability. */
reasoning?: boolean
@@ -116,15 +120,32 @@ function sharedCatalogApi(defaults: ReadonlyMap<string, Model<Api>>): string | u
return apis.size === 1 ? [...apis][0] : undefined
}
/** One route's materialized catalog, plus the request caps its profile chose. */
export interface RouteCatalog {
/** The materialized models in configuration order. */
models: readonly Model<Api>[]
/**
* Per-request output caps this profile explicitly configured, by model id.
*
* Separate from `Model.maxTokens` because the two answer different
* questions: pi-ai requires `maxTokens` as the model's output *capability*,
* while the harness seam's `defaultMaxTokens` is a cap the deployment chose
* to send on requests that name none. Materializing a catalog capability as
* a request default would start capping every request at a number nobody
* picked, so only an explicit configuration lands here.
*/
configuredMaxTokens: ReadonlyMap<string, number>
}
/**
* Materialize one route's catalog by merging the installed catalog defaults
* under the configured entries. A route with no configured `models` serves the
* installed catalog unchanged, which is what keeps an existing
* `providers: { deepseek: { apiKeyEnv: … } }` profile working untouched.
* @param request - the route-level catalog facts.
* @returns the materialized models in configuration order.
* @returns the materialized models and the explicitly configured request caps.
*/
export function resolveRouteModels(request: RouteCatalogRequest): readonly Model<Api>[] {
export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
const { provider } = request
const defaults = catalogModels(provider)
const providerBaseUrl = catalogProvider(provider)?.baseUrl
@@ -141,7 +162,8 @@ export function resolveRouteModels(request: RouteCatalogRequest): readonly Model
}
const routeApi = sharedCatalogApi(defaults)
const seen = new Set<string>()
return entries.map((entry) => {
const configuredMaxTokens = new Map<string, number>()
const models = entries.map((entry) => {
if (entry.id.length === 0) invalid(provider, 'has a model with an empty id')
if (seen.has(entry.id)) invalid(provider, `lists model "${entry.id}" more than once`)
seen.add(entry.id)
@@ -171,6 +193,9 @@ export function resolveRouteModels(request: RouteCatalogRequest): readonly Model
if (!Number.isInteger(maxTokens) || maxTokens <= 0) {
invalid(provider, `model "${entry.id}" maxTokens must be a positive integer`)
}
// Only a value the profile named is a deployment choice; the catalog's is
// the model's capability and stays out of request defaults.
if (entry.maxTokens !== undefined) configuredMaxTokens.set(entry.id, entry.maxTokens)
return {
id: entry.id,
name: entry.name ?? base?.name ?? entry.id,
@@ -190,4 +215,5 @@ export function resolveRouteModels(request: RouteCatalogRequest): readonly Model
...base?.headers === undefined ? {} : { headers: base.headers },
}
})
return { models, configuredMaxTokens }
}

View File

@@ -92,6 +92,12 @@ export interface ResolvedPiAiProviderProfile
* serving requests.
*/
piProvider: Provider
/**
* Per-request output caps this profile explicitly configured, by model id.
* The seam materializes one only into a request that names no cap of its
* own, so a catalog capability must not appear here.
*/
configuredMaxTokens: ReadonlyMap<string, number>
}
/** Plugin configuration: the provider routes this instance owns. */
@@ -200,7 +206,7 @@ export function resolveProfiles(
// always shown route keys, and a catalog route must not silently rename
// itself on every configuration surface just because it gained a profile.
const displayName = source.displayName ?? provider
const models = resolveRouteModels({
const catalog = resolveRouteModels({
provider,
...source.api === undefined ? {} : { api: source.api },
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
@@ -216,12 +222,13 @@ export function resolveProfiles(
retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`),
...rest.headers === undefined ? {} : { headers: { ...rest.headers } },
...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } },
configuredMaxTokens: catalog.configuredMaxTokens,
piProvider: buildProvider({
provider,
displayName,
...source.api === undefined ? {} : { api: source.api },
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
models,
models: catalog.models,
}),
})
}

View File

@@ -44,7 +44,7 @@
import type { Context } from 'cordis'
import { LlmError } from '@deepseek-ai/dsh-llm'
import type { AdapterRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
import type { AdapterRegistrationHandle, DirectoryRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import { PiAiAdapter } from './adapter.ts'
import { catalogProviderIds } from './catalog.ts'
@@ -151,13 +151,21 @@ export function apply(ctx: Context, config: Config): void {
// mounts — dormant or not — so configuration surfaces can offer every
// pi-ai provider before any route exists. Hand-declared routes join it as
// profiles appear, and leave with them.
let directory: (() => void) | undefined
let directory: DirectoryRegistrationHandle | undefined
let directoryFacts: unknown
const ensureDirectory = (): void => {
const entries = directoryEntries(profiles())
if (deepEqualJson(entries, directoryFacts)) return
directory?.()
directory = ctx.llm.registerConfigurableProviders(entries)
// Atomic replace, never dispose-then-register: a route another adapter
// family already declares (a profile keyed `deepseek-official`) would
// otherwise leave this plugin's whole directory withdrawn and the Models
// page empty. The candidate set is validated first, so a collision keeps
// the previous entries serving and only costs a diagnostic.
if (directory === undefined) {
directory = ctx.llm.registerConfigurableProviders(entries)
} else {
directory.replace(entries)
}
directoryFacts = entries
}
ensureDirectory()
@@ -199,8 +207,16 @@ export function apply(ctx: Context, config: Config): void {
onChange: () => {
ensureRegistrationFacts()
// The directory follows the profiles the registry accepted, so a route
// that failed to register is not advertised as configurable.
ensureDirectory()
// that failed to register is not advertised as configurable. A refused
// directory swap is contained here for the same reason the registry's
// is: the previous entries keep serving, and `directoryFacts` stays put
// so returning to a working configuration re-applies.
try {
ensureDirectory()
} catch (error) {
ctx.logger.error('llm-pi-ai: keeping the previous configurable-provider directory after a refused update')
ctx.logger.error(error)
}
},
})
}

View File

@@ -22,12 +22,8 @@
import { createProvider } from '@earendil-works/pi-ai'
import type { Api, ApiKeyAuth, Model, Provider, ProviderStreams } from '@earendil-works/pi-ai'
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy'
import { azureOpenAIResponsesApi } from '@earendil-works/pi-ai/api/azure-openai-responses.lazy'
import { bedrockConverseStreamApi } from '@earendil-works/pi-ai/api/bedrock-converse-stream.lazy'
import { googleGenerativeAIApi } from '@earendil-works/pi-ai/api/google-generative-ai.lazy'
import { googleVertexApi } from '@earendil-works/pi-ai/api/google-vertex.lazy'
import { mistralConversationsApi } from '@earendil-works/pi-ai/api/mistral-conversations.lazy'
import { openAICodexResponsesApi } from '@earendil-works/pi-ai/api/openai-codex-responses.lazy'
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy'
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy'
import { piMessagesApi } from '@earendil-works/pi-ai/api/pi-messages.lazy'
@@ -35,18 +31,24 @@ import { catalogProvider } from './catalog.ts'
/**
* Wire protocols a configured route may name, mapped to pi-ai's lazily loaded
* implementations. The table is pi-ai's own streaming API set: each entry is
* the factory that pi-ai's matching provider factory uses, so a hand-declared
* route reaches exactly the implementation a catalog route would.
* implementations. Each entry is the factory that pi-ai's matching provider
* factory uses, so a hand-declared route reaches exactly the implementation a
* catalog route would.
*
* The table is deliberately narrower than pi-ai's full streaming API set: it
* holds only the protocols a profile can *completely* describe with a key, an
* endpoint, and headers. Bedrock signs with SigV4 over AWS credentials and a
* region, Vertex needs a project, a location, and application-default
* credentials, Azure needs provider environment plus an api-version, and
* Codex authenticates through OAuth — none of which this configuration shape
* can express, so offering them would hand back a provider that cannot
* authenticate. Catalog routes still reach those protocols through their own
* provider; only an explicit override is refused.
*/
const PROTOCOLS: Readonly<Record<string, () => ProviderStreams>> = {
'anthropic-messages': anthropicMessagesApi,
'azure-openai-responses': azureOpenAIResponsesApi,
'bedrock-converse-stream': bedrockConverseStreamApi,
'google-generative-ai': googleGenerativeAIApi,
'google-vertex': googleVertexApi,
'mistral-conversations': mistralConversationsApi,
'openai-codex-responses': openAICodexResponsesApi,
'openai-completions': openAICompletionsApi,
'openai-responses': openAIResponsesApi,
'pi-messages': piMessagesApi,

View File

@@ -1,14 +1,43 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import LlmService, { createUserMessage } from '@deepseek-ai/dsh-llm'
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
import SettingsLocal from '@deepseek-ai/dsh-settings-local'
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
import { resolveProfiles } from '../src/config.ts'
import { buildProvider } from '../src/provider.ts'
import { buildProvider, supportedProtocols } from '../src/provider.ts'
import { assemble } from './assemble.ts'
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
afterEach(async () => { await closeMockServers() })
const homes: string[] = []
afterEach(async () => {
await closeMockServers()
await Promise.all(homes.splice(0).map(dir => rm(dir, { recursive: true, force: true })))
})
/** A throwaway $DSH_HOME with an empty settings document. */
async function home(): Promise<string> {
const dir = await mkdtemp(join(tmpdir(), 'dsh-pi-catalog-'))
homes.push(dir)
await writeFile(join(dir, 'settings.yaml'), '')
return dir
}
/** The dormant composition plus a real settings service, as the product mounts it. */
async function bootWithSettings(dir: string, config: LlmPiAi.Config): Promise<Context> {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(SettingsLocal, { path: join(dir, 'settings.yaml'), watch: false })
await ctx.plugin(LlmPiAi, config)
return ctx
}
/** A complete hand-declared route: nothing about it exists in pi-ai's catalog. */
function gateway(baseURL: string, overrides: Record<string, unknown> = {}): LlmPiAi.Config {
@@ -107,6 +136,19 @@ describe('hand-declared providers', () => {
})).toThrow(/needs a baseURL/)
})
it.each(['bedrock-converse-stream', 'google-vertex', 'azure-openai-responses', 'openai-codex-responses'])(
'refuses %s, whose authentication a profile cannot express',
(api) => {
// These need SigV4 credentials and a region, a project plus ADC, provider
// environment and an api-version, or OAuth — none of which a key, an
// endpoint, and headers can carry, so a route naming one would be built
// unable to authenticate.
expect(supportedProtocols()).not.toContain(api)
expect(() => buildProvider({ provider: 'acme-gateway', displayName: 'Acme', api, models: [] }))
.toThrow(/cannot serve; supported protocols are/)
},
)
it('rejects a protocol this build cannot serve, and a route that names none', () => {
const spec = { provider: 'acme-gateway', displayName: 'Acme Gateway', models: [] }
expect(() => buildProvider({ ...spec, api: 'quantum-telepathy' }))
@@ -206,14 +248,35 @@ describe('catalog routes with per-model configuration', () => {
})
const info = await ctx.llm.resolveModelInfo('deepseek', catalogModel.id)
// The configured field wins; name and output cap still come from the catalog.
// The configured field wins and the name still comes from the catalog. The
// catalog's own output cap is the model's capability, not a cap anyone
// chose, so it must not arrive as the request default.
expect(info.context).toEqual({ contextWindow: 4096 })
expect(info.name).toBe(catalogModel.name)
expect(info.defaultMaxTokens).toBe(catalogModel.maxTokens)
expect(info.defaultMaxTokens).toBeUndefined()
// An explicit list replaces the catalog rather than adding to it.
expect((await ctx.llm.listModels('deepseek')).map(model => model.id)).toEqual([catalogModel.id])
})
it('materializes a request default only from a configured output cap', async () => {
const server = await mockServer([])
const [catalogModel] = getBuiltinModels('deepseek')
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
const ctx = await harness({
providers: {
deepseek: {
apiKey: 'k',
baseURL: server.url,
models: [{ id: catalogModel.id, maxTokens: 4096 }],
},
},
})
// Configuring the cap is the deployment choosing one, so it becomes the
// default the seam materializes into requests that name none.
expect((await ctx.llm.resolveModelInfo('deepseek', catalogModel.id)).defaultMaxTokens).toBe(4096)
})
it('adds a model the installed catalog does not describe to a catalog route', async () => {
const server = await mockServer([{ events: textEvents }])
const ctx = await harness({
@@ -262,6 +325,23 @@ describe('catalog routes with per-model configuration', () => {
expect(model?.contextWindow).toBe(4096)
})
it('delegates both stream methods back to the reused catalog provider', async () => {
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
const resolved = resolveProfiles({ deepseek: { apiKey: 'k', baseURL: `${server.url}/v1` } })
const built = resolved.get('deepseek')?.piProvider
if (built === undefined) throw new Error('the deepseek route built no provider')
const [model] = built.getModels()
if (model === undefined) throw new Error('the deepseek route resolved no models')
const context = { messages: [{ role: 'user' as const, content: 'hi', timestamp: 0 }] }
// `stream` is interface-required and unused by the harness adapter, which
// only calls `streamSimple`; both must still reach the catalog provider.
for await (const _event of built.stream(model, context, { apiKey: 'k' })) { /* drain */ }
for await (const _event of built.streamSimple(model, context, { apiKey: 'k' })) { /* drain */ }
expect(server.paths).toEqual(['/v1/chat/completions', '/v1/chat/completions'])
})
it('keeps each model its own endpoint when the catalog route declares none', () => {
// `opencode` ships no provider-level endpoint: the address lives on every
// catalog model, so the route resolves without any configured baseURL.
@@ -300,3 +380,112 @@ describe('catalog routes with per-model configuration', () => {
expect(server.paths).toEqual(['/v1/chat/completions'])
})
})
describe('resolution snapshots', () => {
it('finishes an in-flight request under the configuration it started with', async () => {
const server = await mockServer([{ events: textEvents }])
let current = resolveProfiles({ deepseek: { apiKey: 'k', baseURL: `${server.url}/v1` } })
let release: () => void = () => {}
const held = new Promise<void>((resolve) => { release = resolve })
const adapter = new PiAiAdapter({
profiles: () => current,
// Credential resolution is the real await inside a stream call, and the
// window a configuration change has to land in.
resolveApiKey: async () => { await held; return 'k' },
})
const chunks: StreamChunk[] = []
const inFlight = (async () => {
for await (const chunk of adapter.stream({
provider: 'deepseek',
model: 'deepseek-v4-flash',
messages: [],
})) chunks.push(chunk)
})()
// The route set changes while the request waits, and something else reads
// the adapter meanwhile, which is what would rebuild a shared collection.
current = resolveProfiles({ openai: { apiKey: 'k', baseURL: `${server.url}/v1` } })
await expect(adapter.listModels('openai')).resolves.not.toHaveLength(0)
release()
await inFlight
// The in-flight request keeps its own snapshot: it reaches the endpoint it
// resolved against instead of failing on a provider that no longer exists.
expect(chunks.at(-1)).toMatchObject({ type: 'finish', reason: { kind: 'stop' } })
expect(server.paths).toEqual(['/v1/chat/completions'])
})
it('serves the next request from the new configuration', async () => {
const first = await mockServer([{ events: textEvents }])
const second = await mockServer([{ events: textEvents }])
let current = resolveProfiles({ deepseek: { apiKey: 'k', baseURL: `${first.url}/v1` } })
const adapter = new PiAiAdapter({ profiles: () => current, resolveApiKey: () => Promise.resolve('k') })
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({
provider: 'deepseek', model: 'deepseek-v4-flash', messages: [],
})) { /* drain */ }
}
await drain()
current = resolveProfiles({ deepseek: { apiKey: 'k', baseURL: `${second.url}/v1` } })
await drain()
expect(first.paths).toHaveLength(1)
expect(second.paths).toHaveLength(1)
})
})
describe('configurable-provider directory', () => {
it('keeps the previous directory when a route collides with another adapter family', async () => {
const dir = await home()
const ctx = await bootWithSettings(dir, {})
// Another adapter family owns this route id, exactly as llm-deepseek does.
ctx.llm.registerConfigurableProviders([
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: 'llm-deepseek', settingsPath: [] },
])
const before = ctx.llm.listConfigurableProviders().length
expect(before).toBeGreaterThan(30)
await ctx.settings.update(settingsNamespace('llm-pi-ai'), {
providers: {
'deepseek-official': {
apiKey: 'k',
api: 'openai-completions',
baseURL: 'https://acme.test/v1',
models: [{ id: 'm', contextWindow: 1, maxTokens: 1 }],
},
},
})
// The refused swap costs a diagnostic, not the directory: every entry the
// page needs is still declared.
expect(ctx.llm.listConfigurableProviders()).toHaveLength(before)
expect(ctx.llm.listConfigurableProviders().find(entry => entry.provider === 'deepseek-official')?.settingsNs)
.toBe('llm-deepseek')
})
it('replaces its entries atomically as declared routes come and go', async () => {
const dir = await home()
const ctx = await bootWithSettings(dir, {})
const catalogOnly = ctx.llm.listConfigurableProviders().length
await ctx.settings.update(settingsNamespace('llm-pi-ai'), {
providers: {
'acme-gateway': {
apiKey: 'k',
displayName: 'Acme Gateway',
api: 'openai-completions',
baseURL: 'https://acme.test/v1',
models: [{ id: 'm', contextWindow: 1, maxTokens: 1 }],
},
},
})
expect(ctx.llm.listConfigurableProviders()).toHaveLength(catalogOnly + 1)
expect(ctx.llm.listConfigurableProviders().find(entry => entry.provider === 'acme-gateway')?.displayName)
.toBe('Acme Gateway')
await ctx.settings.replace(settingsNamespace('llm-pi-ai'), {})
expect(ctx.llm.listConfigurableProviders()).toHaveLength(catalogOnly)
})
})

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
README.md: 21f428fb22c9a59a67d86f446ea866c1629b964a
README.zh.md: 9bd26993bc63b39de3d3a8039fea6b046c875504
README.md: e09ec685ed0ab1e2492749237c277a874eb3b246
README.zh.md: ca98e875a90eb16e32bc405d77cd5b2b56644180

View File

@@ -12,7 +12,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration.
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber.
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber. The handle also carries `replace(entries)`: the candidate set is validated in full before anything moves, so an entry another registration already declares leaves the current set intact, and an empty array is legal there. A plugin whose declared set follows its configuration must use `replace` rather than disposing and re-registering — the latter strands the directory empty whenever the new set is refused.
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` List the declared directory in declaration order; configuration surfaces merge it with `listProviders()` to mark each entry live or dormant.
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.

View File

@@ -12,7 +12,7 @@
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose资源释放。返回的释放器还携带 `replace(providers)`:候选路由集合会在任何东西变动之前完整校验,因此与另一适配器冲突时,当前路由保持注册且继续服务,而替换本身是一个同步区段,不存在可观察的空档。`replace([])` 合法——一个持有零条路由的注册——这与空的初始注册不同。
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。该句柄还带 `replace(entries)`:候选集合会先被整体校验,因此其中若有条目已被另一个注册声明,当前集合原封不动;此处允许传空数组。声明集合随配置变化的插件必须使用 `replace`,而不是先 dispose 再重新注册——后者会在新集合被拒时让目录整个落空。
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` 按声明顺序列出已声明的目录;配置界面将其与 `listProviders()` 合并,为每个条目标注存活或休眠。
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略,并解析 normal 默认值。
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。

View File

@@ -225,6 +225,27 @@ export interface AdapterRegistrationHandle {
replace(providers: string[]): void
}
/**
* A live configurable-provider registration, disposable and atomically
* replaceable — the directory counterpart of {@link AdapterRegistrationHandle}.
*/
export interface DirectoryRegistrationHandle {
/** Withdraw every entry this registration currently holds. */
(): void
/**
* Replace this registration's entries with `entries`. The candidate set is
* validated in full first — an entry another registration already declares,
* a duplicate within the set, or invalid metadata throws and leaves the
* current entries untouched — and the swap is one synchronous section, so no
* reader observes a gap. An empty array is legal here, unlike an empty
* initial registration.
*
* Throws `LlmError` with code `REGISTRATION_DISPOSED` once the registration
* has been disposed.
*/
replace(entries: readonly LlmConfigurableProvider[]): void
}
/**
* The abstract `llm` service: an adapter registry plus a streaming model-call
* surface, interceptable via the `llm/stream` waterfall.
@@ -370,34 +391,61 @@ export class LlmService extends Service {
* entry, or a provider already declared by any registration throws
* `LlmError` without registering the rest. Disposed with the fiber.
* @param entries - every configurable provider this plugin owns.
* @returns the disposer that withdraws all of them.
* @returns a handle that withdraws all of them, and can atomically replace them.
*/
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void {
const dispose = this.ctx.effect(function* (this: LlmService) {
if (entries.length === 0) {
throw new LlmError('a configurable-provider registration must declare at least one provider', 'INVALID_DIRECTORY')
}
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle {
let held: LlmConfigurableProvider[] = []
let disposed = false
/**
* Validate a candidate set in full against everything this registration
* does not already hold, then publish it. Nothing is written until the
* whole set passes, so a refused candidate leaves the current entries in
* place — the property that makes `replace` a swap rather than a
* delete-then-add that can strand the directory empty.
*/
const commit = (candidates: readonly LlmConfigurableProvider[]): void => {
const detached: LlmConfigurableProvider[] = []
for (const entry of entries) {
const own = new Set(held.map(entry => entry.provider))
for (const entry of candidates) {
if (entry.provider.length === 0 || entry.displayName.length === 0 || entry.settingsNs.length === 0) {
throw new LlmError('configurable providers need a non-empty provider, displayName, and settingsNs', 'INVALID_DIRECTORY')
}
if (entry.settingsPath.some(segment => segment.length === 0)) {
throw new LlmError(`configurable provider "${entry.provider}" has an empty settingsPath segment`, 'INVALID_DIRECTORY')
}
if (this.directory.has(entry.provider) || detached.some(seen => seen.provider === entry.provider)) {
if ((this.directory.has(entry.provider) && !own.has(entry.provider))
|| detached.some(seen => seen.provider === entry.provider)) {
throw new LlmError(`configurable provider "${entry.provider}" is already declared`, 'DUPLICATE_DIRECTORY')
}
detached.push({ ...entry, settingsPath: [...entry.settingsPath] })
}
for (const entry of held) this.directory.delete(entry.provider)
for (const entry of detached) this.directory.set(entry.provider, entry)
held = detached
this.emitAdaptersUpdated()
}
const dispose = this.ctx.effect(function* (this: LlmService) {
if (entries.length === 0) {
throw new LlmError('a configurable-provider registration must declare at least one provider', 'INVALID_DIRECTORY')
}
commit(entries)
yield () => {
for (const entry of detached) this.directory.delete(entry.provider)
disposed = true
for (const entry of held) this.directory.delete(entry.provider)
held = []
this.emitAdaptersUpdated()
}
}.bind(this), 'llm.registerConfigurableProviders()')
return () => void dispose()
const handle = ((): void => void dispose()) as DirectoryRegistrationHandle
handle.replace = (next: readonly LlmConfigurableProvider[]): void => {
if (disposed) {
throw new LlmError('this configurable-provider registration was disposed', 'REGISTRATION_DISPOSED')
}
commit(next)
}
return handle
}
/**

View File

@@ -170,6 +170,32 @@ describe('configurable-provider directory', () => {
expect(ctx.llm.listConfigurableProviders()).toEqual([])
})
it('replaces its entries atomically, keeping the old set when a candidate collides', async () => {
const ctx = await setup()
const handle = ctx.llm.registerConfigurableProviders([entry(), entry({ provider: 'second' })])
ctx.llm.registerConfigurableProviders([entry({ provider: 'owned-elsewhere' })])
// A candidate another registration already declares refuses the whole swap.
expect(() =>{ handle.replace([entry({ provider: 'owned-elsewhere' })]); }).toThrow(/already declared/)
expect(ctx.llm.listConfigurableProviders().map(view => view.provider).sort())
.toEqual(['owned-elsewhere', 'second', entry().provider].sort())
// Its own entries are not "already declared" against itself, so a swap that
// keeps one and drops another lands whole.
handle.replace([entry({ displayName: 'Renamed' })])
expect(ctx.llm.listConfigurableProviders().map(view => view.provider).sort())
.toEqual(['owned-elsewhere', entry().provider].sort())
expect(ctx.llm.listConfigurableProviders().find(view => view.provider === entry().provider)?.displayName)
.toBe('Renamed')
// An empty replace is legal, unlike an empty initial registration.
handle.replace([])
expect(ctx.llm.listConfigurableProviders().map(view => view.provider)).toEqual(['owned-elsewhere'])
handle()
expect(() =>{ handle.replace([entry()]); }).toThrow(/was disposed/)
})
it('rejects duplicates within one registration and across registrations', async () => {
const ctx = await setup()
expect(() => ctx.llm.registerConfigurableProviders([entry(), entry()])).toThrow(/already declared/)