fix(llm): capture an immutable snapshot per pi-ai operation
Review found four defects in the declared-provider work. `PiAiAdapter` reused one `Models` collection and mutated it whenever the configuration changed. `Models.streamSimple()` resolves its provider lazily — when the stream is first consumed, which is after the adapter awaits the route's credential — so a configuration change landing in that window let an in-flight request finish under a configuration it never resolved against, or fail on a provider that no longer existed. Each resolution now produces an immutable snapshot and every operation captures one before its first await, which is what makes the seam's per-step freeze (`llm.prepareCall()`) hold end to end: switching models mid-reply takes effect on the next step, never inside the one in flight. `defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`. The two answer different questions: pi-ai requires that field as the model's output capability, while the seam's is a cap the deployment chose to send on requests naming none, so every request had started carrying a number nobody picked. Only an explicitly configured cap reaches the seam now. The configurable-provider directory was refreshed by disposing its registration and making a new one. A candidate set the registry refuses — a profile keyed `deepseek-official`, which llm-deepseek declares — left the whole directory withdrawn and the Models page empty, silently, because the settings callback contains the failure. The seam's registration handle now carries `replace()` with the same validate-first atomicity `registerAdapter` has. The protocol table offered every pi-ai streaming API, including four whose authentication a profile cannot express: Bedrock signs with SigV4 over AWS credentials and a region, Vertex needs a project, a location, and ADC, Azure needs provider environment plus an api-version, and Codex uses OAuth. Offering them handed back routes that cannot authenticate. Catalog routes still reach them through their own provider.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 21f428fb22c9a59a67d86f446ea866c1629b964a
|
||||
README.zh.md: 9bd26993bc63b39de3d3a8039fea6b046c875504
|
||||
README.md: e09ec685ed0ab1e2492749237c277a874eb3b246
|
||||
README.zh.md: ca98e875a90eb16e32bc405d77cd5b2b56644180
|
||||
|
||||
@@ -12,7 +12,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber.
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber. The handle also carries `replace(entries)`: the candidate set is validated in full before anything moves, so an entry another registration already declares leaves the current set intact, and an empty array is legal there. A plugin whose declared set follows its configuration must use `replace` rather than disposing and re-registering — the latter strands the directory empty whenever the new set is refused.
|
||||
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` List the declared directory in declaration order; configuration surfaces merge it with `listProviders()` to mark each entry live or dormant.
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。返回的释放器还携带 `replace(providers)`:候选路由集合会在任何东西变动之前完整校验,因此与另一适配器冲突时,当前路由保持注册且继续服务,而替换本身是一个同步区段,不存在可观察的空档。`replace([])` 合法——一个持有零条路由的注册——这与空的初始注册不同。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace,以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace,以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。该句柄还带 `replace(entries)`:候选集合会先被整体校验,因此其中若有条目已被另一个注册声明,当前集合原封不动;此处允许传空数组。声明集合随配置变化的插件必须使用 `replace`,而不是先 dispose 再重新注册——后者会在新集合被拒时让目录整个落空。
|
||||
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` 按声明顺序列出已声明的目录;配置界面将其与 `listProviders()` 合并,为每个条目标注存活或休眠。
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略,并解析 normal 默认值。
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
|
||||
|
||||
@@ -225,6 +225,27 @@ export interface AdapterRegistrationHandle {
|
||||
replace(providers: string[]): void
|
||||
}
|
||||
|
||||
/**
|
||||
* A live configurable-provider registration, disposable and atomically
|
||||
* replaceable — the directory counterpart of {@link AdapterRegistrationHandle}.
|
||||
*/
|
||||
export interface DirectoryRegistrationHandle {
|
||||
/** Withdraw every entry this registration currently holds. */
|
||||
(): void
|
||||
/**
|
||||
* Replace this registration's entries with `entries`. The candidate set is
|
||||
* validated in full first — an entry another registration already declares,
|
||||
* a duplicate within the set, or invalid metadata throws and leaves the
|
||||
* current entries untouched — and the swap is one synchronous section, so no
|
||||
* reader observes a gap. An empty array is legal here, unlike an empty
|
||||
* initial registration.
|
||||
*
|
||||
* Throws `LlmError` with code `REGISTRATION_DISPOSED` once the registration
|
||||
* has been disposed.
|
||||
*/
|
||||
replace(entries: readonly LlmConfigurableProvider[]): void
|
||||
}
|
||||
|
||||
/**
|
||||
* The abstract `llm` service: an adapter registry plus a streaming model-call
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
@@ -370,34 +391,61 @@ export class LlmService extends Service {
|
||||
* entry, or a provider already declared by any registration throws
|
||||
* `LlmError` without registering the rest. Disposed with the fiber.
|
||||
* @param entries - every configurable provider this plugin owns.
|
||||
* @returns the disposer that withdraws all of them.
|
||||
* @returns a handle that withdraws all of them, and can atomically replace them.
|
||||
*/
|
||||
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (entries.length === 0) {
|
||||
throw new LlmError('a configurable-provider registration must declare at least one provider', 'INVALID_DIRECTORY')
|
||||
}
|
||||
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle {
|
||||
let held: LlmConfigurableProvider[] = []
|
||||
let disposed = false
|
||||
/**
|
||||
* Validate a candidate set in full against everything this registration
|
||||
* does not already hold, then publish it. Nothing is written until the
|
||||
* whole set passes, so a refused candidate leaves the current entries in
|
||||
* place — the property that makes `replace` a swap rather than a
|
||||
* delete-then-add that can strand the directory empty.
|
||||
*/
|
||||
const commit = (candidates: readonly LlmConfigurableProvider[]): void => {
|
||||
const detached: LlmConfigurableProvider[] = []
|
||||
for (const entry of entries) {
|
||||
const own = new Set(held.map(entry => entry.provider))
|
||||
for (const entry of candidates) {
|
||||
if (entry.provider.length === 0 || entry.displayName.length === 0 || entry.settingsNs.length === 0) {
|
||||
throw new LlmError('configurable providers need a non-empty provider, displayName, and settingsNs', 'INVALID_DIRECTORY')
|
||||
}
|
||||
if (entry.settingsPath.some(segment => segment.length === 0)) {
|
||||
throw new LlmError(`configurable provider "${entry.provider}" has an empty settingsPath segment`, 'INVALID_DIRECTORY')
|
||||
}
|
||||
if (this.directory.has(entry.provider) || detached.some(seen => seen.provider === entry.provider)) {
|
||||
if ((this.directory.has(entry.provider) && !own.has(entry.provider))
|
||||
|| detached.some(seen => seen.provider === entry.provider)) {
|
||||
throw new LlmError(`configurable provider "${entry.provider}" is already declared`, 'DUPLICATE_DIRECTORY')
|
||||
}
|
||||
detached.push({ ...entry, settingsPath: [...entry.settingsPath] })
|
||||
}
|
||||
for (const entry of held) this.directory.delete(entry.provider)
|
||||
for (const entry of detached) this.directory.set(entry.provider, entry)
|
||||
held = detached
|
||||
this.emitAdaptersUpdated()
|
||||
}
|
||||
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (entries.length === 0) {
|
||||
throw new LlmError('a configurable-provider registration must declare at least one provider', 'INVALID_DIRECTORY')
|
||||
}
|
||||
commit(entries)
|
||||
yield () => {
|
||||
for (const entry of detached) this.directory.delete(entry.provider)
|
||||
disposed = true
|
||||
for (const entry of held) this.directory.delete(entry.provider)
|
||||
held = []
|
||||
this.emitAdaptersUpdated()
|
||||
}
|
||||
}.bind(this), 'llm.registerConfigurableProviders()')
|
||||
return () => void dispose()
|
||||
|
||||
const handle = ((): void => void dispose()) as DirectoryRegistrationHandle
|
||||
handle.replace = (next: readonly LlmConfigurableProvider[]): void => {
|
||||
if (disposed) {
|
||||
throw new LlmError('this configurable-provider registration was disposed', 'REGISTRATION_DISPOSED')
|
||||
}
|
||||
commit(next)
|
||||
}
|
||||
return handle
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -170,6 +170,32 @@ describe('configurable-provider directory', () => {
|
||||
expect(ctx.llm.listConfigurableProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('replaces its entries atomically, keeping the old set when a candidate collides', async () => {
|
||||
const ctx = await setup()
|
||||
const handle = ctx.llm.registerConfigurableProviders([entry(), entry({ provider: 'second' })])
|
||||
ctx.llm.registerConfigurableProviders([entry({ provider: 'owned-elsewhere' })])
|
||||
|
||||
// A candidate another registration already declares refuses the whole swap.
|
||||
expect(() =>{ handle.replace([entry({ provider: 'owned-elsewhere' })]); }).toThrow(/already declared/)
|
||||
expect(ctx.llm.listConfigurableProviders().map(view => view.provider).sort())
|
||||
.toEqual(['owned-elsewhere', 'second', entry().provider].sort())
|
||||
|
||||
// Its own entries are not "already declared" against itself, so a swap that
|
||||
// keeps one and drops another lands whole.
|
||||
handle.replace([entry({ displayName: 'Renamed' })])
|
||||
expect(ctx.llm.listConfigurableProviders().map(view => view.provider).sort())
|
||||
.toEqual(['owned-elsewhere', entry().provider].sort())
|
||||
expect(ctx.llm.listConfigurableProviders().find(view => view.provider === entry().provider)?.displayName)
|
||||
.toBe('Renamed')
|
||||
|
||||
// An empty replace is legal, unlike an empty initial registration.
|
||||
handle.replace([])
|
||||
expect(ctx.llm.listConfigurableProviders().map(view => view.provider)).toEqual(['owned-elsewhere'])
|
||||
|
||||
handle()
|
||||
expect(() =>{ handle.replace([entry()]); }).toThrow(/was disposed/)
|
||||
})
|
||||
|
||||
it('rejects duplicates within one registration and across registrations', async () => {
|
||||
const ctx = await setup()
|
||||
expect(() => ctx.llm.registerConfigurableProviders([entry(), entry()])).toThrow(/already declared/)
|
||||
|
||||
Reference in New Issue
Block a user