Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(llm): capture an immutable snapshot per pi-ai operation

Browse Source 
Review found four defects in the declared-provider work.

`PiAiAdapter` reused one `Models` collection and mutated it whenever the
configuration changed. `Models.streamSimple()` resolves its provider
lazily — when the stream is first consumed, which is after the adapter
awaits the route's credential — so a configuration change landing in
that window let an in-flight request finish under a configuration it
never resolved against, or fail on a provider that no longer existed.
Each resolution now produces an immutable snapshot and every operation
captures one before its first await, which is what makes the seam's
per-step freeze (`llm.prepareCall()`) hold end to end: switching models
mid-reply takes effect on the next step, never inside the one in flight.

`defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`.
The two answer different questions: pi-ai requires that field as the
model's output capability, while the seam's is a cap the deployment
chose to send on requests naming none, so every request had started
carrying a number nobody picked. Only an explicitly configured cap
reaches the seam now.

The configurable-provider directory was refreshed by disposing its
registration and making a new one. A candidate set the registry refuses
— a profile keyed `deepseek-official`, which llm-deepseek declares —
left the whole directory withdrawn and the Models page empty, silently,
because the settings callback contains the failure. The seam's
registration handle now carries `replace()` with the same
validate-first atomicity `registerAdapter` has.

The protocol table offered every pi-ai streaming API, including four
whose authentication a profile cannot express: Bedrock signs with SigV4
over AWS credentials and a region, Vertex needs a project, a location,
and ADC, Azure needs provider environment plus an api-version, and Codex
uses OAuth. Offering them handed back routes that cannot authenticate.
Catalog routes still reach them through their own provider.
This commit is contained in:
Yichen Jiang
2026-08-04 11:42:35 +08:00
parent d6126c25f2
commit 4c80cab108
21 changed files with 476 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/config-catalog.md
View File

@@ -746,7 +746,11 @@ export interface PiAiModelProfile {
name?: string
/** Maximum combined request and response context in tokens. */
contextWindow?: number
/** Per-request output cap materialized when a caller omits one. */
/**
* Maximum output tokens. Configuring one also makes it this model's
* per-request default; the value inherited from the installed catalog is the
* model's capability and never becomes a request default on its own.
*/
maxTokens?: number
/** Whether the model exposes reasoning; defaults to the catalog capability. */
reasoning?: boolean
@@ -755,7 +759,7 @@ export interface PiAiModelProfile {
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
Source: [`packages/llm/llm-pi-ai/src/config.ts:98`](../packages/llm/llm-pi-ai/src/config.ts)
Source: [`packages/llm/llm-pi-ai/src/config.ts:104`](../packages/llm/llm-pi-ai/src/config.ts)
## `@deepseek-ai/dsh-llm-replay`

8
docs/cordis-catalog/services.md
View File

@@ -834,9 +834,9 @@ listProviders(): LlmProviderInfo[]
* entry, or a provider already declared by any registration throws
* `LlmError` without registering the rest. Disposed with the fiber.
* @param entries - every configurable provider this plugin owns.
* @returns the disposer that withdraws all of them.
* @returns a handle that withdraws all of them, and can atomically replace them.
*/
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void
registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle
/**
* List every declared configurable provider, registered or dormant.
@@ -908,9 +908,9 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
```
Types: [AdapterRegistrationHandle](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmConfigurableProvider](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Types: [AdapterRegistrationHandle](../core-data-structures/core.md) · [DirectoryRegistrationHandle](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmConfigurableProvider](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Source: [`packages/llm/llm/src/index.ts:232`](../../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm/src/index.ts:253`](../../packages/llm/llm/src/index.ts)
## `ctx.permission` — `PermissionService`