fix(llm): size unknown models and refuse a section that cannot be served
Three defects surfaced while driving the Models page. A hand-declared model needed an explicit contextWindow and maxTokens, but a provider listing usually returns ids and nothing else — so the page happily wrote a profile the adapter then rejected, which took the whole namespace down silently. Capacities now fall back to the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768). Both are guesses by construction, which is why they are route fields a deployment corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap. That silent failure was the second defect. A schema-valid profile the adapter could not serve was stored and only rejected later, disabling every route in the namespace with nothing said. `dsh-settings` gains an optional `validate` on registration — a check for what a schema cannot express — and `llm-pi-ai` refuses an unserviceable section at the write that produced it. A stored section that fails keeps the namespace's last good value, as a schema failure already did, so an externally edited document still cannot strand the owner. The plugin's own last-good fallback goes with it: nothing reaching it can fail any more. Third, a model with no reasoning metadata advertised the single level `off`, which pi-ai translates to *omitting* the reasoning option — the same request naming no effort produces. Selecting it disabled nothing, so a provider whose default is to think kept thinking with `off` shown as selected. Such a model now reports no reasoning capability at all, which is the seam's way of saying the control is unavailable, and the per-model `reasoning` flag is gone: without a thinkingLevelMap to spell levels it could only invent them. The protocol table narrows to the three a hand-declared route reaches today, most-reached first so a surface offering a choice defaults to the one gateways actually speak.
This commit is contained in:
@@ -718,6 +718,18 @@ export interface PiAiProviderProfile {
|
||||
* unset fields from the installed model of the same id.
|
||||
*/
|
||||
models?: PiAiModelProfile[]
|
||||
/**
|
||||
* Context capacity for a model this route lists that neither the entry nor
|
||||
* the installed catalog sizes (default 262,144). A guess by construction, so
|
||||
* a deployment whose gateway serves smaller models corrects it here.
|
||||
*/
|
||||
defaultContextWindow?: number
|
||||
/**
|
||||
* Output capability for a model this route lists that neither the entry nor
|
||||
* the installed catalog sizes (default 32,768). This sizes the model; it
|
||||
* never becomes a per-request cap on its own.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -748,18 +760,17 @@ export interface PiAiModelProfile {
|
||||
contextWindow?: number
|
||||
/**
|
||||
* Maximum output tokens. Configuring one also makes it this model's
|
||||
* per-request default; the value inherited from the installed catalog is the
|
||||
* model's capability and never becomes a request default on its own.
|
||||
* per-request default; a value inherited from the installed catalog, or the
|
||||
* route's fallback, is the model's capability and never becomes a request
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/** Whether the model exposes reasoning; defaults to the catalog capability. */
|
||||
reasoning?: boolean
|
||||
}
|
||||
```
|
||||
|
||||
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:104`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:122`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-replay`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user