Merge remote-tracking branch 'origin/master' into worktree/web-multimodal-image-input
# Conflicts: # .agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml # docs/core-data-structures/core.i18n.yaml # docs/module-graph.md # packages/client/ui-conversation/src/client/chat/AssistantMarkdown.tsx # packages/client/ui-conversation/src/client/index.ts # packages/compact/compact-basic/README.i18n.yaml
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: d0968a7366943a36ba428355058ad14169884c0a
|
||||
README.zh.md: d4c401698071bd494b95fc993b42de8b3197edc4
|
||||
README.md: eb67ce889193aadbd694d7aae53e47c7d20703be
|
||||
README.zh.md: b4b3e3c208702fa10e5f434a70608702d0576fbd
|
||||
|
||||
@@ -35,6 +35,15 @@ Configure credentials, the model catalog, and deployment-specific transport sett
|
||||
models:
|
||||
- id: claude-sonnet-4-5
|
||||
contextWindow: 200000
|
||||
# Catalog route with one model reshaped in place; the rest of the
|
||||
# catalog keeps serving (a models list would replace it instead).
|
||||
deepseek:
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
modelOverrides:
|
||||
deepseek-v4-pro:
|
||||
reasoningEfforts:
|
||||
off:
|
||||
high: high
|
||||
# Hand-declared route: pi-ai ships nothing under this key, so the profile
|
||||
# supplies the whole provider.
|
||||
acme-gateway:
|
||||
@@ -42,18 +51,43 @@ Configure credentials, the model catalog, and deployment-specific transport sett
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
# Reasoning dialect for an endpoint whose URL pi-ai cannot recognize.
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
contextWindow: 65536
|
||||
maxTokens: 4096
|
||||
- id: acme-think
|
||||
name: Acme Think
|
||||
contextWindow: 262144
|
||||
maxTokens: 32768
|
||||
# key = selectable level, value = its wire spelling; only off may
|
||||
# leave the value empty (supported, send nothing).
|
||||
reasoningEfforts:
|
||||
off:
|
||||
high: high
|
||||
max: ultra
|
||||
```
|
||||
|
||||
The dict shape makes duplicate routes unrepresentable, and the pre-release array shape (with per-profile `provider` fields) fails load with migration directions. `providers` may also be empty or omitted entirely: the adapter then mounts **dormant** — zero routes, no extra catalog entries — and registers routes the moment the `llm-pi-ai:` settings section supplies profiles, dropping them again when it empties. Dormant or not, the plugin declares every installed catalog provider in the configurable-provider directory (`ctx.llm.listConfigurableProviders()`, settings path `providers.<provider>`), joined with every route the current profiles declare, so configuration surfaces can offer the full catalog before any route exists and can still address a hand-declared one. Each entry carries `declared`: whether pi-ai ships nothing under that key. It follows the installed catalog, never the settings document, because narrowing a shipped provider's models stores a profile too and that route is still one pi-ai knows — only the adapter can tell the two apart, which is why the directory answers rather than leaving a surface to infer it. Which adapters exist is composition; which providers run can be entirely the user's settings document. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; a model the route does not configure fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
|
||||
|
||||
## Catalog resolution
|
||||
|
||||
A profile's `models` list *replaces* the route's installed catalog rather than extending it; omitting it (or leaving it empty) serves that catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a catalog route to two models, correcting one capacity, or adding a model newer than the installed catalog are all one-line edits. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, and `maxTokens`. Pricing and input modalities have no harness consumer and ride the installed entry or are absent. Reasoning is not per-model configurable at all: a bare capability flag would make pi-ai advertise effort levels with no `thinkingLevelMap` to spell them, and no listing endpoint reports a model's reasoning protocol, so reasoning rides the installed catalog entry or is absent.
|
||||
A profile's `models` list *replaces* the route's installed catalog rather than extending it; omitting it (or leaving it empty) serves that catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a catalog route to two models, correcting one capacity, or adding a model newer than the installed catalog are all one-line edits — but declaring any `models` list means every model the route should keep serving must appear in it, an entry of nothing but `id` being enough. The configurable entry fields are `id`, `name`, `contextWindow`, `maxTokens`, `reasoningEfforts`, and `compat`. Pricing and input modalities have no harness consumer and ride the installed entry or are absent.
|
||||
|
||||
`modelOverrides` reshapes individual installed-catalog models without that cost: each key is a catalog model id, each value the same fields a `models` entry takes with the id living in the key, and the rest of the catalog keeps serving untouched — "correct one model, keep the other thirty-seven" as a three-line edit. An override becomes that catalog entry's configuration, so capacities, efforts, and compat resolve through the same path with the same diagnostics and the same request-default semantics as a `models` entry. Overrides are only meaningful on a catalog route serving its catalog: one set beside a `models` list (which already replaces the catalog), on a hand-declared route (whose models are fully spelled in `models`), or naming a model the catalog does not describe is refused rather than skipped, because a silently unchanged model is a typo someone would otherwise hunt for.
|
||||
|
||||
### Per-model reasoning efforts
|
||||
|
||||
`reasoningEfforts` declares a model's selectable thinking levels: each key is a level selectors offer, its value the spelling dispatch sends on the wire, so `high: high` passes the canonical name through while `max: ultra` renames it for a gateway with its own vocabulary. Keys come from pi-ai's level set (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`); a level not declared is not offered. Omitting the field keeps the installed catalog entry's capability (a hand-declared model has none and does not reason); `false` declares a non-reasoning model, which is how a profile strips reasoning from a catalog model its gateway cannot serve; an empty declaration is refused rather than guessing between those two meanings.
|
||||
|
||||
The declaration translates to pi-ai's `Model.reasoning` + `thinkingLevelMap` with every level decided explicitly — undeclared levels are pinned unsupported rather than left to pi-ai's own defaulting, which is asymmetric (an absent key means "supported" for the five base levels but "unsupported" for `xhigh`/`max`) and which a profile author should not need to know. `off` is the one three-state key: left out, selectors offer no Off and an explicit Off request is refused — a request naming no effort still goes out without the parameter, so what the provider then does is its own default; declared with no value (`off:`), Off is offered and selecting it sends nothing — for the `deepseek` dialect an explicit `thinking: {type: "disabled"}` — which also covers a request naming no effort at all; declared with a value (`off: none`), that value goes on the wire as the effort parameter. There is no spelling for restoring a catalog map key to "unset": the declaration is the whole offer, so restate the catalog levels you keep.
|
||||
|
||||
### Reasoning-dispatch compat switches
|
||||
|
||||
How a thinking level travels — `reasoning_effort` alone, DeepSeek's `thinking: {type}` plus effort, z.ai's `thinking` object, and so on — is pi-ai's `compat.thinkingFormat`, which pi-ai guesses from the endpoint URL; a private gateway's URL says nothing, so a DeepSeek-dialect gateway would be spoken to in the OpenAI dialect with no way to correct it. `compat.thinkingFormat` and `compat.supportsReasoningEffort` are therefore configurable on the route (its models' default) and per model (winning per field), resolving model → route → installed catalog entry → pi-ai's URL-derived guess; setting a route-level switch shadows the catalog entry's value for every model on the route, and there is no spelling for handing a field back to the catalog short of restating its value. `thinkingFormat` accepts pi-ai's dispatchable formats except the two `chat-template` variants, which need `chatTemplateKwargs` this configuration does not expose. Both switches exist only on `openai-completions` — the other protocols carry their reasoning shape in the protocol itself — so a model-level switch elsewhere fails resolution, a route-level one skips models of other protocols, and a route with no `openai-completions` model at all is refused. The rest of pi-ai's compat surface (`supportsStore`, `maxTokensField`, …) stays auto-detected and is deliberately not configurable here.
|
||||
|
||||
A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768), so a listing that discloses nothing but ids still yields a serviceable route. Both fallbacks are guesses by construction, which is why they are route fields a deployment whose gateway serves smaller models corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap.
|
||||
|
||||
@@ -71,11 +105,11 @@ Credentials resolve per stream call through `apiKeyEnv` and the optional `ctx.cr
|
||||
|
||||
The adapter exposes each configured route's models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata read from the same pi-ai `Models` collection the request path uses, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, configured output cap, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers. A model's **configured** `maxTokens` becomes the seam's `defaultMaxTokens`, so a request that names no output cap carries the one the deployment chose; a value inherited from the installed catalog is the model's output *capability* and never becomes a request default on its own.
|
||||
|
||||
A model that carries reasoning metadata exposes pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`.
|
||||
A model that carries reasoning metadata — from the installed catalog or from its entry's `reasoningEfforts` — exposes pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`.
|
||||
|
||||
A model **without** that metadata — every hand-declared one, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -152,6 +186,7 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Settings can add or override routes, not remove composition routes** — the user layer merges over the composition `base`, so deleting a `cordis.yml`-provided provider is a composition change; `replace` on the namespace only resets the user layer.
|
||||
- **The layered merge has no delete for dict keys** — the settings seam merges the composition `base` and the user layer per key, recursively, so a `reasoningEfforts` level, `modelOverrides` entry, or `compat` field the base declares cannot be removed by the user layer, only overridden — and for `reasoningEfforts` absence *is* the meaning ("not offered"), so a base-declared level stays offered. This only triggers when a `cordis.yml` entry config declares per-model reasoning fields for the same model the user layer edits; the supported posture is to leave those to the settings document (the shipped composition mounts the adapter dormant), and a `models` list is an array replacing wholesale, which is the in-band escape.
|
||||
- **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work).
|
||||
- **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says, so a model list is only as current as its last edit. Nothing here queries a provider for the models it serves; a route gains a model when someone writes one.
|
||||
- **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround.
|
||||
|
||||
@@ -35,6 +35,15 @@
|
||||
models:
|
||||
- id: claude-sonnet-4-5
|
||||
contextWindow: 200000
|
||||
# Catalog route with one model reshaped in place; the rest of the
|
||||
# catalog keeps serving (a models list would replace it instead).
|
||||
deepseek:
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
modelOverrides:
|
||||
deepseek-v4-pro:
|
||||
reasoningEfforts:
|
||||
off:
|
||||
high: high
|
||||
# Hand-declared route: pi-ai ships nothing under this key, so the profile
|
||||
# supplies the whole provider.
|
||||
acme-gateway:
|
||||
@@ -42,18 +51,43 @@
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
# Reasoning dialect for an endpoint whose URL pi-ai cannot recognize.
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
contextWindow: 65536
|
||||
maxTokens: 4096
|
||||
- id: acme-think
|
||||
name: Acme Think
|
||||
contextWindow: 262144
|
||||
maxTokens: 32768
|
||||
# key = selectable level, value = its wire spelling; only off may
|
||||
# leave the value empty (supported, send nothing).
|
||||
reasoningEfforts:
|
||||
off:
|
||||
high: high
|
||||
max: ultra
|
||||
```
|
||||
|
||||
字典形状使重复路由无法表示，发布前的数组形状（每个 profile 携带 `provider` 字段）会加载失败并给出迁移指引。`providers` 也可以为空或整体省略：适配器将以**休眠**姿态挂载——零路由、模型选择器不多一条——一旦 `llm-pi-ai:` settings 分节提供了 profile 就即时注册路由，分节清空时随之撤销。无论是否休眠，插件都会在可配置提供方目录（`ctx.llm.listConfigurableProviders()`，settings 路径 `providers.<provider>`）中声明每个已安装 catalog 提供方，并与当前 profile 声明的每条路由取并集，因此配置界面既能在任何路由存在之前就提供完整 catalog，也能寻址一条手工声明的路由。每个条目都带上 `declared`：pi-ai 在这个键下是否什么都没有。它跟随已安装 catalog 而非设置文档，因为收窄一个内置提供方的模型同样会存下 profile，而那条路由仍然是 pi-ai 认识的——只有适配器分得清两者，所以由目录直接给出答案，而不是留给界面去猜。哪些适配器存在归组合面；哪些提供方在运行可以完全交给用户的设置文档。向 `ctx.llm` 注册具有原子性：如果与另一适配器已拥有的任何提供方路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；路由未配置的模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
|
||||
|
||||
## Catalog 解析
|
||||
|
||||
profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩充它；省略它（或留空）则原样服务该 catalog。每个条目都会从同 `id` 的已安装模型继承自身未设置的字段，因此把 catalog 路由收窄到两个模型、更正某个容量，或加入一个比已安装 catalog 更新的模型，都是一行编辑。只有 harness 会消费的字段可配置——`id`、`name`、`contextWindow` 与 `maxTokens`。定价与输入模态没有 harness 消费方，因此沿用已安装条目或直接缺席。推理则完全不按模型配置：一个孤立的能力布尔量会让 pi-ai 公布出没有 `thinkingLevelMap` 可供拼写的档位，而且没有任何列表端点会报告模型的推理协议，因此推理沿用已安装 catalog 条目或直接缺席。
|
||||
profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩充它；省略它（或留空）则原样服务该 catalog。每个条目都会从同 `id` 的已安装模型继承自身未设置的字段，因此把 catalog 路由收窄到两个模型、更正某个容量，或加入一个比已安装 catalog 更新的模型，都是一行编辑——但一旦声明了 `models` 列表，该路由要继续服务的每个模型就都必须出现在其中，条目哪怕只写一个 `id` 也足够。可配置的条目字段是 `id`、`name`、`contextWindow`、`maxTokens`、`reasoningEfforts` 与 `compat`。定价与输入模态没有 harness 消费方，因此沿用已安装条目或直接缺席。
|
||||
|
||||
`modelOverrides` 无需这份代价就能就地重塑单个已安装 catalog 模型：每个键是一个 catalog 模型 id，每个值可写 `models` 条目接受的同一批字段，只是 id 落在键上，而 catalog 的其余部分原样继续服务——「改一个模型、其余三十七个原样保留」只是一次三行编辑。一条覆盖会成为该 catalog 条目的配置，因此容量、档位与 compat 沿与 `models` 条目相同的路径解析，携带相同的诊断与相同的请求默认值语义。覆盖只在正服务自身 catalog 的 catalog 路由上才有意义：与 `models` 列表并存的一份（该列表本就替换了 catalog）、落在手工声明路由上的一份（其模型已在 `models` 中完整写出），或点名了 catalog 未描述模型的一份，都会被拒绝而非跳过，因为一个静默保持原样的模型，就是一个否则要有人费力追查的笔误。
|
||||
|
||||
### 按模型的推理档位
|
||||
|
||||
`reasoningEfforts` 声明模型可选的思考级别：每个键是选择器提供的一个档位，其值是分派在协议中发送的拼写，因此 `high: high` 原样透传规范名称，而 `max: ultra` 则为使用自有词汇的网关改名。键取自 pi-ai 的档位集合（`off`、`minimal`、`low`、`medium`、`high`、`xhigh`、`max`）；未声明的档位不会被提供。省略该字段会保留已安装 catalog 条目的能力（手工声明的模型没有这份能力，也不推理）；`false` 声明一个不具备推理能力的模型，profile 正是以此从其网关无法服务的 catalog 模型上剥除推理；空声明会被拒绝，而不是在这两种含义之间去猜。
|
||||
|
||||
该声明会转换为 pi-ai 的 `Model.reasoning` + `thinkingLevelMap`，其中每个档位都被显式决定——未声明的档位一律固定为不支持，而不是留给 pi-ai 自己的默认规则：那套规则并不对称（键缺席对五个基础档位意味着「支持」，对 `xhigh`/`max` 却意味着「不支持」），也本不该要求 profile 作者了解。`off` 是唯一的三态键：不写它，选择器不提供 Off，显式请求 Off 会被拒绝——不点名任何档位的请求仍会在不带该参数的情况下发出，提供方随后做什么是它自己的默认行为；声明而不给值（`off:`），则会提供 Off，选中它时什么也不发送——对 `deepseek` 方言则是一个显式的 `thinking: {type: "disabled"}`——这同时覆盖完全不点名任何档位的请求；声明并给值（`off: none`），该值就会作为档位参数在协议中发送。没有任何写法能把 catalog 映射中的键恢复为「未设置」：这份声明就是对外提供的全部，因此把你要保留的 catalog 档位重述出来。
|
||||
|
||||
### 推理分派的 compat 开关
|
||||
|
||||
思考级别如何在协议中传输——单独一个 `reasoning_effort`、DeepSeek 的 `thinking: {type}` 加上档位、z.ai 的 `thinking` 对象，诸如此类——就是 pi-ai 的 `compat.thinkingFormat`，pi-ai 会从端点 URL 猜测它；私有网关的 URL 什么也说明不了，于是说 DeepSeek 方言的网关只会收到 OpenAI 方言的请求，且无从更正。因此 `compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 既可配置在路由上（作为其模型的默认值），也可按模型配置（逐字段胜出），解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测；设置路由级开关会为路由上的每个模型遮蔽 catalog 条目的值，而且除了重述其值，没有任何写法能把某个字段交还给 catalog。`thinkingFormat` 接受 pi-ai 可分派的各种格式，但不含两个 `chat-template` 变体：它们需要的 `chatTemplateKwargs` 本配置并不暴露。两个开关都只存在于 `openai-completions` 上——其余协议的推理形状由协议本身承载——因此在其他协议的模型上设置模型级开关会使解析失败，路由级开关会跳过其他协议的模型，而完全没有 `openai-completions` 模型的路由则会被拒绝。pi-ai compat 面的其余部分（`supportsStore`、`maxTokensField`……）保持自动检测，特意不在此处开放配置。
|
||||
|
||||
条目与已安装 catalog 都没有给出尺寸的模型，会采用该路由的 `defaultContextWindow`（262,144）与 `defaultMaxTokens`（32,768），因此一份只公布 id 的列表同样能产出可服务的路由。两个回退值本质上都是猜测，这正是它们作为路由字段、供网关服务更小模型的部署一次性更正的原因，而不是埋在适配器里的常量；回退值只用于给模型定尺寸，绝不会变成每请求上限。
|
||||
|
||||
@@ -71,11 +105,11 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每条已配置路由的模型。这是从请求路径所用的同一个 pi-ai `Models` 集合读取的提供方无关 selector 元数据，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找，并返回其身份、上下文窗口、已配置输出上限和可选思考级别，让权威元数据保留在拥有路由的适配器上，而非消费方。模型**已配置**的 `maxTokens` 会成为 seam 的 `defaultMaxTokens`，因此未点名输出上限的请求会携带部署选定的那一个；而从已安装 catalog 继承来的值是模型的输出**能力**，绝不会自行变成请求默认值。
|
||||
|
||||
携带推理元数据的模型会公开 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果，不经筛选或规范化，其中包括 `off`，以及模型对 `xhigh` 或 `max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID；提供方／模型在协议格式中的表示仍保留在 pi-ai 的 `thinkingLevelMap` 中。
|
||||
携带推理元数据的模型——来自已安装 catalog，或来自其条目的 `reasoningEfforts`——会公开 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果，不经筛选或规范化，其中包括 `off`，以及模型对 `xhigh` 或 `max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID；提供方／模型在协议格式中的表示仍保留在 pi-ai 的 `thinkingLevelMap` 中。
|
||||
|
||||
**没有**这份元数据的模型——每一个手工声明的模型，以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档，但 `off` 会被翻译成*省略* reasoning 选项，而那与「不点名任何档位」产出的请求逐字节相同：选它关不掉任何东西，于是自身默认就在思考的提供方，会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用，界面就只剩提供方默认这一项，不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。**描述**一个模型则从不这样失败：同一提供方下各模型接受的档位并不一致，因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」，而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处，而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型，以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档，但 `off` 会被翻译成*省略* reasoning 选项，而那与「不点名任何档位」产出的请求逐字节相同：选它关不掉任何东西，于是自身默认就在思考的提供方，会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用，界面就只剩提供方默认这一项，不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。**描述**一个模型则从不这样失败：同一提供方下各模型接受的档位并不一致，因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」，而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处，而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -152,6 +186,7 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **settings 能新增或覆盖路由，但不能移除组合路由**：用户层合并在组合 `base` 之上，因此删除 `cordis.yml` 提供的提供方属于组合变更；对该 namespace 执行 `replace` 只会重置用户层。
|
||||
- **分层合并对字典键没有删除语义**：settings seam 把组合 `base` 与用户层按键递归合并，因此 base 声明的某个 `reasoningEfforts` 档位、`modelOverrides` 条目或 `compat` 字段，用户层只能覆盖、无法移除——而 `reasoningEfforts` 里缺席本身*就是*语义（「不提供」），于是 base 声明过的档位会一直被提供。只有 `cordis.yml` entry config 为用户层正在编辑的同一模型声明了按模型推理字段才会触发；受支持的姿态是把这些字段留给 settings 文档（shipped 组合以休眠方式挂载该适配器），且 `models` 列表是数组、整体替换，这是体制内的出口。
|
||||
- **`headers` 可能承载一条脱敏器看不见的凭据**：profile 的 `headers` 是纯字符串字典，因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回，并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用；把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。
|
||||
- **路由的 catalog 不会自我刷新**：catalog 就是 `settings.yaml` 所写的内容，因此模型列表的新鲜度只到最近一次编辑为止。这里没有任何环节会去问提供方它服务哪些模型；路由要多一个模型，得有人写进去。
|
||||
- **每条路由只有一种协议格式**：`api` 作用于整条路由，因此混合协议的 catalog 路由（跨 Responses 与 Chat Completions 的 OpenAI 式 catalog）无法承载另一种协议的模型，向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。
|
||||
|
||||
@@ -14,7 +14,15 @@
|
||||
|
||||
import { builtinProviders, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { Api, Model, ModelCost, Provider } from '@earendil-works/pi-ai'
|
||||
import type {
|
||||
Api,
|
||||
Model,
|
||||
ModelCost,
|
||||
ModelThinkingLevel,
|
||||
OpenAICompletionsCompat,
|
||||
Provider,
|
||||
ThinkingLevelMap,
|
||||
} from '@earendil-works/pi-ai'
|
||||
|
||||
/**
|
||||
* Pricing for a model the installed catalog does not describe. The harness
|
||||
@@ -30,6 +38,58 @@ const NO_COST: ModelCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
|
||||
*/
|
||||
const TEXT_ONLY: Model<Api>['input'] = ['text']
|
||||
|
||||
/**
|
||||
* Every pi-ai thinking level, in pi-ai's canonical escalation order. The
|
||||
* `Record` key type is a drift gate: a pi-ai upgrade that adds or removes a
|
||||
* level fails compilation here naming the drifted key, instead of silently
|
||||
* narrowing what a profile may declare.
|
||||
*/
|
||||
const THINKING_LEVEL_GATE: Record<ModelThinkingLevel, true> = {
|
||||
off: true,
|
||||
minimal: true,
|
||||
low: true,
|
||||
medium: true,
|
||||
high: true,
|
||||
xhigh: true,
|
||||
max: true,
|
||||
}
|
||||
|
||||
/** Every pi-ai thinking level a profile may declare, in escalation order. */
|
||||
export const THINKING_LEVELS = Object.keys(THINKING_LEVEL_GATE) as readonly ModelThinkingLevel[]
|
||||
|
||||
/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */
|
||||
type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
|
||||
/**
|
||||
* pi-ai thinking formats a profile cannot name: both drive the request through
|
||||
* `chatTemplateKwargs`, which this configuration does not expose, so offering
|
||||
* them would hand back a format with nothing to say.
|
||||
*/
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
|
||||
/** One reasoning-dispatch wire format a profile may name. */
|
||||
export type PiAiThinkingFormat = Exclude<PiThinkingFormat, WithheldThinkingFormat>
|
||||
|
||||
/**
|
||||
* The nameable reasoning-dispatch formats, most-reached first. The `Record`
|
||||
* key type is a drift gate: a pi-ai upgrade that adds a format (0.84 added
|
||||
* `baseten`) fails compilation here until the format is classified as offered
|
||||
* here or withheld above, so the offer never silently lags the upstream set.
|
||||
*/
|
||||
const THINKING_FORMAT_GATE: Record<PiAiThinkingFormat, true> = {
|
||||
'openai': true,
|
||||
'deepseek': true,
|
||||
'openrouter': true,
|
||||
'together': true,
|
||||
'zai': true,
|
||||
'qwen': true,
|
||||
'string-thinking': true,
|
||||
'ant-ling': true,
|
||||
}
|
||||
|
||||
/** Reasoning-dispatch wire formats a profile may name, most-reached first. */
|
||||
export const SUPPORTED_THINKING_FORMATS = Object.keys(THINKING_FORMAT_GATE) as readonly PiAiThinkingFormat[]
|
||||
|
||||
let providerIndex: Map<string, Provider> | undefined
|
||||
|
||||
/**
|
||||
@@ -71,6 +131,32 @@ export function catalogModels(provider: string): Map<string, Model<Api>> {
|
||||
return new Map(models.map(model => [model.id, model]))
|
||||
}
|
||||
|
||||
/**
|
||||
* Selectable reasoning efforts for one model: each key is a level the model
|
||||
* offers (and selectors show), and its value is the wire spelling dispatch
|
||||
* sends for it. `off` alone may leave its value empty — "supported, send
|
||||
* nothing" — because for most providers not thinking is the parameter's
|
||||
* absence; every other declared level must name a wire value. A level absent
|
||||
* from the dict is not offered.
|
||||
*/
|
||||
export type PiAiReasoningEfforts = Partial<Record<ModelThinkingLevel, string | null>>
|
||||
|
||||
/**
|
||||
* Reasoning-dispatch compatibility switches, set on the route (its models'
|
||||
* default) or per model (winning over the route). Only the switches pi-ai's
|
||||
* reasoning dispatch reads are offered; the rest of pi-ai's compat surface
|
||||
* keeps its baseURL-derived auto-detection. pi-ai types both fields only on
|
||||
* `OpenAICompletionsCompat` — the other wire protocols carry their reasoning
|
||||
* shape in the protocol itself — so resolution rejects a model-level switch
|
||||
* anywhere else, while a route-level default skips past models it cannot fit.
|
||||
*/
|
||||
export interface PiAiCompatProfile {
|
||||
/** Reasoning parameter shape the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
supportsReasoningEffort?: boolean
|
||||
}
|
||||
|
||||
/** One configured model entry: an id plus the catalog fields it overrides. */
|
||||
export interface PiAiModelProfile {
|
||||
/** Model id sent to the provider and accepted by {@link GenerateOptions.model}. */
|
||||
@@ -86,8 +172,27 @@ export interface PiAiModelProfile {
|
||||
* default on its own.
|
||||
*/
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Selectable reasoning efforts. Absent inherits the installed catalog
|
||||
* entry's capability (a hand-declared model has none and does not reason);
|
||||
* `false` declares a non-reasoning model, which is how a profile strips
|
||||
* reasoning from a catalog model its gateway cannot serve; a non-empty dict
|
||||
* declares the offered levels and their wire spellings.
|
||||
*/
|
||||
reasoningEfforts?: false | PiAiReasoningEfforts
|
||||
/** Reasoning-dispatch switches for this model, winning over the route's. */
|
||||
compat?: PiAiCompatProfile
|
||||
}
|
||||
|
||||
/**
|
||||
* Customization of one installed catalog model, keyed by its id in the
|
||||
* route's `modelOverrides` dict — the same fields a `models` entry may set,
|
||||
* with the id living in the key. Unlike a `models` list, overrides leave the
|
||||
* rest of the catalog serving untouched, which is what makes "correct one
|
||||
* model, keep the other thirty-seven" a three-line edit.
|
||||
*/
|
||||
export type PiAiModelOverride = Omit<PiAiModelProfile, 'id'>
|
||||
|
||||
/** The route-level facts model materialization reads. */
|
||||
export interface RouteCatalogRequest {
|
||||
/** Provider route key, stamped onto every materialized model. */
|
||||
@@ -98,6 +203,10 @@ export interface RouteCatalogRequest {
|
||||
baseURL?: string
|
||||
/** Configured catalog; absent means the whole installed catalog for this route. */
|
||||
models?: readonly PiAiModelProfile[]
|
||||
/** Installed-catalog customizations by model id; only meaningful while `models` is absent. */
|
||||
modelOverrides?: Readonly<Record<string, PiAiModelOverride>>
|
||||
/** Reasoning-dispatch switches for every `openai-completions` model on the route; entries override per field. */
|
||||
compat?: PiAiCompatProfile
|
||||
/** Context capacity for a model neither the entry nor the catalog sizes. */
|
||||
defaultContextWindow: number
|
||||
/** Output capability for a model neither the entry nor the catalog sizes. */
|
||||
@@ -123,6 +232,137 @@ function sharedCatalogApi(defaults: ReadonlyMap<string, Model<Api>>): string | u
|
||||
return apis.size === 1 ? [...apis][0] : undefined
|
||||
}
|
||||
|
||||
/** The reasoning fields one materialized model carries. */
|
||||
interface ModelReasoning {
|
||||
/** Whether the model reasons at all; `false` makes pi-ai ignore the map. */
|
||||
reasoning: boolean
|
||||
/** The map dispatch reads; absent only when the installed entry's (or none) applies. */
|
||||
thinkingLevelMap?: ThinkingLevelMap
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one model's reasoning capability from its declared efforts.
|
||||
*
|
||||
* A declared dict translates to pi-ai's `thinkingLevelMap` with every level
|
||||
* decided explicitly: declared levels carry their wire spelling, undeclared
|
||||
* levels are pinned to `null` (unsupported). Pinning matters because pi-ai's
|
||||
* own defaulting is asymmetric — an absent key means "supported" for the five
|
||||
* base levels but "unsupported" for `xhigh`/`max` — and a profile author
|
||||
* should not need to know that. A declared `off` with no value is the one
|
||||
* exception: it stays absent from the map, which pi-ai reads as "supported,
|
||||
* send nothing" — the correct dispatch where not thinking is the parameter's
|
||||
* absence — while `off` with a value sends that value.
|
||||
* @param provider - provider route key, for diagnostics.
|
||||
* @param entry - the configured model entry.
|
||||
* @param base - the installed catalog entry of the same id, when one exists.
|
||||
* @returns the reasoning fields the materialized model carries.
|
||||
*/
|
||||
function resolveModelReasoning(
|
||||
provider: string,
|
||||
entry: PiAiModelProfile,
|
||||
base: Model<Api> | undefined,
|
||||
): ModelReasoning {
|
||||
const efforts = entry.reasoningEfforts
|
||||
if (efforts === undefined) {
|
||||
// Reasoning rides the installed entry or is absent: a bare capability flag
|
||||
// would make pi-ai advertise effort levels with no `thinkingLevelMap` to
|
||||
// spell them, and no listing endpoint reports a model's reasoning
|
||||
// protocol. The entry's map (when any) arrives through the `...base`
|
||||
// spread in the model literal.
|
||||
return { reasoning: base?.reasoning ?? false }
|
||||
}
|
||||
// The installed entry's map may ride along through `...base`; pi-ai never
|
||||
// reads it on a non-reasoning model, so stripping it is not worth a field
|
||||
// enumeration here.
|
||||
if (efforts === false) return { reasoning: false }
|
||||
// A YAML `reasoningEfforts:` left valueless arrives as null through the
|
||||
// schema union — outside the field's declared type, hence the widening —
|
||||
// while an explicit `{}` arrives as an empty dict. Both declare nothing,
|
||||
// and neither is a spelling of "inherit" or "disable".
|
||||
if ((efforts as unknown) === null || Object.keys(efforts).length === 0) {
|
||||
invalid(provider, `model "${entry.id}" has an empty reasoningEfforts; declare the offered levels, set`
|
||||
+ ' false for a non-reasoning model, or omit the field to keep the installed catalog\'s capability')
|
||||
}
|
||||
const declared = THINKING_LEVELS.flatMap((level) => {
|
||||
const wire = efforts[level]
|
||||
return wire === undefined ? [] : [[level, wire] as const]
|
||||
})
|
||||
for (const [level, wire] of declared) {
|
||||
if (wire === null) {
|
||||
if (level !== 'off') {
|
||||
invalid(provider, `model "${entry.id}" reasoningEfforts.${level} needs the wire value dispatch`
|
||||
+ ' should send; only "off" may leave it empty')
|
||||
}
|
||||
} else if (wire.length === 0) {
|
||||
invalid(provider, `model "${entry.id}" reasoningEfforts.${level} must not be an empty string`)
|
||||
}
|
||||
}
|
||||
if (!declared.some(([level]) => level !== 'off')) {
|
||||
invalid(provider, `model "${entry.id}" reasoningEfforts offers no level beyond "off"; declare a thinking`
|
||||
+ ' level, or set reasoningEfforts to false for a non-reasoning model')
|
||||
}
|
||||
const map: ThinkingLevelMap = {}
|
||||
for (const level of THINKING_LEVELS) {
|
||||
const wire = efforts[level]
|
||||
if (wire === undefined) {
|
||||
map[level] = null
|
||||
} else if (wire !== null) {
|
||||
map[level] = wire
|
||||
}
|
||||
}
|
||||
return { reasoning: true, thinkingLevelMap: map }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one model's compat block from the profile's reasoning switches.
|
||||
*
|
||||
* A model switch wins over the route switch; whatever neither sets keeps the
|
||||
* installed entry's value, and a field no layer decides falls through to
|
||||
* pi-ai's baseURL-derived detection. Only an `openai-completions` model takes
|
||||
* the switches at all: a model-level switch on any other protocol fails
|
||||
* resolution, while a route-level default skips past such models — the same
|
||||
* posture as the route-level `reasoning` default, which also must not fail
|
||||
* models it does not fit.
|
||||
* @param provider - provider route key, for diagnostics.
|
||||
* @param entry - the configured model entry.
|
||||
* @param route - the route-level switches, when any.
|
||||
* @param base - the installed catalog entry of the same id, when one exists.
|
||||
* @param api - the model's resolved wire protocol.
|
||||
* @returns a `compat` field to spread into the model, or nothing.
|
||||
*/
|
||||
function resolveModelCompat(
|
||||
provider: string,
|
||||
entry: PiAiModelProfile,
|
||||
route: PiAiCompatProfile | undefined,
|
||||
base: Model<Api> | undefined,
|
||||
api: string,
|
||||
): { compat: OpenAICompletionsCompat } | Record<string, never> {
|
||||
const thinkingFormat = entry.compat?.thinkingFormat ?? route?.thinkingFormat
|
||||
const supportsReasoningEffort = entry.compat?.supportsReasoningEffort ?? route?.supportsReasoningEffort
|
||||
if (thinkingFormat === undefined && supportsReasoningEffort === undefined) return {}
|
||||
if (api !== 'openai-completions') {
|
||||
if (entry.compat?.thinkingFormat !== undefined || entry.compat?.supportsReasoningEffort !== undefined) {
|
||||
invalid(provider, `model "${entry.id}" sets compat reasoning switches, but its api is "${api}";`
|
||||
+ ' thinkingFormat and supportsReasoningEffort exist only on openai-completions')
|
||||
}
|
||||
return {}
|
||||
}
|
||||
// The installed entry's compat matches the entry's OWN api — a route-level
|
||||
// `api` repoint (an anthropic catalog served through an OpenAI-compatible
|
||||
// gateway) leaves `base.compat` in the other protocol's shape, so it is
|
||||
// inherited only while the resolved api still is the entry's. A repointed
|
||||
// model starts from pi-ai's baseURL-derived detection instead, which is
|
||||
// what a protocol change means for every other compat field too.
|
||||
const inherited: OpenAICompletionsCompat | undefined = base?.api === api ? base.compat : undefined
|
||||
return {
|
||||
compat: {
|
||||
...inherited,
|
||||
...thinkingFormat === undefined ? {} : { thinkingFormat },
|
||||
...supportsReasoningEffort === undefined ? {} : { supportsReasoningEffort },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** One route's materialized catalog, plus the request caps its profile chose. */
|
||||
export interface RouteCatalog {
|
||||
/** The materialized models in configuration order. */
|
||||
@@ -156,14 +396,42 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
// schema materializes `[]` for the absent case, and an empty catalog could
|
||||
// serve no request anyway, so both mean "serve the installed catalog".
|
||||
const configured = request.models ?? []
|
||||
const overrides = request.modelOverrides ?? {}
|
||||
// Every miss is refused, never skipped: an override that lands nowhere is a
|
||||
// typo someone would otherwise hunt for in a silently unchanged model.
|
||||
for (const [id, override] of Object.entries(overrides)) {
|
||||
if (id.length === 0) invalid(provider, 'has a modelOverrides entry with an empty model id')
|
||||
if (defaults.size === 0) {
|
||||
invalid(provider, `sets modelOverrides for "${id}", but the installed catalog does not describe this route;`
|
||||
+ ' a declared route spells every model out in its models list')
|
||||
}
|
||||
if (configured.length > 0) {
|
||||
invalid(provider, `sets modelOverrides for "${id}" beside a models list; models already replaces the served`
|
||||
+ ' catalog, so declare the fields on its entries')
|
||||
}
|
||||
if (!defaults.has(id)) {
|
||||
invalid(provider, `modelOverrides names "${id}", which the installed catalog does not describe`)
|
||||
}
|
||||
// The id lives in the dict key; a value carrying its own would quietly
|
||||
// rename the model it meant to customize. The static shape already omits
|
||||
// it — this guards the schema boundary, which passes unknown keys through.
|
||||
if ('id' in override) {
|
||||
invalid(provider, `modelOverrides entry "${id}" sets "id", which is the dict key`)
|
||||
}
|
||||
}
|
||||
// An override becomes the catalog entry's configuration, so everything a
|
||||
// models entry may declare — capacities, efforts, compat — resolves through
|
||||
// the same path with the same diagnostics and request-default semantics.
|
||||
const entries: readonly PiAiModelProfile[] = configured.length > 0
|
||||
? configured
|
||||
: [...defaults.values()].map(model => ({ id: model.id }))
|
||||
: [...defaults.values()].map(model => ({ id: model.id, ...overrides[model.id] }))
|
||||
if (entries.length === 0) {
|
||||
invalid(provider, 'resolves no models; the installed catalog does not describe this route, so its models'
|
||||
+ ' must be listed in configuration')
|
||||
}
|
||||
const routeApi = sharedCatalogApi(defaults)
|
||||
const routeCompatDefined = request.compat?.thinkingFormat !== undefined
|
||||
|| request.compat?.supportsReasoningEffort !== undefined
|
||||
const seen = new Set<string>()
|
||||
const configuredMaxTokens = new Map<string, number>()
|
||||
const models = entries.map((entry) => {
|
||||
@@ -209,15 +477,17 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
api,
|
||||
provider,
|
||||
baseUrl,
|
||||
// Reasoning rides the installed entry or is absent: a bare boolean would
|
||||
// make pi-ai advertise effort levels with no `thinkingLevelMap` to spell
|
||||
// them, and no listing endpoint reports a model's reasoning protocol.
|
||||
reasoning: base?.reasoning ?? false,
|
||||
input: base?.input ?? TEXT_ONLY,
|
||||
cost: base?.cost ?? NO_COST,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
...resolveModelReasoning(provider, entry, base),
|
||||
...resolveModelCompat(provider, entry, request.compat, base, api),
|
||||
}
|
||||
})
|
||||
if (routeCompatDefined && !models.some(model => model.api === 'openai-completions')) {
|
||||
invalid(provider, 'sets compat reasoning switches, but no model on the route speaks openai-completions;'
|
||||
+ ' thinkingFormat and supportsReasoningEffort exist only on that protocol')
|
||||
}
|
||||
return { models, configuredMaxTokens }
|
||||
}
|
||||
|
||||
@@ -21,8 +21,8 @@ import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { resolveRouteModels } from './catalog.ts'
|
||||
import type { PiAiModelProfile } from './catalog.ts'
|
||||
import { resolveRouteModels, SUPPORTED_THINKING_FORMATS, THINKING_LEVELS } from './catalog.ts'
|
||||
import type { PiAiCompatProfile, PiAiModelOverride, PiAiModelProfile, PiAiReasoningEfforts } from './catalog.ts'
|
||||
import { buildProvider, supportedProtocols } from './provider.ts'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
@@ -34,7 +34,13 @@ export const DEFAULT_CONTEXT_WINDOW = 262_144
|
||||
/** Output capability assumed for a model neither configuration nor the catalog sizes. */
|
||||
export const DEFAULT_MAX_TOKENS = 32_768
|
||||
|
||||
export type { PiAiModelProfile } from './catalog.ts'
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiReasoningEfforts,
|
||||
PiAiThinkingFormat,
|
||||
} from './catalog.ts'
|
||||
|
||||
/** Configuration for one pi-ai provider route; the `providers` dict key IS the route. */
|
||||
export interface PiAiProviderProfile {
|
||||
@@ -56,6 +62,22 @@ export interface PiAiProviderProfile {
|
||||
* unset fields from the installed model of the same id.
|
||||
*/
|
||||
models?: PiAiModelProfile[]
|
||||
/**
|
||||
* Installed-catalog customizations by model id: each entry reshapes that
|
||||
* one model with the same fields a {@link models} entry takes, while the
|
||||
* rest of the catalog keeps serving untouched. Only meaningful on a catalog
|
||||
* route with no `models` list — `models` already replaces the catalog, so
|
||||
* an override beside it, on a route the catalog does not ship, or naming a
|
||||
* model the catalog does not describe is refused rather than skipped.
|
||||
*/
|
||||
modelOverrides?: Record<string, PiAiModelOverride>
|
||||
/**
|
||||
* Reasoning-dispatch switches for every `openai-completions` model on this
|
||||
* route; each model's own `compat` overrides per field. What neither sets
|
||||
* keeps the installed catalog entry's value, then pi-ai's baseURL-derived
|
||||
* detection.
|
||||
*/
|
||||
compat?: PiAiCompatProfile
|
||||
/**
|
||||
* Context capacity for a model this route lists that neither the entry nor
|
||||
* the installed catalog sizes (default 262,144). A guess by construction, so
|
||||
@@ -133,23 +155,58 @@ const thinkingBudgets = z.object({
|
||||
high: z.number(),
|
||||
})
|
||||
|
||||
const modelProfile: z<PiAiModelProfile> = z.object({
|
||||
id: z.string().required(),
|
||||
const compatProfile: z<PiAiCompatProfile> = z.object({
|
||||
thinkingFormat: z.union(SUPPORTED_THINKING_FORMATS),
|
||||
supportsReasoningEffort: z.boolean(),
|
||||
})
|
||||
|
||||
/**
|
||||
* Keys are the offered levels, values their wire spellings. A valueless key
|
||||
* (`off:`) survives validation because schemastery passes nullable data
|
||||
* through before any member schema runs — `z.const(null)` only shapes the
|
||||
* error for non-null wrong values and what a configuration surface renders.
|
||||
* Only resolution decides which levels may leave the value empty, so the
|
||||
* diagnostic can name the route and model. The assertion narrows
|
||||
* schemastery's `Dict`, which types every literal key as required; dict
|
||||
* validation is per-present-key, so the runtime shape is the partial record.
|
||||
*/
|
||||
const reasoningEfforts = z.dict(
|
||||
z.union([z.string(), z.const(null)]),
|
||||
z.union(THINKING_LEVELS),
|
||||
) as unknown as z<PiAiReasoningEfforts>
|
||||
|
||||
/** The fields a `models` entry and a `modelOverrides` value share; only the id's home differs. */
|
||||
const modelFields = {
|
||||
name: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
maxTokens: z.number().step(1).min(1),
|
||||
// The union, not a bare dict: schemastery materializes an absent dict as
|
||||
// `{}`, and absent must stay distinguishable — it means "inherit the
|
||||
// installed catalog's capability", while `false` disables reasoning.
|
||||
reasoningEfforts: z.union([z.const(false), reasoningEfforts]),
|
||||
compat: compatProfile,
|
||||
}
|
||||
|
||||
const modelProfile: z<PiAiModelProfile> = z.object({
|
||||
id: z.string().required(),
|
||||
...modelFields,
|
||||
})
|
||||
|
||||
/** A {@link modelProfile} whose id lives in the `modelOverrides` dict key. */
|
||||
const modelOverride: z<PiAiModelOverride> = z.object(modelFields)
|
||||
|
||||
const profile = z.object({
|
||||
apiKeyEnv: z.string().role('credential-ref'),
|
||||
displayName: z.string(),
|
||||
api: z.union(supportedProtocols()),
|
||||
baseURL: z.string(),
|
||||
models: z.array(modelProfile),
|
||||
modelOverrides: z.dict(modelOverride),
|
||||
compat: compatProfile,
|
||||
defaultContextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW),
|
||||
defaultMaxTokens: z.number().step(1).min(1).default(DEFAULT_MAX_TOKENS),
|
||||
headers: z.dict(z.string()),
|
||||
reasoning: z.union(['off', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
|
||||
reasoning: z.union(THINKING_LEVELS),
|
||||
thinkingBudgets,
|
||||
cacheRetention: z.union(['none', 'short', 'long']),
|
||||
transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']),
|
||||
@@ -240,6 +297,8 @@ export function resolveProfiles(
|
||||
...source.api === undefined ? {} : { api: source.api },
|
||||
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
|
||||
...source.models === undefined ? {} : { models: source.models },
|
||||
...source.modelOverrides === undefined ? {} : { modelOverrides: source.modelOverrides },
|
||||
...source.compat === undefined ? {} : { compat: source.compat },
|
||||
defaultContextWindow: source.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW,
|
||||
defaultMaxTokens: source.defaultMaxTokens ?? DEFAULT_MAX_TOKENS,
|
||||
})
|
||||
|
||||
@@ -32,11 +32,24 @@
|
||||
* apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
* api: openai-completions
|
||||
* baseURL: https://gateway.acme.example/v1
|
||||
* # Reasoning dialect for a URL pi-ai cannot recognize.
|
||||
* compat:
|
||||
* thinkingFormat: deepseek
|
||||
* models:
|
||||
* - id: acme-large
|
||||
* name: Acme Large
|
||||
* contextWindow: 65536
|
||||
* maxTokens: 4096
|
||||
* - id: acme-think
|
||||
* name: Acme Think
|
||||
* contextWindow: 262144
|
||||
* maxTokens: 32768
|
||||
* # key = selectable level, value = wire spelling; only off may
|
||||
* # leave the value empty (supported, send nothing).
|
||||
* reasoningEfforts:
|
||||
* off:
|
||||
* high: high
|
||||
* max: ultra
|
||||
* ```
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-pi-ai
|
||||
@@ -56,7 +69,15 @@ import { discoverModels } from './discovery.ts'
|
||||
export { PiAiAdapter } from './adapter.ts'
|
||||
export type { PiAiAdapterOptions } from './adapter.ts'
|
||||
export { Config } from './config.ts'
|
||||
export type { PiAiModelProfile, PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
export type {
|
||||
PiAiCompatProfile,
|
||||
PiAiModelOverride,
|
||||
PiAiModelProfile,
|
||||
PiAiProviderProfile,
|
||||
PiAiReasoningEfforts,
|
||||
PiAiThinkingFormat,
|
||||
ResolvedPiAiProviderProfile,
|
||||
} from './config.ts'
|
||||
export { supportedProtocols } from './provider.ts'
|
||||
|
||||
export const name = 'llm-pi-ai'
|
||||
|
||||
@@ -481,6 +481,185 @@ describe('provider profile lifecycle', () => {
|
||||
.resolves.toMatchObject({ reasoning: { defaultEffort: ReasoningEffortId('off') } })
|
||||
})
|
||||
|
||||
it('serves declared reasoning efforts to selectors and honours the profile default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test/v1',
|
||||
reasoning: 'high',
|
||||
models: [{
|
||||
id: 'acme-think',
|
||||
contextWindow: 65_536,
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: null, low: 'low', high: 'high' },
|
||||
}],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
// Declared levels reach the same seam catalog metadata does, so the
|
||||
// effort picker works for a model pi-ai has never heard of.
|
||||
await expect(ctx.llm.resolveModelInfo('acme-gateway', 'acme-think')).resolves.toMatchObject({
|
||||
reasoning: {
|
||||
efforts: [
|
||||
{ id: ReasoningEffortId('off'), name: 'Off' },
|
||||
{ id: ReasoningEffortId('low'), name: 'Low' },
|
||||
{ id: ReasoningEffortId('high'), name: 'High' },
|
||||
],
|
||||
defaultEffort: ReasoningEffortId('high'),
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('sends the declared wire spelling and refuses undeclared levels before network I/O', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [{
|
||||
id: 'acme-think',
|
||||
contextWindow: 65_536,
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: null, high: 'ultra' },
|
||||
}],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-think',
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: [],
|
||||
})
|
||||
// The declared value, not the canonical level name, goes on the wire.
|
||||
expect(server.requests[0]).toMatchObject({ reasoning_effort: 'ultra' })
|
||||
|
||||
const undeclared = await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-think',
|
||||
reasoningEffort: ReasoningEffortId('max'),
|
||||
messages: [],
|
||||
})
|
||||
expect(undeclared.finish).toMatchObject({
|
||||
kind: 'error',
|
||||
failure: { code: 'UNSUPPORTED_REASONING_EFFORT' },
|
||||
})
|
||||
expect(server.requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('dispatches the compat-switched dialect on a declared route', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
// Without the switch pi-ai guesses the dialect from the endpoint
|
||||
// URL, and a private gateway's URL says nothing.
|
||||
compat: { thinkingFormat: 'deepseek' },
|
||||
models: [{
|
||||
id: 'acme-think',
|
||||
contextWindow: 65_536,
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: null, high: 'high' },
|
||||
}],
|
||||
},
|
||||
},
|
||||
})
|
||||
const prompt = (effort: string): Promise<unknown> => assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-think',
|
||||
reasoningEffort: ReasoningEffortId(effort),
|
||||
messages: [],
|
||||
})
|
||||
|
||||
await prompt('high')
|
||||
expect(server.requests[0]).toMatchObject({ thinking: { type: 'enabled' }, reasoning_effort: 'high' })
|
||||
|
||||
await prompt('off')
|
||||
expect(server.requests[1]).toMatchObject({ thinking: { type: 'disabled' } })
|
||||
expect(server.requests[1]).not.toHaveProperty('reasoning_effort')
|
||||
})
|
||||
|
||||
it('sends a declared off value as the effort parameter instead of omitting it', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [{
|
||||
id: 'acme-think',
|
||||
contextWindow: 65_536,
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: 'none', high: 'high' },
|
||||
}],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
// The adapter strips a selected Off to "no reasoning option", and pi-ai's
|
||||
// dispatch reads thinkingLevelMap.off exactly then — so the declared value
|
||||
// still reaches the wire, which is the README's promise for `off: none`.
|
||||
await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-think',
|
||||
reasoningEffort: ReasoningEffortId('off'),
|
||||
messages: [],
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({ reasoning_effort: 'none' })
|
||||
})
|
||||
|
||||
it('holds back reasoning_effort when the endpoint cannot take it', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
compat: { supportsReasoningEffort: false },
|
||||
models: [{
|
||||
id: 'acme-think',
|
||||
contextWindow: 65_536,
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: null, high: 'high' },
|
||||
}],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-think',
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
messages: [],
|
||||
})
|
||||
expect(server.requests[0]).not.toHaveProperty('reasoning_effort')
|
||||
})
|
||||
|
||||
it('accepts absent credentials for pi-ai ambient authentication', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'ambient-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
|
||||
@@ -10,8 +10,8 @@ import { settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import { createModels } from '@earendil-works/pi-ai'
|
||||
import type { Api, Model, Provider } from '@earendil-works/pi-ai'
|
||||
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import type { Api, Model, OpenAICompletionsCompat, Provider } from '@earendil-works/pi-ai'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { buildProvider, supportedProtocols } from '../src/provider.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
@@ -484,6 +484,248 @@ describe('catalog routes with per-model configuration', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('per-model reasoning efforts', () => {
|
||||
/** One hand-declared route holding exactly the given models. */
|
||||
function declared(models: LlmPiAi.PiAiModelProfile[]): Record<string, LlmPiAi.PiAiProviderProfile> {
|
||||
return { 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models } }
|
||||
}
|
||||
|
||||
/** The first materialized model of one route, or throw. */
|
||||
function modelOf(providers: Record<string, LlmPiAi.PiAiProviderProfile>, route = 'acme-gateway'): Model<Api> {
|
||||
const [model] = resolveProfiles(providers).get(route)?.piProvider.getModels() ?? []
|
||||
if (model === undefined) throw new Error(`route "${route}" resolved no models`)
|
||||
return model
|
||||
}
|
||||
|
||||
it('declares selectable levels with their wire spellings on a hand-declared model', () => {
|
||||
const model = modelOf(declared([{
|
||||
id: 'acme-think',
|
||||
reasoningEfforts: { off: null, low: 'low', high: 'high', max: 'ultra' },
|
||||
}]))
|
||||
|
||||
expect(model.reasoning).toBe(true)
|
||||
// Undeclared levels are pinned null rather than left to pi-ai's own
|
||||
// defaulting, which is asymmetric: an absent key means "supported" for the
|
||||
// five base levels but "unsupported" for xhigh/max. A profile author
|
||||
// should not need to know that. Declared `off` with no value stays absent
|
||||
// from the map — supported, send nothing.
|
||||
expect(model.thinkingLevelMap).toEqual({
|
||||
minimal: null,
|
||||
medium: null,
|
||||
xhigh: null,
|
||||
low: 'low',
|
||||
high: 'high',
|
||||
max: 'ultra',
|
||||
})
|
||||
expect(getSupportedThinkingLevels(model)).toEqual(['off', 'low', 'high', 'max'])
|
||||
})
|
||||
|
||||
it('keeps a declared off value in the map for dispatch to send', () => {
|
||||
const model = modelOf(declared([{ id: 'm', reasoningEfforts: { off: 'none', high: 'high' } }]))
|
||||
expect(model.thinkingLevelMap?.off).toBe('none')
|
||||
expect(getSupportedThinkingLevels(model)).toEqual(['off', 'high'])
|
||||
})
|
||||
|
||||
it('offers exactly the declared keys: leaving off out makes thinking mandatory', () => {
|
||||
const model = modelOf(declared([{ id: 'm', reasoningEfforts: { high: 'high' } }]))
|
||||
expect(getSupportedThinkingLevels(model)).toEqual(['high'])
|
||||
})
|
||||
|
||||
it('narrows a catalog model’s levels in place', () => {
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
expect(getSupportedThinkingLevels(catalogModel as Model<Api>)).toEqual(['off', 'high', 'max'])
|
||||
|
||||
const model = modelOf({
|
||||
deepseek: { models: [{ id: catalogModel.id, reasoningEfforts: { off: null, high: 'high' } }] },
|
||||
}, 'deepseek')
|
||||
|
||||
expect(getSupportedThinkingLevels(model)).toEqual(['off', 'high'])
|
||||
// Only the reasoning fields change; identity and capacities stay catalog.
|
||||
expect(model.name).toBe(catalogModel.name)
|
||||
expect(model.contextWindow).toBe(catalogModel.contextWindow)
|
||||
})
|
||||
|
||||
it('strips reasoning from a catalog model with false', () => {
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
expect(catalogModel.reasoning).toBe(true)
|
||||
|
||||
const model = modelOf({ deepseek: { models: [{ id: catalogModel.id, reasoningEfforts: false }] } }, 'deepseek')
|
||||
|
||||
expect(model.reasoning).toBe(false)
|
||||
expect(getSupportedThinkingLevels(model)).toEqual(['off'])
|
||||
})
|
||||
|
||||
it('inherits the catalog capability when the field is absent', () => {
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
|
||||
const model = modelOf({ deepseek: { models: [{ id: catalogModel.id }] } }, 'deepseek')
|
||||
|
||||
expect(model.reasoning).toBe(catalogModel.reasoning)
|
||||
expect(model.thinkingLevelMap).toEqual(catalogModel.thinkingLevelMap)
|
||||
})
|
||||
|
||||
it('rejects a declaration that offers nothing or spells a level it cannot send', () => {
|
||||
const declare = (efforts: NonNullable<LlmPiAi.PiAiModelProfile['reasoningEfforts']>): (() => unknown) =>
|
||||
() => resolveProfiles(declared([{ id: 'm', reasoningEfforts: efforts }]))
|
||||
|
||||
expect(declare({})).toThrow(/empty reasoningEfforts/)
|
||||
// A YAML `reasoningEfforts:` left valueless arrives as null through the
|
||||
// schema union; it declares nothing and is not a spelling of "inherit".
|
||||
expect(declare(null as never)).toThrow(/empty reasoningEfforts/)
|
||||
expect(declare({ off: null })).toThrow(/offers no level beyond "off"/)
|
||||
expect(declare({ off: 'none' })).toThrow(/offers no level beyond "off"/)
|
||||
expect(declare({ high: null })).toThrow(/only "off" may leave it empty/)
|
||||
expect(declare({ high: '' })).toThrow(/must not be an empty string/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('modelOverrides', () => {
|
||||
const deepseekModel = (): Model<Api> => {
|
||||
const [model] = getBuiltinModels('deepseek')
|
||||
if (model === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
return model
|
||||
}
|
||||
|
||||
it('reshapes one catalog model while the rest of the catalog keeps serving', () => {
|
||||
const catalogSize = getBuiltinModels('deepseek').length
|
||||
const target = deepseekModel()
|
||||
const resolved = resolveProfiles({
|
||||
deepseek: {
|
||||
modelOverrides: {
|
||||
[target.id]: {
|
||||
name: 'DeepSeek (proxied)',
|
||||
maxTokens: 4096,
|
||||
reasoningEfforts: { off: null, high: 'high' },
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
const models = resolved.get('deepseek')?.piProvider.getModels() ?? []
|
||||
const reshaped = models.find(model => model.id === target.id)
|
||||
if (reshaped === undefined) throw new Error('the overridden model vanished from the route')
|
||||
|
||||
// The whole catalog still serves — that is the difference from `models`,
|
||||
// which replaces it.
|
||||
expect(models).toHaveLength(catalogSize)
|
||||
expect(reshaped.name).toBe('DeepSeek (proxied)')
|
||||
expect(getSupportedThinkingLevels(reshaped)).toEqual(['off', 'high'])
|
||||
// An override's cap is explicit configuration, so it becomes the request
|
||||
// default exactly as a models entry's would.
|
||||
expect(resolved.get('deepseek')?.configuredMaxTokens.get(target.id)).toBe(4096)
|
||||
// A sibling the overrides do not name is byte-identical to the catalog.
|
||||
const sibling = models.find(model => model.id !== target.id)
|
||||
expect(sibling?.maxTokens).toBe(getBuiltinModels('deepseek').find(model => model.id === sibling?.id)?.maxTokens)
|
||||
})
|
||||
|
||||
it('refuses every override that lands nowhere instead of skipping it', () => {
|
||||
expect(() => resolveProfiles({
|
||||
deepseek: { modelOverrides: { 'no-such-model': { name: 'ghost' } } },
|
||||
})).toThrow(/which the installed catalog does not describe/)
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm' }],
|
||||
modelOverrides: { m: { name: 'renamed' } },
|
||||
},
|
||||
})).toThrow(/a declared route spells every model out/)
|
||||
const declaredOnly = deepseekModel()
|
||||
expect(() => resolveProfiles({
|
||||
deepseek: {
|
||||
models: [{ id: declaredOnly.id }],
|
||||
modelOverrides: { [declaredOnly.id]: { name: 'renamed' } },
|
||||
},
|
||||
})).toThrow(/models already replaces the served catalog/)
|
||||
expect(() => resolveProfiles({
|
||||
deepseek: { modelOverrides: { '': { name: 'nameless' } } },
|
||||
})).toThrow(/empty model id/)
|
||||
// The dict key is the id; a value smuggling its own would quietly rename
|
||||
// the model it meant to customize. The schema passes unknown keys
|
||||
// through, so resolution is the boundary that refuses it — the variable
|
||||
// indirection mirrors that boundary by sidestepping the literal check.
|
||||
const smuggled = { name: 'x', id: 'other' }
|
||||
expect(() => resolveProfiles({
|
||||
deepseek: { modelOverrides: { [deepseekModel().id]: smuggled } },
|
||||
})).toThrow(/sets "id", which is the dict key/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('reasoning-dispatch compat switches', () => {
|
||||
/** The materialized models of one route, keyed by id. */
|
||||
function modelsOf(providers: Record<string, LlmPiAi.PiAiProviderProfile>, route: string): Map<string, Model<Api>> {
|
||||
const models = resolveProfiles(providers).get(route)?.piProvider.getModels() ?? []
|
||||
return new Map(models.map(model => [model.id, model]))
|
||||
}
|
||||
|
||||
it('applies route switches to every openai-completions model, entries winning per field', () => {
|
||||
const models = modelsOf({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
compat: { thinkingFormat: 'deepseek' },
|
||||
models: [
|
||||
{ id: 'dialect-default', reasoningEfforts: { off: null, high: 'high' } },
|
||||
{ id: 'dialect-odd', compat: { thinkingFormat: 'openai', supportsReasoningEffort: false } },
|
||||
],
|
||||
},
|
||||
}, 'acme-gateway')
|
||||
|
||||
expect(models.get('dialect-default')?.compat).toEqual({ thinkingFormat: 'deepseek' })
|
||||
expect(models.get('dialect-odd')?.compat).toEqual({ thinkingFormat: 'openai', supportsReasoningEffort: false })
|
||||
})
|
||||
|
||||
it('merges the switches over the catalog entry’s own compat instead of replacing it', () => {
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
const inherited = catalogModel.compat as OpenAICompletionsCompat
|
||||
expect(inherited.requiresReasoningContentOnAssistantMessages).toBe(true)
|
||||
|
||||
const models = modelsOf({
|
||||
deepseek: { models: [{ id: catalogModel.id, compat: { thinkingFormat: 'openai' } }] },
|
||||
}, 'deepseek')
|
||||
|
||||
// The one switched field changes; the catalog's other quirks survive,
|
||||
// because configuration has no way to restate them.
|
||||
expect(models.get(catalogModel.id)?.compat).toEqual({ ...inherited, thinkingFormat: 'openai' })
|
||||
})
|
||||
|
||||
it('skips models of other protocols on a mixed route instead of failing them', () => {
|
||||
// xai ships both completions and responses models, so a route-level switch
|
||||
// must land on the former without invalidating the latter.
|
||||
const catalog = getBuiltinModels('xai') as readonly Model<Api>[]
|
||||
const completions = catalog.find(model => model.api === 'openai-completions')
|
||||
const responses = catalog.find(model => model.api === 'openai-responses')
|
||||
if (completions === undefined || responses === undefined) throw new Error('xai no longer ships a mixed catalog')
|
||||
|
||||
const models = modelsOf({
|
||||
xai: {
|
||||
compat: { supportsReasoningEffort: false },
|
||||
models: [{ id: completions.id }, { id: responses.id }],
|
||||
},
|
||||
}, 'xai')
|
||||
|
||||
expect((models.get(completions.id)?.compat as OpenAICompletionsCompat).supportsReasoningEffort).toBe(false)
|
||||
expect(models.get(responses.id)?.compat).toEqual(responses.compat)
|
||||
})
|
||||
|
||||
it('rejects a model-level switch on a protocol that has no such field', () => {
|
||||
expect(() => resolveProfiles({
|
||||
anthropic: {
|
||||
models: [{ id: 'claude-sonnet-4-5', compat: { thinkingFormat: 'openai' } }],
|
||||
},
|
||||
})).toThrow(/exist only on openai-completions/)
|
||||
})
|
||||
|
||||
it('rejects route switches no model on the route can take', () => {
|
||||
expect(() => resolveProfiles({
|
||||
anthropic: { compat: { thinkingFormat: 'openai' } },
|
||||
})).toThrow(/no model on the route speaks openai-completions/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('resolution snapshots', () => {
|
||||
it('finishes an in-flight request under the configuration it started with', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
|
||||
32
packages/llm/llm-pi-ai/tests/config.spec.ts
Normal file
32
packages/llm/llm-pi-ai/tests/config.spec.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Config } from '../src/config.ts'
|
||||
|
||||
describe('reasoning schema boundary', () => {
|
||||
const configWith = (model: Record<string, unknown>): (() => unknown) =>
|
||||
() => Config({
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm', ...model }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
it('rejects a level pi-ai does not know at the write that produced it', () => {
|
||||
expect(configWith({ reasoningEfforts: { ultra: 'x' } })).toThrow(/"off"/)
|
||||
expect(configWith({ reasoningEfforts: { high: 42 } })).toThrow()
|
||||
})
|
||||
|
||||
it('keeps false distinguishable from an absent declaration', () => {
|
||||
type Materialized = { providers: Record<string, { models?: { reasoningEfforts?: unknown }[] }> }
|
||||
const withFalse = configWith({ reasoningEfforts: false })() as Materialized
|
||||
expect(withFalse.providers['acme-gateway']?.models?.[0]?.reasoningEfforts).toBe(false)
|
||||
const absent = configWith({})() as Materialized
|
||||
expect(absent.providers['acme-gateway']?.models?.[0]?.reasoningEfforts).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects a thinking format outside the offered set', () => {
|
||||
expect(configWith({ compat: { thinkingFormat: 'quantum' } })).toThrow(/expected/)
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: d15b2c996d6d47371a3d6c5542eb5c253029dbae
|
||||
README.zh.md: d965f15298f09ff9c2a953a69346c4e83136934b
|
||||
README.md: ddbc2ea482ca0848fb0ee0813839cf5ff1829bcc
|
||||
README.zh.md: 7a3b615d134e27d7c9892d6f411066d89938175b
|
||||
|
||||
@@ -97,5 +97,5 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
- **`APP_IDENTITY.url` names a repository that does not exist yet** — `FIXME`: creating the public `deepseek-ai/deepseek-harness-sdk` repo gates the first release.
|
||||
- **`APP_IDENTITY.url` names a repository that does not exist yet** — the public home must be reachable before release.
|
||||
- **`GenerateOptions.sessionId` is a locally-declared brand** — importing dsh-session's `SessionId` would cycle; a future ids-owning package would dissolve the workaround.
|
||||
|
||||
@@ -97,5 +97,5 @@
|
||||
- **`GenerateOptions` 采样只包含 `temperature`／`maxTokens`／`stop`**：没有 `tool_choice`、`top_p` 或 penalty 字段；有产生方落地时词汇才会增长（见 [已删除惰性旋钮](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)）。
|
||||
- **受产生方约束的变体在实际产生前不会加入**：`prefill`、每工具 `strict`、块 `cache` 提示与 `agent` 消息源变体因没有产生方而被剪除（见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)）。
|
||||
- **`BlockAssembler` 只处理核心块类型**：如果插件添加块类型的流从未由 `block-end` 关闭，`blocks()` 会抛出异常。
|
||||
- **`APP_IDENTITY.url` 指向一个尚不存在的仓库**：`FIXME`：创建公开 `deepseek-ai/deepseek-harness-sdk` 仓库是首次发布的前置条件。
|
||||
- **`APP_IDENTITY.url` 指向一个尚不存在的仓库**：该公开主页必须在首次发布前可访问。
|
||||
- **`GenerateOptions.sessionId` 是本地声明的品牌类型**：导入 dsh-session 的 `SessionId` 会产生循环；未来拥有 id 的包可以消除该权宜之计。
|
||||
|
||||
@@ -40,8 +40,7 @@ export interface AppIdentity {
|
||||
export const APP_IDENTITY: AppIdentity = {
|
||||
product: 'deepseek-harness',
|
||||
version,
|
||||
// FIXME: create the public deepseek-ai/deepseek-harness-sdk repository this
|
||||
// URL promises before the first release ships attribution pointing at it.
|
||||
// TODO(public-home): Ensure this public source repository exists before release.
|
||||
url: 'https://github.com/deepseek-ai/deepseek-harness-sdk',
|
||||
}
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ import type { ReasoningEffortId } from './brand.ts'
|
||||
/** Process-local identities of request objects assembled by dsh-agent-loop. */
|
||||
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
|
||||
|
||||
// TODO(call-config-shape): Revisit which fields are epoch-level for cache reuse
|
||||
// and where provider-specific request options belong.
|
||||
/**
|
||||
* Provider, model, reasoning effort, and sampling scalars of one conversation's
|
||||
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
|
||||
|
||||
Reference in New Issue
Block a user