From 8ccb17690579970ff2430448860f847799c13b78 Mon Sep 17 00:00:00 2001 From: Yichen Jiang Date: Sat, 8 Aug 2026 01:17:10 +0800 Subject: [PATCH] docs: per-model reasoning guide, config catalog, and the feature's Agent Note MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The user guide's model-catalog section teaches the three new knobs at task altitude — declare levels per model, pick the reasoning dialect, reshape catalog models with modelOverrides — with the settings.yaml example exercising all of them, plus an UNSUPPORTED_REASONING_EFFORT troubleshooting row. The generated plugin config catalog picks up the new Config fields, and the bilingual Agent Note records the decision, the alternatives considered, and the schemastery materialization constraint that chose false over {} as the disable spelling. --- ...per-model-reasoning-declarations.i18n.yaml | 6 ++ ...-pi-ai-per-model-reasoning-declarations.md | 33 ++++++++ ...-ai-per-model-reasoning-declarations.zh.md | 33 ++++++++ docs/config-catalog.md | 78 ++++++++++++++++++- docs/user/guide/providers.i18n.yaml | 4 +- docs/user/guide/providers.md | 32 +++++++- docs/user/guide/providers.zh.md | 32 +++++++- 7 files changed, 210 insertions(+), 8 deletions(-) create mode 100644 .agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.i18n.yaml create mode 100644 .agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md create mode 100644 .agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md diff --git a/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.i18n.yaml b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.i18n.yaml new file mode 100644 index 0000000000..3b448f4cf1 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md +2026-08-08-pi-ai-per-model-reasoning-declarations.md: 436b5f3f9f30c1bb1dc5816b12ce1596c5d01ec8 +2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md: 47b34dfd270f90fef2802a00e3632777d5636a73 diff --git a/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md new file mode 100644 index 0000000000..436b5f3f9f --- /dev/null +++ b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md @@ -0,0 +1,33 @@ +# Agent Note: Per-Model Reasoning Declarations in llm-pi-ai + +Status: implemented + +English | [中文](2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md) + +## Problem + +A hand-declared pi-ai route's models materialized with `reasoning: false`, so `getSupportedThinkingLevels` short-circuited to `["off"]`: the composer offered no effort picker for them, and the route-level `reasoning` default — the only reasoning knob a profile had — made every request to such a model fail with `UNSUPPORTED_REASONING_EFFORT` before network I/O. The same route-level knob was also the wrong altitude for catalog routes: one provider's models disagree about which levels they accept (deepseek ships `[off, high, max]` beside catalog models with `xhigh`), so a single per-route level could not be set without breaking part of the route, which is why the Models page stopped writing it entirely (#1860) and left `settings.yaml` with no way to align efforts per model. + +Two adjacent gaps compounded this. pi-ai decides the reasoning *wire dialect* (`compat.thinkingFormat`, `compat.supportsReasoningEffort`) by recognizing the endpoint URL, and a private gateway's URL says nothing — a DeepSeek-dialect gateway was spoken to in the OpenAI dialect with no configuration that could correct it. And the only way to touch one catalog model was the `models` list, which *replaces* the served catalog: narrowing `gpt-5`'s levels meant restating all thirty-eight openai models or silently dropping thirty-seven. + +## Decision + +`PiAiModelProfile` gains `reasoningEfforts`: **each key is a level selectors offer, its value the spelling dispatch sends on the wire**. The declaration translates to pi-ai's `Model.reasoning` + `thinkingLevelMap` with all seven levels decided explicitly — declared levels carry their wire value, undeclared levels are pinned `null` — so the profile author never needs pi-ai's asymmetric defaulting rule (absent means "supported" for the five base levels but "unsupported" for `xhigh`/`max`). `off` is the one three-state key: left out, thinking cannot be turned off; declared valueless, Off is offered and dispatch sends nothing (the `deepseek` dialect sends `thinking: {type: "disabled"}`); declared with a value, that value goes on the wire. `false` declares a non-reasoning model; an empty declaration is refused rather than guessed at. The spelling for "disable" is `false` rather than `{}` because schemastery materializes an absent dict as `{}` — only a `z.union([z.const(false), dict])` keeps absent, disabled, and declared distinguishable, and a bare `reasoningEfforts:` (YAML null) slips through that union unvalidated, so resolution refuses it explicitly. + +`compat.thinkingFormat` and `compat.supportsReasoningEffort` become configurable at two levels — route (its models' default) and model (winning per field) — resolving model → route → installed catalog entry → pi-ai's URL guess. They exist only on `openai-completions` (pi-ai types them nowhere else): a model-level switch on another protocol fails resolution, a route-level default skips such models, and a route with no completions model at all is refused. The two `chat-template` formats stay withheld for want of `chatTemplateKwargs`. Both enums are pinned to pi-ai's types through `Record` drift gates, so the pi-ai upgrade that adds a format (0.84 added `baseten`) fails compilation until the new member is classified. + +`modelOverrides` reshapes individual catalog models without replacing the served set: key = catalog model id, value = a `models` entry minus `id`, materialized by handing the override to the existing entry path so capacities, efforts, compat, and request-default semantics stay identical. Unlike Pi's own config layer, which ignores unknown ids, every override that lands nowhere is refused — beside a `models` list, on a hand-declared route, naming an unknown model, or smuggling an `id` in the value (the schema passes unknown keys through, and a smuggled id would quietly rename the model). + +## Alternatives considered + +- **Pass `reasoning` + `thinkingLevelMap` through verbatim** (pi-ai's own radius-config shape). Rejected by the user for operator confusion: the map's `null`-marks-unsupported convention plus the asymmetric absent-key rule mean the config's meaning depends on knowledge of pi-ai internals; the chosen shape makes the key set itself the offer. +- **A bare level list** (`reasoningEfforts: [off, high]`). Cannot express wire renames, and the catalog's own maps prove renames are real: 66 of 1230 installed map entries are non-identity (`off→none`, `minimal→low`, `low→LOW`, `high→default`). +- **`{}` as the disable spelling.** Unimplementable: schemastery materializes an absent dict as `{}`, so every model without the field would have been force-disabled. +- **Folding this into the route-level `reasoning` knob.** That knob is a *default selection*, not a capability set; it stays, and a declared model's efforts now bound what it can select. + +## Consequences + +- The composer's effort pane works for hand-declared models with zero UI change — `resolveModelInfo` reports declared levels through the same seam catalog metadata uses (pinned by the `declared-reasoning` web scenario). +- #1860's deferred gap — a route-level effort a model cannot take failing its requests — now has an operator remedy: align the model's `reasoningEfforts` or drop the route default. +- There is deliberately no spelling for returning one map key or compat field to "whatever the catalog said": the declaration is the whole offer, so keeping a catalog value means restating it. The README documents this. +- `verify-package-invariants` is untouched: the feature adds configuration resolution, no new events or mutable runtime relations. diff --git a/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md new file mode 100644 index 0000000000..47b34dfd27 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md @@ -0,0 +1,33 @@ +# Agent Note: llm-pi-ai 的按模型推理声明 + +Status: implemented + +[English](2026-08-08-pi-ai-per-model-reasoning-declarations.md) | 中文 + +## 问题 + +手工声明的 pi-ai 路由，其模型物化出来就带着 `reasoning: false`，于是 `getSupportedThinkingLevels` 短路成 `["off"]`：输入框不为它们提供档位选择器，而路由级的 `reasoning` 默认值——当时 profile 仅有的推理旋钮——让发往这类模型的每个请求都在网络 I/O 之前以 `UNSUPPORTED_REASONING_EFFORT` 失败。同一个路由级旋钮对 catalog 路由来说也放错了层级：同一提供方下各模型接受的档位并不一致（deepseek 自带 `[off, high, max]`，旁边就是带 `xhigh` 的 catalog 模型），单个路由级档位怎么设都会弄坏路由的一部分——这正是模型页彻底停写它的原因（#1860），而 `settings.yaml` 也因此没有了任何按模型对齐档位的办法。 + +两个相邻的缺口让问题雪上加霜。pi-ai 靠识别端点 URL 来决定推理的*协议方言*（`compat.thinkingFormat`、`compat.supportsReasoningEffort`），而私有网关的 URL 什么也说明不了——说 DeepSeek 方言的网关只会收到 OpenAI 方言的请求，且没有任何配置能更正它。另外，想动单个 catalog 模型，唯一的手段是 `models` 列表，而它会*替换*所服务的 catalog：收窄 `gpt-5` 的档位，意味着要么重述全部三十八个 openai 模型，要么静默丢掉三十七个。 + +## 决策 + +`PiAiModelProfile` 新增 `reasoningEfforts`：**每个键是选择器提供的一个档位，其值是分派在协议中发送的拼写**。该声明会转换为 pi-ai 的 `Model.reasoning` + `thinkingLevelMap`，七个档位全部显式决定——已声明的档位携带自己的协议值，未声明的档位一律固定为 `null`——因此 profile 作者永远不需要了解 pi-ai 那条不对称的默认规则（键缺席对五个基础档位意味着「支持」，对 `xhigh`/`max` 却意味着「不支持」）。`off` 是唯一的三态键：不写，思考就关不掉；声明而不给值，则提供 Off，分派什么也不发送（`deepseek` 方言发送 `thinking: {type: "disabled"}`）；声明并给值，该值就在协议中发送。`false` 声明一个不具备推理能力的模型；空声明会被拒绝，而不是去猜。「禁用」的拼写取 `false` 而非 `{}`，因为 schemastery 会把缺席的字典物化成 `{}`——只有 `z.union([z.const(false), dict])` 才能让缺席、禁用与已声明三态保持可区分；而裸写的 `reasoningEfforts:`（YAML null）会不经校验地从该 union 溜过去，因此解析对它显式拒绝。 + +`compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 变为两级可配置——路由级（作为其模型的默认值）与模型级（逐字段胜出）——解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测。两者只存在于 `openai-completions` 上（pi-ai 也只在这一协议上为它们建了类型）：在其他协议的模型上设模型级开关会使解析失败，路由级默认值会跳过这类模型，而完全没有 completions 模型的路由则被拒绝。两个 `chat-template` 格式因缺 `chatTemplateKwargs` 而继续保持不开放。两个枚举都经 `Record` 漂移门禁钉在 pi-ai 的类型上，因此新增格式的 pi-ai 升级（0.84 加入了 `baseten`）会编译失败，直到新成员被归类。 + +`modelOverrides` 就地重塑单个 catalog 模型而不替换所服务的集合：键 = catalog 模型 id，值 = 去掉 `id` 的 `models` 条目，物化时把覆盖交给既有的条目路径，因此容量、档位、compat 与请求默认值语义完全一致。与忽略未知 id 的 Pi 自有配置层不同，凡是落不到任何地方的覆盖都会被拒绝——与 `models` 列表并存、写在手工声明的路由上、点名未知模型，或在值里夹带 `id`（schema 会放行未知键，被夹带的 id 会悄悄把模型改名）。 + +## 曾考虑的替代方案 + +- **把 `reasoning` + `thinkingLevelMap` 原样透传**（pi-ai 自家 radius 配置的形状）。用户以运维人员困惑为由否决：map 用 `null` 标记「不支持」的约定，加上不对称的键缺席规则，意味着这份配置的含义取决于对 pi-ai 内部机制的了解；选定的形状则让键集合本身就是对外提供的全部。 +- **裸档位列表**（`reasoningEfforts: [off, high]`）。表达不了协议侧改名，而 catalog 自己的 map 证明改名真实存在：1230 条已安装 map 条目里有 66 条不是恒等映射（`off→none`、`minimal→low`、`low→LOW`、`high→default`）。 +- **用 `{}` 作为禁用拼写。** 无法实现：schemastery 会把缺席的字典物化成 `{}`，于是每个没写该字段的模型都会被强制禁用。 +- **把这件事并进路由级的 `reasoning` 旋钮。** 那个旋钮是*默认选择*，不是能力集合；它保留下来，而已声明模型的档位如今约束着它能选什么。 + +## 后果 + +- 输入框的档位面板对手工声明的模型直接可用，UI 零改动——`resolveModelInfo` 经 catalog 元数据所走的同一 seam 报告已声明档位（由 `declared-reasoning` web 场景钉住）。 +- #1860 暂缓的缺口——模型接不住的路由级档位会让发往它的请求失败——如今有了运维侧补救：对齐该模型的 `reasoningEfforts`，或去掉路由默认值。 +- 刻意不提供任何把单个 map 键或 compat 字段交还给「catalog 原本怎么说」的拼写：这份声明就是对外提供的全部，要保留某个 catalog 值就得重述它。README 记载了这一点。 +- `verify-package-invariants` 原封未动：该功能新增的是配置解析，没有新事件，也没有可变的运行时关系。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 21f38d18e2..ef3721a764 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -767,6 +767,22 @@ export interface PiAiProviderProfile { * unset fields from the installed model of the same id. */ models?: PiAiModelProfile[] + /** + * Installed-catalog customizations by model id: each entry reshapes that + * one model with the same fields a {@link models} entry takes, while the + * rest of the catalog keeps serving untouched. Only meaningful on a catalog + * route with no `models` list — `models` already replaces the catalog, so + * an override beside it, on a route the catalog does not ship, or naming a + * model the catalog does not describe is refused rather than skipped. + */ + modelOverrides?: Record + /** + * Reasoning-dispatch switches for every `openai-completions` model on this + * route; each model's own `compat` overrides per field. What neither sets + * keeps the installed catalog entry's value, then pi-ai's baseURL-derived + * detection. + */ + compat?: PiAiCompatProfile /** * Context capacity for a model this route lists that neither the entry nor * the installed catalog sizes (default 262,144). A guess by construction, so @@ -814,12 +830,70 @@ export interface PiAiModelProfile { * default on its own. */ maxTokens?: number + /** + * Selectable reasoning efforts. Absent inherits the installed catalog + * entry's capability (a hand-declared model has none and does not reason); + * `false` declares a non-reasoning model, which is how a profile strips + * reasoning from a catalog model its gateway cannot serve; a non-empty dict + * declares the offered levels and their wire spellings. + */ + reasoningEfforts?: false | PiAiReasoningEfforts + /** Reasoning-dispatch switches for this model, winning over the route's. */ + compat?: PiAiCompatProfile } + +/** + * Customization of one installed catalog model, keyed by its id in the + * route's `modelOverrides` dict — the same fields a `models` entry may set, + * with the id living in the key. Unlike a `models` list, overrides leave the + * rest of the catalog serving untouched, which is what makes "correct one + * model, keep the other thirty-seven" a three-line edit. + */ +export type PiAiModelOverride = Omit + +/** + * Reasoning-dispatch compatibility switches, set on the route (its models' + * default) or per model (winning over the route). Only the switches pi-ai's + * reasoning dispatch reads are offered; the rest of pi-ai's compat surface + * keeps its baseURL-derived auto-detection. pi-ai types both fields only on + * `OpenAICompletionsCompat` — the other wire protocols carry their reasoning + * shape in the protocol itself — so resolution rejects a model-level switch + * anywhere else, while a route-level default skips past models it cannot fit. + */ +export interface PiAiCompatProfile { + /** Reasoning parameter shape the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */ + thinkingFormat?: PiAiThinkingFormat + /** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */ + supportsReasoningEffort?: boolean +} + +/** + * Selectable reasoning efforts for one model: each key is a level the model + * offers (and selectors show), and its value is the wire spelling dispatch + * sends for it. `off` alone may leave its value empty — "supported, send + * nothing" — because for most providers not thinking is the parameter's + * absence; every other declared level must name a wire value. A level absent + * from the dict is not offered. + */ +export type PiAiReasoningEfforts = Partial> + +/** One reasoning-dispatch wire format a profile may name. */ +export type PiAiThinkingFormat = Exclude + +/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */ +type PiThinkingFormat = NonNullable + +/** + * pi-ai thinking formats a profile cannot name: both drive the request through + * `chatTemplateKwargs`, which this configuration does not expose, so offering + * them would hand back a format with nothing to say. + */ +type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template' ``` -Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`) +Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`) -Source: [`packages/llm/llm-pi-ai/src/config.ts:126`](../packages/llm/llm-pi-ai/src/config.ts) +Source: [`packages/llm/llm-pi-ai/src/config.ts:148`](../packages/llm/llm-pi-ai/src/config.ts) ## `@deepseek-ai/dsh-llm-replay` diff --git a/docs/user/guide/providers.i18n.yaml b/docs/user/guide/providers.i18n.yaml index 82c2f2781a..a24c06b8c5 100644 --- a/docs/user/guide/providers.i18n.yaml +++ b/docs/user/guide/providers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/user/guide/providers.md -providers.md: 5234f5bb03755c11652eb23f3c5d677fa3cddb40 -providers.zh.md: a54819cab8524a6007c335ad70cecd6516bba25b +providers.md: 6f44daf73037f811164f5b22b14a9c39b71d6b1a +providers.zh.md: 6c75d70d485f55ff230f557247ed6be597a8785e diff --git a/docs/user/guide/providers.md b/docs/user/guide/providers.md index 5234f5bb03..6f44daf730 100644 --- a/docs/user/guide/providers.md +++ b/docs/user/guide/providers.md @@ -59,6 +59,16 @@ llm-pi-ai: - id: claude-sonnet-4-5 contextWindow: 200000 + # Catalog route with one model reshaped in place; the rest of the catalog + # keeps serving (a models list would replace it instead). + deepseek: + apiKeyEnv: DEEPSEEK_API_KEY + modelOverrides: + deepseek-v4-pro: + reasoningEfforts: + off: + high: high + # Hand-declared route: pi-ai ships nothing under this key, so the profile # supplies the whole provider. acme-gateway: @@ -66,11 +76,22 @@ llm-pi-ai: apiKeyEnv: ACME_GATEWAY_API_KEY api: openai-completions baseURL: https://gateway.acme.example/v1 + # Reasoning dialect for an endpoint whose URL pi-ai cannot recognize. + compat: + thinkingFormat: deepseek models: - id: acme-large name: Acme Large contextWindow: 65536 maxTokens: 4096 + - id: acme-think + name: Acme Think + # key = level offered in the picker, value = what goes on the wire; + # only off may leave the value empty (supported, send nothing). + reasoningEfforts: + off: + high: high + max: ultra ``` A settings section merges over the matching `cordis.yml` configuration **per provider**, so you can override one field of one route and leave the rest as the composition set them. @@ -79,9 +100,15 @@ A profile the adapter could not serve is refused **where it is written**: a hand ## The model catalog -A profile's `models` list *replaces* that route's installed catalog rather than extending it; omitting it or leaving it empty serves the catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a route to two models, correcting one capacity, or adding a model newer than the installed catalog are each a one-line edit. +A profile's `models` list *replaces* that route's installed catalog rather than extending it; omitting it or leaving it empty serves the catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a route to two models, correcting one capacity, or adding a model newer than the installed catalog are each a one-line edit — but once you declare the list, every model the route should keep serving must appear in it, an entry of nothing but `id` being enough. -Only the four fields the harness consumes are configurable: `id`, `name`, `contextWindow`, and `maxTokens`. Pricing and input modalities have no consumer, and reasoning is not per-model configurable at all — it rides the installed catalog entry. +Reshaping a few catalog models while keeping the rest is `modelOverrides`' job: it is keyed by catalog model id, takes the same fields a `models` entry does, and leaves the rest of the catalog serving untouched. An override naming a model the catalog does not describe — or set beside a `models` list, or on a custom provider — is refused rather than silently skipped. + +The configurable model fields are `id`, `name`, `contextWindow`, `maxTokens`, `reasoningEfforts`, and `compat`. Pricing and input modalities have no consumer and ride the installed entry. + +**Declare reasoning levels per model.** `reasoningEfforts` lists the levels a model offers: each key appears in the composer's effort picker, and its value is what dispatch sends on the wire — `high: high` passes the name through, `max: ultra` renames it for a gateway with its own vocabulary. A level you leave out is not offered. `off` is special: declared without a value, Off appears in the picker and selecting it sends nothing; left out entirely, the model cannot stop thinking. `reasoningEfforts: false` declares a non-reasoning model, which is also how you strip reasoning from a catalog model your gateway cannot serve. Without this field a custom model does not reason and a catalog model keeps its catalog levels. + +**Pick the reasoning dialect.** How a level travels — plain `reasoning_effort`, DeepSeek's `thinking: {type}` plus effort, and so on — is normally guessed from the endpoint URL, and a private gateway's URL says nothing, so a DeepSeek-style gateway would be spoken to in the OpenAI dialect. `compat.thinkingFormat` sets the dialect explicitly, and `compat.supportsReasoningEffort: false` holds the parameter back from an endpoint that rejects it; both work on the route (its models' default) or per model, for `openai-completions` routes only. A model neither the entry nor the catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768). Both are guesses by construction, which is why they are route fields: a deployment whose gateway serves smaller models corrects them once. @@ -114,6 +141,7 @@ If the provider a saved default names is later removed, the composer says **Sele - **`MISSING_CREDENTIAL`** — the variable the profile's `apiKeyEnv` names holds no value. Store the key once through the Models page, or export the variable. - **`UNKNOWN_MODEL`** — the requested model is not in the route's configured catalog. Add it to `models`, or use an id the catalog already carries. +- **`UNSUPPORTED_REASONING_EFFORT`** — the request asked the model for a level it does not offer. Pick a level the composer lists for that model, or declare the missing one in the model's `reasoningEfforts`. - **`settings-rejected`** — the written profile cannot be served, and the message names the route and model. For a hand-declared route, check that `api`, `baseURL`, and `models` are all present. - **Fetching available models answers 401** — the endpoint refused the interrogation. Check the key; if the base URL points at an Anthropic-style gateway, note that the interrogation reads only the OpenAI-compatible `GET /models`, so enter the models by hand instead. diff --git a/docs/user/guide/providers.zh.md b/docs/user/guide/providers.zh.md index a54819cab8..6c75d70d48 100644 --- a/docs/user/guide/providers.zh.md +++ b/docs/user/guide/providers.zh.md @@ -59,6 +59,16 @@ llm-pi-ai: - id: claude-sonnet-4-5 contextWindow: 200000 + # Catalog route with one model reshaped in place; the rest of the catalog + # keeps serving (a models list would replace it instead). + deepseek: + apiKeyEnv: DEEPSEEK_API_KEY + modelOverrides: + deepseek-v4-pro: + reasoningEfforts: + off: + high: high + # Hand-declared route: pi-ai ships nothing under this key, so the profile # supplies the whole provider. acme-gateway: @@ -66,11 +76,22 @@ llm-pi-ai: apiKeyEnv: ACME_GATEWAY_API_KEY api: openai-completions baseURL: https://gateway.acme.example/v1 + # Reasoning dialect for an endpoint whose URL pi-ai cannot recognize. + compat: + thinkingFormat: deepseek models: - id: acme-large name: Acme Large contextWindow: 65536 maxTokens: 4096 + - id: acme-think + name: Acme Think + # key = level offered in the picker, value = what goes on the wire; + # only off may leave the value empty (supported, send nothing). + reasoningEfforts: + off: + high: high + max: ultra ``` settings 段落**逐个提供方**地盖在 `cordis.yml` 的同名配置之上，所以你可以只覆盖某个路由的一个字段，其余保持组合里的样子。 @@ -79,9 +100,15 @@ settings 段落**逐个提供方**地盖在 `cordis.yml` 的同名配置之上 ## 模型目录 -`models` 是**替换**该路由的内置目录，不是往里追加；省略或留空则原样使用内置目录。每个条目会从同 `id` 的内置模型继承自己没写的字段，所以「收窄到两个模型」「更正一个容量」「加一个比内置目录更新的模型」都是一行编辑。 +`models` 是**替换**该路由的内置目录，不是往里追加；省略或留空则原样使用内置目录。每个条目会从同 `id` 的内置模型继承自己没写的字段，所以「收窄到两个模型」「更正一个容量」「加一个比内置目录更新的模型」都是一行编辑——但一旦声明了这份列表，该路由要继续服务的每个模型就都必须出现在其中，条目哪怕只写一个 `id` 也足够。 -可配置的只有 harness 会消费的四个字段：`id`、`name`、`contextWindow`、`maxTokens`。定价与输入模态没有消费方，推理能力也不按模型配置——它随内置目录条目走。 +就地重塑目录里的几个模型、保留其余，归 `modelOverrides` 管：它以目录模型 id 为键，接受与 `models` 条目相同的字段，目录的其余部分原样继续服务。覆盖若点名了目录没有描述的模型，或与 `models` 列表并存，或写在自定义提供方上，都会被拒绝，而不是被静默跳过。 + +可配置的模型字段是 `id`、`name`、`contextWindow`、`maxTokens`、`reasoningEfforts` 与 `compat`。定价与输入模态没有消费方，随内置目录条目走。 + +**按模型声明推理档位。** `reasoningEfforts` 列出模型提供的档位：每个键都会出现在输入框的档位选择器里，其值是分派在协议中实际发送的内容——`high: high` 原样透传名称，`max: ultra` 则为使用自有词汇的网关改名。没写的档位不会被提供。`off` 比较特殊：声明而不给值，选择器里会出现 Off，选中它时什么也不发送；完全不写，模型就无法停止思考。`reasoningEfforts: false` 声明一个不具备推理能力的模型，这也是从网关服务不了的目录模型上剥除推理的办法。不写这个字段，自定义模型不推理，目录模型保留目录给出的档位。 + +**选定推理方言。** 档位如何在协议中传输——单独一个 `reasoning_effort`、DeepSeek 的 `thinking: {type}` 加档位，诸如此类——通常靠端点 URL 来猜，而私有网关的 URL 什么也说明不了，于是 DeepSeek 风格的网关只会收到 OpenAI 方言的请求。`compat.thinkingFormat` 用来显式指定方言，`compat.supportsReasoningEffort: false` 则让该参数不再发给拒绝它的端点；两者既可设在路由上（作为其模型的默认值），也可按模型设置，且仅适用于 `openai-completions` 路由。 两处容量都没给出的模型，取路由级兜底 `defaultContextWindow`（262144）与 `defaultMaxTokens`（32768）。这两个数按定义就是猜测，所以它们是路由字段：网关服务的模型更小时改一次即可。 @@ -114,6 +141,7 @@ api-gateway: - **`MISSING_CREDENTIAL`** — profile 里的 `apiKeyEnv` 指向的变量没有值。用模型页存一次密钥，或导出该环境变量。 - **`UNKNOWN_MODEL`** — 请求的模型不在该路由配置的目录里。把它加进 `models`，或改用目录里已有的 id。 +- **`UNSUPPORTED_REASONING_EFFORT`** — 请求向模型要了一个它不提供的档位。从输入框为该模型列出的档位里挑一个，或把缺的那个声明进该模型的 `reasoningEfforts`。 - **`settings-rejected`** — 写入的 profile 服务不了，错误信息会点名具体的路由和模型。手工声明的路由检查 `api`、`baseURL`、`models` 是否齐全。 - **获取可用模型返回 401** — 端点拒绝了这次探测。检查密钥；若地址指向的是 Anthropic 风格网关，注意探测只读 OpenAI 兼容的 `GET /models`，此时手工填写模型即可。