feat(llm): declare pi-ai providers instead of looking them up
A pi-ai route had to name an installed catalog provider, served that catalog's models verbatim, and could override only the endpoint. An OpenAI-compatible gateway, a self-hosted server, or a model newer than the pinned pi-ai release was therefore unreachable, and a stale context window could not be corrected without upgrading the package. A route is now a declaration whose defaults come from the installed catalog. `catalog.ts` merges that catalog under the profile's own model entries, `provider.ts` builds the pi-ai Provider (reusing the catalog provider when the route keeps its protocol, so implementations this package cannot reconstruct keep working), and the adapter serves every operation from one `createModels()` collection. That also retires the `@earendil-works/pi-ai/compat` import, which pi-ai documents as a temporary entry point it deletes with its ModelManager migration. Credentials stay on the harness seam: the resolved key rides the request as pi-ai's highest-priority auth override, so `Models` holds no credential store and a named-but-missing reference still fails loud instead of falling back to an unrelated ambient key. A model's configured maxTokens now reaches the seam as defaultMaxTokens.
This commit is contained in:
Yichen Jiang
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-03-pi-ai-declared-provider-catalog.md
|
||||
2026-08-03-pi-ai-declared-provider-catalog.md: ef695f6e4c79725400ee39a2d40ead27d6559a8d
|
||||
2026-08-03-pi-ai-declared-provider-catalog.zh.md: 13cfed574f646de07228da80b3e518e31fd1f50b
|
||||
@@ -0,0 +1,47 @@
|
||||
# Agent Note: pi-ai routes are declared providers, not catalog lookups
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-03-pi-ai-declared-provider-catalog.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-llm-pi-ai` treated the pi-ai package's generated catalog as the boundary of what could be configured. A route key had to name an installed provider (`resolveProfiles` rejected anything else), model listing returned `getBuiltinModels(provider)` verbatim, and request-time model resolution looked the id up in that same catalog and overrode only `baseURL`. Three consequences followed, and all three were dead ends rather than gaps: an OpenAI-compatible gateway, a self-hosted server, or a provider newer than the installed catalog could not be configured at all; a model the catalog had not caught up with failed with `UNKNOWN_MODEL` even against a correct endpoint; and a model's context window and output cap were whatever the pinned pi-ai release said, so a deployment could neither correct a stale value nor supply one for a model pi-ai had never described. Upgrading the package was the only way to move any of it.
|
||||
|
||||
The adapter also streamed through `streamSimple` from `@earendil-works/pi-ai/compat`, an entry point whose own module documentation declares it a temporary compatibility surface — its catalog reads are `@deprecated`, and it is deleted when pi-ai finishes its `ModelManager` migration. The three configuration limits and the deprecated dependency have the same fix, because pi-ai's supported runtime (`createModels()` / `createProvider()`) is built around a provider being *declared* rather than looked up.
|
||||
|
||||
## Decision
|
||||
|
||||
A provider route is a **declaration**, and the installed catalog is its default. `resolveProfiles` no longer checks route keys against `getBuiltinProviders()`. Instead each route resolves to a materialized model list plus the pi-ai `Provider` that serves it:
|
||||
|
||||
- `catalog.ts` merges the installed catalog under the profile's own entries. A profile's `models` list *replaces* the route's catalog (an absent or empty list serves it unchanged), and each entry defaults its unset fields from the installed model of the same `id`. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, `maxTokens`, `reasoning`. Pricing and input modalities are absent from the surface because nothing reads them: `replay.ts` zeroes pi-ai's cost metadata and `context.ts` keeps only text blocks. Reasoning-level spellings, OpenAI-compatibility quirks, and model headers ride the installed entry, because restating them in configuration could not be validated.
|
||||
- `provider.ts` builds the route's `Provider`. A catalog route that keeps its catalog protocol **reuses** the installed provider with `getModels()` replaced; every other route is built by `createProvider()` over a protocol table whose entries are the same `@earendil-works/pi-ai/api/*.lazy` factories pi-ai's own provider factories use.
|
||||
- `adapter.ts` owns one `createModels()` collection, re-synced when resolution produces a new profile map, and serves `listModels`, `resolveModel`, and `stream` from it. A model's configured `maxTokens` becomes the seam's `defaultMaxTokens`, so a request naming no output cap now carries the configured one.
|
||||
|
||||
Resolution fails loud and names the route and model at fault: a model the catalog does not describe needs an explicit `contextWindow` and `maxTokens`; a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list. Because the built `Provider` is part of the resolution result, a protocol or model error keeps the last good route set serving, exactly as a bad settings snapshot already did.
|
||||
|
||||
The configurable-provider directory is now the installed catalog **joined with** every route the current profiles declare, re-registered when that set changes. Without the join a hand-declared route would have no settings address and no configuration surface could show or edit it.
|
||||
|
||||
### Credentials stay outside pi-ai
|
||||
|
||||
pi-ai's `Models` carries its own credential concept — a `CredentialStore` keyed by provider id, with `envApiKeyAuth` resolving `credential.key ?? env(VAR)`. Adopting it would have created a second credential source of truth beside `ctx.credentials` and, worse, reintroduced the ambient fallback the harness deliberately forbids: a named-but-missing `apiKeyEnv` must fail with `MISSING_CREDENTIAL` rather than authenticate with whatever unrelated key the environment holds.
|
||||
|
||||
`ModelsImpl.applyAuth` treats `options.apiKey` as the highest-priority auth override, short-circuiting resolution entirely. The harness therefore resolves the route's key through its own seam, as before, and passes the result as the request's `apiKey`; the collection is constructed with no credential store. A catalog route reuses the installed provider's `auth`, which preserves its provider-native ambient discovery for a profile naming no credential. A hand-declared route gets a harness-owned `ApiKeyAuth` that reports configured-but-keyless rather than unconfigured, leaving the requirement to the protocol — which is where it lives: pi-ai's OpenAI-compatible implementation still demands a key or an `Authorization` header, and says so itself.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep `createProvider()` but skip the `Models` collection**, streaming through `provider.streamSimple(model, ctx, {apiKey})`. Smallest diff and the credential path is untouched, but `createProvider`'s `auth` is a required field that this path never invokes — a required-by-signature implementation with no caller. It also leaves `refreshModels` needing a hand-built `RefreshModelsContext`, and keeps the adapter off the runtime pi-ai actually supports.
|
||||
- **Reuse the installed provider for catalog routes and `createProvider()` only for declared ones**, with no shared resolution. Zero risk to catalog behavior, but catalog materialization, endpoint override, and per-model configuration would each exist twice, and a catalog route that repoints its protocol would have to jump paths mid-resolution. The chosen split confines the asymmetry to provider construction, where it is forced by pi-ai not exposing a built provider's API implementations.
|
||||
- **Rebuild every route through `createProvider()`**, including catalog ones. Fully symmetric, but a built `Provider` does not expose its `api`, so the protocol table would become the ceiling on which providers work — Bedrock loads its Smithy module through a separate entry point and would silently stop working.
|
||||
- **Expose pi-ai's whole `Model` shape** (cost, input modalities, `thinkingLevelMap`, `compat`). Maximum configurability, but no current consumer reads those fields, so a configured price or modality would change nothing while reading as supported.
|
||||
- **A runtime dynamic catalog** — `fetchModels` plus `ModelsStore`, refreshed in the background. Rejected for this change: it makes the model list external mutable state needing cache, invalidation, and an offline path, and the product need is a one-shot discovery action whose result the user adopts into `settings.yaml`. That action belongs to the configuration surface and is deferred with it; `settings.yaml` stays the single source of truth for what a route serves.
|
||||
|
||||
## Consequences
|
||||
|
||||
Configuring a provider no longer depends on a pi-ai release. A gateway, a self-hosted server, or a model newer than the pinned catalog is a `settings.yaml` edit, and a stale context window can be corrected in place. The deprecated `/compat` import is gone, so pi-ai deleting it is no longer a breaking event. `defaultMaxTokens` now flows from configuration, closing the case where a request carried no output cap at all.
|
||||
|
||||
What it costs: `settings.yaml` grows for a declared route, because a model the catalog cannot default must state its own capacity. `api` applies to a whole route, so a mixed-protocol catalog route cannot host a model of the other protocol — splitting it across two route keys is the workaround. Nothing queries a provider's `/models`, so a model list is only as current as its last edit. Reported error shape shifts in one case: a route whose auth resolves to nothing now surfaces pi-ai's own diagnostic as an error `finish` chunk before any network call, where the previous adapter sent a keyless request and surfaced the provider's 401.
|
||||
|
||||
## Testing
|
||||
|
||||
`tests/catalog.spec.ts` covers the contract end to end against local mock servers: a hand-declared route streaming to its own endpoint with its own credential, its appearance in the configurable-provider directory, per-model overrides defaulting from the installed catalog, a model added to a catalog route, protocol repointing with and without an endpoint override, catalog-only metadata surviving an override, the keyless posture and its `Authorization`-header workaround, and every resolution failure that names a route or model. `tests/sdk-options.spec.ts` re-targets the SDK boundary from the removed `/compat` import to the protocol table's lazy api module, which also pins that a setup failure arrives as a terminal error chunk rather than a throw. The twin's [design-verification role](2026-06-13-twin-llm-adapters.md) is unchanged.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Agent Note: pi-ai 路由是被声明的提供方，而不是 catalog 查表
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-03-pi-ai-declared-provider-catalog.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-llm-pi-ai` 把 pi-ai 包生成的 catalog 当成了可配置范围的边界。路由键必须点名一个已安装提供方（`resolveProfiles` 拒绝其余一切），模型列举原样返回 `getBuiltinModels(provider)`，请求期的模型解析又在同一份 catalog 里查这个 id、且只覆盖 `baseURL`。由此产生三个后果，而且三个都是死路而非缺口：OpenAI 兼容网关、自建服务，或比已安装 catalog 更新的提供方，根本无法配置；catalog 尚未跟上的模型即便端点正确也会以 `UNKNOWN_MODEL` 失败；模型的上下文窗口与输出上限完全由锁定的 pi-ai 版本决定，部署既无法更正过期值，也无法为 pi-ai 从未描述过的模型补上。要动其中任何一条，只能升级依赖。
|
||||
|
||||
适配器还经 `@earendil-works/pi-ai/compat` 的 `streamSimple` 发起流式请求，而该入口自己的模块文档声明它是临时兼容面——其 catalog 读取标了 `@deprecated`，并会在 pi-ai 完成 `ModelManager` 迁移时被删除。这三条配置限制与这个废弃依赖的解法是同一个，因为 pi-ai 受支持的运行时（`createModels()` / `createProvider()`）正是围绕「提供方是被*声明*出来的，而非查出来的」建立的。
|
||||
|
||||
## Decision
|
||||
|
||||
提供方路由是一份**声明**，已安装 catalog 是它的默认值。`resolveProfiles` 不再拿路由键去核对 `getBuiltinProviders()`，而是把每条路由解析成一份物化模型列表，外加服务它的 pi-ai `Provider`：
|
||||
|
||||
- `catalog.ts` 把已安装 catalog 合并到 profile 自身条目之下。profile 的 `models` 列表*替换*该路由的 catalog（列表缺席或为空则原样服务），每个条目从同 `id` 的已安装模型继承自身未设置的字段。只有 harness 会消费的字段可配置——`id`、`name`、`contextWindow`、`maxTokens`、`reasoning`。定价与输入模态不出现在配置面，因为没有任何读取方：`replay.ts` 把 pi-ai 的成本元数据清零，`context.ts` 只保留文本块。思考级别拼写、OpenAI 兼容性怪癖与模型标头沿用已安装条目，因为在配置里重述它们无法被校验。
|
||||
- `provider.ts` 构造路由的 `Provider`。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方，只替换 `getModels()`；其余路由都由 `createProvider()` 基于一张协议表构造，表中条目正是 pi-ai 自己的提供方工厂所用的 `@earendil-works/pi-ai/api/*.lazy` factory。
|
||||
- `adapter.ts` 持有一个 `createModels()` 集合，在解析产出新的 profile 映射时重新同步，并由它服务 `listModels`、`resolveModel` 与 `stream`。模型已配置的 `maxTokens` 会成为 seam 的 `defaultMaxTokens`，因此未点名输出上限的请求现在会携带已配置的那一个。
|
||||
|
||||
解析失败得响亮，并点名出问题的路由与模型：catalog 未描述的模型需要显式的 `contextWindow` 与 `maxTokens`；catalog 未提供的路由需要 `api`、`baseURL` 和非空的 `models` 列表。由于构造出的 `Provider` 是解析结果的一部分，协议或模型出错时最后可用的路由集合会继续服务——与此前坏的 settings 快照的行为完全一致。
|
||||
|
||||
可配置提供方目录现在是已安装 catalog **与**当前 profile 声明的每条路由的并集，并在该集合变化时重新登记。没有这个并集，手工声明的路由就没有 settings 地址，任何配置界面都无法展示或编辑它。
|
||||
|
||||
### 凭据留在 pi-ai 之外
|
||||
|
||||
pi-ai 的 `Models` 自带一套凭据概念——按提供方 id 索引的 `CredentialStore`，配合 `envApiKeyAuth` 解析 `credential.key ?? env(VAR)`。采用它会在 `ctx.credentials` 之外制造第二个凭据事实源，更糟的是会把 harness 明确禁止的环境回落重新引进来：点名了却取不到的 `apiKeyEnv` 必须以 `MISSING_CREDENTIAL` 失败，而不是用环境里恰好持有的某个无关密钥完成认证。
|
||||
|
||||
`ModelsImpl.applyAuth` 把 `options.apiKey` 视为优先级最高的 auth 覆盖，会整条短路掉解析。因此 harness 一如既往经自身 seam 解析路由密钥，并把结果作为请求的 `apiKey` 传入；该集合构造时不带任何凭据存储。catalog 路由复用已安装提供方的 `auth`，从而为不点名凭据的 profile 保住其提供方原生环境发现。手工声明的路由则获得一个 harness 自有的 `ApiKeyAuth`，它报告「已配置但无密钥」而非「未配置」，把该要求留给协议——那才是它真正所在的位置：pi-ai 的 OpenAI 兼容实现仍要求密钥或 `Authorization` 标头，并且会自己说出来。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **保留 `createProvider()` 但不建 `Models` 集合**，改由 `provider.streamSimple(model, ctx, {apiKey})` 发起。改动最小且凭据路径原封不动，但 `createProvider` 的 `auth` 是必填字段，这条路上它永远不会被调用——一份因签名而必填、却没有调用方的实现。它还让 `refreshModels` 需要手工构造 `RefreshModelsContext`，并使适配器始终不在 pi-ai 真正支持的运行时上。
|
||||
- **catalog 路由复用已安装提供方，只有声明式路由走 `createProvider()`**，且两者不共享解析。对 catalog 行为零风险，但 catalog 物化、端点覆盖与每模型配置这三件事都要各写两遍，而改指协议的 catalog 路由还得在解析中途跳到另一条路径。已采纳的拆法把不对称收敛在提供方构造这一处——那里的不对称是 pi-ai 不暴露已构造提供方的 API 实现所强加的。
|
||||
- **让每条路由都经 `createProvider()` 重建**，包括 catalog 路由。完全对称，但已构造的 `Provider` 不暴露自己的 `api`，于是协议表会成为「哪些提供方能用」的天花板——Bedrock 经独立入口加载其 Smithy 模块，会因此静默失效。
|
||||
- **完整暴露 pi-ai 的 `Model` 形状**（成本、输入模态、`thinkingLevelMap`、`compat`）。可配置性最大，但这些字段当前没有任何读取方，因此配了价格或模态什么也不会改变，却看起来像是受支持的。
|
||||
- **运行时动态 catalog**——`fetchModels` 加 `ModelsStore`，后台刷新。本次变更拒绝：它把模型列表变成需要缓存、失效与离线路径的外部可变状态，而产品需求是一次性的发现动作、其结果由用户采纳进 `settings.yaml`。该动作属于配置界面，与之一并暂缓；`settings.yaml` 始终是「路由服务什么」的唯一事实源。
|
||||
|
||||
## Consequences
|
||||
|
||||
配置一个提供方不再取决于 pi-ai 的发布节奏。网关、自建服务，或比锁定 catalog 更新的模型，都是一次 `settings.yaml` 编辑，过期的上下文窗口也能就地更正。废弃的 `/compat` 导入已经消失，因此 pi-ai 删除它不再是破坏性事件。`defaultMaxTokens` 现在自配置流出，堵上了「请求完全不带输出上限」的情形。
|
||||
|
||||
代价是：声明式路由会让 `settings.yaml` 变长，因为 catalog 无法默认的模型必须自报容量。`api` 作用于整条路由，因此混合协议的 catalog 路由无法承载另一种协议的模型——把它拆成两个路由键是变通办法。没有任何环节查询提供方的 `/models`，因此模型列表的新鲜度只到最近一次编辑为止。有一种情形下报错形状发生变化：auth 解析不出任何值的路由，现在会在任何网络调用之前把 pi-ai 自己的诊断作为错误 `finish` 分片呈现，而此前的适配器会发出无密钥请求并呈现提供方的 401。
|
||||
|
||||
## Testing
|
||||
|
||||
`tests/catalog.spec.ts` 针对本地 mock 服务器端到端覆盖该契约：手工声明的路由带着自己的凭据流向自己的端点、它在可配置提供方目录中的出现、每模型覆盖从已安装 catalog 继承默认值、向 catalog 路由添加模型、带与不带端点覆盖的协议改指、catalog 独有元数据在覆盖后存活、无密钥姿态及其 `Authorization` 标头变通，以及每一种点名路由或模型的解析失败。`tests/sdk-options.spec.ts` 把 SDK 边界从已移除的 `/compat` 导入改指到协议表的 lazy api 模块，同时钉住「setup 失败以终止性错误分片而非抛出的形式抵达」。twin 的[设计验证角色](2026-06-13-twin-llm-adapters.md)不变。
|
||||
@@ -702,8 +702,22 @@ export interface PiAiProviderProfile {
|
||||
apiKey?: string
|
||||
/** Credential reference (environment-variable name) resolved per request through `ctx.credentials`. */
|
||||
apiKeyEnv?: string
|
||||
/** Override the selected catalog model's endpoint without changing its protocol metadata. */
|
||||
/** Name shown by configuration surfaces; defaults to the route key. */
|
||||
displayName?: string
|
||||
/**
|
||||
* Wire protocol every model on this route speaks. Omission keeps each
|
||||
* installed catalog model's own protocol, which is why a catalog route needs
|
||||
* no protocol at all; a route the catalog does not ship must name one.
|
||||
*/
|
||||
api?: string
|
||||
/** Endpoint for this route's models; defaults to the installed catalog's endpoint. */
|
||||
baseURL?: string
|
||||
/**
|
||||
* This route's model catalog. Omission serves the installed catalog for the
|
||||
* route unchanged; an explicit list replaces it, each entry defaulting its
|
||||
* unset fields from the installed model of the same id.
|
||||
*/
|
||||
models?: PiAiModelProfile[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -723,11 +737,25 @@ export interface PiAiProviderProfile {
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** One configured model entry: an id plus the catalog fields it overrides. */
|
||||
export interface PiAiModelProfile {
|
||||
/** Model id sent to the provider and accepted by {@link GenerateOptions.model}. */
|
||||
id: string
|
||||
/** Display name for selectors; defaults to the catalog name, then the id. */
|
||||
name?: string
|
||||
/** Maximum combined request and response context in tokens. */
|
||||
contextWindow?: number
|
||||
/** Per-request output cap materialized when a caller omits one. */
|
||||
maxTokens?: number
|
||||
/** Whether the model exposes reasoning; defaults to the catalog capability. */
|
||||
reasoning?: boolean
|
||||
}
|
||||
```
|
||||
|
||||
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:62`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:98`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-replay`
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 75b2136315aed758f18f7fe82afcd4903f4a7b98
|
||||
README.zh.md: ea67250549f1d23d48455fd185283b00183dd538
|
||||
README.md: e597eedeb4d6e0ebf402b5547f71c9aff370d3dd
|
||||
README.zh.md: e28105f1253c138b9bb0baf5d00e0c7eba0d7b52
|
||||
|
||||
@@ -2,19 +2,20 @@
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
Generic multi-provider adapter for the harness LLM seam backed by [`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai). One plugin instance owns a dict of provider profiles keyed by route; every request selects a profile with `GenerateOptions.provider` and resolves `GenerateOptions.model` dynamically from pi-ai's installed catalog.
|
||||
Generic multi-provider adapter for the harness LLM seam backed by [`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai). One plugin instance owns a dict of provider profiles keyed by route; every request selects a profile with `GenerateOptions.provider` and resolves `GenerateOptions.model` against that route's configured catalog. A route naming an installed pi-ai provider inherits its endpoint, wire protocol, and model catalog as defaults and overrides them field by field; a route pi-ai does not ship is declared outright, so an OpenAI-compatible gateway, a self-hosted server, or a provider newer than the installed catalog is configuration rather than a code change.
|
||||
|
||||
The package root exposes the Cordis plugin contract and `PiAiAdapter`; profile resolution, model construction, replay conversion, and stream conversion remain package-internal.
|
||||
The package root exposes the Cordis plugin contract, `PiAiAdapter`, and `supportedProtocols()`; profile resolution, catalog materialization, provider construction, replay conversion, and stream conversion remain package-internal.
|
||||
|
||||
## Config
|
||||
|
||||
Configure credentials and deployment-specific transport settings per provider, keyed by the provider route itself. Prefer `apiKeyEnv` — a credential *reference* resolved per request — over a literal `apiKey`, so no secret enters this file. Omitting **both** is what delegates authentication to pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. `baseURL` overrides only the endpoint of the selected catalog model, preserving its API family and compatibility metadata, so private proxies such as `https://proxy.example.com:8443` remain supported.
|
||||
Configure credentials, the model catalog, and deployment-specific transport settings per provider, keyed by the provider route itself. Prefer `apiKeyEnv` — a credential *reference* resolved per request — over a literal `apiKey`, so no secret enters this file. Omitting **both** is what leaves the route unauthenticated, which for an installed catalog route means pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. One credential serves every model on its route.
|
||||
|
||||
```yaml
|
||||
- id: llm
|
||||
name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
config:
|
||||
providers:
|
||||
# Catalog route: endpoint, protocol, and models all come from pi-ai.
|
||||
openai:
|
||||
apiKeyEnv: OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
@@ -26,16 +27,37 @@ Configure credentials and deployment-specific transport settings per provider, k
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
# Catalog route with its catalog narrowed to one model and that model's
|
||||
# capacity corrected; every unset field still comes from the catalog.
|
||||
anthropic:
|
||||
apiKeyEnv: ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
openrouter:
|
||||
apiKeyEnv: OPENROUTER_API_KEY
|
||||
headers:
|
||||
X-Deployment: production
|
||||
models:
|
||||
- id: claude-sonnet-4-5
|
||||
contextWindow: 200000
|
||||
# Hand-declared route: pi-ai ships nothing under this key, so the profile
|
||||
# supplies the whole provider.
|
||||
acme-gateway:
|
||||
displayName: Acme Gateway
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
contextWindow: 65536
|
||||
maxTokens: 4096
|
||||
```
|
||||
|
||||
Each dict key must exist in pi-ai's installed catalog; the dict shape makes duplicates unrepresentable, and the pre-release array shape (with per-profile `provider` fields) fails load with migration directions. `providers` may also be empty or omitted entirely: the adapter then mounts **dormant** — zero routes, no extra catalog entries — and registers routes the moment the `llm-pi-ai:` settings section supplies profiles, dropping them again when it empties. Dormant or not, the plugin declares every installed catalog provider in the configurable-provider directory (`ctx.llm.listConfigurableProviders()`, settings path `providers.<provider>`), so configuration surfaces can offer the full catalog before any route exists. Which adapters exist is composition; which providers run can be entirely the user's settings document. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
|
||||
The dict shape makes duplicate routes unrepresentable, and the pre-release array shape (with per-profile `provider` fields) fails load with migration directions. `providers` may also be empty or omitted entirely: the adapter then mounts **dormant** — zero routes, no extra catalog entries — and registers routes the moment the `llm-pi-ai:` settings section supplies profiles, dropping them again when it empties. Dormant or not, the plugin declares every installed catalog provider in the configurable-provider directory (`ctx.llm.listConfigurableProviders()`, settings path `providers.<provider>`), joined with every route the current profiles declare, so configuration surfaces can offer the full catalog before any route exists and can still address a hand-declared one. Which adapters exist is composition; which providers run can be entirely the user's settings document. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; a model the route does not configure fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
|
||||
|
||||
## Catalog resolution
|
||||
|
||||
A profile's `models` list *replaces* the route's installed catalog rather than extending it; omitting it (or leaving it empty) serves that catalog unchanged. Each entry defaults its unset fields from the installed model of the same `id`, so narrowing a catalog route to two models, correcting one capacity, or adding a model newer than the installed catalog are all one-line edits. Only the fields the harness consumes are configurable — `id`, `name`, `contextWindow`, `maxTokens`, and `reasoning`; pricing and input modalities have no harness consumer and ride the installed entry or are absent, while reasoning-level spellings and OpenAI-compatibility quirks have no configuration surface at all because restating them cannot be validated.
|
||||
|
||||
Resolution fails loud, naming the offending route and model, when a route cannot be served: a model the installed catalog does not describe needs an explicit `contextWindow` and `maxTokens`, and a route the catalog does not ship needs `api`, `baseURL`, and a non-empty `models` list. `api` accepts the protocols in `supportedProtocols()` — pi-ai's own streaming API set — and is only needed when the catalog cannot supply one: a model absent from the catalog inherits the protocol its shipped siblings agree on, so adding a model to a single-protocol catalog route restates nothing.
|
||||
|
||||
`baseURL` sets the endpoint of every model on the route, so private proxies such as `https://proxy.example.com:8443` remain supported; a catalog route that omits it keeps each catalog model's own endpoint. Naming `api` on a catalog route repoints the whole route at that protocol, which is how a deployment moves a provider between, say, Responses and Chat Completions.
|
||||
|
||||
## Dynamic configuration (settings + credentials)
|
||||
|
||||
@@ -43,17 +65,21 @@ The adapter reads its profiles through a thunk **once per operation** instead of
|
||||
|
||||
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; exactly that variable without a mounted seam). A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin replaces its registration atomically (same adapter instance, candidate set validated first), so a route another adapter already owns leaves the previous routes serving and reverting to a working configuration re-applies. Provider key order never counts as a change. A live settings snapshot naming an unknown provider (or failing any other resolver bound) keeps the last good profiles and logs the failure; the entry config itself still fails plugin load.
|
||||
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers.
|
||||
The adapter exposes each configured route's models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata read from the same pi-ai `Models` collection the request path uses, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, configured output cap, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers. A model's `maxTokens` becomes the seam's `defaultMaxTokens`, so a request that names no output cap carries the configured one.
|
||||
|
||||
The `reasoning.efforts` list is pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model therefore exposes pi-ai's `off` choice. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKey`, `apiKeyEnv`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKey`, `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
## Provider/model routing and replay
|
||||
|
||||
The selected pi-ai catalog descriptor supplies the protocol implementation. This includes native API differences such as OpenAI models whose descriptor uses the Responses API rather than Chat Completions; the harness adapter does not hardcode endpoint selection by model name.
|
||||
Each resolved route contributes one pi-ai `Provider` to the adapter's `createModels()` collection, and requests reach the provider through `Models.streamSimple()`. A catalog route that keeps its catalog protocol **reuses** the installed provider with its model list replaced, because that provider owns API implementations this package cannot reconstruct — Bedrock loads its Smithy module through a separate entry point — so rebuilding it from parts would silently narrow which providers work. Every other route is built by `createProvider()` over the protocol table behind `supportedProtocols()`, whose entries are the same factories pi-ai's own provider factories use.
|
||||
|
||||
Credentials never enter that collection. The harness resolves a route's key through its own seam before the request reaches pi-ai and passes it as the request's `apiKey` option, which pi-ai treats as the highest-priority auth override; `Models` therefore holds no credential store, and the harness keeps its fail-loud reference semantics. A route naming no credential resolves as configured-but-keyless and leaves the requirement to the protocol, which is where it actually lives.
|
||||
|
||||
The selected model descriptor supplies the protocol implementation. This includes native API differences such as OpenAI models whose descriptor uses the Responses API rather than Chat Completions; the harness adapter does not hardcode endpoint selection by model name.
|
||||
|
||||
Successful assistant responses store a versioned, lossless-JSON replay state beside their durable provider/model provenance. At request time, `LlmService` passes replay state only when the historical provider route and target provider route are currently owned by this same `PiAiAdapter` instance. The adapter validates the state and restores pi-ai response ids and provider signatures even when the target provider or model changes; pi-ai then decides which metadata its target API can reuse. History without replay state is translated as foreign provider-neutral content and never impersonates a native pi-ai response.
|
||||
|
||||
@@ -109,7 +135,9 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
|
||||
- **Settings can add or override routes, not remove composition routes** — the user layer merges over the composition `base`, so deleting a `cordis.yml`-provided provider is a composition change; `replace` on the namespace only resets the user layer.
|
||||
- **`headers` can carry a credential the redactor never sees** — the profile's `headers` dict is plain strings, so `Authorization` or `api-key` set there is returned verbatim by a redacted `describe()` and rendered by any configuration UI. Store credentials as `apiKeyEnv` references; making the dict write-only is deferred with the rest of the [wire-boundary work](../llm/README.md#known-limitations-and-deferred-work).
|
||||
- **Catalog membership is required** — custom model ids that are absent from the installed pi-ai catalog fail with `UNKNOWN_MODEL`, even when a provider profile supplies a custom endpoint.
|
||||
- **Model discovery is configuration, not a provider query** — the route's catalog is whatever `settings.yaml` says; nothing fetches a provider's `/models` endpoint, so a model list is only as current as its last edit. A one-shot discovery action that offers a provider's live list for the user to adopt belongs to the configuration surface and is deferred with it.
|
||||
- **One wire protocol per route** — `api` applies to the whole route, so a mixed-protocol catalog route (an OpenAI-style catalog spanning Responses and Chat Completions) cannot host a model of the other protocol, and adding a model such a route does not describe requires naming `api` and moving every model onto it. Splitting the provider across two route keys is the workaround.
|
||||
- **An unauthenticated route depends on its protocol** — naming no credential resolves the route as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder `apiKey` or an `Authorization` entry in `headers`.
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes.
|
||||
|
||||
@@ -2,19 +2,20 @@
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
基于 [`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai) 的 harness LLM（大语言模型）seam 通用多提供方适配器。一个插件实例拥有一份以路由为键的提供方 profile 字典；每个请求使用 `GenerateOptions.provider` 选择 profile，并从 pi-ai 已安装 catalog 中动态解析 `GenerateOptions.model`。
|
||||
基于 [`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai) 的 harness LLM（大语言模型）seam 通用多提供方适配器。一个插件实例拥有一份以路由为键的提供方 profile 字典；每个请求使用 `GenerateOptions.provider` 选择 profile，并针对该路由已配置的 catalog 解析 `GenerateOptions.model`。点名了已安装 pi-ai 提供方的路由会继承其端点、协议格式与模型 catalog 作为默认值，并逐字段覆盖；pi-ai 未提供的路由则整体声明出来，因此接入 OpenAI 兼容网关、自建服务，或比已安装 catalog 更新的提供方，都属于配置而非改代码。
|
||||
|
||||
包根入口导出 Cordis 插件契约与 `PiAiAdapter`；profile 解析、模型构造、回放转换和流转换保留在包内部。
|
||||
包（package）根入口导出 Cordis 插件契约、`PiAiAdapter` 与 `supportedProtocols()`；profile 解析、catalog 物化、提供方构造、回放转换和流转换保留在包内部。
|
||||
|
||||
## 配置
|
||||
|
||||
按提供方配置凭据与部署特定传输设置，并以提供方路由本身为键。优先使用 `apiKeyEnv`——按请求解析的凭据*引用*——而非字面 `apiKey`，让机密不进入该文件。**两者**都省略，才会把认证委托给 pi-ai 的提供方原生环境发现；已配置却解析不出任何值的引用则相反，会让请求以 `MISSING_CREDENTIAL` 失败，因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。`baseURL` 只会覆盖所选 catalog 模型的端点，保留其 API 家族与兼容性元数据，因此仍支持 `https://proxy.example.com:8443` 等私有 proxy。
|
||||
按提供方配置凭据、模型 catalog 与部署特定传输设置，并以提供方路由本身为键。优先使用 `apiKeyEnv`——按请求解析的凭据*引用*——而非字面 `apiKey`，让机密不进入该文件。**两者**都省略，才会让该路由处于未认证状态；对已安装 catalog 路由而言，这意味着交给 pi-ai 的提供方原生环境发现。已配置却解析不出任何值的引用则相反，会让请求以 `MISSING_CREDENTIAL` 失败，因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。一条凭据服务该路由下的全部模型。
|
||||
|
||||
```yaml
|
||||
- id: llm
|
||||
name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
config:
|
||||
providers:
|
||||
# Catalog route: endpoint, protocol, and models all come from pi-ai.
|
||||
openai:
|
||||
apiKeyEnv: OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
@@ -26,16 +27,37 @@
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
# Catalog route with its catalog narrowed to one model and that model's
|
||||
# capacity corrected; every unset field still comes from the catalog.
|
||||
anthropic:
|
||||
apiKeyEnv: ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
openrouter:
|
||||
apiKeyEnv: OPENROUTER_API_KEY
|
||||
headers:
|
||||
X-Deployment: production
|
||||
models:
|
||||
- id: claude-sonnet-4-5
|
||||
contextWindow: 200000
|
||||
# Hand-declared route: pi-ai ships nothing under this key, so the profile
|
||||
# supplies the whole provider.
|
||||
acme-gateway:
|
||||
displayName: Acme Gateway
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
contextWindow: 65536
|
||||
maxTokens: 4096
|
||||
```
|
||||
|
||||
每个字典键都必须存在于 pi-ai 已安装 catalog 中；字典形状使重复项无法表示，发布前的数组形状（每个 profile 携带 `provider` 字段）会加载失败并给出迁移指引。`providers` 也可以为空或整体省略：适配器将以**休眠**姿态挂载——零路由、模型选择器不多一条——一旦 `llm-pi-ai:` settings 分节提供了 profile 就即时注册路由，分节清空时随之撤销。无论是否休眠，插件都会在可配置提供方目录（`ctx.llm.listConfigurableProviders()`，settings 路径 `providers.<provider>`）中声明每个已安装 catalog 提供方，因此配置界面可以在任何路由存在之前就提供完整 catalog。哪些适配器存在归组合面；哪些提供方在运行可以完全交给用户的设置文档。向 `ctx.llm` 注册具有原子性：如果与另一适配器已拥有的任何提供方路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；未知模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
|
||||
字典形状使重复路由无法表示，发布前的数组形状（每个 profile 携带 `provider` 字段）会加载失败并给出迁移指引。`providers` 也可以为空或整体省略：适配器将以**休眠**姿态挂载——零路由、模型选择器不多一条——一旦 `llm-pi-ai:` settings 分节提供了 profile 就即时注册路由，分节清空时随之撤销。无论是否休眠，插件都会在可配置提供方目录（`ctx.llm.listConfigurableProviders()`，settings 路径 `providers.<provider>`）中声明每个已安装 catalog 提供方，并与当前 profile 声明的每条路由取并集，因此配置界面既能在任何路由存在之前就提供完整 catalog，也能寻址一条手工声明的路由。哪些适配器存在归组合面；哪些提供方在运行可以完全交给用户的设置文档。向 `ctx.llm` 注册具有原子性：如果与另一适配器已拥有的任何提供方路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；路由未配置的模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
|
||||
|
||||
## Catalog 解析
|
||||
|
||||
profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩充它；省略它（或留空）则原样服务该 catalog。每个条目都会从同 `id` 的已安装模型继承自身未设置的字段，因此把 catalog 路由收窄到两个模型、更正某个容量，或加入一个比已安装 catalog 更新的模型，都是一行编辑。只有 harness 会消费的字段可配置——`id`、`name`、`contextWindow`、`maxTokens` 与 `reasoning`；定价与输入模态没有 harness 消费方，因此沿用已安装条目或直接缺席，而思考级别的协议拼写与 OpenAI 兼容性怪癖则完全没有配置面，因为重述它们无法被校验。
|
||||
|
||||
解析会失败得响亮，并点名出问题的路由与模型：已安装 catalog 未描述的模型需要显式的 `contextWindow` 与 `maxTokens`，catalog 未提供的路由则需要 `api`、`baseURL` 和非空的 `models` 列表。`api` 接受 `supportedProtocols()` 中的协议——即 pi-ai 自己的流式 API 集合——且仅在 catalog 无法提供协议时才需要：catalog 中不存在的模型会继承其同门模型一致同意的协议，因此向单协议 catalog 路由添加模型无需重述任何内容。
|
||||
|
||||
`baseURL` 设定该路由下每个模型的端点，因此仍支持 `https://proxy.example.com:8443` 等私有 proxy；省略它的 catalog 路由会保留每个 catalog 模型自己的端点。在 catalog 路由上点名 `api` 会把整条路由改指到该协议，这正是部署把某个提供方在 Responses 与 Chat Completions 之间迁移的方式。
|
||||
|
||||
## 动态配置（settings + credentials）
|
||||
|
||||
@@ -43,17 +65,21 @@
|
||||
|
||||
凭据按每次 stream 调用解析：非空的字面 `apiKey` 优先，其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`（活跃环境之下的 `$DSH_HOME/.env`；未挂载 seam 时恰好读取该环境变量）。只有完全没有点名任何凭据的 profile——仅限这一种情况——才交给 pi-ai 的环境发现。路由集合与每条路由捕获的重试策略是注册级事实：两者任一变化时，插件都会原子地替换自己的注册（同一适配器实例，候选集合先经校验），因此某条路由若已被另一适配器占有，先前的路由会继续服务，而改回可用配置时注册会重新生效。提供方键的顺序绝不算作变化。存活 settings 快照若点名未知提供方（或违反任何其他 resolver 约束），则保留最后可用 profile 并记录失败；entry 配置本身仍会使插件加载失败。
|
||||
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每个已配置提供方已安装的 pi-ai 模型。这是从 `getModels(provider)` 派生的提供方无关 selector 元数据；请求时解析仍会执行权威 catalog 查找，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找，并返回其身份、上下文窗口和可选思考级别，让权威元数据保留在拥有路由的适配器上，而非消费方。
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每条已配置路由的模型。这是从请求路径所用的同一个 pi-ai `Models` 集合读取的提供方无关 selector 元数据，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找，并返回其身份、上下文窗口、已配置输出上限和可选思考级别，让权威元数据保留在拥有路由的适配器上，而非消费方。模型的 `maxTokens` 会成为 seam 的 `defaultMaxTokens`，因此未点名输出上限的请求会携带已配置的那一个。
|
||||
|
||||
`reasoning.efforts` 列表是 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果，不经筛选或规范化，其中包括 `off`，以及模型对 `xhigh` 或 `max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID；提供方／模型在协议格式中的表示仍保留在 pi-ai 的 `thinkingLevelMap` 中。因此，不具备推理（reasoning）能力的模型也会公开 pi-ai 的 `off` 选项。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；任何未出现在确切模型能力中的显式值都会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKey`、`apiKeyEnv`、`baseURL`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKey`、`apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
## 提供方／模型路由与回放
|
||||
|
||||
所选 pi-ai catalog descriptor 提供协议实现。这包括原生 API 差异，例如 descriptor 使用 Responses API 而非 Chat Completions 的 OpenAI 模型；harness 适配器不会按模型名称硬编码端点选择。
|
||||
每条已解析路由都会向适配器的 `createModels()` 集合贡献一个 pi-ai `Provider`，请求经 `Models.streamSimple()` 抵达提供方。保持 catalog 协议不变的 catalog 路由会**复用**已安装提供方，只替换其模型列表，因为该提供方持有本包无法重建的 API 实现——Bedrock 经由独立入口加载其 Smithy 模块——从零件重建会静默收窄可用提供方的范围。其余路由都由 `createProvider()` 基于 `supportedProtocols()` 背后的协议表构造，表中条目正是 pi-ai 自己的提供方工厂所用的同一批 factory。
|
||||
|
||||
凭据绝不进入该集合。harness 在请求抵达 pi-ai 之前经自身 seam 解析路由密钥，并作为请求的 `apiKey` 选项传入，而 pi-ai 将其视为优先级最高的 auth 覆盖；因此 `Models` 不持有任何凭据存储，harness 也保住了自己失败得响亮的引用语义。没有点名任何凭据的路由会解析为「已配置但无密钥」，把该要求留给协议——那才是它真正所在的位置。
|
||||
|
||||
所选模型 descriptor 提供协议实现。这包括原生 API 差异，例如 descriptor 使用 Responses API 而非 Chat Completions 的 OpenAI 模型；harness 适配器不会按模型名称硬编码端点选择。
|
||||
|
||||
成功的 assistant 响应会在自身持久提供方／模型溯源旁存储经版本化的无损 JSON 回放状态。请求时，`LlmService` 只有在历史提供方路由与目标提供方路由当前由同一个 `PiAiAdapter` 实例拥有时，才会传递回放状态。即使目标提供方或模型改变，适配器也会验证状态并恢复 pi-ai 响应 id 与提供方 signature；随后由 pi-ai 判定目标 API 可以复用哪些元数据。没有回放状态的历史会被转换为外来的、与提供方无关的内容，绝不伪装为原生 pi-ai 响应。
|
||||
|
||||
@@ -109,7 +135,9 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
|
||||
|
||||
- **settings 能新增或覆盖路由，但不能移除组合路由**：用户层合并在组合 `base` 之上，因此删除 `cordis.yml` 提供的提供方属于组合变更；对该 namespace 执行 `replace` 只会重置用户层。
|
||||
- **`headers` 可能承载一条脱敏器看不见的凭据**：profile 的 `headers` 是纯字符串字典，因此设在其中的 `Authorization` 或 `api-key` 会被脱敏后的 `describe()` 原样返回，并被任何配置 UI 渲染出来。请把凭据存为 `apiKeyEnv` 引用；把该字典整体改为只写与其余[协议边界工作](../llm/README.md#known-limitations-and-deferred-work)一并暂缓。
|
||||
- **必须属于 catalog**：已安装 pi-ai catalog 中不存在的自定义模型 id 会以 `UNKNOWN_MODEL` 失败，即使提供方 profile 配置了自定义端点。
|
||||
- **模型发现属于配置，不是提供方查询**：路由的 catalog 就是 `settings.yaml` 所写的内容；没有任何环节会去拉取提供方的 `/models` 端点，因此模型列表的新鲜度只到最近一次编辑为止。把提供方实时列表呈给用户采纳的一次性发现动作属于配置界面，与之一并暂缓。
|
||||
- **每条路由只有一种协议格式**：`api` 作用于整条路由，因此混合协议的 catalog 路由（跨 Responses 与 Chat Completions 的 OpenAI 式 catalog）无法承载另一种协议的模型，向这类路由添加它未描述的模型必须点名 `api` 并把全部模型一起迁过去。把该提供方拆成两个路由键是变通办法。
|
||||
- **未认证路由取决于其协议**：不点名凭据会让路由解析为「已配置但无密钥」，但 pi-ai 的 OpenAI 兼容实现仍要求 API key 或 `Authorization` 标头，因此无鉴权的本地服务需要一个占位 `apiKey`，或在 `headers` 中给出 `Authorization` 条目。
|
||||
- **不支持 `GenerateOptions.stop`**：pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence，因此适配器会拒绝该字段。
|
||||
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**：提供方特定位置由 pi-ai 决定，而非由 harness 拥有的协议覆盖决定。
|
||||
- **无法获取提供方 HTTP 状态**：pi-ai 错误事件不会在所有提供方上公开稳定 HTTP 状态；失败只公开稳定 harness 错误 code。
|
||||
|
||||
@@ -1,17 +1,25 @@
|
||||
/**
|
||||
* Generic pi-ai-backed implementation of the Harness LLM seam.
|
||||
*
|
||||
* The adapter owns one pi-ai `Models` collection and keeps it in step with the
|
||||
* resolved profiles: each route contributes the `Provider` its resolution built,
|
||||
* so model lookup, protocol dispatch, and request auth all reach pi-ai through
|
||||
* its supported runtime rather than the deprecated global compatibility entry.
|
||||
*
|
||||
* Credentials stay outside that collection. The harness resolves a route's key
|
||||
* through its own seam and passes it as the request's `apiKey` option, which
|
||||
* pi-ai treats as the highest-priority auth override — so `Models` never holds
|
||||
* a credential store and the harness keeps its fail-loud reference semantics.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/adapter
|
||||
*/
|
||||
|
||||
import { streamSimple } from '@earendil-works/pi-ai/compat'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all'
|
||||
import { getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import { createModels, getSupportedThinkingLevels } from '@earendil-works/pi-ai'
|
||||
import type {
|
||||
Api,
|
||||
Model,
|
||||
ModelThinkingLevel,
|
||||
MutableModels,
|
||||
SimpleStreamOptions,
|
||||
ThinkingLevel,
|
||||
} from '@earendil-works/pi-ai'
|
||||
@@ -40,29 +48,15 @@ export interface PiAiAdapterOptions {
|
||||
profiles: () => ReadonlyMap<string, ResolvedPiAiProviderProfile>
|
||||
/**
|
||||
* Resolve the credential for one already-resolved profile; called once per
|
||||
* stream call and frozen for that call. `undefined` defers to pi-ai's
|
||||
* provider-native ambient discovery, which the plugin allows only for a
|
||||
* profile naming no credential at all; a named reference that misses throws
|
||||
* `LlmError` `MISSING_CREDENTIAL` rather than falling back.
|
||||
* stream call and frozen for that call. `undefined` defers to the route's own
|
||||
* pi-ai auth, which for an installed catalog route is its provider-native
|
||||
* ambient discovery; the plugin allows that only for a profile naming no
|
||||
* credential at all, because a named reference that misses throws `LlmError`
|
||||
* `MISSING_CREDENTIAL` rather than falling back.
|
||||
*/
|
||||
resolveApiKey: (provider: string, profile: ResolvedPiAiProviderProfile) => Promise<string | undefined>
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a catalog model dynamically and apply only the configured endpoint
|
||||
* override, preserving the catalog's API/capability/compatibility metadata.
|
||||
*/
|
||||
function resolvePiModel(
|
||||
profile: ResolvedPiAiProviderProfile,
|
||||
modelId: string,
|
||||
): Model<Api> {
|
||||
const model = getBuiltinModels(profile.provider as BuiltinProvider).find(candidate => candidate.id === modelId) as Model<Api> | undefined
|
||||
if (model === undefined) {
|
||||
throw new LlmError(`pi-ai provider "${profile.provider}" has no catalog model "${modelId}"`, 'UNKNOWN_MODEL')
|
||||
}
|
||||
return profile.baseURL === undefined ? model : { ...model, baseUrl: profile.baseURL }
|
||||
}
|
||||
|
||||
/** Copy profile stream knobs into pi-ai's common option vocabulary. */
|
||||
function profileOptions(
|
||||
profile: ResolvedPiAiProviderProfile,
|
||||
@@ -108,28 +102,65 @@ function requestHeaders(headers: Readonly<Record<string, string>> | undefined):
|
||||
}
|
||||
|
||||
/**
|
||||
* pi-ai-backed multi-provider adapter. Model descriptors are resolved for each
|
||||
* request, so models need not be registered during the Cordis lifecycle.
|
||||
* pi-ai-backed multi-provider adapter. Each operation reads the current
|
||||
* profiles, so a configuration change reaches the next request without a
|
||||
* restart; model descriptors come from the collection those profiles built.
|
||||
*/
|
||||
export class PiAiAdapter extends LlmAdapter {
|
||||
private readonly models: MutableModels = createModels()
|
||||
private registered: ReadonlyMap<string, ResolvedPiAiProviderProfile> | undefined
|
||||
|
||||
constructor(private readonly config: PiAiAdapterOptions) {
|
||||
super()
|
||||
}
|
||||
|
||||
/**
|
||||
* The `Models` collection for the current profiles. Resolution memoizes its
|
||||
* result, so an unchanged configuration is recognized by identity and the
|
||||
* collection is rebuilt only when the route set or any profile actually
|
||||
* changes.
|
||||
*/
|
||||
private collection(): MutableModels {
|
||||
const profiles = this.config.profiles()
|
||||
if (profiles === this.registered) return this.models
|
||||
this.models.clearProviders()
|
||||
for (const profile of profiles.values()) this.models.setProvider(profile.piProvider)
|
||||
this.registered = profiles
|
||||
return this.models
|
||||
}
|
||||
|
||||
/** The profile for one route, or the seam's own not-owned failure. */
|
||||
private profileOf(provider: string): ResolvedPiAiProviderProfile {
|
||||
const profile = this.config.profiles().get(provider)
|
||||
if (profile === undefined) {
|
||||
throw new LlmError(`pi-ai adapter does not own provider "${provider}"`, 'NO_ADAPTER')
|
||||
}
|
||||
return profile
|
||||
}
|
||||
|
||||
/** The configured descriptor for one exact route/model pair. */
|
||||
private modelOf(provider: string, model: string): Model<Api> {
|
||||
this.profileOf(provider)
|
||||
const resolved = this.collection().getModel(provider, model)
|
||||
if (resolved === undefined) {
|
||||
throw new LlmError(`pi-ai provider "${provider}" has no configured model "${model}"`, 'UNKNOWN_MODEL')
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
override providerRetryPolicy(provider: string): ResolvedRetryPolicy | undefined {
|
||||
return this.config.profiles().get(provider)?.retryPolicy
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
const profile = this.config.profiles().get(provider)
|
||||
if (profile === undefined) {
|
||||
return Promise.reject(new LlmError(`pi-ai adapter does not own provider "${provider}"`, 'NO_ADAPTER'))
|
||||
}
|
||||
return Promise.resolve(getBuiltinModels(profile.provider as BuiltinProvider).map(model => ({
|
||||
provider,
|
||||
id: model.id,
|
||||
name: model.name,
|
||||
})))
|
||||
return Promise.resolve().then(() => {
|
||||
this.profileOf(provider)
|
||||
return this.collection().getModels(provider).map(model => ({
|
||||
provider,
|
||||
id: model.id,
|
||||
name: model.name,
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
override resolveModel(
|
||||
@@ -137,15 +168,9 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const profile = this.config.profiles().get(provider)
|
||||
if (profile === undefined) {
|
||||
return Promise.reject(new LlmError(
|
||||
`pi-ai adapter does not own provider "${provider}"`,
|
||||
'NO_ADAPTER',
|
||||
))
|
||||
}
|
||||
return Promise.resolve().then(() => {
|
||||
const resolvedModel = resolvePiModel(profile, model)
|
||||
const profile = this.profileOf(provider)
|
||||
const resolvedModel = this.modelOf(provider, model)
|
||||
const levels = getSupportedThinkingLevels(resolvedModel)
|
||||
const defaultLevel = resolveReasoningLevel(resolvedModel, profile.reasoning)
|
||||
return {
|
||||
@@ -153,6 +178,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
id: model,
|
||||
name: resolvedModel.name,
|
||||
context: { contextWindow: resolvedModel.contextWindow },
|
||||
defaultMaxTokens: resolvedModel.maxTokens,
|
||||
reasoning: {
|
||||
efforts: levels.map(level => ({
|
||||
id: ReasoningEffortId(level),
|
||||
@@ -170,14 +196,13 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
if (options.stop !== undefined) {
|
||||
throw new LlmError('llm-pi-ai does not support GenerateOptions.stop', 'UNSUPPORTED_OPTION')
|
||||
}
|
||||
// One resolution per stream call: the profile snapshot and the credential
|
||||
// freeze here and hold for this whole request, so an in-flight stream
|
||||
// never observes a configuration change and the next call re-resolves.
|
||||
const profile = this.config.profiles().get(options.provider)
|
||||
if (profile === undefined) {
|
||||
throw new LlmError(`pi-ai adapter does not own provider "${options.provider}"`, 'NO_ADAPTER')
|
||||
}
|
||||
const model = resolvePiModel(profile, options.model)
|
||||
// One resolution per stream call: the profile snapshot, the model
|
||||
// descriptor, and the credential freeze here and hold for this whole
|
||||
// request, so an in-flight stream never observes a configuration change and
|
||||
// the next call re-resolves.
|
||||
const profile = this.profileOf(options.provider)
|
||||
const collection = this.collection()
|
||||
const model = this.modelOf(options.provider, options.model)
|
||||
const reasoning = resolveReasoningLevel(
|
||||
model,
|
||||
options.reasoningEffort ?? profile.reasoning,
|
||||
@@ -192,7 +217,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
using watchdog = idleWatchdog(upstream, streamIdleTimeoutMs, 'LLM_STREAM_IDLE_TIMEOUT')
|
||||
|
||||
try {
|
||||
const events = streamSimple(model, toPiContext(options), {
|
||||
const events = collection.streamSimple(model, toPiContext(options), {
|
||||
...profileOptions(profile, reasoning, apiKey),
|
||||
...options.temperature === undefined ? {} : { temperature: options.temperature },
|
||||
...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },
|
||||
|
||||
193
packages/llm/llm-pi-ai/src/catalog.ts
Normal file
193
packages/llm/llm-pi-ai/src/catalog.ts
Normal file
@@ -0,0 +1,193 @@
|
||||
/**
|
||||
* Materialization of one provider route's model catalog. The installed pi-ai
|
||||
* catalog supplies defaults keyed by model id, and a profile's own model
|
||||
* entries override them field by field, so a route naming a catalog provider
|
||||
* stays configuration-free while a route pi-ai has never heard of is fully
|
||||
* describable from `settings.yaml`.
|
||||
*
|
||||
* Every pi-ai `Model` field the harness cannot default is required here rather
|
||||
* than at request time: an unserviceable route fails while its configuration is
|
||||
* being resolved, which is the earliest point that can name the offending key.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/catalog
|
||||
*/
|
||||
|
||||
import { builtinProviders, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { Api, Model, ModelCost, Provider } from '@earendil-works/pi-ai'
|
||||
|
||||
/**
|
||||
* Pricing for a model the installed catalog does not describe. The harness
|
||||
* never reads pi-ai's cost metadata — `replay.ts` zeroes it and no consumer
|
||||
* reports spend — so this is the absence of a fact, not a configurable rate.
|
||||
*/
|
||||
const NO_COST: ModelCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
|
||||
|
||||
/**
|
||||
* Input modalities for a model the installed catalog does not describe. The
|
||||
* request converter keeps only text blocks, so text is the adapter's actual
|
||||
* capability rather than a deployment choice.
|
||||
*/
|
||||
const TEXT_ONLY: Model<Api>['input'] = ['text']
|
||||
|
||||
let providerIndex: Map<string, Provider> | undefined
|
||||
|
||||
/**
|
||||
* Installed catalog providers by id, constructed once. Each entry owns the API
|
||||
* implementations for its own models, which is why a catalog route reuses this
|
||||
* provider instead of being rebuilt from parts.
|
||||
* @returns the catalog provider index.
|
||||
*/
|
||||
function catalogProviders(): Map<string, Provider> {
|
||||
providerIndex ??= new Map(builtinProviders().map(provider => [provider.id, provider]))
|
||||
return providerIndex
|
||||
}
|
||||
|
||||
/**
|
||||
* The installed catalog provider for one route, when pi-ai ships one.
|
||||
* @param provider - provider route key.
|
||||
* @returns the catalog provider, or `undefined` for a route pi-ai does not ship.
|
||||
*/
|
||||
export function catalogProvider(provider: string): Provider | undefined {
|
||||
return catalogProviders().get(provider)
|
||||
}
|
||||
|
||||
/**
|
||||
* Every provider route the installed pi-ai catalog ships.
|
||||
* @returns the catalog provider ids.
|
||||
*/
|
||||
export function catalogProviderIds(): readonly string[] {
|
||||
return getBuiltinProviders()
|
||||
}
|
||||
|
||||
/**
|
||||
* The installed catalog models for one route, indexed by model id.
|
||||
* @param provider - provider route key.
|
||||
* @returns catalog models by id; empty for a route pi-ai does not ship.
|
||||
*/
|
||||
export function catalogModels(provider: string): Map<string, Model<Api>> {
|
||||
if (!catalogProviders().has(provider)) return new Map()
|
||||
const models = getBuiltinModels(provider as BuiltinProvider) as Model<Api>[]
|
||||
return new Map(models.map(model => [model.id, model]))
|
||||
}
|
||||
|
||||
/** One configured model entry: an id plus the catalog fields it overrides. */
|
||||
export interface PiAiModelProfile {
|
||||
/** Model id sent to the provider and accepted by {@link GenerateOptions.model}. */
|
||||
id: string
|
||||
/** Display name for selectors; defaults to the catalog name, then the id. */
|
||||
name?: string
|
||||
/** Maximum combined request and response context in tokens. */
|
||||
contextWindow?: number
|
||||
/** Per-request output cap materialized when a caller omits one. */
|
||||
maxTokens?: number
|
||||
/** Whether the model exposes reasoning; defaults to the catalog capability. */
|
||||
reasoning?: boolean
|
||||
}
|
||||
|
||||
/** The route-level facts model materialization reads. */
|
||||
export interface RouteCatalogRequest {
|
||||
/** Provider route key, stamped onto every materialized model. */
|
||||
provider: string
|
||||
/** Wire protocol override; absent defers to each catalog model's own API. */
|
||||
api?: string
|
||||
/** Endpoint override; absent defers to the catalog model, then the catalog provider. */
|
||||
baseURL?: string
|
||||
/** Configured catalog; absent means the whole installed catalog for this route. */
|
||||
models?: readonly PiAiModelProfile[]
|
||||
}
|
||||
|
||||
/** Report a route the deployment cannot serve, naming the settings key at fault. */
|
||||
function invalid(provider: string, detail: string): never {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" ${detail}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* The one wire protocol a catalog route's shipped models agree on. This is what
|
||||
* lets a deployment add a model the installed catalog has not caught up with —
|
||||
* a provider's newest release — without restating the protocol its siblings
|
||||
* already use. A route whose shipped models disagree (an OpenAI-style catalog
|
||||
* spanning Responses and Chat Completions) has no such answer, so a model it
|
||||
* does not describe must name its protocol at the route.
|
||||
*/
|
||||
function sharedCatalogApi(defaults: ReadonlyMap<string, Model<Api>>): string | undefined {
|
||||
const apis = new Set<string>()
|
||||
for (const model of defaults.values()) apis.add(model.api)
|
||||
return apis.size === 1 ? [...apis][0] : undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Materialize one route's catalog by merging the installed catalog defaults
|
||||
* under the configured entries. A route with no configured `models` serves the
|
||||
* installed catalog unchanged, which is what keeps an existing
|
||||
* `providers: { deepseek: { apiKeyEnv: … } }` profile working untouched.
|
||||
* @param request - the route-level catalog facts.
|
||||
* @returns the materialized models in configuration order.
|
||||
*/
|
||||
export function resolveRouteModels(request: RouteCatalogRequest): readonly Model<Api>[] {
|
||||
const { provider } = request
|
||||
const defaults = catalogModels(provider)
|
||||
const providerBaseUrl = catalogProvider(provider)?.baseUrl
|
||||
// An absent `models` key and an empty one are the same request: the config
|
||||
// schema materializes `[]` for the absent case, and an empty catalog could
|
||||
// serve no request anyway, so both mean "serve the installed catalog".
|
||||
const configured = request.models ?? []
|
||||
const entries: readonly PiAiModelProfile[] = configured.length > 0
|
||||
? configured
|
||||
: [...defaults.values()].map(model => ({ id: model.id }))
|
||||
if (entries.length === 0) {
|
||||
invalid(provider, 'resolves no models; the installed catalog does not describe this route, so its models'
|
||||
+ ' must be listed in configuration')
|
||||
}
|
||||
const routeApi = sharedCatalogApi(defaults)
|
||||
const seen = new Set<string>()
|
||||
return entries.map((entry) => {
|
||||
if (entry.id.length === 0) invalid(provider, 'has a model with an empty id')
|
||||
if (seen.has(entry.id)) invalid(provider, `lists model "${entry.id}" more than once`)
|
||||
seen.add(entry.id)
|
||||
const base = defaults.get(entry.id)
|
||||
const api = request.api ?? base?.api ?? routeApi
|
||||
if (api === undefined) {
|
||||
invalid(provider, `model "${entry.id}" needs an api; the installed catalog does not describe it, so set the`
|
||||
+ ' route\'s api to the wire protocol its endpoint speaks')
|
||||
}
|
||||
const baseUrl = request.baseURL ?? base?.baseUrl ?? providerBaseUrl
|
||||
if (baseUrl === undefined) {
|
||||
invalid(provider, `model "${entry.id}" needs a baseURL; the installed catalog does not describe this route`)
|
||||
}
|
||||
const contextWindow = entry.contextWindow ?? base?.contextWindow
|
||||
if (contextWindow === undefined) {
|
||||
invalid(provider, `model "${entry.id}" needs a contextWindow; without it the harness cannot detect overflow`
|
||||
+ ' or size compaction')
|
||||
}
|
||||
if (!Number.isInteger(contextWindow) || contextWindow <= 0) {
|
||||
invalid(provider, `model "${entry.id}" contextWindow must be a positive integer`)
|
||||
}
|
||||
const maxTokens = entry.maxTokens ?? base?.maxTokens
|
||||
if (maxTokens === undefined) {
|
||||
invalid(provider, `model "${entry.id}" needs a maxTokens; it is the output cap materialized into requests`
|
||||
+ ' that omit one')
|
||||
}
|
||||
if (!Number.isInteger(maxTokens) || maxTokens <= 0) {
|
||||
invalid(provider, `model "${entry.id}" maxTokens must be a positive integer`)
|
||||
}
|
||||
return {
|
||||
id: entry.id,
|
||||
name: entry.name ?? base?.name ?? entry.id,
|
||||
api,
|
||||
provider,
|
||||
baseUrl,
|
||||
reasoning: entry.reasoning ?? base?.reasoning ?? false,
|
||||
input: base?.input ?? TEXT_ONLY,
|
||||
cost: base?.cost ?? NO_COST,
|
||||
contextWindow,
|
||||
maxTokens,
|
||||
// Catalog-only metadata: reasoning-level spellings and OpenAI-compatibility
|
||||
// quirks have no configuration surface, so they ride the catalog entry or
|
||||
// are absent for a model pi-ai has never described.
|
||||
...base?.thinkingLevelMap === undefined ? {} : { thinkingLevelMap: base.thinkingLevelMap },
|
||||
...base?.compat === undefined ? {} : { compat: base.compat },
|
||||
...base?.headers === undefined ? {} : { headers: base.headers },
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -3,29 +3,55 @@
|
||||
* Profiles are a dict keyed by provider route, so the composition base and a
|
||||
* user-settings layer merge per provider and the route set is structural.
|
||||
*
|
||||
* A route key is not required to name an installed pi-ai provider. When it does,
|
||||
* that provider's endpoint, protocol, display name, and model catalog are the
|
||||
* profile's defaults and the profile overrides them field by field; when it does
|
||||
* not, the profile is the whole provider declaration. Resolution therefore ends
|
||||
* in a built pi-ai `Provider` per route: everything a request needs is decided
|
||||
* once, while the configuration key that made a route unserviceable can still be
|
||||
* named in the failure.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/config
|
||||
*/
|
||||
|
||||
import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { CacheRetention, ModelThinkingLevel, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import type { CacheRetention, ModelThinkingLevel, Provider, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import z from 'schemastery'
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { resolveRouteModels } from './catalog.ts'
|
||||
import type { PiAiModelProfile } from './catalog.ts'
|
||||
import { buildProvider, supportedProtocols } from './provider.ts'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
|
||||
export type { PiAiModelProfile } from './catalog.ts'
|
||||
|
||||
/** Configuration for one pi-ai provider route; the `providers` dict key IS the route. */
|
||||
export interface PiAiProviderProfile {
|
||||
/** Literal provider credential; prefer {@link apiKeyEnv}. With both absent pi-ai uses its provider-native ambient discovery. */
|
||||
apiKey?: string
|
||||
/** Credential reference (environment-variable name) resolved per request through `ctx.credentials`. */
|
||||
apiKeyEnv?: string
|
||||
/** Override the selected catalog model's endpoint without changing its protocol metadata. */
|
||||
/** Name shown by configuration surfaces; defaults to the route key. */
|
||||
displayName?: string
|
||||
/**
|
||||
* Wire protocol every model on this route speaks. Omission keeps each
|
||||
* installed catalog model's own protocol, which is why a catalog route needs
|
||||
* no protocol at all; a route the catalog does not ship must name one.
|
||||
*/
|
||||
api?: string
|
||||
/** Endpoint for this route's models; defaults to the installed catalog's endpoint. */
|
||||
baseURL?: string
|
||||
/**
|
||||
* This route's model catalog. Omission serves the installed catalog for the
|
||||
* route unchanged; an explicit list replaces it, each entry defaulting its
|
||||
* unset fields from the installed model of the same id.
|
||||
*/
|
||||
models?: PiAiModelProfile[]
|
||||
/** Provider request headers; Harness attribution wins reserved names. */
|
||||
headers?: Record<string, string>
|
||||
/** Provider-neutral pi-ai reasoning level. */
|
||||
@@ -47,15 +73,25 @@ export interface PiAiProviderProfile {
|
||||
}
|
||||
|
||||
/** Validated profile with its route stamped and every adapter-owned default resolved. */
|
||||
export interface ResolvedPiAiProviderProfile extends Omit<PiAiProviderProfile, 'apiKeyEnv' | 'retryPolicy'> {
|
||||
/** pi-ai provider catalog name and Harness route key (the configuration dict key). */
|
||||
export interface ResolvedPiAiProviderProfile
|
||||
extends Omit<PiAiProviderProfile, 'apiKeyEnv' | 'retryPolicy' | 'models' | 'displayName'> {
|
||||
/** Harness route key and the `Models` collection key (the configuration dict key). */
|
||||
provider: string
|
||||
/** Resolved display name for selectors and configuration surfaces. */
|
||||
displayName: string
|
||||
/** Validated credential reference, when one is configured. */
|
||||
apiKeyEnv?: CredentialRef
|
||||
/** Positive finite provider-idle interval after defaulting. */
|
||||
streamIdleTimeoutMs: number
|
||||
/** Immutable retry policy captured with this provider route. */
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
/**
|
||||
* The pi-ai provider this route registers, built from the resolved models.
|
||||
* Construction happens here so an unserviceable protocol or an underspecified
|
||||
* model fails with the rest of resolution, leaving the last good route set
|
||||
* serving requests.
|
||||
*/
|
||||
piProvider: Provider
|
||||
}
|
||||
|
||||
/** Plugin configuration: the provider routes this instance owns. */
|
||||
@@ -75,10 +111,21 @@ const thinkingBudgets = z.object({
|
||||
high: z.number(),
|
||||
})
|
||||
|
||||
const modelProfile: z<PiAiModelProfile> = z.object({
|
||||
id: z.string().required(),
|
||||
name: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
maxTokens: z.number().step(1).min(1),
|
||||
reasoning: z.boolean(),
|
||||
})
|
||||
|
||||
const profile = z.object({
|
||||
apiKey: z.string().role('secret'),
|
||||
apiKeyEnv: z.string().role('credential-ref'),
|
||||
displayName: z.string(),
|
||||
api: z.union(supportedProtocols()),
|
||||
baseURL: z.string(),
|
||||
models: z.array(modelProfile),
|
||||
headers: z.dict(z.string()),
|
||||
reasoning: z.union(['off', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
|
||||
thinkingBudgets,
|
||||
@@ -95,11 +142,29 @@ export const Config: z<Config> = z.object({
|
||||
providers: z.dict(profile).default({}),
|
||||
})
|
||||
|
||||
/** Reject a pre-release profile shape, naming the replacement. */
|
||||
function rejectRemovedFields(provider: string, source: PiAiProviderProfile): void {
|
||||
const legacy = source as PiAiProviderProfile & {
|
||||
provider?: unknown
|
||||
maxRetries?: unknown
|
||||
maxRetryDelayMs?: unknown
|
||||
}
|
||||
if ('provider' in legacy) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" sets "provider", which moved to the providers dict key`)
|
||||
}
|
||||
if ('maxRetries' in legacy || 'maxRetryDelayMs' in legacy) {
|
||||
throw new Error(
|
||||
`llm-pi-ai: provider "${provider}" sets maxRetries or maxRetryDelayMs, which were removed;`
|
||||
+ ' compose agent recovery with dsh-llm-retry',
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate profiles against the installed pi-ai catalog and return a detached
|
||||
* route-keyed map suitable for per-request reads. This is the one explicit
|
||||
* resolve step, so an omitted dict resolves to the empty (dormant) route set
|
||||
* here rather than through a hidden fallback.
|
||||
* Validate profiles and return a detached route-keyed map suitable for
|
||||
* per-request reads. This is the one explicit resolve step, so an omitted dict
|
||||
* resolves to the empty (dormant) route set here rather than through a hidden
|
||||
* fallback, and each route's models and pi-ai provider are materialized once.
|
||||
* @param providers - configured provider profiles keyed by route.
|
||||
* @returns validated profiles in configuration order.
|
||||
*/
|
||||
@@ -110,28 +175,19 @@ export function resolveProfiles(
|
||||
throw new Error('llm-pi-ai: providers is now a dict keyed by provider route, not an array of profiles')
|
||||
}
|
||||
const entries = Object.entries(providers ?? {})
|
||||
const supported = new Set<string>(getBuiltinProviders())
|
||||
const resolved = new Map<string, ResolvedPiAiProviderProfile>()
|
||||
for (const [provider, source] of entries) {
|
||||
const legacy = source as PiAiProviderProfile & {
|
||||
provider?: unknown
|
||||
maxRetries?: unknown
|
||||
maxRetryDelayMs?: unknown
|
||||
}
|
||||
if ('provider' in legacy) {
|
||||
throw new Error('llm-pi-ai: the profile "provider" field moved to the providers dict key')
|
||||
}
|
||||
if ('maxRetries' in legacy || 'maxRetryDelayMs' in legacy) {
|
||||
throw new Error('llm-pi-ai: maxRetries and maxRetryDelayMs were removed; compose agent recovery with dsh-llm-retry')
|
||||
}
|
||||
rejectRemovedFields(provider, source)
|
||||
if (provider.length === 0) throw new Error('llm-pi-ai: provider names must be non-empty')
|
||||
if (!supported.has(provider)) throw new Error(`llm-pi-ai: unknown pi-ai provider "${provider}"`)
|
||||
if (source.apiKey !== undefined && source.apiKey.trim().length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" has an empty apiKey; omit it to use ambient authentication`)
|
||||
}
|
||||
if (source.baseURL !== undefined && source.baseURL.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" has an empty baseURL`)
|
||||
}
|
||||
if (source.displayName !== undefined && source.displayName.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" has an empty displayName`)
|
||||
}
|
||||
const streamIdleTimeoutMs = source.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
|
||||
if (!Number.isFinite(streamIdleTimeoutMs)
|
||||
|| streamIdleTimeoutMs <= 0
|
||||
@@ -140,15 +196,33 @@ export function resolveProfiles(
|
||||
`llm-pi-ai: provider "${provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
const { apiKeyEnv, retryPolicy, ...rest } = source
|
||||
// The route key, not the installed provider's own name: the directory has
|
||||
// always shown route keys, and a catalog route must not silently rename
|
||||
// itself on every configuration surface just because it gained a profile.
|
||||
const displayName = source.displayName ?? provider
|
||||
const models = resolveRouteModels({
|
||||
provider,
|
||||
...source.api === undefined ? {} : { api: source.api },
|
||||
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
|
||||
...source.models === undefined ? {} : { models: source.models },
|
||||
})
|
||||
const { apiKeyEnv, retryPolicy, models: _models, displayName: _displayName, ...rest } = source
|
||||
resolved.set(provider, {
|
||||
...rest,
|
||||
provider,
|
||||
displayName,
|
||||
...apiKeyEnv === undefined ? {} : { apiKeyEnv: credentialRef(apiKeyEnv) },
|
||||
streamIdleTimeoutMs,
|
||||
retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`),
|
||||
...rest.headers === undefined ? {} : { headers: { ...rest.headers } },
|
||||
...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } },
|
||||
piProvider: buildProvider({
|
||||
provider,
|
||||
displayName,
|
||||
...source.api === undefined ? {} : { api: source.api },
|
||||
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
|
||||
models,
|
||||
}),
|
||||
})
|
||||
}
|
||||
return resolved
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
/**
|
||||
* Generic pi-ai-backed LLM adapter plugin. One plugin instance owns a dict of
|
||||
* provider routes; requests select a profile by provider and resolve the
|
||||
* model dynamically from pi-ai's installed catalog. Profile facts resolve per
|
||||
* request over the optional `llm-pi-ai` user-settings section and the
|
||||
* optional credential seam, so a changed key, endpoint, or knob reaches the
|
||||
* next request without a restart; a changed *route set* (or a route's
|
||||
* provider routes; a route naming an installed pi-ai provider inherits that
|
||||
* provider's endpoint, protocol, and model catalog as defaults, and a route
|
||||
* pi-ai does not ship is declared outright. Profile facts resolve per request
|
||||
* over the optional `llm-pi-ai` user-settings section and the optional
|
||||
* credential seam, so a changed key, endpoint, model, or knob reaches the next
|
||||
* request without a restart; a changed *route set* (or a route's
|
||||
* registration-captured retry policy) re-registers the same adapter instance
|
||||
* in place.
|
||||
*
|
||||
@@ -13,34 +14,48 @@
|
||||
* name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
* config:
|
||||
* providers:
|
||||
* # Catalog route: everything but the credential comes from pi-ai.
|
||||
* openai:
|
||||
* apiKeyEnv: OPENAI_API_KEY
|
||||
* retryPolicy:
|
||||
* mode: normal
|
||||
* maxRetries: 2
|
||||
* # Catalog route with the catalog narrowed and one capacity corrected.
|
||||
* anthropic:
|
||||
* apiKeyEnv: ANTHROPIC_API_KEY
|
||||
* openrouter:
|
||||
* apiKeyEnv: OPENROUTER_API_KEY
|
||||
* baseURL: https://proxy.example.com/v1
|
||||
* models:
|
||||
* - id: claude-sonnet-4-5
|
||||
* contextWindow: 200000
|
||||
* # Hand-declared route: pi-ai ships nothing under this key.
|
||||
* acme-gateway:
|
||||
* displayName: Acme Gateway
|
||||
* apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
* api: openai-completions
|
||||
* baseURL: https://gateway.acme.example/v1
|
||||
* models:
|
||||
* - id: acme-large
|
||||
* name: Acme Large
|
||||
* contextWindow: 65536
|
||||
* maxTokens: 4096
|
||||
* ```
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-pi-ai
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { AdapterRegistrationHandle } from '@deepseek-ai/dsh-llm'
|
||||
import type { AdapterRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
|
||||
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import { PiAiAdapter } from './adapter.ts'
|
||||
import { catalogProviderIds } from './catalog.ts'
|
||||
import { Config, resolveProfiles } from './config.ts'
|
||||
import type { ResolvedPiAiProviderProfile } from './config.ts'
|
||||
|
||||
export { PiAiAdapter } from './adapter.ts'
|
||||
export type { PiAiAdapterOptions } from './adapter.ts'
|
||||
export { Config } from './config.ts'
|
||||
export type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
export type { PiAiModelProfile, PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
export { supportedProtocols } from './provider.ts'
|
||||
|
||||
export const name = 'llm-pi-ai'
|
||||
export const inject = ['llm']
|
||||
@@ -58,6 +73,26 @@ function registrationFacts(profiles: ReadonlyMap<string, ResolvedPiAiProviderPro
|
||||
.sort((left, right) => left.provider.localeCompare(right.provider))
|
||||
}
|
||||
|
||||
/**
|
||||
* The configurable-provider directory: every installed catalog route, plus
|
||||
* every route the current profiles declare. A hand-declared route has no
|
||||
* catalog entry, so without this union it would have no settings address and
|
||||
* configuration surfaces could neither show nor edit it.
|
||||
* @param profiles - the currently resolved provider profiles.
|
||||
* @returns the directory entries in catalog order, declared routes last.
|
||||
*/
|
||||
function directoryEntries(
|
||||
profiles: ReadonlyMap<string, ResolvedPiAiProviderProfile>,
|
||||
): LlmConfigurableProvider[] {
|
||||
const entries = new Map<string, LlmConfigurableProvider>()
|
||||
const declare = (provider: string, displayName: string): void => {
|
||||
entries.set(provider, { provider, displayName, settingsNs: NS, settingsPath: ['providers', provider] })
|
||||
}
|
||||
for (const provider of catalogProviderIds()) declare(provider, provider)
|
||||
for (const [provider, profile] of profiles) declare(provider, profile.displayName)
|
||||
return [...entries.values()]
|
||||
}
|
||||
|
||||
/** Register one generic pi-ai adapter for all configured provider routes. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
let current: () => Config = () => config
|
||||
@@ -114,13 +149,18 @@ export function apply(ctx: Context, config: Config): void {
|
||||
const adapter = new PiAiAdapter({ profiles, resolveApiKey })
|
||||
// The full installed catalog is configurable from the moment the plugin
|
||||
// mounts — dormant or not — so configuration surfaces can offer every
|
||||
// pi-ai provider before any route exists.
|
||||
ctx.llm.registerConfigurableProviders(getBuiltinProviders().map(provider => ({
|
||||
provider,
|
||||
displayName: provider,
|
||||
settingsNs: NS,
|
||||
settingsPath: ['providers', provider],
|
||||
})))
|
||||
// pi-ai provider before any route exists. Hand-declared routes join it as
|
||||
// profiles appear, and leave with them.
|
||||
let directory: (() => void) | undefined
|
||||
let directoryFacts: unknown
|
||||
const ensureDirectory = (): void => {
|
||||
const entries = directoryEntries(profiles())
|
||||
if (deepEqualJson(entries, directoryFacts)) return
|
||||
directory?.()
|
||||
directory = ctx.llm.registerConfigurableProviders(entries)
|
||||
directoryFacts = entries
|
||||
}
|
||||
ensureDirectory()
|
||||
// Route effects bind to this apply fiber via the stable `ctx` reference,
|
||||
// even when a swap runs inside the scoped settings callback below. A bare
|
||||
// mount (zero routes) is the dormant posture: nothing registers until a
|
||||
@@ -156,6 +196,11 @@ export function apply(ctx: Context, config: Config): void {
|
||||
setSource: (source) => {
|
||||
current = source
|
||||
},
|
||||
onChange: ensureRegistrationFacts,
|
||||
onChange: () => {
|
||||
ensureRegistrationFacts()
|
||||
// The directory follows the profiles the registry accepted, so a route
|
||||
// that failed to register is not advertised as configurable.
|
||||
ensureDirectory()
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
155
packages/llm/llm-pi-ai/src/provider.ts
Normal file
155
packages/llm/llm-pi-ai/src/provider.ts
Normal file
@@ -0,0 +1,155 @@
|
||||
/**
|
||||
* Construction of the pi-ai `Provider` that one configured route registers into
|
||||
* the adapter's `Models` collection.
|
||||
*
|
||||
* Two constructions, one decision: a route the installed catalog ships, whose
|
||||
* profile does not override the wire protocol, **reuses that catalog provider**
|
||||
* with its models replaced — the catalog provider owns API implementations this
|
||||
* package cannot reconstruct (Bedrock loads its Smithy module through a
|
||||
* separate entry point), so rebuilding it from parts would silently narrow
|
||||
* which providers work. Every other route — one pi-ai has never heard of, or a
|
||||
* catalog route pointed at a different protocol — is built by `createProvider`
|
||||
* over the protocol table below.
|
||||
*
|
||||
* Credentials never reach this module's storage: the harness resolves a route's
|
||||
* key through `ctx.credentials` before the request enters pi-ai and hands it
|
||||
* over as a stream option, which `Models` presents to `resolve()` as the
|
||||
* credential key.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/provider
|
||||
*/
|
||||
|
||||
import { createProvider } from '@earendil-works/pi-ai'
|
||||
import type { Api, ApiKeyAuth, Model, Provider, ProviderStreams } from '@earendil-works/pi-ai'
|
||||
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy'
|
||||
import { azureOpenAIResponsesApi } from '@earendil-works/pi-ai/api/azure-openai-responses.lazy'
|
||||
import { bedrockConverseStreamApi } from '@earendil-works/pi-ai/api/bedrock-converse-stream.lazy'
|
||||
import { googleGenerativeAIApi } from '@earendil-works/pi-ai/api/google-generative-ai.lazy'
|
||||
import { googleVertexApi } from '@earendil-works/pi-ai/api/google-vertex.lazy'
|
||||
import { mistralConversationsApi } from '@earendil-works/pi-ai/api/mistral-conversations.lazy'
|
||||
import { openAICodexResponsesApi } from '@earendil-works/pi-ai/api/openai-codex-responses.lazy'
|
||||
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy'
|
||||
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy'
|
||||
import { piMessagesApi } from '@earendil-works/pi-ai/api/pi-messages.lazy'
|
||||
import { catalogProvider } from './catalog.ts'
|
||||
|
||||
/**
|
||||
* Wire protocols a configured route may name, mapped to pi-ai's lazily loaded
|
||||
* implementations. The table is pi-ai's own streaming API set: each entry is
|
||||
* the factory that pi-ai's matching provider factory uses, so a hand-declared
|
||||
* route reaches exactly the implementation a catalog route would.
|
||||
*/
|
||||
const PROTOCOLS: Readonly<Record<string, () => ProviderStreams>> = {
|
||||
'anthropic-messages': anthropicMessagesApi,
|
||||
'azure-openai-responses': azureOpenAIResponsesApi,
|
||||
'bedrock-converse-stream': bedrockConverseStreamApi,
|
||||
'google-generative-ai': googleGenerativeAIApi,
|
||||
'google-vertex': googleVertexApi,
|
||||
'mistral-conversations': mistralConversationsApi,
|
||||
'openai-codex-responses': openAICodexResponsesApi,
|
||||
'openai-completions': openAICompletionsApi,
|
||||
'openai-responses': openAIResponsesApi,
|
||||
'pi-messages': piMessagesApi,
|
||||
}
|
||||
|
||||
/**
|
||||
* Every wire protocol a configured route may name, sorted for stable
|
||||
* diagnostics and configuration surfaces.
|
||||
* @returns the supported protocol identifiers.
|
||||
*/
|
||||
export function supportedProtocols(): readonly string[] {
|
||||
return Object.keys(PROTOCOLS).sort()
|
||||
}
|
||||
|
||||
/**
|
||||
* Api-key auth for a route the harness authenticates itself. `Models` calls
|
||||
* this after the adapter has already resolved the route's credential, so a
|
||||
* missing key here is not this layer's failure: a named-but-unresolvable
|
||||
* reference has already failed the request with `MISSING_CREDENTIAL`, and a
|
||||
* route naming no credential at all is deliberately unauthenticated. Reporting
|
||||
* it as configured hands the decision to the protocol, which is where the
|
||||
* requirement actually lives — pi-ai's OpenAI-compatible implementation, for
|
||||
* one, still insists on a key or an `Authorization` header of its own.
|
||||
* @param name - display name used as the resolution's status label.
|
||||
* @returns the api-key auth for a harness-authenticated route.
|
||||
*/
|
||||
function harnessApiKeyAuth(name: string): ApiKeyAuth {
|
||||
return {
|
||||
name,
|
||||
resolve: ({ credential }) => Promise.resolve({
|
||||
auth: credential?.key === undefined ? {} : { apiKey: credential.key },
|
||||
source: name,
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/** The resolved route facts provider construction reads. */
|
||||
export interface ProviderSpec {
|
||||
/** Provider route key; also the `Models` collection key and each model's `provider`. */
|
||||
provider: string
|
||||
/** Display name for selectors and status labels. */
|
||||
displayName: string
|
||||
/** Wire protocol override; absent means each model keeps its catalog protocol. */
|
||||
api?: string
|
||||
/** Endpoint override already applied to {@link models}; kept for provider-level display. */
|
||||
baseURL?: string
|
||||
/** The route's materialized models, in configuration order. */
|
||||
models: readonly Model<Api>[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Reuse an installed catalog provider with this route's models and identity.
|
||||
* Model dispatch stays with the catalog provider, so its API implementations,
|
||||
* compatibility quirks, and ambient credential discovery are preserved exactly.
|
||||
* Catalog-owned dynamic refresh is dropped: this route's catalog is the
|
||||
* settings document, and a background refresh would contradict it.
|
||||
*/
|
||||
function reuseCatalogProvider(base: Provider, spec: ProviderSpec): Provider {
|
||||
// Provider-level `baseUrl` is display metadata: pi-ai routes every request
|
||||
// through `Model.baseUrl`, which model resolution has already overridden.
|
||||
const baseUrl = spec.baseURL ?? base.baseUrl
|
||||
return {
|
||||
id: spec.provider,
|
||||
name: spec.displayName,
|
||||
...baseUrl === undefined ? {} : { baseUrl },
|
||||
auth: base.auth,
|
||||
getModels: () => spec.models,
|
||||
// Delegated rather than copied: the catalog provider stays the receiver, so
|
||||
// an implementation holding state on itself keeps working.
|
||||
stream: (model, context, options) => base.stream(model, context, options),
|
||||
streamSimple: (model, context, options) => base.streamSimple(model, context, options),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the pi-ai provider for one resolved route.
|
||||
* @param spec - the resolved route facts.
|
||||
* @returns the provider to register in the adapter's `Models` collection.
|
||||
* @throws Error when the route names a wire protocol this build cannot serve.
|
||||
*/
|
||||
export function buildProvider(spec: ProviderSpec): Provider {
|
||||
const catalog = catalogProvider(spec.provider)
|
||||
// A catalog route keeping its catalog protocol reuses the catalog provider;
|
||||
// an explicit protocol means the deployment is repointing the route at a
|
||||
// different wire format, which only the protocol table can serve.
|
||||
if (catalog !== undefined && spec.api === undefined) return reuseCatalogProvider(catalog, spec)
|
||||
|
||||
// Every model on this path carries the route's protocol: model resolution
|
||||
// requires one for a route the catalog cannot default, and an explicit one
|
||||
// replaces each catalog model's own. So the route has a single API.
|
||||
const factory = spec.api === undefined ? undefined : PROTOCOLS[spec.api]
|
||||
if (factory === undefined) {
|
||||
throw new Error(
|
||||
`llm-pi-ai: provider "${spec.provider}" names api "${spec.api}", which this build cannot serve;`
|
||||
+ ` supported protocols are ${supportedProtocols().join(', ')}`,
|
||||
)
|
||||
}
|
||||
return createProvider({
|
||||
id: spec.provider,
|
||||
name: spec.displayName,
|
||||
...spec.baseURL === undefined ? {} : { baseUrl: spec.baseURL },
|
||||
auth: { apiKey: harnessApiKeyAuth(spec.displayName) },
|
||||
models: spec.models,
|
||||
api: factory(),
|
||||
})
|
||||
}
|
||||
@@ -400,12 +400,14 @@ describe('provider profile lifecycle', () => {
|
||||
expect(server.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('validates empty, unknown, legacy-shaped, and explicitly blank profiles', () => {
|
||||
it('validates empty, underspecified, legacy-shaped, and explicitly blank profiles', () => {
|
||||
// Empty and omitted dicts are the dormant zero-route posture, not errors.
|
||||
expect(resolveProfiles({}).size).toBe(0)
|
||||
expect(resolveProfiles(undefined).size).toBe(0)
|
||||
expect(() => resolveProfiles({ '': {} })).toThrow(/non-empty/)
|
||||
expect(() => resolveProfiles({ 'not-real': {} })).toThrow(/unknown/)
|
||||
// A route the installed catalog does not ship is allowed, but it has no
|
||||
// defaults to fall back on: it must describe its own models.
|
||||
expect(() => resolveProfiles({ 'not-real': {} })).toThrow(/resolves no models/)
|
||||
// The pre-release array shape and its per-profile provider field fail
|
||||
// loud with migration directions instead of half-working.
|
||||
expect(() => resolveProfiles([{ provider: 'openai' }] as never)).toThrow(/dict keyed by provider/)
|
||||
|
||||
302
packages/llm/llm-pi-ai/tests/catalog.spec.ts
Normal file
302
packages/llm/llm-pi-ai/tests/catalog.spec.ts
Normal file
@@ -0,0 +1,302 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { createUserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { buildProvider } from '../src/provider.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
|
||||
|
||||
afterEach(async () => { await closeMockServers() })
|
||||
|
||||
/** A complete hand-declared route: nothing about it exists in pi-ai's catalog. */
|
||||
function gateway(baseURL: string, overrides: Record<string, unknown> = {}): LlmPiAi.Config {
|
||||
return {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKey: 'gw-key',
|
||||
displayName: 'Acme Gateway',
|
||||
api: 'openai-completions',
|
||||
baseURL,
|
||||
models: [{ id: 'acme-large', name: 'Acme Large', contextWindow: 65_536, maxTokens: 4096 }],
|
||||
...overrides,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
async function harness(config: LlmPiAi.Config): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, config)
|
||||
return ctx
|
||||
}
|
||||
|
||||
describe('hand-declared providers', () => {
|
||||
it('serves a route pi-ai has never heard of from its own declaration', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(gateway(`${server.url}/v1`))
|
||||
|
||||
const result = await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model: 'acme-large',
|
||||
messages: [createUserMessage({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})],
|
||||
})
|
||||
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(server.paths).toEqual(['/v1/chat/completions'])
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer gw-key')
|
||||
})
|
||||
|
||||
it('lists and resolves the declared models rather than a catalog', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = await harness(gateway(`${server.url}/v1`))
|
||||
|
||||
expect(await ctx.llm.listModels('acme-gateway')).toEqual([
|
||||
{ provider: 'acme-gateway', id: 'acme-large', name: 'Acme Large' },
|
||||
])
|
||||
const info = await ctx.llm.resolveModelInfo('acme-gateway', 'acme-large')
|
||||
expect(info).toMatchObject({
|
||||
provider: 'acme-gateway',
|
||||
id: 'acme-large',
|
||||
name: 'Acme Large',
|
||||
context: { contextWindow: 65_536 },
|
||||
defaultMaxTokens: 4096,
|
||||
})
|
||||
})
|
||||
|
||||
it('joins the configurable-provider directory so a settings surface can reach it', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = await harness(gateway(`${server.url}/v1`))
|
||||
|
||||
expect(ctx.llm.listConfigurableProviders()).toContainEqual({
|
||||
provider: 'acme-gateway',
|
||||
displayName: 'Acme Gateway',
|
||||
settingsNs: 'llm-pi-ai',
|
||||
settingsPath: ['providers', 'acme-gateway'],
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects a model whose capacity the catalog cannot supply', () => {
|
||||
const declare = (model: LlmPiAi.PiAiModelProfile): (() => unknown) =>
|
||||
() => resolveProfiles({ 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models: [model] } })
|
||||
|
||||
expect(declare({ id: 'acme-large', maxTokens: 1 })).toThrow(/needs a contextWindow/)
|
||||
expect(declare({ id: 'acme-large', contextWindow: 1 })).toThrow(/needs a maxTokens/)
|
||||
expect(declare({ id: '', contextWindow: 1, maxTokens: 1 })).toThrow(/empty id/)
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'dup', contextWindow: 1, maxTokens: 1 }, { id: 'dup', contextWindow: 2, maxTokens: 2 }],
|
||||
},
|
||||
})).toThrow(/more than once/)
|
||||
})
|
||||
|
||||
it('rejects a declaration that names no wire protocol or endpoint', () => {
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': { baseURL: 'https://acme.test', models: [{ id: 'm', contextWindow: 1, maxTokens: 1 }] },
|
||||
})).toThrow(/needs an api/)
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': { api: 'openai-completions', models: [{ id: 'm', contextWindow: 1, maxTokens: 1 }] },
|
||||
})).toThrow(/needs a baseURL/)
|
||||
})
|
||||
|
||||
it('rejects a protocol this build cannot serve, and a route that names none', () => {
|
||||
const spec = { provider: 'acme-gateway', displayName: 'Acme Gateway', models: [] }
|
||||
expect(() => buildProvider({ ...spec, api: 'quantum-telepathy' }))
|
||||
.toThrow(/cannot serve; supported protocols are/)
|
||||
expect(() => buildProvider(spec)).toThrow(/cannot serve; supported protocols are/)
|
||||
})
|
||||
|
||||
it('leaves an unauthenticated route to its protocol rather than inventing a credential', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
// Naming no credential is the deliberately unauthenticated posture — a
|
||||
// named reference that resolved to nothing would have failed with
|
||||
// MISSING_CREDENTIAL long before this point. The route resolves as
|
||||
// configured and the protocol decides: pi-ai's OpenAI-compatible
|
||||
// implementation wants a key or an Authorization header of its own, and
|
||||
// says so instead of the harness guessing a placeholder.
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
'local-llm': {
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [{ id: 'qwen3', contextWindow: 32_768, maxTokens: 2048 }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const result = await assemble(ctx, { provider: 'local-llm', model: 'qwen3', messages: [] })
|
||||
expect(result.finish).toMatchObject({
|
||||
kind: 'error',
|
||||
failure: { message: 'No API key for provider: local-llm' },
|
||||
})
|
||||
expect(server.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('authenticates an unauthenticated route through a configured header', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
'local-llm': {
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
headers: { Authorization: 'Bearer local' },
|
||||
models: [{ id: 'qwen3', contextWindow: 32_768, maxTokens: 2048 }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const result = await assemble(ctx, { provider: 'local-llm', model: 'qwen3', messages: [] })
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer local')
|
||||
})
|
||||
|
||||
it('rejects a capacity that is not a positive integer', () => {
|
||||
const declare = (model: LlmPiAi.PiAiModelProfile): (() => unknown) =>
|
||||
() => resolveProfiles({ 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models: [model] } })
|
||||
|
||||
expect(declare({ id: 'm', contextWindow: 0, maxTokens: 1 })).toThrow(/contextWindow must be a positive integer/)
|
||||
expect(declare({ id: 'm', contextWindow: 1.5, maxTokens: 1 })).toThrow(/contextWindow must be a positive integer/)
|
||||
expect(declare({ id: 'm', contextWindow: 1, maxTokens: 0 })).toThrow(/maxTokens must be a positive integer/)
|
||||
expect(declare({ id: 'm', contextWindow: 1, maxTokens: 1.5 })).toThrow(/maxTokens must be a positive integer/)
|
||||
})
|
||||
|
||||
it('names the route key when no displayName is configured', () => {
|
||||
const resolved = resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'm', contextWindow: 1, maxTokens: 1 }],
|
||||
},
|
||||
})
|
||||
expect(resolved.get('acme-gateway')?.displayName).toBe('acme-gateway')
|
||||
expect(() => resolveProfiles({ 'acme-gateway': { displayName: '' } })).toThrow(/empty displayName/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('catalog routes with per-model configuration', () => {
|
||||
it('serves the installed catalog untouched when the profile lists no models', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = await harness({ providers: { deepseek: { apiKey: 'k', baseURL: server.url } } })
|
||||
|
||||
const listed = await ctx.llm.listModels('deepseek')
|
||||
expect(listed.map(model => model.id).sort())
|
||||
.toEqual(getBuiltinModels('deepseek').map(model => model.id).sort())
|
||||
})
|
||||
|
||||
it('overrides one catalog model field and defaults the rest from the catalog', async () => {
|
||||
const server = await mockServer([])
|
||||
const [catalogModel] = getBuiltinModels('deepseek')
|
||||
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
deepseek: {
|
||||
apiKey: 'k',
|
||||
baseURL: server.url,
|
||||
models: [{ id: catalogModel.id, contextWindow: 4096 }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const info = await ctx.llm.resolveModelInfo('deepseek', catalogModel.id)
|
||||
// The configured field wins; name and output cap still come from the catalog.
|
||||
expect(info.context).toEqual({ contextWindow: 4096 })
|
||||
expect(info.name).toBe(catalogModel.name)
|
||||
expect(info.defaultMaxTokens).toBe(catalogModel.maxTokens)
|
||||
// An explicit list replaces the catalog rather than adding to it.
|
||||
expect((await ctx.llm.listModels('deepseek')).map(model => model.id)).toEqual([catalogModel.id])
|
||||
})
|
||||
|
||||
it('adds a model the installed catalog does not describe to a catalog route', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
deepseek: {
|
||||
apiKey: 'k',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [{ id: 'deepseek-preview', contextWindow: 200_000, maxTokens: 8192 }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const result = await assemble(ctx, { provider: 'deepseek', model: 'deepseek-preview', messages: [] })
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
// The catalog route keeps its catalog protocol, so the new model reaches
|
||||
// the same endpoint shape the shipped models use.
|
||||
expect(server.paths).toEqual(['/v1/chat/completions'])
|
||||
})
|
||||
|
||||
it('fails an unconfigured model id before any provider request', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
deepseek: { apiKey: 'k', baseURL: server.url, models: [{ id: 'deepseek-preview', contextWindow: 1, maxTokens: 1 }] },
|
||||
},
|
||||
})
|
||||
|
||||
await expect(assemble(ctx, { provider: 'deepseek', model: 'not-configured', messages: [] }))
|
||||
.rejects.toMatchObject({ code: 'UNKNOWN_MODEL' })
|
||||
expect(server.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('preserves catalog-only model metadata the profile cannot express', () => {
|
||||
// Some catalog models carry provider-required request headers; overriding a
|
||||
// capacity must not drop them, because configuration has no way to restate
|
||||
// them.
|
||||
const headered = (getBuiltinModels('nvidia') as { id: string; headers?: unknown }[])
|
||||
.find(model => model.headers !== undefined)
|
||||
if (headered === undefined) throw new Error('the installed catalog ships no nvidia model with headers')
|
||||
|
||||
const resolved = resolveProfiles({
|
||||
nvidia: { models: [{ id: headered.id, contextWindow: 4096 }] },
|
||||
})
|
||||
const [model] = resolved.get('nvidia')?.piProvider.getModels() ?? []
|
||||
expect(model?.headers).toEqual(headered.headers)
|
||||
expect(model?.contextWindow).toBe(4096)
|
||||
})
|
||||
|
||||
it('keeps each model its own endpoint when the catalog route declares none', () => {
|
||||
// `opencode` ships no provider-level endpoint: the address lives on every
|
||||
// catalog model, so the route resolves without any configured baseURL.
|
||||
const resolved = resolveProfiles({ opencode: {} })
|
||||
const models = resolved.get('opencode')?.piProvider.getModels() ?? []
|
||||
expect(models.length).toBeGreaterThan(0)
|
||||
expect(models.every(model => model.baseUrl.length > 0)).toBe(true)
|
||||
expect(resolved.get('opencode')?.piProvider.baseUrl).toBeUndefined()
|
||||
})
|
||||
|
||||
it('repoints a catalog route at another wire protocol without restating its endpoint', () => {
|
||||
const resolved = resolveProfiles({ openai: { api: 'openai-completions' } })
|
||||
const models = resolved.get('openai')?.piProvider.getModels() ?? []
|
||||
// The protocol changes for the whole route; each model keeps the catalog
|
||||
// endpoint it already had.
|
||||
expect(models.every(model => model.api === 'openai-completions')).toBe(true)
|
||||
expect(models.every(model => model.baseUrl === 'https://api.openai.com/v1')).toBe(true)
|
||||
})
|
||||
|
||||
it('repoints a catalog route at another wire protocol', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness({
|
||||
providers: {
|
||||
// openai's catalog models speak the Responses API; naming the protocol
|
||||
// explicitly moves the whole route onto Chat Completions.
|
||||
openai: {
|
||||
apiKey: 'k',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [{ id: 'gpt-4.1', contextWindow: 100_000, maxTokens: 4096 }],
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
|
||||
expect(server.paths).toEqual(['/v1/chat/completions'])
|
||||
})
|
||||
})
|
||||
@@ -1,41 +1,74 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
const streamSimple = vi.hoisted(() => vi.fn())
|
||||
|
||||
// The 0.81 SDK moved `streamSimple` to the compat entry; the adapter imports it
|
||||
// from there, so the mock must target the same specifier.
|
||||
vi.mock('@earendil-works/pi-ai/compat', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@earendil-works/pi-ai/compat')>()
|
||||
return { ...actual, streamSimple }
|
||||
})
|
||||
// A hand-declared route is built by `createProvider` over the protocol table in
|
||||
// `src/provider.ts`, so the table's lazy api module is the SDK boundary this
|
||||
// test can observe. A catalog route dispatches through pi-ai's own provider and
|
||||
// would not see this mock.
|
||||
vi.mock('@earendil-works/pi-ai/api/openai-completions.lazy', () => ({
|
||||
openAICompletionsApi: () => ({ stream: streamSimple, streamSimple }),
|
||||
}))
|
||||
|
||||
import { PiAiAdapter } from '../src/adapter.ts'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
|
||||
afterEach(() => { streamSimple.mockReset() })
|
||||
|
||||
/** A hand-declared OpenAI-compatible route with one fully described model. */
|
||||
function gatewayAdapter(): PiAiAdapter {
|
||||
return new PiAiAdapter({
|
||||
profiles: () => resolveProfiles({
|
||||
'local-gateway': {
|
||||
apiKey: 'test-key',
|
||||
api: 'openai-completions',
|
||||
baseURL: 'http://127.0.0.1:9/v1',
|
||||
models: [{ id: 'local-model', contextWindow: 8192, maxTokens: 1024 }],
|
||||
},
|
||||
}),
|
||||
resolveApiKey: () => Promise.resolve('test-key'),
|
||||
})
|
||||
}
|
||||
|
||||
async function drain(adapter: PiAiAdapter): Promise<StreamChunk[]> {
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of adapter.stream({
|
||||
provider: 'local-gateway',
|
||||
model: 'local-model',
|
||||
messages: [],
|
||||
})) chunks.push(chunk)
|
||||
return chunks
|
||||
}
|
||||
|
||||
describe('pi-ai SDK retry boundary', () => {
|
||||
it('pins one SDK attempt even when the installed provider currently defaults to zero retries', async () => {
|
||||
const failure = new Error('mock SDK boundary')
|
||||
streamSimple.mockReturnValue({
|
||||
async * [Symbol.asyncIterator](): AsyncGenerator<never> {
|
||||
throw failure
|
||||
},
|
||||
})
|
||||
const adapter = new PiAiAdapter({
|
||||
profiles: () => resolveProfiles({ openai: { apiKey: 'test-key' } }),
|
||||
resolveApiKey: () => Promise.resolve('test-key'),
|
||||
})
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({
|
||||
provider: 'openai',
|
||||
model: 'gpt-4.1',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
}
|
||||
streamSimple.mockImplementation(() => { throw new Error('mock SDK boundary') })
|
||||
|
||||
const chunks = await drain(gatewayAdapter())
|
||||
|
||||
await expect(drain()).rejects.toBe(failure)
|
||||
expect(streamSimple).toHaveBeenCalledOnce()
|
||||
expect(streamSimple.mock.calls[0]?.[2]).toMatchObject({ maxRetries: 0 })
|
||||
expect(streamSimple.mock.calls[0]?.[2]).toMatchObject({ maxRetries: 0, apiKey: 'test-key' })
|
||||
// pi-ai reports a setup failure as a terminal in-stream error rather than
|
||||
// throwing, which the converter turns into the harness error finish.
|
||||
expect(chunks.at(-1)).toMatchObject({
|
||||
type: 'finish',
|
||||
reason: { kind: 'error', failure: { message: 'mock SDK boundary' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('dispatches a hand-declared route to the endpoint and model its configuration describes', async () => {
|
||||
streamSimple.mockImplementation(() => { throw new Error('mock SDK boundary') })
|
||||
|
||||
await drain(gatewayAdapter())
|
||||
|
||||
expect(streamSimple.mock.calls[0]?.[0]).toMatchObject({
|
||||
id: 'local-model',
|
||||
provider: 'local-gateway',
|
||||
api: 'openai-completions',
|
||||
baseUrl: 'http://127.0.0.1:9/v1',
|
||||
contextWindow: 8192,
|
||||
maxTokens: 1024,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user