Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(credentials,llm): remove speculative mutation and route lifecycle

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-31 01:08:54 +08:00
parent 16802cd612
commit f9f8148e79
100 changed files with 559 additions and 2752 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
packages/llm/llm-deepseek/README.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
README.md: ab44b61e300ca65cc4dd3507ad7262cd08edcfce
README.zh.md: 4ecaf361fdb396f9f8079476240b5e9353a73f5e
README.md: 8532dab4731e25b4af777217ad7c5ffad521d924
README.zh.md: 922fb50ae063ae0a1f90db3b915dc55096448a5c

12
packages/llm/llm-deepseek/README.md
View File

@@ -47,12 +47,12 @@ The same exact-model result exposes ordered `off`, `high`, and `max` efforts und
## Dynamic configuration (settings + credentials)
Connection facts are not frozen at load. `resolveAdapterOptions` is the one explicit resolve step from raw config to validated facts, and the adapter re-reads them through a thunk **once per operation**: base URL, catalog, request defaults, and idle budget all take effect on the next request, while an in-flight stream keeps the facts it started with. Two optional seams feed that thunk:
Request facts are not frozen at load. `resolveAdapterOptions` is the one explicit resolve step from raw config to validated facts, and the adapter re-reads them through a thunk **once per operation**: base URL, catalog, request defaults, and idle budget take effect on the next operation, while an in-flight stream keeps the facts it started with. The `deepseek` route and its retry policy remain fixed by the plugin composition. Two optional seams feed the request facts:
- **`ctx.settings`** — the plugin registers the `llm-deepseek` namespace with this same `Config` schema and its `cordis.yml` entry as the composition `base`, so a `llm-deepseek:` section in the user settings document overrides any field without a restart. Without a mounted settings service the entry config alone drives the adapter, unchanged. A live settings snapshot that passes the schema but fails a beyond-schema bound (a duplicate catalog id, a broken thinking/effort pair) keeps the last good facts and logs the failure; the entry config itself still fails plugin load.
- **`ctx.credentials`** — the API key resolves per stream call, from the *same* resolved snapshot that supplies the endpoint: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the credential seam (`$DSH_HOME/.env` under the live environment), then — only without a mounted seam — the raw environment variable. Because credential facts travel with the connection facts, a settings snapshot the resolver rejects contributes neither its endpoint nor its key: the whole previous generation keeps serving. A request with no key anywhere fails with `MISSING_CREDENTIAL` naming every configuration entry point, while the route stays registered and the catalog stays browsable — first-run onboarding is "browse models, store the key, prompt again", with no restart between.
- **`ctx.settings`** — the plugin registers the `llm-deepseek` namespace with this same `Config` schema and its `cordis.yml` entry as the composition `base`. Without a mounted settings service the entry config alone drives the adapter. A live settings snapshot that passes the schema but fails a beyond-schema bound (a duplicate catalog id, a broken thinking/effort pair) keeps the last good request facts and logs the failure; the entry config itself still fails plugin load.
- **`ctx.credentials`** — the API key resolves per stream call, from the *same* resolved snapshot that supplies the endpoint: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the credential seam (`$DSH_HOME/.env` under the live environment), then — only without a mounted seam — the raw environment variable. Because credential facts travel with the connection facts, a rejected settings snapshot contributes neither its endpoint nor its key. A request with no key anywhere fails with `MISSING_CREDENTIAL`; after the operator supplies the named environment or dotenv value, the next request resolves it without a restart.
The one registration-captured fact is the retry policy: when its resolved value changes, the plugin re-registers the route in place (same adapter instance, one synchronous section), so `ctx.llm.providerRetryPolicy('deepseek')` always reports the current policy.
`ctx.llm.providerRetryPolicy('deepseek')` reports the policy captured from the composition entry at registration.
## App attribution
@@ -72,7 +72,7 @@ Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA`
## Testing
Unit suites run against a local `node:http` mock SSE server (no network), including dynamic `high`/`off`/`max` selection, structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers (next-request base-URL/key pickup, literal precedence, keyless onboarding, last-good snapshots, retry-policy re-registration), and `tests/loader-composition.spec.ts` boots the full chain from a test-only `cordis.yml` through the actual Loader and edits `settings.yaml`/`.env` on disk. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback and a request whose key exists only in a credentials-local document.
Unit suites run against a local `node:http` mock SSE server (no network), including dynamic `high`/`off`/`max` selection, structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers (next-request base-URL/key pickup, literal precedence, keyless onboarding, last-good snapshots, and composition-fixed retry policy), and `tests/loader-composition.spec.ts` boots the full chain from a test-only `cordis.yml` through the actual Loader and edits `settings.yaml`/`.env` on disk. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback and a request whose key exists only in a credentials-local document.
## Model Experience
@@ -107,7 +107,7 @@ Loop-retained response blocks append to the next request and preserve its earlie
## Known Limitations and Deferred Work
- **A settings `models` list replaces the composition list wholesale** — settings-layer merging is per-field, and arrays are one field; per-entry catalog merging would need a keyed shape.
- **`Config.apiKey` is schema-tagged `role('secret')` but not yet masked anywhere** — the settings `describe()` envelope returns values verbatim; the wire/UI layer that must redact secret-role fields ships with the settings RPC surface.
- **`Config.apiKey` is schema-tagged `role('secret')` but not masked by `ctx.settings.describe()`** — do not expose that envelope to an untrusted UI without redacting secret-role fields.
- **`tool_choice` is not mapped** — not part of the core vocabulary (MVP cut, shared with the pi-ai twin).
- **Requests use raw `fetch`, not `@cordisjs/plugin-http`** — no shared proxy/interception configuration; adoption is deferred until a second adapter wants it (`TODO(http)`).
- **Serialization flattens user and tool-result content to text blocks** — plugin-added block types are skipped, and empty tool output crosses the wire as the literal `(no output)`.

12
packages/llm/llm-deepseek/README.zh.md
View File

@@ -47,12 +47,12 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
## 动态配置（settings + credentials）
连接事实不在加载时冻结。`resolveAdapterOptions` 是从原始配置到已校验事实的唯一显式 resolve 步骤，适配器经由一个 thunk **每操作重读一次**：base URL、catalog、请求默认值与 idle 预算都在下一次请求生效，进行中的流则保持其起始事实。两个可选 seam 供给该 thunk：
请求事实不在加载时冻结。`resolveAdapterOptions` 是从原始配置到已校验事实的唯一显式 resolve 步骤，适配器经由一个 thunk **每操作重读一次**：base URL、catalog、请求默认值与 idle 预算都在下一次操作生效，进行中的流则保持其起始事实。`deepseek` 路由及其重试策略始终由插件组合固定。两个可选 seam 为请求事实供值：
- **`ctx.settings`**——插件用同一份 `Config` schema 注册 `llm-deepseek` namespace，并以其 `cordis.yml` 条目为组合 `base`，因此用户设置文档中的 `llm-deepseek:` 分节可以免重启覆盖任何字段。未挂载 settings 服务时，仅由 entry 配置驱动适配器，行为不变。存活 settings 快照若通过 schema 却违反 schema 之外的约束（重复的 catalog id、无法成立的 thinking／推理强度组合），则保留最后可用事实并记录失败；entry 配置本身仍会使插件加载失败。
- **`ctx.credentials`**——API 密钥按每次 stream 调用解析，取自与端点*同一*份解析后的快照：非空的字面 `apiKey` 优先，其次经凭据 seam 解析 `apiKeyEnv`（活跃环境之下的 `$DSH_HOME/.env`），最后——仅在未挂载 seam 时——读取原始环境变量。由于凭据事实与连接事实同行，被 resolver 拒绝的 settings 快照既不贡献自己的端点，也不贡献自己的密钥：整个先前世代继续服务。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败，并点名每个配置入口，同时路由保持注册、catalog 保持可浏览——首次运行的上手流程就是「浏览模型、存入密钥、再次发起提示」，中间无需任何重启。
- **`ctx.settings`**：插件用同一份 `Config` schema 注册 `llm-deepseek` namespace，并以其 `cordis.yml` 条目为组合 `base`。未挂载 settings 服务时，仅由 entry 配置驱动适配器。存活 settings 快照若通过 schema 却违反 schema 之外的约束（重复的 catalog id、无法成立的 thinking／推理强度组合），则保留最后可用的请求事实并记录失败；entry 配置本身仍会使插件加载失败。
- **`ctx.credentials`**：API 密钥按每次 stream 调用解析，取自与端点*同一*份解析后的快照：非空的字面 `apiKey` 优先，其次经凭据 seam 解析 `apiKeyEnv`（活跃环境之下的 `$DSH_HOME/.env`），最后仅在未挂载 seam 时读取原始环境变量。由于凭据事实与连接事实同行，被拒绝的 settings 快照既不贡献自己的端点，也不贡献自己的密钥。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败；操作者为点名的环境变量或 dotenv 值供值后，下一次请求无需重启即可解析它。
唯一在注册期捕获的事实是重试策略：其解析值变化时，插件原地重新注册该路由（同一适配器实例、一个同步区段），因此 `ctx.llm.providerRetryPolicy('deepseek')` 始终报告当前策略。
`ctx.llm.providerRetryPolicy('deepseek')` 报告注册时从组合配置项捕获的策略。
## 应用归因
@@ -72,7 +72,7 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
## 测试
单元套件使用本地 `node:http` mock SSE 服务器（无网络），覆盖动态 `high`／`off`／`max` 选择、结构化 HTTP 事实、格式错误／截断流、调用方 abort、连接失败，以及 idle 超时确实会 abort 实际 body 的证明。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider（下一请求即生效的 base-URL／密钥拾取、字面值优先、无密钥上手、最后可用快照、重试策略重注册），`tests/loader-composition.spec.ts` 则从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起完整链路，并在磁盘上编辑 `settings.yaml`/`.env`。真实 API 覆盖位于 `tests/adapter.e2e.ts`（`pnpm run test:e2e`，需有 key 才会运行）：V4 Flash + V4 Pro，覆盖思考启用／禁用与两种官方 effort 级别，包括思考 + 工具往返与推理回传，以及密钥仅存在于 credentials-local 文档中的请求。
单元套件使用本地 `node:http` mock SSE 服务器（无网络），覆盖动态 `high`／`off`／`max` 选择、结构化 HTTP 事实、格式错误／截断流、调用方 abort、连接失败，以及 idle 超时确实会 abort 实际 body 的证明。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider（下一请求即生效的 base-URL／密钥拾取、字面值优先、无密钥上手、最后可用快照，以及由组合固定的重试策略），`tests/loader-composition.spec.ts` 则从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起完整链路，并在磁盘上编辑 `settings.yaml`/`.env`。真实 API 覆盖位于 `tests/adapter.e2e.ts`（`pnpm run test:e2e`，需有 key 才会运行）：V4 Flash + V4 Pro，覆盖思考启用／禁用与两种官方 effort 级别，包括思考 + 工具往返与推理回传，以及密钥仅存在于 credentials-local 文档中的请求。
## 模型体验
@@ -107,7 +107,7 @@ loop 保留的响应块会追加到下一个请求，并保留其较早可复用
## 已知限制与暂缓事项
- **settings 的 `models` 列表会整体替换组合列表**：settings 层按字段合并，而数组是单个字段；按条目合并 catalog 需要带键的形状。
- **`Config.apiKey` 已在 schema 中标注 `role('secret')`，但尚未在任何地方脱敏**：settings 的 `describe()` 信封原样返回值；负责对 secret 角色字段脱敏的 wire／UI 层将随 settings RPC 面一起交付。
- **`Config.apiKey` 已在 schema 中标注 `role('secret')`，但未由 `ctx.settings.describe()` 脱敏**：在对 secret 角色字段脱敏之前，不要向不受信任的 UI 暴露该信封。
- **未映射 `tool_choice`**：它不属于核心词汇（MVP 取舍，与 pi-ai twin 共享）。
- **请求使用原始 `fetch`，而非 `@cordisjs/plugin-http`**：没有共享 proxy／拦截配置；采用暂缓到第二个适配器需要该功能时（`TODO(http)`）。
- **序列化会将 user 与工具结果内容展平为文本块**：会跳过插件添加的块类型，空工具输出会以字面 `(no output)` 通过协议发送。

37
packages/llm/llm-deepseek/src/index.ts
View File

@@ -6,8 +6,7 @@
* key through the optional credential seam (`ctx.credentials`), so a changed
* base URL, catalog, or key reaches the very next request without restarting
* anything, while an in-flight stream keeps the facts it started with. The
* one registration-captured fact — the retry policy — re-registers the route
* in place when it changes.
* registration-captured facts stay composition-fixed.
* @module @deepseek-ai/dsh-llm-deepseek
*/
@@ -16,7 +15,7 @@ import z from 'schemastery'
import { LlmError, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
import { credentialRef } from '@deepseek-ai/dsh-credentials'
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import { installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
import type { DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.ts'
@@ -167,12 +166,13 @@ export function resolveAdapterOptions(config: Config): ResolvedDeepSeekOptions {
}
export function apply(ctx: Context, config: Config): void {
const compositionOptions = resolveAdapterOptions(config)
let current: () => Config = () => config
let lastRaw: Config | undefined
let lastGood: ResolvedDeepSeekOptions | undefined
let lastRaw: Config = config
let lastGood = compositionOptions
const options = (): ResolvedDeepSeekOptions => {
const raw = current()
if (raw === lastRaw && lastGood !== undefined) return lastGood
if (raw === lastRaw) return lastGood
try {
const next = resolveAdapterOptions(raw)
lastRaw = raw
@@ -182,14 +182,12 @@ export function apply(ctx: Context, config: Config): void {
// Static composition resolves before anything registers, so this branch
// only sees a live settings snapshot failing a beyond-schema bound:
// keep serving the last good facts and say so once per bad snapshot.
if (lastGood === undefined) throw error
lastRaw = raw
ctx.logger.error('llm-deepseek: keeping the last good configuration after an invalid settings section')
ctx.logger.error(error)
return lastGood
}
}
options()
const resolveApiKey = async (connection: ResolvedDeepSeekOptions): Promise<string> => {
// Every credential fact comes from the caller's snapshot, so a rejected
@@ -199,7 +197,7 @@ export function apply(ctx: Context, config: Config): void {
const credentials = ctx.get('credentials')
if (credentials !== undefined) {
const hit = await credentials.resolve(ref)
if (hit !== undefined) return hit.value
if (hit !== undefined) return hit
} else {
// Without the seam, keep the historical ambient fallback so a plain
// cordis.yml composition works from the environment alone.
@@ -207,33 +205,18 @@ export function apply(ctx: Context, config: Config): void {
if (ambient !== undefined && ambient.length > 0) return ambient
}
throw new LlmError(
`llm-deepseek: no API key for provider route "${PROVIDER}"; store ${ref} through the credentials`
+ ` service (the web Models page writes it), export ${ref} in the launching environment, or — as a`
+ ' last resort — set a literal "apiKey" in the llm-deepseek settings section',
`llm-deepseek: no API key for provider route "${PROVIDER}"; provide ${ref} through the credential`
+ ' provider or launching environment, or set a literal "apiKey" in the llm-deepseek settings section',
'MISSING_CREDENTIAL',
)
}
const adapter = new DeepSeekAdapter({ options, resolveApiKey })
// Route effects bind to this apply fiber via the stable `ctx` reference,
// even when a swap runs inside the scoped settings callback below.
let disposeRoute = ctx.llm.registerAdapter([PROVIDER], adapter)
let registeredPolicy = options().retryPolicy
const ensureRegistrationFacts = (): void => {
const policy = options().retryPolicy
if (deepEqualJson(policy, registeredPolicy)) return
// The registry captures the retry policy at registration, so it is the one
// fact per-request resolution cannot refresh: swap the registration in one
// synchronous section (same adapter instance, no NO_ADAPTER window).
disposeRoute()
disposeRoute = ctx.llm.registerAdapter([PROVIDER], adapter)
registeredPolicy = policy
}
ctx.llm.registerAdapter([PROVIDER], adapter)
installSettingsSection(ctx, NS, Config, config, {
setSource: (source) => {
current = source
},
onChange: ensureRegistrationFacts,
})
}

2
packages/llm/llm-deepseek/tests/adapter.e2e.ts
View File

@@ -69,7 +69,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(LlmService)
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env'), watch: false })
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env') })
await ctx.plugin(LlmDeepSeek, {})
const result = await assemble(ctx, {

5
packages/llm/llm-deepseek/tests/adapter.spec.ts
View File

@@ -817,10 +817,9 @@ describe('plugin registration and config', () => {
await expect(ctx.llm.listModels('deepseek')).resolves.toHaveLength(2)
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
.rejects.toMatchObject({ code: 'MISSING_CREDENTIAL' })
// The guidance leads with the credential store — the path that keeps the
// secret out of configuration files — and mentions a literal key last.
// The guidance names real external sources and the literal escape hatch.
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
.rejects.toThrow(/store DEEPSEEK_API_KEY through the credentials service.*as a last resort.*"apiKey"/s)
.rejects.toThrow(/provide DEEPSEEK_API_KEY through the credential provider.*"apiKey"/s)
})
it('reads the ambient variable when no credentials seam is mounted', async () => {

26
packages/llm/llm-deepseek/tests/dynamic-config.spec.ts
View File

@@ -4,7 +4,6 @@ import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import LlmService from '@deepseek-ai/dsh-llm'
import { credentialRef } from '@deepseek-ai/dsh-credentials'
import { CredentialsLocal } from '@deepseek-ai/dsh-credentials-local'
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
import { SettingsLocal } from '@deepseek-ai/dsh-settings-local'
@@ -13,7 +12,6 @@ import { assemble } from './assemble.ts'
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
const NS = settingsNamespace('llm-deepseek')
const KEY_REF = credentialRef('DEEPSEEK_API_KEY')
const cleanups: Array<() => Promise<void>> = []
@@ -36,9 +34,8 @@ interface Harness {
/**
* Real dynamic composition: llm + settings-local + credentials-local +
* llm-deepseek over one temp harness home. `watch: false` keeps every change
* flowing through the in-process write path, which is deterministic; external
* file watching is the providers' own covered concern.
* llm-deepseek over one temp harness home. Settings updates use their owning
* write path; credentials are edited externally and read on demand.
*/
async function boot(dir: string, config: object): Promise<Harness> {
const ctx = new Context()
@@ -48,7 +45,7 @@ async function boot(dir: string, config: object): Promise<Harness> {
await ctx.plugin(LlmService)
const settingsFiber = ctx.plugin(SettingsLocal, { path: join(dir, 'settings.yaml'), watch: false })
await settingsFiber
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env'), watch: false })
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env') })
await ctx.plugin(LlmDeepSeek, config)
return { ctx, settingsFiber }
}
@@ -70,7 +67,7 @@ describe('request-level dynamic configuration', () => {
expect(serverA.headers[0]?.authorization).toBe('Bearer first-key')
await ctx.settings.update(NS, { baseURL: serverB.url })
await ctx.credentials.set(KEY_REF, 'second-key')
await writeFile(join(dir, '.env'), 'DEEPSEEK_API_KEY=second-key\n')
await prompt(ctx)
// No restart, no re-registration: the next request resolved both facts.
@@ -97,7 +94,7 @@ describe('request-level dynamic configuration', () => {
const { ctx } = await boot(dir, { baseURL: server.url })
await expect(prompt(ctx)).rejects.toMatchObject({ code: 'MISSING_CREDENTIAL' })
await ctx.credentials.set(KEY_REF, 'sk-arrived')
await writeFile(join(dir, '.env'), 'DEEPSEEK_API_KEY=sk-arrived\n')
await prompt(ctx)
expect(server.headers[0]?.authorization).toBe('Bearer sk-arrived')
})
@@ -113,12 +110,19 @@ describe('request-level dynamic configuration', () => {
])
})
it('re-registers the route in place when the captured retry policy changes', async () => {
it('keeps the registration retry policy composition-fixed', async () => {
const dir = await home()
const { ctx } = await boot(dir, { apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
const { ctx } = await boot(dir, {
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
retryPolicy: {
mode: 'always',
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
},
})
await ctx.settings.update(NS, {
retryPolicy: { mode: 'always', backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 } },
retryPolicy: { mode: 'normal', maxRetries: 0 },
})
expect(ctx.llm.providerRetryPolicy('deepseek')).toEqual({
mode: 'always',

24
packages/llm/llm-deepseek/tests/loader-composition.spec.ts
View File

@@ -17,7 +17,6 @@ import { Context } from 'cordis'
import Loader from '@cordisjs/plugin-loader'
import Include from '@cordisjs/plugin-include'
import LlmService from '@deepseek-ai/dsh-llm'
import { credentialRef } from '@deepseek-ai/dsh-credentials'
import CredentialsLocal from '@deepseek-ai/dsh-credentials-local'
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
import SettingsLocal from '@deepseek-ai/dsh-settings-local'
@@ -26,7 +25,6 @@ import { assemble } from './assemble.ts'
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
const NS = settingsNamespace('llm-deepseek')
const KEY_REF = credentialRef('DEEPSEEK_API_KEY')
let root: string | undefined
let context: Context | undefined
@@ -69,7 +67,6 @@ async function loadComposition(
" name: '@deepseek-ai/dsh-credentials-local'",
' config:',
` path: ${JSON.stringify(envPath)}`,
' debounceMs: 10',
]
: [],
'- id: llm-deepseek',
@@ -117,22 +114,19 @@ describe('llm-deepseek real dynamic composition', () => {
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
expect(serverA.headers[0]?.authorization).toBe('Bearer boot-key')
// External edits, exactly as a user or the web UI would leave them on disk.
// External edits, exactly as a user would leave them on disk.
await writeFile(settingsPath, `llm-deepseek:\n baseURL: ${serverB.url}\n`)
await vi.waitFor(() => {
expect((ctx.get('settings')!.get(NS) as { baseURL?: string }).baseURL).toBe(serverB.url)
}, { timeout: 5000 })
await writeFile(envPath, 'DEEPSEEK_API_KEY=rotated-key\n')
await vi.waitFor(async () => {
expect(await ctx.get('credentials')!.resolve(KEY_REF)).toEqual({ value: 'rotated-key', source: 'file' })
}, { timeout: 5000 })
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
expect(serverA.requests).toHaveLength(1)
expect(serverB.headers[0]?.authorization).toBe('Bearer rotated-key')
})
it('keeps a stored key writable and rotatable across a real restart', async () => {
it('reads a stored key and an external rotation across a real restart', async () => {
// No ambient DEEPSEEK_API_KEY: the shipped surfaces no longer hoist
// $DSH_HOME/.env into process.env, so a stored key must stay file-sourced.
vi.stubEnv('DEEPSEEK_API_KEY', '')
@@ -140,23 +134,15 @@ describe('llm-deepseek real dynamic composition', () => {
const second = await mockServer([{ kind: 'sse', events: textEvents }])
const boot = await loadComposition({ withDynamic: true, baseURL: first.url })
const home = root!
await boot.ctx.get('credentials')!.set(KEY_REF, 'stored-by-ui')
expect(await boot.ctx.get('credentials')!.describe(KEY_REF))
.toEqual({ configured: true, source: 'file', writable: true })
await writeFile(boot.envPath, 'DEEPSEEK_API_KEY=stored-directly\n')
await assemble(boot.ctx, { model: 'deepseek-v4-flash', messages: [] })
expect(first.headers[0]?.authorization).toBe('Bearer stored-by-ui')
expect(first.headers[0]?.authorization).toBe('Bearer stored-directly')
await boot.ctx.fiber.dispose()
context = undefined
// Restart over the same harness home.
const restarted = await loadComposition({ withDynamic: true, baseURL: second.url, reuseRoot: home })
const credentials = restarted.ctx.get('credentials')!
// The stored key is still the provider's own writable file entry — not a
// read-only launch override, which is what hoisting it would have made it.
expect(await credentials.resolve(KEY_REF)).toEqual({ value: 'stored-by-ui', source: 'file' })
expect(await credentials.describe(KEY_REF)).toEqual({ configured: true, source: 'file', writable: true })
// Rotation still works after the restart, and the next request uses it.
await credentials.set(KEY_REF, 'rotated-after-restart')
await writeFile(restarted.envPath, 'DEEPSEEK_API_KEY=rotated-after-restart\n')
await assemble(restarted.ctx, { model: 'deepseek-v4-flash', messages: [] })
expect(second.headers[0]?.authorization).toBe('Bearer rotated-after-restart')
})

4
packages/llm/llm-pi-ai/README.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: 0099c9acd39cd2d471936505726d68423f351c76
README.zh.md: 7cb4f5fcbc1c7a67b77d690031cc7d553569433f
README.md: e8b7adf122946fc22f231fafb521866cbacdc652
README.zh.md: 5bbf034267f6e276bc6552c5e731f6369cd97aee

12
packages/llm/llm-pi-ai/README.md
View File

@@ -35,13 +35,13 @@ Configure credentials and deployment-specific transport settings per provider, k
X-Deployment: production
```
Each dict key must exist in pi-ai's installed catalog; the dict shape makes duplicates unrepresentable, and the pre-release array shape (with per-profile `provider` fields) fails load with migration directions. `providers` may also be empty or omitted entirely: the adapter then mounts **dormant** — zero routes, no extra catalog entries — and registers routes the moment the `llm-pi-ai:` settings section supplies profiles, dropping them again when it empties. Which adapters exist is composition; which providers run can be entirely the user's settings document. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
Each dict key must exist in pi-ai's installed catalog; the dict shape makes duplicates unrepresentable, and the pre-release array shape (with per-profile `provider` fields) fails load with migration directions. Composition must provide at least one route. Registration with `ctx.llm` is all-or-nothing: a collision with any route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
## Dynamic configuration (settings + credentials)
The adapter reads its profiles through a thunk **once per operation** instead of freezing them at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`, and because `providers` is a dict, the base and the user's `llm-pi-ai:` settings section merge **per provider**: a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. Without a mounted settings service the entry config alone drives the adapter, unchanged.
The adapter reads its profiles through a thunk **once per operation** instead of freezing request facts at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`. The user layer can override request-level fields of a composition route, such as its endpoint, credential reference, headers, or transport controls, effective on the next operation. Provider routes and retry policies remain composition-fixed; a settings snapshot that changes either is rejected as one generation. Without a mounted settings service the entry config alone drives the adapter.
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; exactly that variable without a mounted seam). A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin replaces its registration atomically (same adapter instance, candidate set validated first), so a route another adapter already owns leaves the previous routes serving and reverting to a working configuration re-applies. Provider key order never counts as a change. A live settings snapshot naming an unknown provider (or failing any other resolver bound) keeps the last good profiles and logs the failure; the entry config itself still fails plugin load.
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; exactly that variable without a mounted seam). A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. A live settings snapshot that changes registration facts, names an unknown provider, or fails another resolver bound keeps the last good profiles and logs the failure; the entry config itself fails plugin load.
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers.
@@ -77,7 +77,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
## Testing
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers: a settings-born route registers live and drops when the user layer resets, `apiKeyEnv` credentials rotate between requests, and an unknown-provider snapshot keeps the last good profiles. `tests/loader-composition.spec.ts` boots the dormant posture from a test-only `cordis.yml` through the actual Loader and registers its route from an on-disk `settings.yaml` edit. Real-API coverage remains key-gated under `pnpm run test:e2e`.
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers: endpoint and `apiKeyEnv` changes reach later requests while routes and retry policy stay composition-fixed. `tests/loader-composition.spec.ts` boots that chain from a test-only `cordis.yml` through the actual Loader and edits `settings.yaml`/`.env` on disk. Real-API coverage remains key-gated under `pnpm run test:e2e`.
## Model Experience
@@ -111,8 +111,8 @@ Recorded response content appends to the next request and does not invalidate it
## Known Limitations and Deferred Work
- **Settings can add or override routes, not remove composition routes** — the user layer merges over the composition `base`, so deleting a `cordis.yml`-provided provider is a composition change; `replace` on the namespace only resets the user layer.
- **`apiKey` is schema-tagged `role('secret')` but not yet masked anywhere** — the settings `describe()` envelope returns values verbatim; the wire/UI layer that must redact secret-role fields ships with the settings RPC surface.
- **Settings cannot add or remove routes** — provider ownership and retry policy are composition facts; the user layer can only change request-level fields of existing routes.
- **`apiKey` is schema-tagged `role('secret')` but not masked by `ctx.settings.describe()`** — do not expose that envelope to an untrusted UI without redacting secret-role fields.
- **Catalog membership is required** — custom model ids that are absent from the installed pi-ai catalog fail with `UNKNOWN_MODEL`, even when a provider profile supplies a custom endpoint.
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.

12
packages/llm/llm-pi-ai/README.zh.md
View File

@@ -35,13 +35,13 @@
X-Deployment: production
```
每个字典键都必须存在于 pi-ai 已安装 catalog 中；字典形状使重复项无法表示，发布前的数组形状（每个 profile 携带 `provider` 字段）会加载失败并给出迁移指引。`providers` 也可以为空或整体省略：适配器将以**休眠**姿态挂载——零路由、模型选择器不多一条——一旦 `llm-pi-ai:` settings 分节提供了 profile 就即时注册路由，分节清空时随之撤销。哪些适配器存在归组合面；哪些提供方在运行可以完全交给用户的设置文档。向 `ctx.llm` 注册具有原子性：如果与另一适配器已拥有的任何提供方路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；未知模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
每个字典键都必须存在于 pi-ai 已安装 catalog 中；字典形状使重复项无法表示，发布前的数组形状（每个 profile 携带 `provider` 字段）会加载失败并给出迁移指引。组合必须提供至少一条路由。向 `ctx.llm` 注册要么全部成功，要么全部不生效：如果与另一适配器已拥有的任何路由冲突，插件会加载失败，不注册剩余路由。模型 id 不是生命周期配置；未知模型会在发起任何提供方请求前以 `LlmError('UNKNOWN_MODEL')` 失败。
## 动态配置（settings + credentials）
适配器经由一个 thunk **每操作读取一次** profile，而非在构造期冻结。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace，并以其 `cordis.yml` 条目为组合 `base`；由于 `providers` 是字典，base 与用户的 `llm-pi-ai:` settings 分节**按提供方**合并：用户可以新增路由、覆盖组合路由的单个字段，或把路由指向另一个 proxy，全部在下一次请求生效，无需重启。未挂载 settings 服务时，仅由 entry 配置驱动适配器，行为不变。
适配器经由一个 thunk **每操作读取一次** profile，而非在构造期冻结请求事实。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace，并以其 `cordis.yml` 条目为组合 `base`。用户层可以覆盖组合路由的请求级字段，例如端点、凭据引用、标头或传输控制项，并在下一次操作生效。提供方路由与重试策略始终由组合固定；settings 快照若更改任一项，就会整代被拒绝。未挂载 settings 服务时，仅由 entry 配置驱动适配器。
凭据按每次 stream 调用解析：非空的字面 `apiKey` 优先，其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`（活跃环境之下的 `$DSH_HOME/.env`；未挂载 seam 时恰好读取该环境变量）。只有完全没有点名任何凭据的 profile——仅限这一种情况——才交给 pi-ai 的环境发现。路由集合与每条路由捕获的重试策略是注册级事实：两者任一变化时，插件都会原子地替换自己的注册（同一适配器实例，候选集合先经校验），因此某条路由若已被另一适配器占有，先前的路由会继续服务，而改回可用配置时注册会重新生效。提供方键的顺序绝不算作变化。存活 settings 快照若点名未知提供方（或违反任何其他 resolver 约束），则保留最后可用 profile 并记录失败；entry 配置本身仍会使插件加载失败。
凭据按每次 stream 调用解析：非空的字面 `apiKey` 优先，其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`（活跃环境之下的 `$DSH_HOME/.env`；未挂载 seam 时恰好读取该环境变量）。只有完全没有点名任何凭据的 profile（仅限这一种情况），才交给 pi-ai 的环境发现。存活 settings 快照若更改注册事实、点名未知提供方或违反其他 resolver 约束，则保留最后可用 profile 并记录失败；entry 配置本身会使插件加载失败。
适配器通过 `ctx.llm.listModels(provider)` 公开每个已配置提供方已安装的 pi-ai 模型。这是从 `getModels(provider)` 派生的提供方无关 selector 元数据；请求时解析仍会执行权威 catalog 查找，因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找，并返回其身份、上下文窗口和可选思考级别，让权威元数据保留在拥有路由的适配器上，而非消费方。
@@ -77,7 +77,7 @@ pi-ai 会安装多个提供方 SDK，并延迟加载 catalog 模型所选的 SDK
## 测试
单元测试使用重定向到本地 mock 服务器的 pi-ai catalog 模型，覆盖提供方／profile 路由、每次适配器调用只发起一个协议请求、idle-timeout 响应终止、调用方 abort、原生 API 选择、端点覆盖、归因、转换、回放状态验证，以及一个适配器实例内的跨提供方／模型回放。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider：settings 里新生的路由实时完成注册，并在用户层重置时随之移除，`apiKeyEnv` 凭据在两次请求之间轮换，点名未知提供方的快照则保留最后可用 profile。`tests/loader-composition.spec.ts` 从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起休眠姿态，并从磁盘上的一次 `settings.yaml` 编辑注册出它的路由。真实 API 覆盖仍需 key 才会启用，并通过 `pnpm run test:e2e` 运行。
单元测试使用重定向到本地 mock 服务器的 pi-ai catalog 模型，覆盖提供方／profile 路由、每次适配器调用只发起一个协议请求、idle-timeout 响应终止、调用方 abort、原生 API 选择、端点覆盖、归因、转换、回放状态验证，以及一个适配器实例内的跨提供方／模型回放。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider：端点与 `apiKeyEnv` 变更会作用于后续请求，而路由与重试策略始终由组合固定。`tests/loader-composition.spec.ts` 从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起该链路，并在磁盘上编辑 `settings.yaml`/`.env`。真实 API 覆盖仍需 key 才会启用，并通过 `pnpm run test:e2e` 运行。
## 模型体验
@@ -111,8 +111,8 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
## 已知限制与暂缓事项
- **settings 能新增或覆盖路由，但不能移除组合路由**：用户层合并在组合 `base` 之上，因此删除 `cordis.yml` 提供的提供方属于组合变更；对该 namespace 执行 `replace` 只会重置用户层。
- **`apiKey` 已在 schema 中标注 `role('secret')`，但尚未在任何地方脱敏**：settings 的 `describe()` 信封原样返回值；负责对 secret 角色字段脱敏的 wire／UI 层将随 settings RPC 面一起交付。
- **settings 无法新增或移除路由**：提供方所有权与重试策略属于组合事实；用户层只能更改现有路由的请求级字段。
- **`apiKey` 已在 schema 中标注 `role('secret')`，但未由 `ctx.settings.describe()` 脱敏**：在对 secret 角色字段脱敏之前，不要向不受信任的 UI 暴露该信封。
- **必须属于 catalog**：已安装 pi-ai catalog 中不存在的自定义模型 id 会以 `UNKNOWN_MODEL` 失败，即使提供方 profile 配置了自定义端点。
- **不支持 `GenerateOptions.stop`**：pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence，因此适配器会拒绝该字段。
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**：提供方特定位置由 pi-ai 决定，而非由 harness 拥有的协议覆盖决定。

20
packages/llm/llm-pi-ai/src/config.ts
View File

@@ -1,7 +1,7 @@
/**
* Configuration schema and provider-profile validation for the pi-ai adapter.
* Profiles are a dict keyed by provider route, so the composition base and a
* user-settings layer merge per provider and the route set is structural.
* user-settings layer merge per provider.
*
* @module dsh-llm-pi-ai/config
*/
@@ -60,12 +60,8 @@ export interface ResolvedPiAiProviderProfile extends Omit<PiAiProviderProfile, '
/** Plugin configuration: the provider routes this instance owns. */
export interface Config {
/**
* pi-ai provider routes, keyed by provider. An empty (or omitted) dict is
* the dormant settings-driven posture: the adapter mounts with no routes
* and registers them the moment a settings section supplies profiles.
*/
providers?: Record<string, PiAiProviderProfile>
/** Non-empty pi-ai provider routes, keyed by provider and fixed by composition. */
providers: Record<string, PiAiProviderProfile>
}
const thinkingBudgets = z.object({
@@ -92,24 +88,24 @@ const profile = z.object({
/** Runtime schema for {@link Config}. */
export const Config: z<Config> = z.object({
providers: z.dict(profile).default({}),
providers: z.dict(profile).required(),
})
/**
* Validate profiles against the installed pi-ai catalog and return a detached
* route-keyed map suitable for per-request reads. This is the one explicit
* resolve step, so an omitted dict resolves to the empty (dormant) route set
* here rather than through a hidden fallback.
* resolve step; a composition must name at least one route.
* @param providers - configured provider profiles keyed by route.
* @returns validated profiles in configuration order.
*/
export function resolveProfiles(
providers: Readonly<Record<string, PiAiProviderProfile>> | undefined,
providers: Readonly<Record<string, PiAiProviderProfile>>,
): Map<string, ResolvedPiAiProviderProfile> {
if (Array.isArray(providers)) {
throw new Error('llm-pi-ai: providers is now a dict keyed by provider route, not an array of profiles')
}
const entries = Object.entries(providers ?? {})
const entries = Object.entries(providers)
if (entries.length === 0) throw new Error('llm-pi-ai: providers must contain at least one profile')
const supported = new Set<string>(getBuiltinProviders())
const resolved = new Map<string, ResolvedPiAiProviderProfile>()
for (const [provider, source] of entries) {

57
packages/llm/llm-pi-ai/src/index.ts
View File

@@ -3,10 +3,9 @@
* provider routes; requests select a profile by provider and resolve the
* model dynamically from pi-ai's installed catalog. Profile facts resolve per
* request over the optional `llm-pi-ai` user-settings section and the
* optional credential seam, so a changed key, endpoint, or knob reaches the
* next request without a restart; a changed *route set* (or a route's
* registration-captured retry policy) re-registers the same adapter instance
* in place.
* optional credential seam, so a changed key, endpoint, or request knob
* reaches the next request without a restart. Provider routes and retry
* policies stay composition-fixed.
*
* ```yaml
* - id: llm
@@ -30,7 +29,6 @@
import type { Context } from 'cordis'
import { LlmError } from '@deepseek-ai/dsh-llm'
import type { AdapterRegistrationHandle } from '@deepseek-ai/dsh-llm'
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import { PiAiAdapter } from './adapter.ts'
import { Config, resolveProfiles } from './config.ts'
@@ -59,14 +57,19 @@ function registrationFacts(profiles: ReadonlyMap<string, ResolvedPiAiProviderPro
/** Register one generic pi-ai adapter for all configured provider routes. */
export function apply(ctx: Context, config: Config): void {
const compositionProfiles = resolveProfiles(config.providers)
const compositionFacts = registrationFacts(compositionProfiles)
let current: () => Config = () => config
let lastRaw: Config | undefined
let lastGood: ReadonlyMap<string, ResolvedPiAiProviderProfile> | undefined
let lastRaw: Config = config
let lastGood: ReadonlyMap<string, ResolvedPiAiProviderProfile> = compositionProfiles
const profiles = (): ReadonlyMap<string, ResolvedPiAiProviderProfile> => {
const raw = current()
if (raw === lastRaw && lastGood !== undefined) return lastGood
if (raw === lastRaw) return lastGood
try {
const next = resolveProfiles(raw.providers)
if (!deepEqualJson(registrationFacts(next), compositionFacts)) {
throw new Error('llm-pi-ai: provider routes and retry policies are composition-fixed')
}
lastRaw = raw
lastGood = next
return next
@@ -74,14 +77,12 @@ export function apply(ctx: Context, config: Config): void {
// Static composition resolves before anything registers, so this branch
// only sees a live settings snapshot failing catalog or bound checks:
// keep serving the last good profiles and say so once per bad snapshot.
if (lastGood === undefined) throw error
lastRaw = raw
ctx.logger.error('llm-pi-ai: keeping the last good profiles after an invalid settings section')
ctx.logger.error(error)
return lastGood
}
}
profiles()
const resolveApiKey = async (
provider: string,
@@ -97,55 +98,25 @@ export function apply(ctx: Context, config: Config): void {
if (ref === undefined) return undefined
const credentials = ctx.get('credentials')
const hit = credentials !== undefined
? (await credentials.resolve(ref))?.value
? await credentials.resolve(ref)
// Without the seam, read exactly the named variable so a plain
// cordis.yml composition works from the environment alone.
: process.env[ref]
if (hit !== undefined && hit.length > 0) return hit
throw new LlmError(
`llm-pi-ai: no credential for provider route "${provider}"; its profile resolves ${ref}, which is not`
+ ` set — store ${ref} through the credentials service (the web Models page writes it) or export it,`
+ ` set — provide ${ref} through the credential provider or launching environment,`
+ ' and remove apiKeyEnv only if this provider should authenticate from pi-ai\'s own environment discovery',
'MISSING_CREDENTIAL',
)
}
const adapter = new PiAiAdapter({ profiles, resolveApiKey })
// Route effects bind to this apply fiber via the stable `ctx` reference,
// even when a swap runs inside the scoped settings callback below. A bare
// mount (zero routes) is the dormant posture: nothing registers until a
// settings section supplies profiles, and routes drop when it empties.
let registration: AdapterRegistrationHandle | undefined
let registeredFacts: unknown
const ensureRegistrationFacts = (): void => {
const facts = registrationFacts(profiles())
if (deepEqualJson(facts, registeredFacts)) return
// The registry captures the route set and each route's retry policy at
// registration, so a change to either must re-register. The swap is
// atomic (same adapter instance, validated before anything moves): a
// conflicting route leaves the previous routes serving requests, and
// `registeredFacts` only advances once the registry actually holds the
// new set — so returning to a working configuration always re-applies.
const routes = [...profiles().keys()]
if (registration === undefined) {
// Dormant bare mount: nothing is registered until a section supplies
// profiles, and an empty section keeps it that way.
if (routes.length === 0) {
registeredFacts = facts
return
}
registration = ctx.llm.registerAdapter(routes, adapter)
} else {
registration.replace(routes)
}
registeredFacts = facts
}
ensureRegistrationFacts()
ctx.llm.registerAdapter([...compositionProfiles.keys()], adapter)
installSettingsSection(ctx, NS, Config, config, {
setSource: (source) => {
current = source
},
onChange: ensureRegistrationFacts,
})
}

4
packages/llm/llm-pi-ai/tests/adapter.spec.ts
View File

@@ -401,9 +401,7 @@ describe('provider profile lifecycle', () => {
})
it('validates empty, unknown, legacy-shaped, and explicitly blank profiles', () => {
// Empty and omitted dicts are the dormant zero-route posture, not errors.
expect(resolveProfiles({}).size).toBe(0)
expect(resolveProfiles(undefined).size).toBe(0)
expect(() => resolveProfiles({})).toThrow(/at least one profile/)
expect(() => resolveProfiles({ '': {} })).toThrow(/non-empty/)
expect(() => resolveProfiles({ 'not-real': {} })).toThrow(/unknown/)
// The pre-release array shape and its per-profile provider field fail

163
packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts
View File

@@ -3,8 +3,7 @@ import { Context } from 'cordis'
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import LlmService, { LlmAdapter } from '@deepseek-ai/dsh-llm'
import { credentialRef } from '@deepseek-ai/dsh-credentials'
import LlmService from '@deepseek-ai/dsh-llm'
import { CredentialsLocal } from '@deepseek-ai/dsh-credentials-local'
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
import { SettingsLocal } from '@deepseek-ai/dsh-settings-local'
@@ -13,15 +12,6 @@ import { assemble } from './assemble.ts'
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
const NS = settingsNamespace('llm-pi-ai')
/** Minimal foreign adapter: only needs to own a route the pi-ai plugin then wants. */
class StubAdapter extends LlmAdapter {
override async * stream(): AsyncIterable<never> {
throw new Error('stub adapter must never stream')
}
}
const cleanups: Array<() => Promise<void>> = []
afterEach(async () => {
@@ -36,154 +26,75 @@ async function home(): Promise<string> {
return dir
}
/** Real dynamic composition mirroring the deepseek twin's harness. */
/** Real dynamic composition mirroring the DeepSeek twin's harness. */
async function boot(dir: string, config: LlmPiAi.Config): Promise<Context> {
const ctx = new Context()
cleanups.push(async () => {
await ctx.fiber.dispose()
})
cleanups.push(async () => { await ctx.fiber.dispose() })
await ctx.plugin(LlmService)
await ctx.plugin(SettingsLocal, { path: join(dir, 'settings.yaml'), watch: false })
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env'), watch: false })
await ctx.plugin(CredentialsLocal, { path: join(dir, '.env') })
await ctx.plugin(LlmPiAi, config)
return ctx
}
describe('request-level dynamic profiles', () => {
it('mounts bare and dormant, then registers routes the moment settings supply providers', async () => {
vi.stubEnv('PI_DYNAMIC_KEY', '')
const dir = await home()
await writeFile(join(dir, '.env'), 'PI_DYNAMIC_KEY=pk-from-settings\n')
const server = await mockServer([{ events: textEvents }])
// The exact product posture: `- id: llm-pi-ai` with no config at all.
const ctx = await boot(dir, {})
expect(ctx.llm.listProviders()).toEqual([])
await ctx.settings.update(NS, {
providers: { deepseek: { apiKeyEnv: 'PI_DYNAMIC_KEY', baseURL: server.url } },
})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek'])
await expect(ctx.llm.listModels('deepseek')).resolves.not.toHaveLength(0)
const result = await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
expect(server.headers[0]?.authorization).toBe('Bearer pk-from-settings')
// Emptying the user layer returns the adapter to its dormant state.
await ctx.settings.replace(NS, {})
expect(ctx.llm.listProviders()).toEqual([])
})
it('adds a provider route from settings and drops it when the user layer resets', async () => {
const dir = await home()
const server = await mockServer([{ events: textEvents }])
const ctx = await boot(dir, {
providers: { openai: { apiKey: 'k', baseURL: 'http://127.0.0.1:1/v1' } },
})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
await ctx.settings.update(NS, {
providers: { deepseek: { apiKey: 'live-key', baseURL: server.url } },
})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai', 'deepseek'])
const result = await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
expect(server.headers[0]?.authorization).toBe('Bearer live-key')
// Reset the user layer: the settings-born route unregisters, the
// composition route stays.
await ctx.settings.replace(NS, {})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
await expect(assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] }))
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
})
it('rotates the per-request credential referenced by apiKeyEnv', async () => {
it('uses the next endpoint and credential while keeping the route fixed', async () => {
vi.stubEnv('PI_DYNAMIC_KEY', '')
const dir = await home()
await writeFile(join(dir, '.env'), 'PI_DYNAMIC_KEY=pk-one\n')
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
const serverA = await mockServer([{ events: textEvents }])
const serverB = await mockServer([{ events: textEvents }])
const ctx = await boot(dir, {
providers: { deepseek: { apiKeyEnv: 'PI_DYNAMIC_KEY', baseURL: server.url } },
providers: { deepseek: { apiKeyEnv: 'PI_DYNAMIC_KEY', baseURL: serverA.url } },
})
await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(server.headers[0]?.authorization).toBe('Bearer pk-one')
expect(serverA.headers[0]?.authorization).toBe('Bearer pk-one')
await ctx.credentials.set(credentialRef('PI_DYNAMIC_KEY'), 'pk-two')
await ctx.settings.update(NS, { providers: { deepseek: { baseURL: serverB.url } } })
await writeFile(join(dir, '.env'), 'PI_DYNAMIC_KEY=pk-two\n')
await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(server.headers[1]?.authorization).toBe('Bearer pk-two')
expect(serverA.requests).toHaveLength(1)
expect(serverB.headers[0]?.authorization).toBe('Bearer pk-two')
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek'])
})
it('re-registers routes in place when a captured retry policy changes', async () => {
it('rejects settings-born routes and keeps the composition profile serving', async () => {
const dir = await home()
const ctx = await boot(dir, { providers: { openai: {} } })
const server = await mockServer([{ events: textEvents }])
const ctx = await boot(dir, {
providers: { openai: { apiKey: 'pk', baseURL: `${server.url}/v1` } },
})
await ctx.settings.update(NS, {
providers: { anthropic: { apiKey: 'other' } },
})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
expect(server.paths).toEqual(['/v1/responses'])
})
it('keeps the registration retry policy composition-fixed', async () => {
const dir = await home()
const ctx = await boot(dir, {
providers: {
openai: {
retryPolicy: { mode: 'always', backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 } },
retryPolicy: {
mode: 'always',
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
},
},
},
})
await ctx.settings.update(NS, {
providers: { openai: { retryPolicy: { mode: 'normal', maxRetries: 0 } } },
})
expect(ctx.llm.providerRetryPolicy('openai')).toEqual({
mode: 'always',
initialDelayMs: 25,
maxDelayMs: 100,
jitterRatio: 0.2,
})
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
})
it('keeps the last good profiles when a settings snapshot names an unknown provider', async () => {
const dir = await home()
const ctx = await boot(dir, { providers: { openai: {} } })
// Schema-valid but catalog-invalid: the resolver rejects it and the
// last good route set keeps serving.
await ctx.settings.update(NS, { providers: { 'not-a-real-provider': {} } })
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
})
it('keeps serving its routes when a settings-born route collides with another adapter', async () => {
const dir = await home()
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
const ctx = await boot(dir, { providers: { openai: { apiKey: 'pk', baseURL: `${server.url}/v1` } } })
// Another adapter owns `anthropic`; the registry must refuse to hand it over.
ctx.llm.registerAdapter(['anthropic'], new StubAdapter())
await ctx.settings.update(NS, {
providers: {
openai: { apiKey: 'pk', baseURL: `${server.url}/v1` },
anthropic: { apiKey: 'other' },
},
})
// The conflicting swap was refused whole: the previous route set still
// owns openai (an eager dispose would have dropped it), and anthropic
// still belongs to its original adapter.
expect(ctx.llm.listProviders().map(provider => provider.id).sort()).toEqual(['anthropic', 'openai'])
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
expect(result.finish.kind).toBe('error')
expect(server.paths).toEqual(['/v1/responses'])
// Reverting to the working configuration re-applies, even though its
// facts equal the ones the registry already holds.
await ctx.settings.replace(NS, {})
expect(ctx.llm.listProviders().map(provider => provider.id).sort()).toEqual(['anthropic', 'openai'])
await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
expect(server.paths).toEqual(['/v1/responses', '/v1/responses'])
})
it('ignores a settings document that merely reorders its provider keys', async () => {
const dir = await home()
const ctx = await boot(dir, { providers: { openai: {}, anthropic: {} } })
const before = ctx.llm.listProviders().map(provider => provider.id)
// Same routes, different YAML key order: nothing about the registration
// changed, so no swap should happen at all.
await ctx.settings.update(NS, { providers: { anthropic: {}, openai: {} } })
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(before)
})
})

54
packages/llm/llm-pi-ai/tests/loader-composition.spec.ts
View File

@@ -1,11 +1,6 @@
/**
* Real-composition guard for the dormant pi-ai posture: LlmService,
* settings-local, credentials-local, and a bare `llm-pi-ai` row boot from a
* test-only cordis.yml through the actual Loader + Include path, an external
* edit of settings.yaml registers the route live, and the next request
* carries the credential the .env supplies. A hand-mounted `ctx.plugin` cannot
* catch Loader export-shape failures, which is why the twin adapter has the
* same guard.
* Real-composition guard for a configured pi-ai route through Loader + Include.
* Settings may change request facts, while the route stays composition-owned.
*/
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
@@ -18,6 +13,7 @@ import Loader from '@cordisjs/plugin-loader'
import Include from '@cordisjs/plugin-include'
import LlmService from '@deepseek-ai/dsh-llm'
import CredentialsLocal from '@deepseek-ai/dsh-credentials-local'
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
import SettingsLocal from '@deepseek-ai/dsh-settings-local'
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
import { assemble } from './assemble.ts'
@@ -25,6 +21,7 @@ import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
let root: string | undefined
let context: Context | undefined
const NS = settingsNamespace('llm-pi-ai')
afterEach(async () => {
await context?.fiber.dispose()
@@ -35,12 +32,12 @@ afterEach(async () => {
vi.unstubAllEnvs()
})
/** Boot the dormant composition: a bare `llm-pi-ai` row with no config at all. */
async function loadComposition(): Promise<{ ctx: Context; settingsPath: string }> {
async function loadComposition(baseURL: string): Promise<{ ctx: Context; settingsPath: string; envPath: string }> {
root = await mkdtemp(join(tmpdir(), 'dsh-pi-composition-'))
const settingsPath = join(root, 'settings.yaml')
const envPath = join(root, '.env')
await writeFile(settingsPath, '# personal settings\n')
await writeFile(join(root, '.env'), 'PI_COMPOSITION_KEY=key-from-store\n')
await writeFile(envPath, 'PI_COMPOSITION_KEY=key-from-store\n')
const configPath = join(root, 'cordis.yml')
await writeFile(configPath, [
@@ -54,10 +51,14 @@ async function loadComposition(): Promise<{ ctx: Context; settingsPath: string }
'- id: credentials',
" name: '@deepseek-ai/dsh-credentials-local'",
' config:',
` path: ${JSON.stringify(join(root, '.env'))}`,
' debounceMs: 10',
` path: ${JSON.stringify(envPath)}`,
'- id: llm-pi-ai',
" name: '@deepseek-ai/dsh-llm-pi-ai'",
' config:',
' providers:',
' deepseek:',
' apiKeyEnv: PI_COMPOSITION_KEY',
` baseURL: ${baseURL}`,
'',
].join('\n'))
@@ -84,33 +85,34 @@ async function loadComposition(): Promise<{ ctx: Context; settingsPath: string }
config: { path: pathToFileURL(configPath).href },
})
await ctx.loader.await()
return { ctx, settingsPath }
return { ctx, settingsPath, envPath }
}
describe('llm-pi-ai real dormant composition', () => {
it('boots with zero routes and registers one the moment settings supply a profile', async () => {
describe('llm-pi-ai real composition', () => {
it('keeps its route while external settings and credential edits reach the next request', async () => {
vi.stubEnv('PI_COMPOSITION_KEY', '')
const server = await mockServer([{ events: textEvents }])
const { ctx, settingsPath } = await loadComposition()
const serverA = await mockServer([{ events: textEvents }])
const serverB = await mockServer([{ events: textEvents }])
const { ctx, settingsPath, envPath } = await loadComposition(serverA.url)
// The shipped posture: the adapter exists, no route does.
expect(ctx.llm.listProviders()).toEqual([])
await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(serverA.headers[0]?.authorization).toBe('Bearer key-from-store')
// Exactly what the web Models page leaves on disk.
await writeFile(settingsPath, [
'llm-pi-ai:',
' providers:',
' deepseek:',
' apiKeyEnv: PI_COMPOSITION_KEY',
` baseURL: ${server.url}`,
` baseURL: ${serverB.url}`,
'',
].join('\n'))
await writeFile(envPath, 'PI_COMPOSITION_KEY=rotated-key\n')
await vi.waitFor(() => {
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek'])
expect((ctx.get('settings')!.get(NS) as { providers?: { deepseek?: { baseURL?: string } } })
.providers?.deepseek?.baseURL).toBe(serverB.url)
}, { timeout: 5000 })
const result = await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
expect(server.headers[0]?.authorization).toBe('Bearer key-from-store')
await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek'])
expect(serverB.headers[0]?.authorization).toBe('Bearer rotated-key')
})
})

4
packages/llm/llm-retry/README.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md
README.md: 7a86652a794e70c4dfd00ab7427730387e3ec949
README.zh.md: c66c04806597c11b5c64dcb01443bb84f489e0f5
README.md: 233ccb0b744f7ec52379f823286c1c0638d91ee2
README.zh.md: 86260785128f7089be7bcbae08b7e858427a68ae

2
packages/llm/llm-retry/README.md
View File

@@ -8,7 +8,7 @@ Each provider adapter owns an optional nested `retryPolicy`, captured when its r
Both modes use bounded exponential backoff with symmetric jitter. A valid `providerRetryAfterMs` at or below `maxDelayMs` replaces local backoff without jitter. An over-cap provider delay makes normal mode delegate, while always mode uses its configured local backoff so it cannot terminate on that instruction.
Before waiting, the plugin appends a non-surface `llm/retry` event with the provider, mode, canonical resolved-policy key, failure, and scheduled delay. The key includes every behavior-affecting field and sorts normal-mode codes because eligibility uses set membership. Retry numbers continue only across events with the same provider and complete policy key, so a route replacement with different limits, code membership, or backoff starts its own history. Normal events include the finite maximum; always events omit it, and UIs render `∞`. After the wait, the listener returns `{ kind: 'retry' }`, and the loop closes the failed turn and opens a retry turn over the same durable history. Cancellation and plugin disposal abort active backoff, drain active delegated recovery before applying the abort, and make a callback captured before disposal fail closed.
Before waiting, the plugin appends a non-surface `llm/retry` event with the provider, mode, canonical resolved-policy key, failure, and scheduled delay. The key includes every behavior-affecting field and sorts normal-mode codes because eligibility uses set membership. Retry numbers continue only across events with the same provider and complete policy key, so a later registration with different limits, code membership, or backoff starts its own history. Normal events include the finite maximum; always events omit it, and UIs render `∞`. After the wait, the listener returns `{ kind: 'retry' }`, and the loop closes the failed turn and opens a retry turn over the same durable history. Cancellation and plugin disposal abort active backoff, drain active delegated recovery before applying the abort, and make a callback captured before disposal fail closed.
The separately published `./invariant` companion checks that every retry record names the current open turn and latest closed step, matches the failed request's durable provider, carries non-empty provider and policy identities, has mode-specific bounds, a unique step record, the correct provider-policy retry number, and a bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.

2
packages/llm/llm-retry/README.zh.md
View File

@@ -8,7 +8,7 @@
两种 mode 都使用带对称 jitter 的有界指数退避。有效 `providerRetryAfterMs` 不超过 `maxDelayMs` 时会替换本地退避，并且不加 jitter。超出上限的提供方延迟会使 normal mode 继续委托；always mode 则改用已配置的本地退避，避免该指令终止重试。
等待前，插件会追加一条不进入表层的 `llm/retry` 事件，其中包含提供方、mode、已解析策略的规范 key、失败和计划延迟。该 key 包含所有影响行为的字段，并对 normal mode 的 code 排序，因为合格性采用集合成员关系判断。只有提供方与完整策略 key 都相同的事件才会延续重试编号；因此，用限制、code 成员关系或退避不同的路由替换后，会开始自己的历史。normal 事件包含有限上限；always 事件省略该上限，UI 会渲染 `∞`。等待结束后，监听器返回 `{ kind: 'retry' }`，循环关闭失败轮次，并在同一持久历史上开启重试轮次。取消与插件 dispose 会中止活跃退避，在应用中止前排空活跃的委托恢复，并使 dispose 前捕获的 callback 只能以失败结束。
等待前，插件会追加一条不进入表层的 `llm/retry` 事件，其中包含提供方、mode、已解析策略的规范 key、失败和计划延迟。该 key 包含所有影响行为的字段，并对 normal mode 的 code 排序，因为合格性采用集合成员关系判断。只有提供方与完整策略 key 都相同的事件才会延续重试编号；因此，后续注册若采用不同的限制、code 成员关系或退避，就会开始自己的历史。normal 事件包含有限上限；always 事件省略该上限，UI 会渲染 `∞`。等待结束后，监听器返回 `{ kind: 'retry' }`，循环关闭失败轮次，并在同一持久历史上开启重试轮次。取消与插件 dispose 会中止活跃退避，在应用中止前排空活跃的委托恢复，并使 dispose 前捕获的 callback 只能以失败结束。
单独发布的 `./invariant` 配套模块会检查每个重试记录是否指向当前开启轮次及其最新已关闭步骤，是否与失败请求的持久提供方匹配，是否携带非空的提供方与策略标识，是否满足 mode 特定边界，是否拥有唯一步骤记录和正确的提供方策略重试编号，以及是否携带有界定时器延迟。完整 jitter 可以在下界调度为零毫秒。

4
packages/llm/llm/README.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
README.md: 5b0c1b2dcafeefaad25f1714e4a1783430370118
README.zh.md: 5f5c8142ec829e8ca8cfd40e6caa341ae0a33c7d
README.md: 095a5dc48ad1e762aabf71db0216fa50aceb6211
README.zh.md: baeb0214501e9c2a13a0a272a37a6ead75f6c8d6

4
packages/llm/llm/README.md
View File

@@ -10,7 +10,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
### Public API
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration.
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for a non-empty, composition-owned route set. Registration is all-or-nothing, is disposed with the calling fiber, and returns an explicit disposer.
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
@@ -19,7 +19,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route is later disposed or replaced. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route later changes ownership. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.

4
packages/llm/llm/README.zh.md
View File

@@ -10,7 +10,7 @@
### 公开 API
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功，要么全部不生效，并且会随调用 fiber 一起 dispose（资源释放）。返回的释放器还携带 `replace(providers)`：候选路由集合会在任何东西变动之前完整校验，因此与另一适配器冲突时，当前路由保持注册且继续服务，而替换本身是一个同步区段，不存在可观察的空档。`replace([])` 合法——一个持有零条路由的注册——这与空的初始注册不同。
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为一组非空且由组合拥有的路由注册一个适配器实例。注册要么全部成功，要么全部不生效，会随调用 fiber 一起 dispose（资源释放），并返回显式释放器。
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略，并解析 normal 默认值。
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
@@ -19,7 +19,7 @@
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始分片（token 级增量）。消费方使用 `BlockAssembler` 将分片组装为块／消息。
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误，并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误；`llmFailureOf(stream, value)` 返回关联的不可变 `LlmFailure`；`llmRetryPolicyOf(stream)` 返回在该边界选中的确切注册所对应的不可变策略，即使之后释放或替换路由也不变。未到达最终适配器的调用没有服务策略。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器原有的带代码 `Error`。
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误，并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误；`llmFailureOf(stream, value)` 返回关联的不可变 `LlmFailure`；`llmRetryPolicyOf(stream)` 返回在该边界选中的确切注册所对应的不可变策略，即使路由后来更换所有者也不变。未到达最终适配器的调用没有服务策略。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器原有的带代码 `Error`。
提供方与模型元数据是发现接口，不是路由白名单。`registerAdapter()` 仍拥有提供方排他性，并为每条路由捕获适配器的重试策略；适配器则可以接受 `listModels()` 中不存在的模型 id，消费方禁止因模型未列出而拒绝请求。返回的 selector 元数据与输入脱离，无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。

114
packages/llm/llm/src/index.ts
View File

@@ -184,30 +184,6 @@ export abstract class LlmAdapter {
abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
/**
* What {@link LlmService.registerAdapter} returns: the disposer, plus an
* atomic route replacement for the same adapter instance.
*/
export interface AdapterRegistrationHandle {
/** Release every route this registration currently holds. */
(): void
/**
* Replace this registration's routes with `providers`, keeping the same
* adapter instance. The candidate set is validated in full first — a
* conflict with another adapter, an invalid name, or bad provider metadata
* throws and leaves the current routes untouched — and the swap itself is
* one synchronous section, so no request can observe a gap. An empty array
* is legal here (a settings section that emptied holds zero routes while
* staying registered), unlike an empty initial registration.
*
* Throws `LlmError` with code `REGISTRATION_DISPOSED` once the registration
* has been released: its routes are gone and its disposer has already run,
* so anything registered afterwards would have no owner left to release it.
* @param providers - the complete next route set for this registration.
*/
replace(providers: string[]): void
}
/**
* The abstract `llm` service: an adapter registry plus a streaming model-call
* surface, interceptable via the `llm/stream` waterfall.
@@ -225,79 +201,39 @@ export class LlmService extends Service {
* Disposed with the fiber.
* @param providers - every provider route this adapter should serve.
* @param adapter - the adapter that streams calls for those providers.
* @returns the disposer, carrying {@link AdapterRegistrationHandle.replace}.
* @returns the disposer that unregisters all routes.
*/
registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle {
// The routes this registration currently holds; `replace` rewrites it, and
// the disposer releases whatever it holds at disposal time.
const owned = new Set<string>()
// The disposer has run: `owned` being empty cannot say so on its own,
// because `replace([])` legally leaves a live registration holding none.
let released = false
registerAdapter(providers: string[], adapter: LlmAdapter): () => void {
const dispose = this.ctx.effect(function* (this: LlmService) {
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
this.commitRoutes(owned, this.prepareRoutes(providers, adapter, owned))
const unique = new Set<string>()
const registrations: AdapterRegistration[] = []
for (const provider of providers) {
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
if (unique.has(provider) || this.adapters.has(provider)) {
throw new LlmError(`an adapter for provider "${provider}" is already registered`, 'DUPLICATE_ADAPTER')
}
const info = adapter.providerInfo(provider)
if (typeof info.id !== 'string' || info.id !== provider || typeof info.name !== 'string' || info.name.length === 0) {
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
}
unique.add(provider)
const retryPolicy = adapter.providerRetryPolicy(provider)
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
registrations.push({
adapter,
provider: { id: info.id, name: info.name },
retryPolicy,
})
}
for (const registration of registrations) this.adapters.set(registration.provider.id, registration)
yield () => {
released = true
for (const provider of owned) this.adapters.delete(provider)
owned.clear()
for (const provider of providers) this.adapters.delete(provider)
}
}.bind(this), 'llm.registerAdapter()')
// ctx.effect's disposer returns Promise<void>; our disposer API is
// synchronous fire-and-forget — discard the (always-resolved) promise.
const handle = (() => void dispose()) as AdapterRegistrationHandle
handle.replace = (next: string[]): void => {
// Registering here would leak: the effect's disposer already ran, so
// nothing remains to release whatever this call would put in the map.
if (released) {
throw new LlmError('a disposed adapter registration cannot replace its routes', 'REGISTRATION_DISPOSED')
}
this.commitRoutes(owned, this.prepareRoutes(next, adapter, owned))
}
return handle
}
/**
* Validate one candidate route set for `adapter`, treating routes this
* registration already holds as available. Nothing is mutated: a rejected
* candidate leaves the registry exactly as it was.
*/
private prepareRoutes(providers: string[], adapter: LlmAdapter, owned: ReadonlySet<string>): AdapterRegistration[] {
const unique = new Set<string>()
const registrations: AdapterRegistration[] = []
for (const provider of providers) {
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
if (unique.has(provider) || (this.adapters.has(provider) && !owned.has(provider))) {
throw new LlmError(`an adapter for provider "${provider}" is already registered`, 'DUPLICATE_ADAPTER')
}
const info = adapter.providerInfo(provider)
if (typeof info.id !== 'string' || info.id !== provider || typeof info.name !== 'string' || info.name.length === 0) {
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
}
unique.add(provider)
const retryPolicy = adapter.providerRetryPolicy(provider)
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
registrations.push({
adapter,
provider: { id: info.id, name: info.name },
retryPolicy,
})
}
return registrations
}
/**
* Swap this registration's routes for the prepared ones in one synchronous
* section, so no observer can see the registry between the release and the
* re-registration.
*/
private commitRoutes(owned: Set<string>, registrations: readonly AdapterRegistration[]): void {
for (const provider of owned) this.adapters.delete(provider)
owned.clear()
for (const registration of registrations) {
this.adapters.set(registration.provider.id, registration)
owned.add(registration.provider.id)
}
return () => void dispose()
}
/**

29
packages/llm/llm/tests/service.spec.ts
View File

@@ -217,7 +217,7 @@ describe('LlmService', () => {
)
})
it('keeps the serving registration policy on an in-flight call after route replacement', async () => {
it('keeps the serving registration policy on an in-flight call after route re-registration', async () => {
const oldPolicy = resolveRetryPolicy({ mode: 'always' }, 'old retryPolicy')
const newPolicy = resolveRetryPolicy({ mode: 'normal', maxRetries: 0 }, 'new retryPolicy')
const entered = Promise.withResolvers<undefined>()
@@ -1382,31 +1382,4 @@ describe('LlmService', () => {
expect(ctx.llm.listProviders()).toEqual([])
})
it('refuses to replace routes on a registration that was already released', async () => {
// The leak this prevents: the effect's disposer has run, so a route added
// afterwards would sit in the registry with nothing left to release it.
const ctx = new Context()
await ctx.plugin(LlmService)
const handle = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
handle()
expect(() => { handle.replace(['leaked']) })
.toThrow(/disposed adapter registration cannot replace its routes/)
expect(ctx.llm.listProviders()).toEqual([])
})
it('still allows an empty route set on a live registration', async () => {
// `replace([])` is the settings-section-emptied case: legal, and it must
// not be mistaken for disposal by the guard above.
const ctx = new Context()
await ctx.plugin(LlmService)
const handle = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
handle.replace([])
expect(ctx.llm.listProviders()).toEqual([])
handle.replace(['m2'])
expect(ctx.llm.listProviders()).toEqual([{ id: 'm2', name: 'm2' }])
handle()
expect(ctx.llm.listProviders()).toEqual([])
})
})