Merge branch 'worktree-llm-dynamic-config' into worktree-llm-web-config
# Conflicts: # docs/cordis-catalog/services.md # docs/core-data-structures/core.i18n.yaml # docs/event-producer-consumer.md # examples/headless-agent/tests/headless.snapshot.ts # examples/headless-agent/tests/snapshots/missing-credential/stream-json.expected.jsonl # packages/llm/llm-deepseek/README.i18n.yaml # packages/llm/llm-deepseek/src/index.ts # packages/llm/llm-pi-ai/README.i18n.yaml # packages/llm/llm-pi-ai/src/index.ts # packages/llm/llm/README.i18n.yaml # packages/llm/llm/src/index.ts
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 186739a3b0ee423afac27ef43c41f42a0e07ee84
|
||||
README.zh.md: 844fd1422339ad88640a948a735ac1cf9f130ff0
|
||||
README.md: 11bed74b4952624208f23f093b787eb978cfef69
|
||||
README.zh.md: 5fa75ad3434fe9610ba8005d436af51fd7a134f9
|
||||
|
||||
@@ -50,7 +50,7 @@ The same exact-model result exposes ordered `off`, `high`, and `max` efforts und
|
||||
Connection facts are not frozen at load. `resolveAdapterOptions` is the one explicit resolve step from raw config to validated facts, and the adapter re-reads them through a thunk **once per operation**: base URL, catalog, request defaults, and idle budget all take effect on the next request, while an in-flight stream keeps the facts it started with. Two optional seams feed that thunk:
|
||||
|
||||
- **`ctx.settings`** — the plugin registers the `llm-deepseek` namespace with this same `Config` schema and its `cordis.yml` entry as the composition `base`, so a `llm-deepseek:` section in the user settings document overrides any field without a restart. Without a mounted settings service the entry config alone drives the adapter, unchanged. A live settings snapshot that passes the schema but fails a beyond-schema bound (a duplicate catalog id, a broken thinking/effort pair) keeps the last good facts and logs the failure; the entry config itself still fails plugin load.
|
||||
- **`ctx.credentials`** — the API key resolves per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the credential seam (`$DSH_HOME/.env` under the live environment), then — only without a mounted seam — the raw environment variable. A request with no key anywhere fails with `MISSING_CREDENTIAL` naming every configuration entry point, while the route stays registered and the catalog stays browsable — first-run onboarding is "browse models, store the key, prompt again", with no restart between.
|
||||
- **`ctx.credentials`** — the API key resolves per stream call, from the *same* resolved snapshot that supplies the endpoint: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the credential seam (`$DSH_HOME/.env` under the live environment), then — only without a mounted seam — the raw environment variable. Because credential facts travel with the connection facts, a settings snapshot the resolver rejects contributes neither its endpoint nor its key: the whole previous generation keeps serving. A request with no key anywhere fails with `MISSING_CREDENTIAL` naming every configuration entry point, while the route stays registered and the catalog stays browsable — first-run onboarding is "browse models, store the key, prompt again", with no restart between.
|
||||
|
||||
The one registration-captured fact is the retry policy: when its resolved value changes, the plugin re-registers the route in place (same adapter instance, one synchronous section), so `ctx.llm.providerRetryPolicy('deepseek-official')` always reports the current policy.
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
|
||||
连接事实不在加载时冻结。`resolveAdapterOptions` 是从原始配置到已校验事实的唯一显式 resolve 步骤,适配器经由一个 thunk **每操作重读一次**:base URL、catalog、请求默认值与 idle 预算都在下一次请求生效,进行中的流则保持其起始事实。两个可选 seam 供给该 thunk:
|
||||
|
||||
- **`ctx.settings`**——插件用同一份 `Config` schema 注册 `llm-deepseek` namespace,并以其 `cordis.yml` 条目为组合 `base`,因此用户设置文档中的 `llm-deepseek:` 分节可以免重启覆盖任何字段。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。存活 settings 快照若通过 schema 却违反 schema 之外的约束(重复的 catalog id、无法成立的 thinking/推理强度组合),则保留最后可用事实并记录失败;entry 配置本身仍会使插件加载失败。
|
||||
- **`ctx.credentials`**——API 密钥按每次 stream 调用解析:非空的字面 `apiKey` 优先,其次经凭据 seam 解析 `apiKeyEnv`(活跃环境之下的 `$DSH_HOME/.env`),最后——仅在未挂载 seam 时——读取原始环境变量。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败,并点名每个配置入口,同时路由保持注册、catalog 保持可浏览——首次运行的上手流程就是「浏览模型、存入密钥、再次发起提示」,中间无需任何重启。
|
||||
- **`ctx.credentials`**——API 密钥按每次 stream 调用解析,取自与端点*同一*份解析后的快照:非空的字面 `apiKey` 优先,其次经凭据 seam 解析 `apiKeyEnv`(活跃环境之下的 `$DSH_HOME/.env`),最后——仅在未挂载 seam 时——读取原始环境变量。由于凭据事实与连接事实同行,被 resolver 拒绝的 settings 快照既不贡献自己的端点,也不贡献自己的密钥:整个先前世代继续服务。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败,并点名每个配置入口,同时路由保持注册、catalog 保持可浏览——首次运行的上手流程就是「浏览模型、存入密钥、再次发起提示」,中间无需任何重启。
|
||||
|
||||
唯一在注册期捕获的事实是重试策略:其解析值变化时,插件原地重新注册该路由(同一适配器实例、一个同步区段),因此 `ctx.llm.providerRetryPolicy('deepseek-official')` 始终报告当前策略。
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ import type {
|
||||
ResolvedRetryPolicy,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { serializeRequest } from './serialize.ts'
|
||||
import type { RequestDefaults } from './serialize.ts'
|
||||
@@ -45,6 +46,14 @@ export interface DeepSeekCatalogModel {
|
||||
export interface DeepSeekConnectionOptions {
|
||||
/** Endpoint base; `/chat/completions` is appended. */
|
||||
baseURL: string
|
||||
/**
|
||||
* Literal API key of this same resolution, when the configuration carried
|
||||
* one. Travelling with the endpoint is the point: a request can never pair
|
||||
* one generation's URL with another generation's secret.
|
||||
*/
|
||||
apiKey?: string
|
||||
/** Credential reference of this same resolution, resolved per request when no literal key exists. */
|
||||
apiKeyEnv: CredentialRef
|
||||
/** Request defaults applied to every call (thinking mode, effort). */
|
||||
defaults: RequestDefaults
|
||||
/** Positive context capacity used when the selected model has no exact value. */
|
||||
@@ -62,11 +71,12 @@ export interface DeepSeekAdapterOptions {
|
||||
/** Current validated connection facts; called once per operation. */
|
||||
options: () => DeepSeekConnectionOptions
|
||||
/**
|
||||
* Resolve the bearer token for one request; called once per stream call and
|
||||
* frozen for that call. Throws `LlmError` `MISSING_CREDENTIAL` when no key
|
||||
* is available anywhere.
|
||||
* Resolve the bearer token for the connection facts of one request. The
|
||||
* snapshot is passed in — never re-read — so the key can only ever come
|
||||
* from the same resolution as the endpoint it is sent to. Throws `LlmError`
|
||||
* `MISSING_CREDENTIAL` when no key is available anywhere.
|
||||
*/
|
||||
resolveApiKey: () => Promise<string>
|
||||
resolveApiKey: (connection: DeepSeekConnectionOptions) => Promise<string>
|
||||
}
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
@@ -189,8 +199,10 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
// One resolution per stream call: connection facts and the credential
|
||||
// freeze here and hold for this whole request, so an in-flight stream
|
||||
// never observes a configuration change and the next call re-resolves.
|
||||
// The key resolves *from this snapshot*, so an endpoint and the secret
|
||||
// sent to it can never come from different configuration generations.
|
||||
const connection = this.config.options()
|
||||
const apiKey = await this.config.resolveApiKey()
|
||||
const apiKey = await this.config.resolveApiKey(connection)
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
? consumer.signal
|
||||
|
||||
@@ -16,7 +16,6 @@ import z from 'schemastery'
|
||||
import { LlmError, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
|
||||
@@ -32,6 +31,8 @@ export const inject = ['llm']
|
||||
|
||||
const NS = settingsNamespace('llm-deepseek')
|
||||
const DEFAULT_API_KEY_ENV = 'DEEPSEEK_API_KEY'
|
||||
/** The single provider route this plugin owns. */
|
||||
const PROVIDER = 'deepseek-official'
|
||||
|
||||
const DEFAULT_MODELS: DeepSeekCatalogModel[] = [
|
||||
{ id: 'deepseek-v4-flash', name: 'DeepSeek-V4-Flash', contextWindow: 256_000 },
|
||||
@@ -89,11 +90,13 @@ export const Config: z<Config> = z.object({
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
export const PUBLIC_BASE_URL = 'https://api.deepseek.com'
|
||||
|
||||
/** Connection facts plus the plugin-consumed credential reference. */
|
||||
export interface ResolvedDeepSeekOptions extends DeepSeekConnectionOptions {
|
||||
/** Reference resolved per request when no literal key is configured. */
|
||||
apiKeyEnv: CredentialRef
|
||||
}
|
||||
/**
|
||||
* One resolution's complete request facts. Connection and credential facts
|
||||
* are one value on purpose: a snapshot the resolver rejects keeps the whole
|
||||
* previous generation, so a request can never pair a stale endpoint with a
|
||||
* newer key.
|
||||
*/
|
||||
export type ResolvedDeepSeekOptions = DeepSeekConnectionOptions
|
||||
|
||||
/** Resolve, validate, and detach the advisory model catalog. */
|
||||
function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): DeepSeekCatalogModel[] {
|
||||
@@ -147,6 +150,7 @@ export function resolveAdapterOptions(config: Config): ResolvedDeepSeekOptions {
|
||||
)
|
||||
}
|
||||
return {
|
||||
...config.apiKey !== undefined && config.apiKey.length > 0 ? { apiKey: config.apiKey } : {},
|
||||
apiKeyEnv: credentialRef(config.apiKeyEnv ?? DEFAULT_API_KEY_ENV),
|
||||
baseURL: config.baseURL ?? process.env.DEEPSEEK_BASE_URL ?? PUBLIC_BASE_URL,
|
||||
defaults: {
|
||||
@@ -187,10 +191,11 @@ export function apply(ctx: Context, config: Config): void {
|
||||
}
|
||||
options()
|
||||
|
||||
const resolveApiKey = async (): Promise<string> => {
|
||||
const raw = current()
|
||||
if (raw.apiKey !== undefined && raw.apiKey.length > 0) return raw.apiKey
|
||||
const ref = options().apiKeyEnv
|
||||
const resolveApiKey = async (connection: ResolvedDeepSeekOptions): Promise<string> => {
|
||||
// Every credential fact comes from the caller's snapshot, so a rejected
|
||||
// settings generation cannot leak its key onto the previous endpoint.
|
||||
if (connection.apiKey !== undefined) return connection.apiKey
|
||||
const ref = connection.apiKeyEnv
|
||||
const credentials = ctx.get('credentials')
|
||||
if (credentials !== undefined) {
|
||||
const hit = await credentials.resolve(ref)
|
||||
@@ -202,19 +207,20 @@ export function apply(ctx: Context, config: Config): void {
|
||||
if (ambient !== undefined && ambient.length > 0) return ambient
|
||||
}
|
||||
throw new LlmError(
|
||||
'llm-deepseek: no API key for provider route "deepseek-official"; set the llm-deepseek "apiKey" setting,'
|
||||
+ ` store ${ref} with the credentials service, or export ${ref}`,
|
||||
`llm-deepseek: no API key for provider route "${PROVIDER}"; store ${ref} through the credentials`
|
||||
+ ` service (the web Models page writes it), export ${ref} in the launching environment, or — as a`
|
||||
+ ' last resort — set a literal "apiKey" in the llm-deepseek settings section',
|
||||
'MISSING_CREDENTIAL',
|
||||
)
|
||||
}
|
||||
|
||||
const adapter = new DeepSeekAdapter({ options, resolveApiKey })
|
||||
ctx.llm.registerConfigurableProviders([
|
||||
{ provider: 'deepseek-official', displayName: 'DeepSeek', settingsNs: NS, settingsPath: [] },
|
||||
{ provider: PROVIDER, displayName: 'DeepSeek', settingsNs: NS, settingsPath: [] },
|
||||
])
|
||||
// Route effects bind to this apply fiber via the stable `ctx` reference,
|
||||
// even when a swap runs inside the scoped settings callback below.
|
||||
let disposeRoute = ctx.llm.registerAdapter(['deepseek-official'], adapter)
|
||||
let disposeRoute = ctx.llm.registerAdapter([PROVIDER], adapter)
|
||||
let registeredPolicy = options().retryPolicy
|
||||
const ensureRegistrationFacts = (): void => {
|
||||
const policy = options().retryPolicy
|
||||
@@ -223,17 +229,10 @@ export function apply(ctx: Context, config: Config): void {
|
||||
// fact per-request resolution cannot refresh: swap the registration in one
|
||||
// synchronous section (same adapter instance, no NO_ADAPTER window).
|
||||
disposeRoute()
|
||||
disposeRoute = ctx.llm.registerAdapter(['deepseek-official'], adapter)
|
||||
disposeRoute = ctx.llm.registerAdapter([PROVIDER], adapter)
|
||||
registeredPolicy = policy
|
||||
}
|
||||
|
||||
void resolveApiKey().then(() => undefined, () => {
|
||||
// Expected on a first boot with dynamic sources: the route stays
|
||||
// registered (the catalog is browsable) and each request fails with the
|
||||
// actionable MISSING_CREDENTIAL message until a key arrives.
|
||||
ctx.logger.warn('llm-deepseek: no API key resolved yet for route "deepseek-official"; requests will fail until one is configured')
|
||||
})
|
||||
|
||||
installSettingsSection(ctx, NS, Config, config, {
|
||||
setSource: (source) => {
|
||||
current = source
|
||||
|
||||
@@ -824,8 +824,31 @@ describe('plugin registration and config', () => {
|
||||
await expect(ctx.llm.listModels('deepseek-official')).resolves.toHaveLength(2)
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toMatchObject({ code: 'MISSING_CREDENTIAL' })
|
||||
// The guidance leads with the credential store — the path that keeps the
|
||||
// secret out of configuration files — and mentions a literal key last.
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/store DEEPSEEK_API_KEY with the credentials service, or export DEEPSEEK_API_KEY/)
|
||||
.rejects.toThrow(/store DEEPSEEK_API_KEY through the credentials service.*as a last resort.*"apiKey"/s)
|
||||
})
|
||||
|
||||
it('reads the ambient variable when no credentials seam is mounted', async () => {
|
||||
// The plain cordis.yml composition: no credential provider, the key in
|
||||
// the launching environment.
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'ambient-key')
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { baseURL: server.url })
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer ambient-key')
|
||||
})
|
||||
|
||||
it('treats an empty ambient variable as no key when no credentials seam is mounted', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { baseURL: 'http://127.0.0.1:1' })
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toMatchObject({ code: 'MISSING_CREDENTIAL' })
|
||||
})
|
||||
|
||||
it('prefers explicit config over env for key and base URL', async () => {
|
||||
|
||||
@@ -143,6 +143,29 @@ describe('request-level dynamic configuration', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('sends the whole last-good snapshot when a rejected one changed both the key and the URL', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const dir = await home()
|
||||
const good = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const rejected = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const { ctx } = await boot(dir, { apiKey: 'good-key', baseURL: good.url })
|
||||
|
||||
// One snapshot moves the endpoint AND the literal key, and fails the
|
||||
// resolve step beyond the schema (duplicate catalog ids).
|
||||
await ctx.settings.update(NS, {
|
||||
apiKey: 'rejected-key',
|
||||
baseURL: rejected.url,
|
||||
models: [{ id: 'dup' }, { id: 'dup' }],
|
||||
})
|
||||
|
||||
await prompt(ctx)
|
||||
// The rejected generation contributes nothing: not its endpoint, and — the
|
||||
// regression this pins — not its key either.
|
||||
expect(rejected.requests).toHaveLength(0)
|
||||
expect(good.requests).toHaveLength(1)
|
||||
expect(good.headers[0]?.authorization).toBe('Bearer good-key')
|
||||
})
|
||||
|
||||
it('falls back to the composition entry when settings detach', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const dir = await home()
|
||||
|
||||
@@ -41,12 +41,15 @@ afterEach(async () => {
|
||||
})
|
||||
|
||||
async function loadComposition(
|
||||
options: { withDynamic: boolean; baseURL: string },
|
||||
options: { withDynamic: boolean; baseURL: string; reuseRoot?: string },
|
||||
): Promise<{ ctx: Context; settingsPath: string; envPath: string }> {
|
||||
root = await mkdtemp(join(tmpdir(), 'dsh-llm-composition-'))
|
||||
// A reused root is the restart case: the same harness home, its documents
|
||||
// exactly as the previous process left them.
|
||||
const fresh = options.reuseRoot === undefined
|
||||
root = options.reuseRoot ?? await mkdtemp(join(tmpdir(), 'dsh-llm-composition-'))
|
||||
const settingsPath = join(root, 'settings.yaml')
|
||||
const envPath = join(root, '.env')
|
||||
if (options.withDynamic) {
|
||||
if (options.withDynamic && fresh) {
|
||||
await writeFile(settingsPath, '# personal settings\n')
|
||||
await writeFile(envPath, 'DEEPSEEK_API_KEY=boot-key\n')
|
||||
}
|
||||
@@ -129,6 +132,35 @@ describe('llm-deepseek real dynamic composition', () => {
|
||||
expect(serverB.headers[0]?.authorization).toBe('Bearer rotated-key')
|
||||
})
|
||||
|
||||
it('keeps a stored key writable and rotatable across a real restart', async () => {
|
||||
// No ambient DEEPSEEK_API_KEY: the shipped surfaces no longer hoist
|
||||
// $DSH_HOME/.env into process.env, so a stored key must stay file-sourced.
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const first = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const second = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const boot = await loadComposition({ withDynamic: true, baseURL: first.url })
|
||||
const home = root!
|
||||
await boot.ctx.get('credentials')!.set(KEY_REF, 'stored-by-ui')
|
||||
expect(await boot.ctx.get('credentials')!.describe(KEY_REF))
|
||||
.toEqual({ configured: true, source: 'file', writable: true })
|
||||
await assemble(boot.ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(first.headers[0]?.authorization).toBe('Bearer stored-by-ui')
|
||||
await boot.ctx.fiber.dispose()
|
||||
context = undefined
|
||||
|
||||
// Restart over the same harness home.
|
||||
const restarted = await loadComposition({ withDynamic: true, baseURL: second.url, reuseRoot: home })
|
||||
const credentials = restarted.ctx.get('credentials')!
|
||||
// The stored key is still the provider's own writable file entry — not a
|
||||
// read-only launch override, which is what hoisting it would have made it.
|
||||
expect(await credentials.resolve(KEY_REF)).toEqual({ value: 'stored-by-ui', source: 'file' })
|
||||
expect(await credentials.describe(KEY_REF)).toEqual({ configured: true, source: 'file', writable: true })
|
||||
// Rotation still works after the restart, and the next request uses it.
|
||||
await credentials.set(KEY_REF, 'rotated-after-restart')
|
||||
await assemble(restarted.ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(second.headers[0]?.authorization).toBe('Bearer rotated-after-restart')
|
||||
})
|
||||
|
||||
it('boots the same adapter without settings or credentials entries on entry config alone', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: f2d030087c9cd704724ce4adf38f3f8111e87063
|
||||
README.zh.md: bcecd3394693a95e1fe2127b44e6f60030518cca
|
||||
README.md: 75442e2f1f6578ed458d05302b1cb6b063e26092
|
||||
README.zh.md: 3baed4e30bbce6ce21c52da79369ad096bbbb752
|
||||
|
||||
@@ -8,7 +8,7 @@ The package root exposes the Cordis plugin contract and `PiAiAdapter`; profile r
|
||||
|
||||
## Config
|
||||
|
||||
Configure credentials and deployment-specific transport settings per provider, keyed by the provider route itself. Prefer `apiKeyEnv` — a credential *reference* resolved per request — over a literal `apiKey`, so no secret enters this file; omitting both delegates authentication to pi-ai's provider-native ambient discovery. `baseURL` overrides only the endpoint of the selected catalog model, preserving its API family and compatibility metadata, so private proxies such as `https://proxy.example.com:8443` remain supported.
|
||||
Configure credentials and deployment-specific transport settings per provider, keyed by the provider route itself. Prefer `apiKeyEnv` — a credential *reference* resolved per request — over a literal `apiKey`, so no secret enters this file. Omitting **both** is what delegates authentication to pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. `baseURL` overrides only the endpoint of the selected catalog model, preserving its API family and compatibility metadata, so private proxies such as `https://proxy.example.com:8443` remain supported.
|
||||
|
||||
```yaml
|
||||
- id: llm
|
||||
@@ -41,7 +41,7 @@ Each dict key must exist in pi-ai's installed catalog; the dict shape makes dupl
|
||||
|
||||
The adapter reads its profiles through a thunk **once per operation** instead of freezing them at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`, and because `providers` is a dict, the base and the user's `llm-pi-ai:` settings section merge **per provider**: a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. Without a mounted settings service the entry config alone drives the adapter, unchanged.
|
||||
|
||||
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; the raw environment variable without a mounted seam), then pi-ai's ambient discovery. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin re-registers the same adapter instance in one synchronous section, so `ctx.llm.listProviders()` and `providerRetryPolicy()` always reflect the current configuration. A live settings snapshot naming an unknown provider (or failing any other resolver bound) keeps the last good profiles and logs the failure; the entry config itself still fails plugin load.
|
||||
Credentials resolve per stream call: a non-empty literal `apiKey` wins, then `apiKeyEnv` through the optional `ctx.credentials` seam (`$DSH_HOME/.env` under the live environment; exactly that variable without a mounted seam). A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin replaces its registration atomically (same adapter instance, candidate set validated first), so a route another adapter already owns leaves the previous routes serving and reverting to a working configuration re-applies. Provider key order never counts as a change. A live settings snapshot naming an unknown provider (or failing any other resolver bound) keeps the last good profiles and logs the failure; the entry config itself still fails plugin load.
|
||||
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers.
|
||||
|
||||
@@ -77,7 +77,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers: a settings-born route registers live and drops when the user layer resets, `apiKeyEnv` credentials rotate between requests, and an unknown-provider snapshot keeps the last good profiles. Real-API coverage remains key-gated under `pnpm run test:e2e`.
|
||||
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers: a settings-born route registers live and drops when the user layer resets, `apiKeyEnv` credentials rotate between requests, and an unknown-provider snapshot keeps the last good profiles. `tests/loader-composition.spec.ts` boots the dormant posture from a test-only `cordis.yml` through the actual Loader and registers its route from an on-disk `settings.yaml` edit. Real-API coverage remains key-gated under `pnpm run test:e2e`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
## 配置
|
||||
|
||||
按提供方配置凭据与部署特定传输设置,并以提供方路由本身为键。优先使用 `apiKeyEnv`——按请求解析的凭据*引用*——而非字面 `apiKey`,让机密不进入该文件;两者都省略则把认证委托给 pi-ai 的提供方原生环境发现。`baseURL` 只会覆盖所选 catalog 模型的端点,保留其 API 家族与兼容性元数据,因此仍支持 `https://proxy.example.com:8443` 等私有 proxy。
|
||||
按提供方配置凭据与部署特定传输设置,并以提供方路由本身为键。优先使用 `apiKeyEnv`——按请求解析的凭据*引用*——而非字面 `apiKey`,让机密不进入该文件。**两者**都省略,才会把认证委托给 pi-ai 的提供方原生环境发现;已配置却解析不出任何值的引用则相反,会让请求以 `MISSING_CREDENTIAL` 失败,因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。`baseURL` 只会覆盖所选 catalog 模型的端点,保留其 API 家族与兼容性元数据,因此仍支持 `https://proxy.example.com:8443` 等私有 proxy。
|
||||
|
||||
```yaml
|
||||
- id: llm
|
||||
@@ -41,7 +41,7 @@
|
||||
|
||||
适配器经由一个 thunk **每操作读取一次** profile,而非在构造期冻结。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace,并以其 `cordis.yml` 条目为组合 `base`;由于 `providers` 是字典,base 与用户的 `llm-pi-ai:` settings 分节**按提供方**合并:用户可以新增路由、覆盖组合路由的单个字段,或把路由指向另一个 proxy,全部在下一次请求生效,无需重启。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。
|
||||
|
||||
凭据按每次 stream 调用解析:非空的字面 `apiKey` 优先,其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`(活跃环境之下的 `$DSH_HOME/.env`;未挂载 seam 时为原始环境变量),最后是 pi-ai 的环境发现。路由集合与每条路由捕获的重试策略是注册级事实:两者任一变化时,插件都会在一个同步区段内重新注册同一适配器实例,因此 `ctx.llm.listProviders()` 与 `providerRetryPolicy()` 始终反映当前配置。存活 settings 快照若点名未知提供方(或违反任何其他 resolver 约束),则保留最后可用 profile 并记录失败;entry 配置本身仍会使插件加载失败。
|
||||
凭据按每次 stream 调用解析:非空的字面 `apiKey` 优先,其次经可选的 `ctx.credentials` seam 解析 `apiKeyEnv`(活跃环境之下的 `$DSH_HOME/.env`;未挂载 seam 时恰好读取该环境变量)。只有完全没有点名任何凭据的 profile——仅限这一种情况——才交给 pi-ai 的环境发现。路由集合与每条路由捕获的重试策略是注册级事实:两者任一变化时,插件都会原子地替换自己的注册(同一适配器实例,候选集合先经校验),因此某条路由若已被另一适配器占有,先前的路由会继续服务,而改回可用配置时注册会重新生效。提供方键的顺序绝不算作变化。存活 settings 快照若点名未知提供方(或违反任何其他 resolver 约束),则保留最后可用 profile 并记录失败;entry 配置本身仍会使插件加载失败。
|
||||
|
||||
适配器通过 `ctx.llm.listModels(provider)` 公开每个已配置提供方已安装的 pi-ai 模型。这是从 `getModels(provider)` 派生的提供方无关 selector 元数据;请求时解析仍会执行权威 catalog 查找,因此发现不会创建第二个模型注册表。`ctx.llm.resolveModelInfo(provider, model)` 会执行一次精确 descriptor 查找,并返回其身份、上下文窗口和可选思考级别,让权威元数据保留在拥有路由的适配器上,而非消费方。
|
||||
|
||||
@@ -77,7 +77,7 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试使用重定向到本地 mock 服务器的 pi-ai catalog 模型,覆盖提供方/profile 路由、每次适配器调用只发起一个协议请求、idle-timeout 响应终止、调用方 abort、原生 API 选择、端点覆盖、归因、转换、回放状态验证,以及一个适配器实例内的跨提供方/模型回放。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider:settings 里新生的路由实时完成注册,并在用户层重置时随之移除,`apiKeyEnv` 凭据在两次请求之间轮换,点名未知提供方的快照则保留最后可用 profile。真实 API 覆盖仍需 key 才会启用,并通过 `pnpm run test:e2e` 运行。
|
||||
单元测试使用重定向到本地 mock 服务器的 pi-ai catalog 模型,覆盖提供方/profile 路由、每次适配器调用只发起一个协议请求、idle-timeout 响应终止、调用方 abort、原生 API 选择、端点覆盖、归因、转换、回放状态验证,以及一个适配器实例内的跨提供方/模型回放。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider:settings 里新生的路由实时完成注册,并在用户层重置时随之移除,`apiKeyEnv` 凭据在两次请求之间轮换,点名未知提供方的快照则保留最后可用 profile。`tests/loader-composition.spec.ts` 从仅测试用的 `cordis.yml` 出发,经真实 Loader 拉起休眠姿态,并从磁盘上的一次 `settings.yaml` 编辑注册出它的路由。真实 API 覆盖仍需 key 才会启用,并通过 `pnpm run test:e2e` 运行。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -41,9 +41,11 @@ export interface PiAiAdapterOptions {
|
||||
/**
|
||||
* Resolve the credential for one already-resolved profile; called once per
|
||||
* stream call and frozen for that call. `undefined` defers to pi-ai's
|
||||
* provider-native ambient discovery.
|
||||
* provider-native ambient discovery, which the plugin allows only for a
|
||||
* profile naming no credential at all; a named reference that misses throws
|
||||
* `LlmError` `MISSING_CREDENTIAL` rather than falling back.
|
||||
*/
|
||||
resolveApiKey: (profile: ResolvedPiAiProviderProfile) => Promise<string | undefined>
|
||||
resolveApiKey: (provider: string, profile: ResolvedPiAiProviderProfile) => Promise<string | undefined>
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -180,7 +182,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
model,
|
||||
options.reasoningEffort ?? profile.reasoning,
|
||||
)
|
||||
const apiKey = await this.config.resolveApiKey(profile)
|
||||
const apiKey = await this.config.resolveApiKey(options.provider, profile)
|
||||
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
|
||||
@@ -30,7 +30,8 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { AdapterRegistrationHandle } from '@deepseek-ai/dsh-llm'
|
||||
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import { PiAiAdapter } from './adapter.ts'
|
||||
import { Config, resolveProfiles } from './config.ts'
|
||||
@@ -46,9 +47,15 @@ export const inject = ['llm']
|
||||
|
||||
const NS = settingsNamespace('llm-pi-ai')
|
||||
|
||||
/** The registry captures these per route; a change here must re-register. */
|
||||
/**
|
||||
* The registry captures these per route; a change here must re-register.
|
||||
* Sorted by provider so a settings document that merely reorders its keys is
|
||||
* not mistaken for a route change.
|
||||
*/
|
||||
function registrationFacts(profiles: ReadonlyMap<string, ResolvedPiAiProviderProfile>): unknown {
|
||||
return [...profiles.entries()].map(([provider, profile]) => ({ provider, retryPolicy: profile.retryPolicy }))
|
||||
return [...profiles.entries()]
|
||||
.map(([provider, profile]) => ({ provider, retryPolicy: profile.retryPolicy }))
|
||||
.sort((left, right) => left.provider.localeCompare(right.provider))
|
||||
}
|
||||
|
||||
/** Register one generic pi-ai adapter for all configured provider routes. */
|
||||
@@ -77,17 +84,31 @@ export function apply(ctx: Context, config: Config): void {
|
||||
}
|
||||
profiles()
|
||||
|
||||
const resolveApiKey = async (profile: ResolvedPiAiProviderProfile): Promise<string | undefined> => {
|
||||
const resolveApiKey = async (
|
||||
provider: string,
|
||||
profile: ResolvedPiAiProviderProfile,
|
||||
): Promise<string | undefined> => {
|
||||
if (profile.apiKey !== undefined) return profile.apiKey
|
||||
const ref = profile.apiKeyEnv
|
||||
// Only a profile that names no credential at all defers to pi-ai's
|
||||
// provider-native discovery. Once one is named, a miss must fail loud:
|
||||
// handing pi-ai `undefined` would let it pick up an unrelated ambient key
|
||||
// (OPENAI_API_KEY and friends), billing another tenant for a request the
|
||||
// deployment meant to authenticate differently.
|
||||
if (ref === undefined) return undefined
|
||||
const credentials = ctx.get('credentials')
|
||||
if (credentials !== undefined) return (await credentials.resolve(ref))?.value
|
||||
// Without the seam, keep an ambient fallback so a plain cordis.yml
|
||||
// composition works from the environment alone; an empty variable defers
|
||||
// to pi-ai's own provider-native discovery like an absent one.
|
||||
const ambient = process.env[ref]
|
||||
return ambient !== undefined && ambient.length > 0 ? ambient : undefined
|
||||
const hit = credentials !== undefined
|
||||
? (await credentials.resolve(ref))?.value
|
||||
// Without the seam, read exactly the named variable so a plain
|
||||
// cordis.yml composition works from the environment alone.
|
||||
: process.env[ref]
|
||||
if (hit !== undefined && hit.length > 0) return hit
|
||||
throw new LlmError(
|
||||
`llm-pi-ai: no credential for provider route "${provider}"; its profile resolves ${ref}, which is not`
|
||||
+ ` set — store ${ref} through the credentials service (the web Models page writes it) or export it,`
|
||||
+ ' and remove apiKeyEnv only if this provider should authenticate from pi-ai\'s own environment discovery',
|
||||
'MISSING_CREDENTIAL',
|
||||
)
|
||||
}
|
||||
|
||||
const adapter = new PiAiAdapter({ profiles, resolveApiKey })
|
||||
@@ -104,18 +125,29 @@ export function apply(ctx: Context, config: Config): void {
|
||||
// even when a swap runs inside the scoped settings callback below. A bare
|
||||
// mount (zero routes) is the dormant posture: nothing registers until a
|
||||
// settings section supplies profiles, and routes drop when it empties.
|
||||
let disposeRoutes: (() => void) | undefined
|
||||
let registration: AdapterRegistrationHandle | undefined
|
||||
let registeredFacts: unknown
|
||||
const ensureRegistrationFacts = (): void => {
|
||||
const facts = registrationFacts(profiles())
|
||||
if (deepEqualJson(facts, registeredFacts)) return
|
||||
// The registry captures the route set and each route's retry policy at
|
||||
// registration: swap the registration in one synchronous section (same
|
||||
// adapter instance, no NO_ADAPTER window).
|
||||
disposeRoutes?.()
|
||||
disposeRoutes = undefined
|
||||
// registration, so a change to either must re-register. The swap is
|
||||
// atomic (same adapter instance, validated before anything moves): a
|
||||
// conflicting route leaves the previous routes serving requests, and
|
||||
// `registeredFacts` only advances once the registry actually holds the
|
||||
// new set — so returning to a working configuration always re-applies.
|
||||
const routes = [...profiles().keys()]
|
||||
if (routes.length > 0) disposeRoutes = ctx.llm.registerAdapter(routes, adapter)
|
||||
if (registration === undefined) {
|
||||
// Dormant bare mount: nothing is registered until a section supplies
|
||||
// profiles, and an empty section keeps it that way.
|
||||
if (routes.length === 0) {
|
||||
registeredFacts = facts
|
||||
return
|
||||
}
|
||||
registration = ctx.llm.registerAdapter(routes, adapter)
|
||||
} else {
|
||||
registration.replace(routes)
|
||||
}
|
||||
registeredFacts = facts
|
||||
}
|
||||
ensureRegistrationFacts()
|
||||
|
||||
@@ -27,7 +27,7 @@ async function harness(baseURL: string, overrides: Record<string, unknown> = {})
|
||||
function adapterOf(providers: Record<string, LlmPiAi.PiAiProviderProfile>): PiAiAdapter {
|
||||
return new PiAiAdapter({
|
||||
profiles: () => resolveProfiles(providers),
|
||||
resolveApiKey: profile => Promise.resolve(profile.apiKey),
|
||||
resolveApiKey: (_provider, profile) => Promise.resolve(profile.apiKey),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -385,13 +385,19 @@ describe('provider profile lifecycle', () => {
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer custom-ref-key')
|
||||
})
|
||||
|
||||
it('treats an empty apiKeyEnv variable as absent and defers to SDK ambient discovery', async () => {
|
||||
it('fails a named-but-missing apiKeyEnv instead of using another ambient key', async () => {
|
||||
// The exact confusion this guards: the named reference is empty while an
|
||||
// unrelated provider key sits in the environment. Deferring to pi-ai's own
|
||||
// discovery here would authenticate as another tenant.
|
||||
vi.stubEnv('PI_CUSTOM_REF_KEY', '')
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'ambient-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, { apiKey: undefined, apiKeyEnv: 'PI_CUSTOM_REF_KEY' })
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer ambient-key')
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toMatchObject({ code: 'MISSING_CREDENTIAL' })
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/provider route "deepseek".*PI_CUSTOM_REF_KEY/s)
|
||||
expect(server.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('validates empty, unknown, legacy-shaped, and explicitly blank profiles', () => {
|
||||
|
||||
@@ -3,7 +3,7 @@ import { Context } from 'cordis'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import { CredentialsLocal } from '@deepseek-ai/dsh-credentials-local'
|
||||
import { settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
@@ -14,6 +14,14 @@ import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
|
||||
|
||||
const NS = settingsNamespace('llm-pi-ai')
|
||||
|
||||
/** Minimal foreign adapter: only needs to own a route the pi-ai plugin then wants. */
|
||||
class StubAdapter extends LlmAdapter {
|
||||
|
||||
override async * stream(): AsyncIterable<never> {
|
||||
throw new Error('stub adapter must never stream')
|
||||
}
|
||||
}
|
||||
|
||||
const cleanups: Array<() => Promise<void>> = []
|
||||
|
||||
afterEach(async () => {
|
||||
@@ -147,4 +155,45 @@ describe('request-level dynamic profiles', () => {
|
||||
await ctx.settings.update(NS, { providers: { 'not-a-real-provider': {} } })
|
||||
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
|
||||
})
|
||||
|
||||
it('keeps serving its routes when a settings-born route collides with another adapter', async () => {
|
||||
const dir = await home()
|
||||
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
|
||||
const ctx = await boot(dir, { providers: { openai: { apiKey: 'pk', baseURL: `${server.url}/v1` } } })
|
||||
// Another adapter owns `anthropic`; the registry must refuse to hand it over.
|
||||
ctx.llm.registerAdapter(['anthropic'], new StubAdapter())
|
||||
|
||||
await ctx.settings.update(NS, {
|
||||
providers: {
|
||||
openai: { apiKey: 'pk', baseURL: `${server.url}/v1` },
|
||||
anthropic: { apiKey: 'other' },
|
||||
},
|
||||
})
|
||||
|
||||
// The conflicting swap was refused whole: the previous route set still
|
||||
// owns openai (an eager dispose would have dropped it), and anthropic
|
||||
// still belongs to its original adapter.
|
||||
expect(ctx.llm.listProviders().map(provider => provider.id).sort()).toEqual(['anthropic', 'openai'])
|
||||
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
|
||||
expect(result.finish.kind).toBe('error')
|
||||
expect(server.paths).toEqual(['/v1/responses'])
|
||||
|
||||
// Reverting to the working configuration re-applies, even though its
|
||||
// facts equal the ones the registry already holds.
|
||||
await ctx.settings.replace(NS, {})
|
||||
expect(ctx.llm.listProviders().map(provider => provider.id).sort()).toEqual(['anthropic', 'openai'])
|
||||
await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
|
||||
expect(server.paths).toEqual(['/v1/responses', '/v1/responses'])
|
||||
})
|
||||
|
||||
it('ignores a settings document that merely reorders its provider keys', async () => {
|
||||
const dir = await home()
|
||||
const ctx = await boot(dir, { providers: { openai: {}, anthropic: {} } })
|
||||
const before = ctx.llm.listProviders().map(provider => provider.id)
|
||||
|
||||
// Same routes, different YAML key order: nothing about the registration
|
||||
// changed, so no swap should happen at all.
|
||||
await ctx.settings.update(NS, { providers: { anthropic: {}, openai: {} } })
|
||||
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(before)
|
||||
})
|
||||
})
|
||||
|
||||
116
packages/llm/llm-pi-ai/tests/loader-composition.spec.ts
Normal file
116
packages/llm/llm-pi-ai/tests/loader-composition.spec.ts
Normal file
@@ -0,0 +1,116 @@
|
||||
/**
|
||||
* Real-composition guard for the dormant pi-ai posture: LlmService,
|
||||
* settings-local, credentials-local, and a bare `llm-pi-ai` row boot from a
|
||||
* test-only cordis.yml through the actual Loader + Include path, an external
|
||||
* edit of settings.yaml registers the route live, and the next request
|
||||
* carries the credential the .env supplies. A hand-mounted `ctx.plugin` cannot
|
||||
* catch Loader export-shape failures, which is why the twin adapter has the
|
||||
* same guard.
|
||||
*/
|
||||
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import Include from '@cordisjs/plugin-include'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import CredentialsLocal from '@deepseek-ai/dsh-credentials-local'
|
||||
import SettingsLocal from '@deepseek-ai/dsh-settings-local'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { assemble } from './assemble.ts'
|
||||
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
|
||||
|
||||
let root: string | undefined
|
||||
let context: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await context?.fiber.dispose()
|
||||
context = undefined
|
||||
if (root !== undefined) await rm(root, { recursive: true, force: true })
|
||||
root = undefined
|
||||
await closeMockServers()
|
||||
vi.unstubAllEnvs()
|
||||
})
|
||||
|
||||
/** Boot the dormant composition: a bare `llm-pi-ai` row with no config at all. */
|
||||
async function loadComposition(): Promise<{ ctx: Context; settingsPath: string }> {
|
||||
root = await mkdtemp(join(tmpdir(), 'dsh-pi-composition-'))
|
||||
const settingsPath = join(root, 'settings.yaml')
|
||||
await writeFile(settingsPath, '# personal settings\n')
|
||||
await writeFile(join(root, '.env'), 'PI_COMPOSITION_KEY=key-from-store\n')
|
||||
|
||||
const configPath = join(root, 'cordis.yml')
|
||||
await writeFile(configPath, [
|
||||
'- id: llm',
|
||||
" name: 'test-llm-service'",
|
||||
'- id: settings',
|
||||
" name: '@deepseek-ai/dsh-settings-local'",
|
||||
' config:',
|
||||
` path: ${JSON.stringify(settingsPath)}`,
|
||||
' debounceMs: 10',
|
||||
'- id: credentials',
|
||||
" name: '@deepseek-ai/dsh-credentials-local'",
|
||||
' config:',
|
||||
` path: ${JSON.stringify(join(root, '.env'))}`,
|
||||
' debounceMs: 10',
|
||||
'- id: llm-pi-ai',
|
||||
" name: '@deepseek-ai/dsh-llm-pi-ai'",
|
||||
'',
|
||||
].join('\n'))
|
||||
|
||||
const ctx = new Context()
|
||||
context = ctx
|
||||
ctx.baseUrl = pathToFileURL(root).href + '/'
|
||||
await ctx.plugin(Loader)
|
||||
ctx.loader.builtins.include = Include
|
||||
const modules = new Map<string, unknown>([
|
||||
['test-llm-service', LlmService],
|
||||
['@deepseek-ai/dsh-settings-local', SettingsLocal],
|
||||
['@deepseek-ai/dsh-credentials-local', CredentialsLocal],
|
||||
['@deepseek-ai/dsh-llm-pi-ai', LlmPiAi],
|
||||
])
|
||||
ctx.loader.internal = {
|
||||
version: 'v2',
|
||||
async import(specifier: string) {
|
||||
if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`)
|
||||
return modules.get(specifier)
|
||||
},
|
||||
} as unknown as NonNullable<typeof ctx.loader.internal>
|
||||
await ctx.loader.create({
|
||||
name: 'cordis:include',
|
||||
config: { path: pathToFileURL(configPath).href },
|
||||
})
|
||||
await ctx.loader.await()
|
||||
return { ctx, settingsPath }
|
||||
}
|
||||
|
||||
describe('llm-pi-ai real dormant composition', () => {
|
||||
it('boots with zero routes and registers one the moment settings supply a profile', async () => {
|
||||
vi.stubEnv('PI_COMPOSITION_KEY', '')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const { ctx, settingsPath } = await loadComposition()
|
||||
|
||||
// The shipped posture: the adapter exists, no route does.
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
|
||||
// Exactly what the web Models page leaves on disk.
|
||||
await writeFile(settingsPath, [
|
||||
'llm-pi-ai:',
|
||||
' providers:',
|
||||
' deepseek:',
|
||||
' apiKeyEnv: PI_COMPOSITION_KEY',
|
||||
` baseURL: ${server.url}`,
|
||||
'',
|
||||
].join('\n'))
|
||||
await vi.waitFor(() => {
|
||||
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['deepseek'])
|
||||
}, { timeout: 5000 })
|
||||
|
||||
const result = await assemble(ctx, { provider: 'deepseek', model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer key-from-store')
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 12bf3ca9901a7027111761a6d523862f9f7e150b
|
||||
README.zh.md: d39b6c518f332e0778e10b73bdcf1ddd25433ab3
|
||||
README.md: b338f4d07ae4c5e3dd9a04ad0765fb8b6d6a99a7
|
||||
README.zh.md: 11d4c92573e2bdba74d1f7e04b445a9039824267
|
||||
|
||||
@@ -10,7 +10,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
### Public API
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` Declare provider routes an adapter plugin can activate through configuration — registered or dormant — each naming its owning settings namespace and the path to its profile inside that section. All-or-nothing (`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`), disposed with the calling fiber.
|
||||
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` List the declared directory in declaration order; configuration surfaces merge it with `listProviders()` to mark each entry live or dormant.
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
### 公开 API
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。返回的释放器还携带 `replace(providers)`:候选路由集合会在任何东西变动之前完整校验,因此与另一适配器冲突时,当前路由保持注册且继续服务,而替换本身是一个同步区段,不存在可观察的空档。`replace([])` 合法——一个持有零条路由的注册——这与空的初始注册不同。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): () => void` 声明适配器插件可通过配置激活的提供方路由——无论已注册还是休眠——每个条目指明其所属 settings namespace,以及 profile 在该分节内的路径。要么全部成功,要么全部不生效(`INVALID_DIRECTORY`/`DUPLICATE_DIRECTORY`),并随调用 fiber dispose。
|
||||
- `ctx.llm.listConfigurableProviders(): LlmConfigurableProvider[]` 按声明顺序列出已声明的目录;配置界面将其与 `listProviders()` 合并,为每个条目标注存活或休眠。
|
||||
|
||||
@@ -196,6 +196,26 @@ export abstract class LlmAdapter {
|
||||
abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
}
|
||||
|
||||
/**
|
||||
* What {@link LlmService.registerAdapter} returns: the disposer, plus an
|
||||
* atomic route replacement for the same adapter instance.
|
||||
*/
|
||||
export interface AdapterRegistrationHandle {
|
||||
/** Release every route this registration currently holds. */
|
||||
(): void
|
||||
/**
|
||||
* Replace this registration's routes with `providers`, keeping the same
|
||||
* adapter instance. The candidate set is validated in full first — a
|
||||
* conflict with another adapter, an invalid name, or bad provider metadata
|
||||
* throws and leaves the current routes untouched — and the swap itself is
|
||||
* one synchronous section, so no request can observe a gap. An empty array
|
||||
* is legal here (a settings section that emptied holds zero routes while
|
||||
* staying registered), unlike an empty initial registration.
|
||||
* @param providers - the complete next route set for this registration.
|
||||
*/
|
||||
replace(providers: string[]): void
|
||||
}
|
||||
|
||||
/**
|
||||
* The abstract `llm` service: an adapter registry plus a streaming model-call
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
@@ -235,41 +255,74 @@ export class LlmService extends Service {
|
||||
* Disposed with the fiber.
|
||||
* @param providers - every provider route this adapter should serve.
|
||||
* @param adapter - the adapter that streams calls for those providers.
|
||||
* @returns the disposer that unregisters all of them.
|
||||
* @returns the disposer, carrying {@link AdapterRegistrationHandle.replace}.
|
||||
*/
|
||||
registerAdapter(providers: string[], adapter: LlmAdapter): () => void {
|
||||
registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle {
|
||||
// The routes this registration currently holds; `replace` rewrites it, and
|
||||
// the disposer releases whatever it holds at disposal time.
|
||||
const owned = new Set<string>()
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
throw new LlmError(`an adapter for provider "${provider}" is already registered`, 'DUPLICATE_ADAPTER')
|
||||
}
|
||||
const info = adapter.providerInfo(provider)
|
||||
if (typeof info.id !== 'string' || info.id !== provider || typeof info.name !== 'string' || info.name.length === 0) {
|
||||
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
|
||||
}
|
||||
unique.add(provider)
|
||||
const retryPolicy = adapter.providerRetryPolicy(provider)
|
||||
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
|
||||
registrations.push({
|
||||
adapter,
|
||||
provider: { id: info.id, name: info.name },
|
||||
retryPolicy,
|
||||
})
|
||||
}
|
||||
for (const registration of registrations) this.adapters.set(registration.provider.id, registration)
|
||||
this.emitAdaptersUpdated()
|
||||
this.commitRoutes(owned, this.prepareRoutes(providers, adapter, owned))
|
||||
yield () => {
|
||||
for (const provider of providers) this.adapters.delete(provider)
|
||||
for (const provider of owned) this.adapters.delete(provider)
|
||||
owned.clear()
|
||||
this.emitAdaptersUpdated()
|
||||
}
|
||||
}.bind(this), 'llm.registerAdapter()')
|
||||
// ctx.effect's disposer returns Promise<void>; our disposer API is
|
||||
// synchronous fire-and-forget — discard the (always-resolved) promise.
|
||||
return () => void dispose()
|
||||
const handle = (() => void dispose()) as AdapterRegistrationHandle
|
||||
handle.replace = (next: string[]): void => {
|
||||
this.commitRoutes(owned, this.prepareRoutes(next, adapter, owned))
|
||||
}
|
||||
return handle
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate one candidate route set for `adapter`, treating routes this
|
||||
* registration already holds as available. Nothing is mutated: a rejected
|
||||
* candidate leaves the registry exactly as it was.
|
||||
*/
|
||||
private prepareRoutes(providers: string[], adapter: LlmAdapter, owned: ReadonlySet<string>): AdapterRegistration[] {
|
||||
const unique = new Set<string>()
|
||||
const registrations: AdapterRegistration[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || (this.adapters.has(provider) && !owned.has(provider))) {
|
||||
throw new LlmError(`an adapter for provider "${provider}" is already registered`, 'DUPLICATE_ADAPTER')
|
||||
}
|
||||
const info = adapter.providerInfo(provider)
|
||||
if (typeof info.id !== 'string' || info.id !== provider || typeof info.name !== 'string' || info.name.length === 0) {
|
||||
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
|
||||
}
|
||||
unique.add(provider)
|
||||
const retryPolicy = adapter.providerRetryPolicy(provider)
|
||||
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
|
||||
registrations.push({
|
||||
adapter,
|
||||
provider: { id: info.id, name: info.name },
|
||||
retryPolicy,
|
||||
})
|
||||
}
|
||||
return registrations
|
||||
}
|
||||
|
||||
/**
|
||||
* Swap this registration's routes for the prepared ones in one synchronous
|
||||
* section, so no observer can see the registry between the release and the
|
||||
* re-registration. The route set's one mutation point is also where
|
||||
* `llm/adapters-updated` is published, so a `replace` announces itself
|
||||
* exactly like a first registration.
|
||||
*/
|
||||
private commitRoutes(owned: Set<string>, registrations: readonly AdapterRegistration[]): void {
|
||||
for (const provider of owned) this.adapters.delete(provider)
|
||||
owned.clear()
|
||||
for (const registration of registrations) {
|
||||
this.adapters.set(registration.provider.id, registration)
|
||||
owned.add(registration.provider.id)
|
||||
}
|
||||
this.emitAdaptersUpdated()
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user