Merge remote-tracking branch 'origin/master' into codex/enforce-tool-cancellation
# Conflicts: # docs/event-producer-consumer.md # examples/acp-agent/tests/snapshots/bash-spill/session.jsonl # examples/acp-agent/tests/snapshots/escalation-approved/session.jsonl # examples/acp-agent/tests/snapshots/escalation-rejected/session.jsonl # examples/acp-agent/tests/snapshots/fs-escalation-approved/session.jsonl # examples/acp-agent/tests/snapshots/hook-cc-pretool-ask/session.jsonl # packages/context/workspace-context/tests/workspace-context.spec.ts # packages/core/agent/src/index.ts # packages/support/invariants/tests/invariants.spec.ts # packages/ui/acp/src/index.ts # packages/ui/tui/tests/harness.ts # packages/ui/tui/tests/tui.spec.ts
This commit is contained in:
@@ -10,4 +10,4 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|
||||
| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
|
||||
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership.
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the interface or consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
|
||||
@@ -20,11 +20,15 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
name: DeepSeek V4 Flash
|
||||
contextWindow: 128000
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
```
|
||||
|
||||
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns it only for an exact configured id; omission or an unlisted pass-through model returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` wire field is not sent and the server applies its own default for the model. The only accepted values are `high` and `max` (DeepSeek's official effort levels). It is meaningful only with thinking enabled (the provider default).
|
||||
|
||||
|
||||
@@ -11,17 +11,23 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
@@ -30,6 +36,7 @@
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
|
||||
@@ -6,7 +6,13 @@
|
||||
*/
|
||||
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { serializeRequest } from './serialize.ts'
|
||||
import type { RequestDefaults } from './serialize.ts'
|
||||
@@ -22,6 +28,8 @@ export interface DeepSeekCatalogModel {
|
||||
name?: string
|
||||
/** Optional selector detail for deployments with similar model variants. */
|
||||
description?: string
|
||||
/** Known combined request/response context capacity; omitted when deployment metadata is unavailable. */
|
||||
contextWindow?: number
|
||||
}
|
||||
|
||||
/** Constructor options for {@link DeepSeekAdapter}; the plugin's `apply` resolves them from Config + environment. */
|
||||
@@ -111,6 +119,14 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
})))
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const contextWindow = this.options.models?.find(entry => entry.id === model)?.contextWindow
|
||||
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
|
||||
@@ -21,8 +21,8 @@ export const name = 'llm-deepseek'
|
||||
export const inject = ['llm']
|
||||
|
||||
const DEFAULT_MODELS: DeepSeekCatalogModel[] = [
|
||||
{ id: 'deepseek-v4-flash' },
|
||||
{ id: 'deepseek-v4-pro' },
|
||||
{ id: 'deepseek-v4-flash', contextWindow: 128_000 },
|
||||
{ id: 'deepseek-v4-pro', contextWindow: 128_000 },
|
||||
]
|
||||
|
||||
/**
|
||||
@@ -50,6 +50,7 @@ const catalogModel: z<DeepSeekCatalogModel> = z.object({
|
||||
id: z.string().required(),
|
||||
name: z.string(),
|
||||
description: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
})
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
@@ -72,12 +73,19 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee
|
||||
if (model.name !== undefined && model.name.length === 0) {
|
||||
throw new Error(`llm-deepseek: catalog model "${model.id}" has an empty name`)
|
||||
}
|
||||
if (model.contextWindow !== undefined
|
||||
&& (!Number.isInteger(model.contextWindow) || model.contextWindow <= 0)) {
|
||||
throw new Error(
|
||||
`llm-deepseek: catalog model "${model.id}" contextWindow must be a positive integer`,
|
||||
)
|
||||
}
|
||||
if (seen.has(model.id)) throw new Error(`llm-deepseek: duplicate catalog model "${model.id}"`)
|
||||
seen.add(model.id)
|
||||
return {
|
||||
id: model.id,
|
||||
...model.name === undefined ? {} : { name: model.name },
|
||||
...model.description === undefined ? {} : { description: model.description },
|
||||
...model.contextWindow === undefined ? {} : { contextWindow: model.contextWindow },
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
30
packages/llm/llm-deepseek/src/invariant.ts
Normal file
30
packages/llm/llm-deepseek/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-llm-deepseek`.
|
||||
* @module @deepseek-ai/dsh-llm-deepseek/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'llm-deepseek-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this package exposes no independent event sequence or mutable data relation
|
||||
* beyond contracts enforced at its owning seam.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -517,6 +517,8 @@ describe('plugin registration and config', () => {
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'deepseek-v4-flash' },
|
||||
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'deepseek-v4-pro' },
|
||||
])
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'deepseek-v4-flash'))
|
||||
.resolves.toEqual({ contextWindow: 128_000 })
|
||||
})
|
||||
|
||||
it('uses the default model catalog when apply is called directly', async () => {
|
||||
@@ -536,14 +538,23 @@ describe('plugin registration and config', () => {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
models: [
|
||||
{ id: 'private-fast' },
|
||||
{ id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget' },
|
||||
{ id: 'private-fast', contextWindow: 32_000 },
|
||||
{
|
||||
id: 'private-reasoner',
|
||||
name: 'Private Reasoner',
|
||||
description: 'Higher reasoning budget',
|
||||
contextWindow: 64_000,
|
||||
},
|
||||
],
|
||||
})
|
||||
await expect(ctx.llm.listModels('deepseek')).resolves.toEqual([
|
||||
{ provider: 'deepseek', id: 'private-fast', name: 'private-fast' },
|
||||
{ provider: 'deepseek', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget' },
|
||||
])
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'private-fast'))
|
||||
.resolves.toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'arbitrary-unlisted'))
|
||||
.resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('allows an explicit empty model catalog', async () => {
|
||||
@@ -560,6 +571,8 @@ describe('plugin registration and config', () => {
|
||||
it.each([
|
||||
[[{ id: '' }], /ids must be non-empty/],
|
||||
[[{ id: 'm', name: '' }], /empty name/],
|
||||
[[{ id: 'm', contextWindow: 0 }], /contextWindow/],
|
||||
[[{ id: 'm', contextWindow: 1.5 }], /contextWindow/],
|
||||
[[{ id: 'm' }, { id: 'm' }], /duplicate catalog model/],
|
||||
] as const)('rejects invalid advisory model config', async (models, message) => {
|
||||
const ctx = new Context()
|
||||
@@ -572,6 +585,19 @@ describe('plugin registration and config', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('rejects invalid context capacity when apply is called directly', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
expect(() => {
|
||||
LlmDeepSeek.apply(ctx, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
models: [{ id: 'invalid-context', contextWindow: 0 }],
|
||||
})
|
||||
}).toThrow(/contextWindow must be a positive integer/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('falls back to DEEPSEEK_API_KEY and DEEPSEEK_BASE_URL env vars', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://127.0.0.1:1')
|
||||
|
||||
@@ -20,6 +20,9 @@
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
|
||||
@@ -28,7 +28,7 @@ Configure credentials and deployment-specific transport settings per provider. O
|
||||
|
||||
Each provider name must exist in pi-ai's installed catalog and may appear only once in this plugin instance. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
|
||||
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry.
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelContext(provider, model)` performs the same exact descriptor lookup and returns its context window, keeping capacity metadata on the route-owning adapter rather than a consuming plugin.
|
||||
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
|
||||
@@ -11,17 +11,23 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
@@ -31,6 +37,7 @@
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm-deepseek": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
|
||||
@@ -15,7 +15,7 @@ import type {
|
||||
SimpleStreamOptions,
|
||||
} from '@earendil-works/pi-ai'
|
||||
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelContext, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveProfiles } from './config.ts'
|
||||
import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
@@ -89,6 +89,22 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
})))
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const profile = this.profiles.get(provider)
|
||||
if (profile === undefined) {
|
||||
return Promise.reject(new LlmError(
|
||||
`pi-ai adapter does not own provider "${provider}"`,
|
||||
'NO_ADAPTER',
|
||||
))
|
||||
}
|
||||
return Promise.resolve().then(() => ({
|
||||
contextWindow: resolveModel(profile, model).contextWindow,
|
||||
}))
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (options.stop !== undefined) {
|
||||
throw new LlmError('llm-pi-ai does not support GenerateOptions.stop', 'UNSUPPORTED_OPTION')
|
||||
|
||||
30
packages/llm/llm-pi-ai/src/invariant.ts
Normal file
30
packages/llm/llm-pi-ai/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-llm-pi-ai`.
|
||||
* @module @deepseek-ai/dsh-llm-pi-ai/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm-pi-ai'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'llm-pi-ai-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this package exposes no independent event sequence or mutable data relation
|
||||
* beyond contracts enforced at its owning seam.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -322,6 +322,9 @@ describe('provider profile lifecycle', () => {
|
||||
provider: 'openai', id: 'gpt-4.1', name: 'GPT-4.1',
|
||||
})
|
||||
expect(models.every(model => model.provider === 'openai')).toBe(true)
|
||||
const context = await ctx.llm.resolveModelContext('openai', 'gpt-4.1')
|
||||
expect(context).toBeDefined()
|
||||
expect(typeof context?.contextWindow).toBe('number')
|
||||
})
|
||||
|
||||
it('accepts absent credentials for pi-ai ambient authentication', async () => {
|
||||
@@ -373,6 +376,10 @@ describe('provider profile lifecycle', () => {
|
||||
it('constructs the adapter directly and rejects routes it does not own', async () => {
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai' }] })
|
||||
await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(adapter.resolveModelContext('anthropic', 'claude-sonnet-4'))
|
||||
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(adapter.resolveModelContext('openai', 'not-a-catalog-model'))
|
||||
.rejects.toMatchObject({ code: 'UNKNOWN_MODEL' })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of adapter.stream({ provider: 'anthropic', model: 'claude-sonnet-4', messages: [] })) { /* drain */ }
|
||||
})()).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
|
||||
@@ -20,6 +20,9 @@
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
|
||||
@@ -6,6 +6,8 @@ The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, an
|
||||
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward.
|
||||
|
||||
The separately published `./invariant` companion checks that every retry record names the current open turn and its latest closed step, has a unique step record and increasing retry number, and carries a positive bounded retry budget and non-negative bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
config:
|
||||
|
||||
@@ -11,10 +11,15 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
@@ -22,6 +27,7 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
@@ -35,6 +41,7 @@
|
||||
"@cordisjs/plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
|
||||
97
packages/llm/llm-retry/src/invariant.ts
Normal file
97
packages/llm/llm-retry/src/invariant.ts
Normal file
@@ -0,0 +1,97 @@
|
||||
/** Package-owned durable retry-event invariants. @module @deepseek-ai/dsh-llm-retry/invariant */
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
import type {} from './index.ts'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm-retry'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'llm-retry-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/** Validate one retry record against the open turn and most recently closed step. */
|
||||
function validateRetry(
|
||||
history: readonly SessionEvent[],
|
||||
event: SessionEvent<'llm/retry'>,
|
||||
fail: InvariantFailure,
|
||||
): void {
|
||||
const { turn, step, retry, maxRetries, delayMs } = event.data
|
||||
if (!Number.isSafeInteger(retry) || retry < 1) {
|
||||
fail('llm/retry retry must be a positive safe integer')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 1 || retry > maxRetries) {
|
||||
fail(`llm/retry retry ${retry} must not exceed a positive safe maxRetries ${maxRetries}`)
|
||||
}
|
||||
if (!(delayMs >= 0 && delayMs <= MAX_TIMER_DELAY_MS)) {
|
||||
fail(`llm/retry delayMs must be within 0..${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
|
||||
const currentTurnEvents: SessionEvent[] = []
|
||||
let openTurn: number | undefined
|
||||
for (const prior of history.slice().reverse()) {
|
||||
if (prior.type === 'turn/end') fail('llm/retry must be appended inside an open turn')
|
||||
if (prior.type === 'turn/start') {
|
||||
openTurn = prior.data.turn
|
||||
break
|
||||
}
|
||||
currentTurnEvents.push(prior)
|
||||
}
|
||||
if (openTurn === undefined) fail('llm/retry must be appended inside an open turn')
|
||||
if (turn !== openTurn) {
|
||||
fail(`llm/retry names turn ${turn}, but the open turn is ${openTurn}`)
|
||||
}
|
||||
|
||||
let closedStep: number | undefined
|
||||
for (const prior of currentTurnEvents) {
|
||||
if (prior.type === 'step/start') {
|
||||
fail(`llm/retry must follow step/end, but step ${prior.data.step} is still open`)
|
||||
}
|
||||
if (prior.type === 'step/end') {
|
||||
closedStep = prior.data.step
|
||||
break
|
||||
}
|
||||
}
|
||||
if (closedStep === undefined || step !== closedStep) {
|
||||
fail(`llm/retry names step ${step}, but the latest closed step is ${String(closedStep)}`)
|
||||
}
|
||||
|
||||
const priorRetries = currentTurnEvents
|
||||
.filter((prior): prior is SessionEvent<'llm/retry'> => prior.type === 'llm/retry')
|
||||
if (priorRetries.some(prior => prior.data.step === step)) {
|
||||
fail(`llm/retry duplicates the retry record for turn ${turn}/step ${step}`)
|
||||
}
|
||||
const priorRetry = priorRetries[0]
|
||||
if (priorRetry !== undefined && retry <= priorRetry.data.retry) {
|
||||
fail(`llm/retry retry ${retry} must increase after retry ${priorRetry.data.retry}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate every retry record already present in one loaded session. */
|
||||
function validateSession(session: Session, fail: InvariantFailure): void {
|
||||
for (const [index, event] of session.events.entries()) {
|
||||
if (event.type === 'llm/retry') validateRetry(session.events.slice(0, index), event, fail)
|
||||
}
|
||||
}
|
||||
|
||||
/** Install validation for loaded and newly appended retry records. */
|
||||
const install: InvariantInstaller = Object.assign((ctx: Context, fail: InvariantFailure) => {
|
||||
for (const session of ctx.sessions.list()) validateSession(session, fail)
|
||||
ctx.on('session/created', (session) => { validateSession(session, fail) }, { global: true })
|
||||
ctx.on('internal/dispatch', (_mode, eventName, args) => {
|
||||
if (eventName !== 'session/event') return
|
||||
const [session, event] = args as [Session, SessionEvent]
|
||||
if (event.type === 'llm/retry') validateRetry(session.events, event, fail)
|
||||
}, { global: true })
|
||||
}, { inject: ['sessions'] })
|
||||
|
||||
/**
|
||||
* Register the LLM retry invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
148
packages/llm/llm-retry/tests/invariant.spec.ts
Normal file
148
packages/llm/llm-retry/tests/invariant.spec.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import InvariantService from '@deepseek-ai/dsh-invariants'
|
||||
import * as RetryInvariant from '@deepseek-ai/dsh-llm-retry/invariant'
|
||||
|
||||
async function setup(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(InvariantService)
|
||||
await ctx.plugin(RetryInvariant)
|
||||
return ctx
|
||||
}
|
||||
|
||||
function closeStep(ctx: Context, id: string, turn = 1, step = 1) {
|
||||
const session = ctx.sessions.create(SessionId(id))
|
||||
session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn, step })
|
||||
session.append('step/end', { turn, step })
|
||||
return session
|
||||
}
|
||||
|
||||
const failure = { message: 'provider busy', code: 'RATE_LIMIT', status: 429 }
|
||||
|
||||
describe('llm-retry invariants', () => {
|
||||
it('accepts increasing retry records for successive closed steps and ignores unrelated events', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-valid')
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 500, failure,
|
||||
})
|
||||
session.append('step/start', { turn: 1, step: 2 })
|
||||
session.append('step/end', { turn: 1, step: 2 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 2, maxRetries: 2, delayMs: 1_000, failure,
|
||||
})
|
||||
const zeroDelay = closeStep(ctx, 'retry-invariant-zero-delay')
|
||||
zeroDelay.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 1, delayMs: 0, failure,
|
||||
})
|
||||
}).not.toThrow()
|
||||
expect(() => { ctx.emit('tools/change') }).not.toThrow()
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ retry: 0, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1.5, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1, maxRetries: 0, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 1, maxRetries: 1.5, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 3, maxRetries: 2, delayMs: 1 }, /must not exceed/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: -1 }, /delayMs/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: MAX_TIMER_DELAY_MS + 1 }, /delayMs/],
|
||||
])('rejects invalid retry bounds %#', async (data, message) => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, `retry-invariant-bounds-${data.retry}-${data.maxRetries}-${data.delayMs}`)
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...data, failure })
|
||||
}).toThrow(message)
|
||||
})
|
||||
|
||||
it('rejects retry records outside the matching closed-step boundary', async () => {
|
||||
const ctx = await setup()
|
||||
const absent = ctx.sessions.create(SessionId('retry-invariant-no-turn'))
|
||||
expect(() => {
|
||||
absent.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/inside an open turn/)
|
||||
|
||||
const wrongTurn = closeStep(ctx, 'retry-invariant-wrong-turn')
|
||||
expect(() => {
|
||||
wrongTurn.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/open turn is 1/)
|
||||
|
||||
const openStep = ctx.sessions.create(SessionId('retry-invariant-open-step'))
|
||||
openStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
openStep.append('step/start', { turn: 1, step: 1 })
|
||||
expect(() => {
|
||||
openStep.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/step 1 is still open/)
|
||||
|
||||
const noStep = ctx.sessions.create(SessionId('retry-invariant-no-step'))
|
||||
noStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
expect(() => {
|
||||
noStep.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/latest closed step is undefined/)
|
||||
|
||||
const wrongStep = closeStep(ctx, 'retry-invariant-wrong-step')
|
||||
expect(() => {
|
||||
wrongStep.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/latest closed step is 1/)
|
||||
|
||||
const closedTurn = closeStep(ctx, 'retry-invariant-closed-turn')
|
||||
closedTurn.append('turn/end', { turn: 1, reason: { kind: 'aborted' } })
|
||||
expect(() => {
|
||||
closedTurn.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
it('rejects duplicate and non-increasing retry records', async () => {
|
||||
const ctx = await setup()
|
||||
const duplicate = closeStep(ctx, 'retry-invariant-duplicate')
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
expect(() => {
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 2, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/duplicates the retry record/)
|
||||
|
||||
const nonIncreasing = closeStep(ctx, 'retry-invariant-non-increasing')
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
nonIncreasing.append('step/start', { turn: 1, step: 2 })
|
||||
nonIncreasing.append('step/end', { turn: 1, step: 2 })
|
||||
expect(() => {
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/must increase/)
|
||||
})
|
||||
|
||||
it('validates existing histories on late registration', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = ctx.sessions.create(SessionId('retry-invariant-late'))
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).rejects.toThrow(/inside an open turn/)
|
||||
})
|
||||
})
|
||||
@@ -232,6 +232,29 @@ describe('bounded transient retry policy', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('accepts the zero-delay lower jitter bound', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('busy', 'SERVER'),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
jitterRatio: 1,
|
||||
}, undefined, { random: () => 0 }))
|
||||
const agent = context.agentLoop.create(SessionId('retry-zero-delay'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
expect((await scheduled).data.delayMs).toBe(0)
|
||||
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.runAllTimersAsync()
|
||||
await idle
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('uses a bounded provider Retry-After verbatim and delegates an over-cap instruction', async () => {
|
||||
vi.useFakeTimers()
|
||||
const accepted = new ScriptedAdapter([
|
||||
|
||||
@@ -26,6 +26,9 @@
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
|
||||
@@ -11,12 +11,15 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
|
||||
### Events
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
@@ -25,7 +28,7 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -11,11 +11,16 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
@@ -23,10 +28,12 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,7 +7,15 @@
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { GenerateOptions, LlmFailure, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
} from './types.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
@@ -122,6 +130,20 @@ export abstract class LlmAdapter {
|
||||
return Promise.resolve([])
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity for one model accepted by this adapter. Absence
|
||||
* means the adapter does not know the capacity, not that routing is invalid.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns provider-owned context metadata, or `undefined` when unavailable.
|
||||
*/
|
||||
resolveModelContext(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as raw chunks. The only required method.
|
||||
* @param options - the fully-assembled request; implementations must honor `options.signal`.
|
||||
@@ -217,6 +239,29 @@ export class LlmService extends Service {
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity from the adapter that owns one exact route.
|
||||
* This query is independent of the advisory model catalog: an unlisted model
|
||||
* may return metadata, while `undefined` never rejects later routing.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @returns detached context metadata, or `undefined` when the adapter has none.
|
||||
*/
|
||||
async resolveModelContext(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
|
||||
if (context === undefined) return undefined
|
||||
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_CONTEXT',
|
||||
)
|
||||
}
|
||||
return { contextWindow: context.contextWindow }
|
||||
}
|
||||
|
||||
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
|
||||
95
packages/llm/llm/src/invariant.ts
Normal file
95
packages/llm/llm/src/invariant.ts
Normal file
@@ -0,0 +1,95 @@
|
||||
/** Package-owned LLM stream-protocol invariants. @module @deepseek-ai/dsh-llm/invariant */
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
import type { ContentBlockType, StreamChunk } from './types.ts'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'llm-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/** Require one chunk index to be a non-negative safe integer. */
|
||||
function validateIndex(index: number, fail: InvariantFailure): void {
|
||||
if (!Number.isSafeInteger(index) || index < 0) {
|
||||
fail(`LLM stream block index must be a non-negative safe integer, got ${index}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Require a delta to address an open block of its matching type. */
|
||||
function validateDelta(
|
||||
open: ReadonlyMap<number, ContentBlockType>,
|
||||
index: number,
|
||||
expected: ContentBlockType,
|
||||
fail: InvariantFailure,
|
||||
): void {
|
||||
validateIndex(index, fail)
|
||||
const actual = open.get(index)
|
||||
if (actual !== expected) {
|
||||
fail(`${expected} delta at index ${index} requires an open ${expected} block, got ${String(actual)}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Wrap one provider stream and enforce its grammar as chunks are consumed. */
|
||||
async function* validateStream(
|
||||
source: AsyncIterable<StreamChunk>,
|
||||
fail: InvariantFailure,
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const open = new Map<number, ContentBlockType>()
|
||||
let usageSeen = false
|
||||
let finished = false
|
||||
for await (const chunk of source) {
|
||||
if (finished) fail(`LLM stream emitted ${chunk.type} after terminal finish`)
|
||||
switch (chunk.type) {
|
||||
case 'block-start':
|
||||
validateIndex(chunk.index, fail)
|
||||
if (open.has(chunk.index)) fail(`LLM stream repeated block-start index ${chunk.index}`)
|
||||
open.set(chunk.index, chunk.blockType)
|
||||
break
|
||||
case 'text-delta':
|
||||
validateDelta(open, chunk.index, 'text', fail)
|
||||
break
|
||||
case 'reasoning-delta':
|
||||
validateDelta(open, chunk.index, 'reasoning', fail)
|
||||
break
|
||||
case 'tool-call-delta':
|
||||
validateDelta(open, chunk.index, 'tool-call', fail)
|
||||
break
|
||||
case 'block-end': {
|
||||
validateIndex(chunk.index, fail)
|
||||
const blockType = open.get(chunk.index)
|
||||
if (blockType === undefined) fail(`LLM stream block-end index ${chunk.index} has no open block`)
|
||||
if (chunk.block.type !== blockType) {
|
||||
fail(`LLM stream block-end index ${chunk.index} closes ${chunk.block.type}, expected ${blockType}`)
|
||||
}
|
||||
open.delete(chunk.index)
|
||||
break
|
||||
}
|
||||
case 'usage':
|
||||
if (usageSeen) fail('LLM stream emitted usage more than once')
|
||||
usageSeen = true
|
||||
break
|
||||
case 'finish':
|
||||
if (open.size > 0) fail(`LLM stream finished with ${open.size} open block(s)`)
|
||||
finished = true
|
||||
break
|
||||
}
|
||||
yield chunk
|
||||
}
|
||||
if (!finished) fail('LLM stream ended without a terminal finish chunk')
|
||||
}
|
||||
|
||||
/** Install validation around every provider stream. */
|
||||
const install: InvariantInstaller = (ctx, fail) => {
|
||||
ctx.on('llm/stream', (_options, next) => validateStream(next(), fail), { global: true, prepend: true })
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the LLM invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
@@ -155,6 +155,12 @@ export interface LlmModelInfo {
|
||||
description?: string
|
||||
}
|
||||
|
||||
/** Provider-owned context capacity for one exact provider/model route. */
|
||||
export interface LlmModelContext {
|
||||
/** Maximum combined request and response context in tokens. */
|
||||
contextWindow: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
|
||||
86
packages/llm/llm/tests/invariant.spec.ts
Normal file
86
packages/llm/llm/tests/invariant.spec.ts
Normal file
@@ -0,0 +1,86 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmInvariant from '@deepseek-ai/dsh-llm/invariant'
|
||||
import InvariantService from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
async function setup(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(InvariantService)
|
||||
await ctx.plugin(LlmInvariant)
|
||||
return ctx
|
||||
}
|
||||
|
||||
const options: GenerateOptions = { provider: 'mock', model: 'mock', messages: [] }
|
||||
|
||||
async function* source(chunks: readonly StreamChunk[]): AsyncIterable<StreamChunk> {
|
||||
yield* chunks
|
||||
}
|
||||
|
||||
async function consume(ctx: Context, chunks: readonly StreamChunk[]): Promise<StreamChunk[]> {
|
||||
const stream = ctx.waterfall(ctx as never, 'llm/stream', options, () => source(chunks))
|
||||
const consumed: StreamChunk[] = []
|
||||
for await (const chunk of stream) consumed.push(chunk)
|
||||
return consumed
|
||||
}
|
||||
|
||||
const finish: StreamChunk = { type: 'finish', reason: { kind: 'stop' } }
|
||||
|
||||
describe('LLM stream invariants', () => {
|
||||
it('accepts a complete interleaved stream grammar', async () => {
|
||||
const ctx = await setup()
|
||||
const chunks: StreamChunk[] = [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'a' },
|
||||
{ type: 'block-start', index: 1, blockType: 'reasoning' },
|
||||
{ type: 'reasoning-delta', index: 1, text: 'b' },
|
||||
{ type: 'block-end', index: 1, block: { type: 'reasoning', text: 'b' } },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'a' } },
|
||||
{ type: 'block-start', index: 2, blockType: 'tool-call' },
|
||||
{ type: 'tool-call-delta', index: 2, id: CallId('c1'), name: 'echo', argumentsDelta: '{}' },
|
||||
{ type: 'block-end', index: 2, block: { type: 'tool-call', id: CallId('c1'), name: 'echo', arguments: '{}' } },
|
||||
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } },
|
||||
finish,
|
||||
]
|
||||
await expect(consume(ctx, chunks)).resolves.toEqual(chunks)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[[{ type: 'block-start', index: -1, blockType: 'text' }, finish], /non-negative safe integer/],
|
||||
[[
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
], /repeated block-start/],
|
||||
[[{ type: 'text-delta', index: 0, text: 'x' }], /requires an open text block/],
|
||||
[[
|
||||
{ type: 'block-start', index: 0, blockType: 'reasoning' },
|
||||
{ type: 'text-delta', index: 0, text: 'x' },
|
||||
], /got reasoning/],
|
||||
[[{ type: 'block-end', index: 0, block: { type: 'text', text: '' } }], /has no open block/],
|
||||
[[
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: '' } },
|
||||
], /closes reasoning, expected text/],
|
||||
[[
|
||||
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } },
|
||||
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } },
|
||||
], /usage more than once/],
|
||||
[[{ type: 'block-start', index: 0, blockType: 'text' }, finish], /finished with 1 open block/],
|
||||
[[finish, { type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }], /usage after terminal finish/],
|
||||
[[], /ended without a terminal finish/],
|
||||
] as Array<[StreamChunk[], RegExp]>)('rejects malformed stream %#', async (chunks, message) => {
|
||||
const ctx = await setup()
|
||||
await expect(consume(ctx, chunks)).rejects.toThrow(message)
|
||||
})
|
||||
|
||||
it('preserves provider exceptions without inventing a missing-finish failure', async () => {
|
||||
const ctx = await setup()
|
||||
const stream = ctx.waterfall(ctx as never, 'llm/stream', options, async function* () {
|
||||
throw new Error('provider failed')
|
||||
})
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* consume */ }
|
||||
})()).rejects.toThrow('provider failed')
|
||||
})
|
||||
})
|
||||
@@ -13,7 +13,7 @@ import LlmService, {
|
||||
ProviderRequestId,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
constructor(private script: StreamChunk[]) {
|
||||
@@ -48,6 +48,7 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
constructor(
|
||||
private readonly provider: LlmProviderInfo,
|
||||
private readonly models: readonly LlmModelInfo[],
|
||||
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
|
||||
) {
|
||||
super(SCRIPT)
|
||||
}
|
||||
@@ -59,11 +60,19 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
override listModels(_provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(this.contexts[model])
|
||||
}
|
||||
}
|
||||
|
||||
const SCRIPT: StreamChunk[] = [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'hi' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
|
||||
@@ -644,8 +653,42 @@ describe('LlmService', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it('resolves detached model context independently of advisory catalog membership', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const source = { contextWindow: 32_000 }
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{ unlisted: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
source.contextWindow = 64_000
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it.each([0, -1, 1.5, Number.NaN])(
|
||||
'rejects invalid adapter model context %s',
|
||||
async (contextWindow) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[],
|
||||
{ model: { contextWindow } },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelContext('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
|
||||
},
|
||||
)
|
||||
|
||||
it.each([
|
||||
[{ id: 1, name: 'Name' }, 'non-string id'],
|
||||
[{ id: 'other', name: 'Name' }, 'mismatched id'],
|
||||
@@ -694,13 +737,14 @@ describe('LlmService', () => {
|
||||
const inner = next()
|
||||
return (async function * () {
|
||||
yield { type: 'block-start', index: 99, blockType: 'text' } satisfies StreamChunk
|
||||
yield { type: 'block-end', index: 99, block: { type: 'text', text: '' } } satisfies StreamChunk
|
||||
yield * inner
|
||||
})()
|
||||
})
|
||||
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of ctx.llm.stream({ provider: 'test-model', model: 'dynamic-model', messages: [] })) chunks.push(chunk)
|
||||
expect(chunks).toHaveLength(4)
|
||||
expect(chunks).toHaveLength(6)
|
||||
expect(chunks[0]).toMatchObject({ index: 99 })
|
||||
})
|
||||
|
||||
|
||||
@@ -16,6 +16,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../util/brand"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -4,11 +4,7 @@ Replay-aware token measurement through the singleton `ctx.tokenMeter` service. I
|
||||
|
||||
## Configuration
|
||||
|
||||
| Key | Default | Contract |
|
||||
|---|---:|---|
|
||||
| `contextWindow` | `128000` | Positive integer service-wide context capacity. |
|
||||
|
||||
The estimator intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. `contextWindow` is the only deployment setting. Direct construction validates it; Loader mounts first apply the package's Schemastery shape validation. Unrecognized top-level keys are rejected.
|
||||
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelContext()`.
|
||||
|
||||
## Measurement contract
|
||||
|
||||
@@ -30,13 +26,7 @@ Usage accounting sums disjoint input, cache-read, cache-write, and output bucket
|
||||
- name: '@deepseek-ai/dsh-compact-basic'
|
||||
```
|
||||
|
||||
Both plugins have usable defaults. A deployment with a different capacity configures the meter once:
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-token-meter'
|
||||
config:
|
||||
contextWindow: 32768
|
||||
```
|
||||
Both plugins have usable defaults. The meter remains independent of model routing and optional compaction. A deployment configures capacity on its LLM adapter and compaction policy on `dsh-compact-basic`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -11,17 +11,23 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
@@ -30,6 +36,7 @@
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
|
||||
@@ -19,12 +19,6 @@ import type {
|
||||
|
||||
export type * from './types.ts'
|
||||
|
||||
/** Default service-wide provider context capacity. */
|
||||
const DEFAULT_CONTEXT_WINDOW = 128_000
|
||||
|
||||
/** Complete public configuration key set. */
|
||||
const TOKEN_METER_CONFIG_KEYS: ReadonlySet<string> = new Set(['contextWindow'])
|
||||
|
||||
/** Fixed text-density estimate used until exact tokenization is needed. */
|
||||
const CHARS_PER_TOKEN = 4
|
||||
|
||||
@@ -74,28 +68,10 @@ function optionalHeaderEquals(
|
||||
/** Reject stale or misspelled keys before defaults can hide them. */
|
||||
function validateConfigKeys(config: TokenMeterConfig): void {
|
||||
for (const key of Object.keys(config)) {
|
||||
if (!TOKEN_METER_CONFIG_KEYS.has(key)) {
|
||||
throw new Error(
|
||||
`TokenMeterConfig: unknown key "${key}" (allowed: contextWindow)`,
|
||||
)
|
||||
}
|
||||
throw new Error(`TokenMeterConfig: unknown key "${key}" (no settings are supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolve and validate the one service-wide context capacity. */
|
||||
function resolveContextWindow(config: TokenMeterConfig): number {
|
||||
validateConfigKeys(config)
|
||||
const contextWindow = config.contextWindow === undefined
|
||||
? DEFAULT_CONTEXT_WINDOW
|
||||
: config.contextWindow
|
||||
if (!Number.isInteger(contextWindow) || contextWindow <= 0) {
|
||||
throw new Error(
|
||||
`TokenMeterConfig: contextWindow (${contextWindow}) must be a positive integer`,
|
||||
)
|
||||
}
|
||||
return contextWindow
|
||||
}
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
tokenMeter: TokenMeterService
|
||||
@@ -104,18 +80,15 @@ declare module 'cordis' {
|
||||
|
||||
/** Replay owner for one service-wide estimator and isolated per-session folds. */
|
||||
export class TokenMeterService extends Service {
|
||||
static Config: z<TokenMeterConfig> = z.object({
|
||||
contextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW),
|
||||
})
|
||||
|
||||
/** Provider context-window capacity used by pressure consumers. */
|
||||
readonly contextWindow: number
|
||||
// Schemastery preserves untrusted loader keys on an empty object schema;
|
||||
// the public type excludes settings while validateConfigKeys rejects them.
|
||||
static Config: z<TokenMeterConfig> = z.object({}) as unknown as z<TokenMeterConfig>
|
||||
|
||||
private readonly states = new WeakMap<Session, ReplayState>()
|
||||
|
||||
constructor(ctx: Context, config: TokenMeterConfig = {}) {
|
||||
super(ctx, 'tokenMeter')
|
||||
this.contextWindow = resolveContextWindow(config)
|
||||
validateConfigKeys(config)
|
||||
|
||||
// Readers catch up independently, while eager observation bounds ordinary
|
||||
// read latency without creating state for sessions no consumer has read.
|
||||
|
||||
30
packages/llm/token-meter/src/invariant.ts
Normal file
30
packages/llm/token-meter/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-token-meter`.
|
||||
* @module @deepseek-ai/dsh-token-meter/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-token-meter'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'token-meter-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: token estimates are per-call outputs and the private session cache is
|
||||
* invalidated at its event mutation boundary; neither exposes an independent observation stream.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -6,11 +6,8 @@
|
||||
|
||||
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Token-meter plugin configuration. */
|
||||
export interface TokenMeterConfig {
|
||||
/** Service-wide context-window capacity in tokens. Defaults to `128000`. */
|
||||
contextWindow?: number
|
||||
}
|
||||
/** Token-meter plugin configuration; the fixed estimator has no settings. */
|
||||
export type TokenMeterConfig = Record<string, never>
|
||||
|
||||
/** The baseline from which a signed surface delta produces current pressure. */
|
||||
export type TokenMeasurementBaseline =
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { describe, expect, expectTypeOf, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, Message, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
@@ -87,29 +87,18 @@ function expectSurfaceTotal(measurement: TokenMeasurement): void {
|
||||
}
|
||||
|
||||
describe('TokenMeterService configuration and registration', () => {
|
||||
it('provides one zero-config context window', () => {
|
||||
const service = meter()
|
||||
expect(service.contextWindow).toBe(128_000)
|
||||
it('exposes an empty public configuration type', () => {
|
||||
expectTypeOf<{}>().toExtend<TokenMeterConfig>()
|
||||
expectTypeOf<{ contextWindow: number }>().not.toExtend<TokenMeterConfig>()
|
||||
})
|
||||
|
||||
it('accepts one service-wide context-window override', () => {
|
||||
expect(meter({ contextWindow: 32_000 }).contextWindow).toBe(32_000)
|
||||
})
|
||||
|
||||
it.each(['models', 'contextWidow'])('rejects unknown top-level config key %s', (key) => {
|
||||
expect(() => meter({ [key]: {} }))
|
||||
.toThrow(`TokenMeterConfig: unknown key "${key}"`)
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ contextWindow: 0 },
|
||||
{ contextWindow: -1 },
|
||||
{ contextWindow: 1.5 },
|
||||
{ contextWindow: Number.NaN },
|
||||
{ contextWindow: null },
|
||||
] as unknown as TokenMeterConfig[])('rejects invalid context capacity %#', (config) => {
|
||||
expect(() => meter(config)).toThrow(/contextWindow .* positive integer/)
|
||||
})
|
||||
it.each(['models', 'contextWindow', 'contextWidow'])(
|
||||
'rejects stale or unknown top-level config key %s',
|
||||
(key) => {
|
||||
expect(() => meter({ [key]: {} } as unknown as TokenMeterConfig))
|
||||
.toThrow(`TokenMeterConfig: unknown key "${key}"`)
|
||||
},
|
||||
)
|
||||
|
||||
it('registers and unregisters ctx.tokenMeter with its plugin fiber', async () => {
|
||||
const ctx = new Context()
|
||||
@@ -123,7 +112,7 @@ describe('TokenMeterService configuration and registration', () => {
|
||||
|
||||
describe('TokenMeterService pricing', () => {
|
||||
it('prices every built-in content shape and merge-extended blocks with one fixed heuristic', () => {
|
||||
const service = meter({ contextWindow: 100 })
|
||||
const service = meter()
|
||||
const blocks: ContentBlock[] = [
|
||||
{ type: 'text', text: 'abcd' },
|
||||
{ type: 'reasoning', text: 'ab' },
|
||||
@@ -331,7 +320,7 @@ describe('replay anchors and surface folds', () => {
|
||||
})
|
||||
|
||||
it('keeps only the latest successful request anchor across model switches', () => {
|
||||
const service = meter({ contextWindow: 1_000 })
|
||||
const service = meter()
|
||||
const session = new Session(SessionId('switch'))
|
||||
const alphaHeader = header('alpha', { system: 'same envelope' })
|
||||
appendSuccessfulCall(session, alphaHeader, { usage: USAGE, providerText: 'alpha' })
|
||||
@@ -631,19 +620,24 @@ describe('malformed replay and listener lifecycle', () => {
|
||||
})
|
||||
const firstFiber = await ctx.plugin(TokenMeterService)
|
||||
activeMeter = ctx.tokenMeter
|
||||
const session = ctx.sessions.create(SessionId('listener-order'))
|
||||
const session = ctx.sessions.create(SessionId('listener-order'), { seed: [{
|
||||
type: 'turn/start',
|
||||
seq: 0,
|
||||
time: 1,
|
||||
data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } },
|
||||
}] })
|
||||
activeMeter.measure(session)
|
||||
session.append('user/message', {
|
||||
content: [{ type: 'text', text: 'one' }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
expect(revisions).toEqual([1])
|
||||
expect(activeMeter.measure(session).logRevision).toBe(1)
|
||||
expect(revisions).toEqual([2])
|
||||
expect(activeMeter.measure(session).logRevision).toBe(2)
|
||||
|
||||
await firstFiber.dispose()
|
||||
const secondFiber = await ctx.plugin(TokenMeterService)
|
||||
activeMeter = ctx.tokenMeter
|
||||
expect(activeMeter.measure(session).logRevision).toBe(1)
|
||||
expect(activeMeter.measure(session).logRevision).toBe(2)
|
||||
await secondFiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -22,6 +22,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user