Merge latest origin/master into compact-post-step-overflow-recovery
# Conflicts: # docs/agent-lifecycle.md # docs/architecture.md # docs/cookbook/extension-cookbook.i18n.yaml # docs/cordis-catalog/events.md # docs/cordis-catalog/services.md # docs/event-producer-consumer.md # docs/rfc/INDEX.md # docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md # docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml # docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md # docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md # docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md # docs/rfc/implemented/feature/2026-07-07-session-prefix.md # packages/compact/compact-basic/README.md # packages/compact/compact-basic/src/config.ts # packages/compact/compact-basic/src/index.ts # packages/compact/compact-basic/src/summarizer.ts # packages/compact/compact-basic/tests/compact-basic.spec.ts # packages/compact/compact-basic/tests/compact-loop-repro.spec.ts # packages/compact/compact/README.md # packages/cordis/tool-cordis/src/api-catalog.ts # packages/core/agent-loop/src/loop.ts # packages/core/agent-loop/tests/cancel.spec.ts # packages/llm/llm-deepseek/src/adapter.ts # packages/llm/llm-pi-ai/README.md # packages/llm/llm-pi-ai/src/stream.ts # packages/llm/llm-pi-ai/tests/convert.spec.ts # packages/llm/llm/README.md # packages/llm/llm/src/index.ts # packages/llm/llm/tests/service.spec.ts # scripts/gen-doc-graphs.ts
This commit is contained in:
Tianyi Cui
@@ -8,12 +8,15 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
### Public API
|
||||
|
||||
- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber.
|
||||
- `ctx.llm.models(): string[]` — model names with a registered adapter.
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves and privately tags errors from final adapter selection, synchronous dispatch, iterator construction, and iteration. `isLlmAdapterFailure(value)` exposes that provenance without classifying `llm/stream` middleware or downstream consumer failures as provider failures, and without replacing the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
### Events
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
@@ -22,18 +25,18 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. The core set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it.
|
||||
Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. Assistant messages produced by the loop also carry provider/model provenance and optional adapter-private replay state. Before dispatch, `LlmService` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers. The core block set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it.
|
||||
|
||||
Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages.
|
||||
|
||||
### Call configuration (`call-config.ts`)
|
||||
|
||||
`LlmCallConfig` is the model + sampling scalars of one conversation's requests (`model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite).
|
||||
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite).
|
||||
|
||||
### App attribution (`attribution.ts`)
|
||||
|
||||
@@ -44,12 +47,12 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract.
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
|
||||
|
||||
### Real adapters
|
||||
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) uses `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) uses hand-rolled fetch/SSE for the `deepseek` route, while [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) dynamically resolves configured provider/model pairs through `@earendil-works/pi-ai`. Both follow the `StreamChunk` conventions in `types.ts`: usage precedes finish, tool arguments remain raw strings, and errors take one of two sanctioned paths. See [the twin LLM adapters](../../../docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the design rationale.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -36,14 +36,13 @@ export class BlockAssembler {
|
||||
private order: number[] = []
|
||||
private _usage: TokenUsage | undefined
|
||||
private _finish: FinishReason | undefined
|
||||
private _replayState: unknown = undefined
|
||||
|
||||
/**
|
||||
* Feed one chunk. Returns the completed block when the chunk closes one
|
||||
* (an explicit `block-end`), otherwise undefined.
|
||||
* Feed one chunk into the assembly state.
|
||||
* @param chunk - the next raw chunk, in stream order.
|
||||
* @returns the authoritative block from the first `block-end` at its index; undefined for every other chunk.
|
||||
*/
|
||||
push(chunk: StreamChunk): ContentBlock | undefined {
|
||||
push(chunk: StreamChunk): void {
|
||||
switch (chunk.type) {
|
||||
case 'block-start': {
|
||||
if (!this.partials.has(chunk.index)) {
|
||||
@@ -77,7 +76,7 @@ export class BlockAssembler {
|
||||
// and the final assembled block in agreement.
|
||||
if (partial.block) return
|
||||
partial.block = chunk.block
|
||||
return chunk.block
|
||||
return
|
||||
}
|
||||
case 'usage': {
|
||||
this._usage = chunk.usage
|
||||
@@ -85,6 +84,7 @@ export class BlockAssembler {
|
||||
}
|
||||
case 'finish': {
|
||||
this._finish = chunk.reason
|
||||
this._replayState = chunk.replayState
|
||||
return
|
||||
}
|
||||
default: return assertNever(chunk, 'BlockAssembler.push')
|
||||
@@ -142,6 +142,11 @@ export class BlockAssembler {
|
||||
return this._finish ?? { kind: 'stop' }
|
||||
}
|
||||
|
||||
/** Adapter-private replay state from the terminal finish chunk, if any. */
|
||||
get replayState(): unknown {
|
||||
return this._replayState
|
||||
}
|
||||
|
||||
/**
|
||||
* The assembled assistant message.
|
||||
* @returns an assistant-role message over `blocks()` (same open-block assembly rules).
|
||||
|
||||
@@ -1,17 +1,18 @@
|
||||
/**
|
||||
* Conversation call configuration and freeze utilities. Model and sampling
|
||||
* values are request-header state that can affect cache reuse; request
|
||||
* waterfalls replace them and the loop logs changes instead of allowing
|
||||
* silent per-call drift.
|
||||
* Conversation call configuration and freeze utilities. Provider routing,
|
||||
* model, and sampling values are request-header state that can affect cache
|
||||
* reuse; request waterfalls replace them and the loop logs changed snapshots
|
||||
* instead of allowing silent per-call drift.
|
||||
* @module dsh-llm/call-config
|
||||
*/
|
||||
|
||||
/**
|
||||
* Model + sampling scalars of one conversation's requests. Every field maps
|
||||
* Provider + model + sampling scalars of one conversation's requests. Every field maps
|
||||
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
|
||||
* from the logged header rather than accepting these per call.
|
||||
*/
|
||||
export interface LlmCallConfig {
|
||||
provider: string
|
||||
model: string
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
@@ -21,13 +22,13 @@ export interface LlmCallConfig {
|
||||
/**
|
||||
* Field-wise equality over {@link LlmCallConfig} — the comparison a caller
|
||||
* runs to decide whether a proposed configuration is a real change (worth a
|
||||
* logged header delta) or the held one restated.
|
||||
* logged header snapshot) or the held one restated.
|
||||
* @param a - one configuration.
|
||||
* @param b - the other.
|
||||
* @returns whether every field (including the `stop` list, element-wise) matches.
|
||||
*/
|
||||
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
|
||||
if (a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
|
||||
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
|
||||
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
|
||||
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
|
||||
}
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { GenerateOptions, StreamChunk } from './types.ts'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
|
||||
@@ -44,12 +45,10 @@ declare module 'cordis' {
|
||||
|
||||
/**
|
||||
* Typed error for LLM-related failures. Extends {@link HarnessError}, so the
|
||||
* `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy;
|
||||
* `status` carries the HTTP status when the error originated from a non-2xx
|
||||
* provider response (absent for protocol/usage errors that have no HTTP status).
|
||||
* `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy.
|
||||
*/
|
||||
export class LlmError extends HarnessError {
|
||||
constructor(message: string, code: string, public status?: number, options?: ErrorOptions) {
|
||||
constructor(message: string, code: string, options?: ErrorOptions) {
|
||||
super(message, code, options)
|
||||
this.name = 'LlmError'
|
||||
}
|
||||
@@ -57,11 +56,31 @@ export class LlmError extends HarnessError {
|
||||
|
||||
/**
|
||||
* Provider-wire adapter for the harness message and stream vocabulary. Register implementations
|
||||
* with `ctx.llm.registerAdapter(models, adapter)`. Every provider HTTP request must include
|
||||
* with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include
|
||||
* `attributionHeaders()`; prove that at the wire or library header-hook boundary. The hand-rolled
|
||||
* DeepSeek and pi-ai adapters intentionally exercise this contract through different internals.
|
||||
*/
|
||||
export abstract class LlmAdapter {
|
||||
/**
|
||||
* Describe one provider route owned by this adapter.
|
||||
* @param provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @returns detached display metadata whose id must equal `provider`.
|
||||
*/
|
||||
providerInfo(provider: string): LlmProviderInfo {
|
||||
return { id: provider, name: provider }
|
||||
}
|
||||
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
* consumers must not turn absence into request rejection.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @returns discoverable models in adapter-preferred order.
|
||||
*/
|
||||
listModels(_provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve([])
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as raw chunks. The only required method.
|
||||
* @param options - the fully-assembled request; implementations must honor `options.signal`.
|
||||
@@ -75,30 +94,40 @@ export abstract class LlmAdapter {
|
||||
* surface, interceptable via the `llm/stream` waterfall.
|
||||
*/
|
||||
export class LlmService extends Service {
|
||||
private adapters = new Map<string, LlmAdapter>()
|
||||
private adapters = new Map<string, { adapter: LlmAdapter; provider: LlmProviderInfo }>()
|
||||
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'llm')
|
||||
}
|
||||
|
||||
/**
|
||||
* Register an adapter for the given model names. Throws `LlmError` with code
|
||||
* `DUPLICATE_ADAPTER` if any model already has an adapter (all-or-nothing).
|
||||
* Register an adapter for the given provider routes. Throws `LlmError` with code
|
||||
* `DUPLICATE_ADAPTER` if any provider already has an adapter (all-or-nothing).
|
||||
* Disposed with the fiber.
|
||||
* @param models - every model name this adapter should serve.
|
||||
* @param adapter - the adapter that streams calls for those models.
|
||||
* @param providers - every provider route this adapter should serve.
|
||||
* @param adapter - the adapter that streams calls for those providers.
|
||||
* @returns the disposer that unregisters all of them.
|
||||
*/
|
||||
registerAdapter(models: string[], adapter: LlmAdapter): () => void {
|
||||
registerAdapter(providers: string[], adapter: LlmAdapter): () => void {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
for (const model of models) {
|
||||
if (this.adapters.has(model)) {
|
||||
throw new LlmError(`an adapter for model "${model}" is already registered`, 'DUPLICATE_ADAPTER')
|
||||
if (providers.length === 0) throw new LlmError('an adapter must register at least one provider', 'INVALID_ADAPTER')
|
||||
const unique = new Set<string>()
|
||||
const registrations: { adapter: LlmAdapter; provider: LlmProviderInfo }[] = []
|
||||
for (const provider of providers) {
|
||||
if (provider.length === 0) throw new LlmError('adapter provider names must be non-empty', 'INVALID_ADAPTER')
|
||||
if (unique.has(provider) || this.adapters.has(provider)) {
|
||||
throw new LlmError(`an adapter for provider "${provider}" is already registered`, 'DUPLICATE_ADAPTER')
|
||||
}
|
||||
const info = adapter.providerInfo(provider)
|
||||
if (typeof info.id !== 'string' || info.id !== provider || typeof info.name !== 'string' || info.name.length === 0) {
|
||||
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
|
||||
}
|
||||
unique.add(provider)
|
||||
registrations.push({ adapter, provider: { id: info.id, name: info.name } })
|
||||
}
|
||||
for (const model of models) this.adapters.set(model, adapter)
|
||||
for (const registration of registrations) this.adapters.set(registration.provider.id, registration)
|
||||
yield () => {
|
||||
for (const model of models) this.adapters.delete(model)
|
||||
for (const provider of providers) this.adapters.delete(provider)
|
||||
}
|
||||
}.bind(this), 'llm.registerAdapter()')
|
||||
// ctx.effect's disposer returns Promise<void>; our disposer API is
|
||||
@@ -107,17 +136,66 @@ export class LlmService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Model names with a registered adapter.
|
||||
* @returns the registered names, in registration order.
|
||||
* Describe provider routes with a registered adapter.
|
||||
* @returns detached provider metadata in registration order.
|
||||
*/
|
||||
models(): string[] {
|
||||
return [...this.adapters.keys()]
|
||||
listProviders(): LlmProviderInfo[] {
|
||||
return [...this.adapters.values()].map(({ provider }) => ({ ...provider }))
|
||||
}
|
||||
|
||||
private adapter(model: string): LlmAdapter {
|
||||
const adapter = this.adapters.get(model)
|
||||
if (!adapter) throw new LlmError(`no adapter registered for model "${model}"`, 'NO_ADAPTER')
|
||||
return adapter
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @returns detached model metadata in adapter-preferred order.
|
||||
*/
|
||||
async listModels(provider: string): Promise<LlmModelInfo[]> {
|
||||
const adapter = this.registration(provider).adapter
|
||||
const models = await adapter.listModels(provider)
|
||||
const seen = new Set<string>()
|
||||
return models.map((model) => {
|
||||
if (
|
||||
typeof model.provider !== 'string'
|
||||
|| model.provider !== provider
|
||||
|| typeof model.id !== 'string'
|
||||
|| model.id.length === 0
|
||||
|| typeof model.name !== 'string'
|
||||
|| model.name.length === 0
|
||||
|| (model.description !== undefined && typeof model.description !== 'string')
|
||||
|| seen.has(model.id)
|
||||
) {
|
||||
throw new LlmError(`adapter returned invalid or duplicate model metadata for provider "${provider}"`, 'INVALID_CATALOG')
|
||||
}
|
||||
seen.add(model.id)
|
||||
return {
|
||||
provider: model.provider,
|
||||
id: model.id,
|
||||
name: model.name,
|
||||
...model.description === undefined ? {} : { description: model.description },
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
|
||||
const registration = this.adapters.get(provider)
|
||||
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
|
||||
return registration
|
||||
}
|
||||
|
||||
/** Remove replay state whose historical route is owned by another adapter. */
|
||||
private forAdapter(options: GenerateOptions, adapter: LlmAdapter): GenerateOptions {
|
||||
const messages: Message[] = options.messages.map((message) => {
|
||||
const provenance = message.provenance
|
||||
if (message.role !== 'assistant' || provenance?.replayState === undefined) return message
|
||||
if (this.adapters.get(provenance.provider)?.adapter === adapter) return message
|
||||
return {
|
||||
...message,
|
||||
provenance: { provider: provenance.provider, model: provenance.model },
|
||||
}
|
||||
})
|
||||
if (messages.every((message, index) => message === options.messages[index])) return options
|
||||
const filtered = { ...options, messages }
|
||||
return Object.isFrozen(options) ? deepFreeze(filtered) : filtered
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -131,7 +209,8 @@ export class LlmService extends Service {
|
||||
private async * adapterStream(options: GenerateOptions): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const stream = this.adapter(options.model).stream(options)
|
||||
const adapter = this.registration(options.provider).adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(error)
|
||||
@@ -169,11 +248,12 @@ export class LlmService extends Service {
|
||||
/**
|
||||
* Stream one model call as raw chunks (token-level deltas). Throws
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.model`. Dispatches through the `llm/stream` waterfall. Final
|
||||
* adapter dispatch/iteration failures retain their original Error identity
|
||||
* and are tagged so the agent loop can distinguish them from middleware
|
||||
* failures without widening request recovery to plugin code.
|
||||
* @param options - the full request; `options.model` selects the adapter.
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged for narrow agent-loop request recovery;
|
||||
* middleware failures remain untagged.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
|
||||
@@ -53,10 +53,29 @@ export type ContentBlockType = keyof ContentBlockMap
|
||||
/** Any known content block, derived from {@link ContentBlockMap}; switch on `type` and fall through unknowns (merge-extensible). */
|
||||
export type ContentBlock = ContentBlockMap[ContentBlockType]
|
||||
|
||||
/** A single message in a conversation history. */
|
||||
/** Provider ownership and adapter-private replay data for an assistant message. */
|
||||
export interface AssistantProvenance {
|
||||
/** Provider route that produced the message. */
|
||||
provider: string
|
||||
/** Provider model id that produced the message. */
|
||||
model: string
|
||||
/**
|
||||
* Lossless-JSON adapter state needed to replay the provider response.
|
||||
* `LlmService` exposes it to a target adapter only when that adapter instance
|
||||
* currently owns both this historical provider and the target provider.
|
||||
*/
|
||||
replayState?: unknown
|
||||
}
|
||||
|
||||
/**
|
||||
* A single message in a conversation history. Loop-derived assistant messages
|
||||
* always carry provenance; callers may omit it on hand-built foreign history.
|
||||
*/
|
||||
export interface Message {
|
||||
role: 'system' | 'user' | 'assistant'
|
||||
content: ContentBlock[]
|
||||
/** Present only on assistant messages produced by a routed adapter. */
|
||||
provenance?: AssistantProvenance
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -102,6 +121,26 @@ export interface TokenUsage {
|
||||
reasoningTokens?: number
|
||||
}
|
||||
|
||||
/** Display metadata for one registered provider route. */
|
||||
export interface LlmProviderInfo {
|
||||
/** Provider route key used by {@link GenerateOptions.provider}. */
|
||||
id: string
|
||||
/** Human-readable provider name for selectors and diagnostics. */
|
||||
name: string
|
||||
}
|
||||
|
||||
/** One adapter-discovered model; catalog membership is advisory, not request validation. */
|
||||
export interface LlmModelInfo {
|
||||
/** Provider route that owns this model entry. */
|
||||
provider: string
|
||||
/** Model id passed to {@link GenerateOptions.model}. */
|
||||
id: string
|
||||
/** Human-readable model name for selectors. */
|
||||
name: string
|
||||
/** Optional user-facing distinction from otherwise similar models. */
|
||||
description?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
@@ -116,7 +155,12 @@ export type StreamChunk =
|
||||
| { type: 'tool-call-delta'; index: number; id: CallId; name?: string; argumentsDelta: string }
|
||||
| { type: 'block-end'; index: number; block: ContentBlock }
|
||||
| { type: 'usage'; usage: TokenUsage }
|
||||
| { type: 'finish'; reason: FinishReason }
|
||||
| {
|
||||
type: 'finish'
|
||||
reason: FinishReason
|
||||
/** Adapter-private lossless-JSON state for replaying a successful response. */
|
||||
replayState?: unknown
|
||||
}
|
||||
|
||||
/**
|
||||
* JSON-schema description of a tool, as sent to the model.
|
||||
@@ -134,6 +178,8 @@ export interface ToolSchema {
|
||||
|
||||
/** A single model request, fully assembled. */
|
||||
export interface GenerateOptions {
|
||||
/** Registered provider route selecting the adapter instance. */
|
||||
provider: string
|
||||
model: string
|
||||
/**
|
||||
* Ordered conversation messages, exactly as the provider sees them (after
|
||||
|
||||
@@ -29,12 +29,12 @@ describe('BlockAssembler', () => {
|
||||
expect(assembler.message().role).toBe('assistant')
|
||||
})
|
||||
|
||||
it('returns the completed block from push() on block-end', () => {
|
||||
it('records the completed block from block-end', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
expect(assembler.push({ type: 'block-start', index: 0, blockType: 'text' })).toBeUndefined()
|
||||
expect(assembler.push({ type: 'text-delta', index: 0, text: 'hi' })).toBeUndefined()
|
||||
const block = assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } })
|
||||
expect(block).toEqual({ type: 'text', text: 'hi' })
|
||||
assembler.push({ type: 'block-start', index: 0, blockType: 'text' })
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'hi' })
|
||||
assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } })
|
||||
expect(assembler.blocks()).toEqual([{ type: 'text', text: 'hi' }])
|
||||
})
|
||||
|
||||
it('tolerates deltas without explicit block-start/end', () => {
|
||||
@@ -57,8 +57,8 @@ describe('BlockAssembler', () => {
|
||||
// push a delta first to guarantee the partial exists
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'hi' })
|
||||
// block-end's ensure() must find the existing partial (the second branch path)
|
||||
const block = assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } })
|
||||
expect(block).toEqual({ type: 'text', text: 'hi' })
|
||||
assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } })
|
||||
expect(assembler.blocks()).toEqual([{ type: 'text', text: 'hi' }])
|
||||
})
|
||||
|
||||
it('throws from assemble() when a partial has an unhandled blockType', () => {
|
||||
@@ -128,7 +128,7 @@ describe('assertNever', () => {
|
||||
|
||||
it('BlockAssembler.push rejects chunks outside the closed StreamChunk union', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
expect(() => assembler.push({ type: 'rogue-chunk' } as unknown as StreamChunk))
|
||||
expect(() => { assembler.push({ type: 'rogue-chunk' } as unknown as StreamChunk) })
|
||||
.toThrow('unreachable variant in BlockAssembler.push')
|
||||
})
|
||||
})
|
||||
@@ -140,26 +140,8 @@ describe('BlockAssembler duplicate-close contract', () => {
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'first' } },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'second' } },
|
||||
]
|
||||
const streaming = new BlockAssembler()
|
||||
const closed = []
|
||||
for (const chunk of chunks) {
|
||||
const block = streaming.push(chunk)
|
||||
if (block) closed.push(block)
|
||||
}
|
||||
|
||||
const oneShot = new BlockAssembler()
|
||||
for (const chunk of chunks) oneShot.push(chunk)
|
||||
|
||||
expect(closed).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(oneShot.blocks()).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(closed).toEqual(oneShot.blocks())
|
||||
})
|
||||
|
||||
it('push returns undefined for a duplicate block-end (it closed nothing)', () => {
|
||||
const a = new BlockAssembler()
|
||||
expect(a.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'x' } }))
|
||||
.toEqual({ type: 'text', text: 'x' })
|
||||
expect(a.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'y' } }))
|
||||
.toBeUndefined()
|
||||
const assembler = new BlockAssembler()
|
||||
for (const chunk of chunks) assembler.push(chunk)
|
||||
expect(assembler.blocks()).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
/**
|
||||
* call-config unit tests: field-wise LlmCallConfig equality (the real-change
|
||||
* detector behind logged header deltas) and the deepFreeze ownership helper
|
||||
* detector behind logged changed headers) and the deepFreeze ownership helper
|
||||
* the loop applies to every built request.
|
||||
*/
|
||||
|
||||
@@ -9,14 +9,16 @@ import { callConfigEquals, deepFreeze } from '../src/call-config.ts'
|
||||
|
||||
describe('callConfigEquals', () => {
|
||||
it('compares every field, including the stop list element-wise', () => {
|
||||
expect(callConfigEquals({ model: 'm' }, { model: 'm' })).toBe(true)
|
||||
expect(callConfigEquals({ model: 'm' }, { model: 'x' })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', temperature: 0.5 }, { model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', maxTokens: 1 }, { model: 'm', maxTokens: 2 })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', stop: ['a'] }, { model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', stop: ['a'] }, { model: 'm', stop: ['a', 'b'] })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', stop: ['a'] }, { model: 'm', stop: ['b'] })).toBe(false)
|
||||
expect(callConfigEquals({ model: 'm', stop: ['a', 'b'] }, { model: 'm', stop: ['a', 'b'] })).toBe(true)
|
||||
const base = { provider: 'p', model: 'm' }
|
||||
expect(callConfigEquals(base, base)).toBe(true)
|
||||
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
|
||||
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
|
||||
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, { ...base, stop: ['a', 'b'] })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a'] }, { ...base, stop: ['b'] })).toBe(false)
|
||||
expect(callConfigEquals({ ...base, stop: ['a', 'b'] }, { ...base, stop: ['a', 'b'] })).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import LlmService, {
|
||||
LlmError,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
constructor(private script: StreamChunk[]) {
|
||||
@@ -20,6 +21,32 @@ class ScriptedAdapter extends LlmAdapter {
|
||||
}
|
||||
}
|
||||
|
||||
class RecordingAdapter extends ScriptedAdapter {
|
||||
lastOptions: GenerateOptions | undefined
|
||||
|
||||
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.lastOptions = options
|
||||
yield * super.stream(options)
|
||||
}
|
||||
}
|
||||
|
||||
class CatalogAdapter extends ScriptedAdapter {
|
||||
constructor(
|
||||
private readonly provider: LlmProviderInfo,
|
||||
private readonly models: readonly LlmModelInfo[],
|
||||
) {
|
||||
super(SCRIPT)
|
||||
}
|
||||
|
||||
override providerInfo(_provider: string): LlmProviderInfo {
|
||||
return this.provider
|
||||
}
|
||||
|
||||
override listModels(_provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
}
|
||||
|
||||
const SCRIPT: StreamChunk[] = [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'hi' },
|
||||
@@ -46,19 +73,19 @@ describe('LlmService', () => {
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
ctx.llm.registerAdapter(['test-provider'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) chunks.push(chunk)
|
||||
for await (const chunk of ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })) chunks.push(chunk)
|
||||
expect(chunks).toEqual(SCRIPT)
|
||||
})
|
||||
|
||||
it('throws NO_ADAPTER for unregistered models', async () => {
|
||||
it('throws NO_ADAPTER for unregistered providers', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _ of ctx.llm.stream({ model: 'nope', messages: [] })) { /* drain */ }
|
||||
for await (const _ of ctx.llm.stream({ provider: 'nope', model: 'any-model', messages: [] })) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -97,7 +124,7 @@ describe('LlmService', () => {
|
||||
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ }
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -121,7 +148,7 @@ describe('LlmService', () => {
|
||||
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ }
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -154,7 +181,7 @@ describe('LlmService', () => {
|
||||
|
||||
const failure = (async (): Promise<unknown> => {
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ }
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
return error
|
||||
}
|
||||
@@ -196,7 +223,7 @@ describe('LlmService', () => {
|
||||
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) break
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) break
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -221,7 +248,7 @@ describe('LlmService', () => {
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
let chunks = 0
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) {
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) {
|
||||
chunks += 1
|
||||
break
|
||||
}
|
||||
@@ -234,7 +261,7 @@ describe('LlmService', () => {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
// Third-party adapters can reject with arbitrary values; normalization is under test.
|
||||
// Third-party adapters can reject with arbitrary values.
|
||||
// eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors
|
||||
return { next: () => Promise.reject('plain provider failure') }
|
||||
},
|
||||
@@ -247,7 +274,7 @@ describe('LlmService', () => {
|
||||
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) { /* drain */ }
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -265,7 +292,7 @@ describe('LlmService', () => {
|
||||
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) throw downstream
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) throw downstream
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
@@ -281,10 +308,80 @@ describe('LlmService', () => {
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
inner.llm.registerAdapter(['scoped-model'], new ScriptedAdapter(SCRIPT))
|
||||
}, { inject: ['llm'] }))
|
||||
expect(ctx.llm.models()).toEqual(['scoped-model'])
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'scoped-model', name: 'scoped-model' }])
|
||||
|
||||
await fiber.dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('discovers detached provider and advisory model metadata', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const provider = { id: 'catalog', name: 'Catalog Provider' }
|
||||
const model = { provider: 'catalog', id: 'fast', name: 'Fast', description: 'Low latency' }
|
||||
ctx.llm.registerAdapter(['catalog'], new CatalogAdapter(provider, [model]))
|
||||
|
||||
const providers = ctx.llm.listProviders()
|
||||
const models = await ctx.llm.listModels('catalog')
|
||||
expect(providers).toEqual([provider])
|
||||
expect(models).toEqual([model])
|
||||
|
||||
providers[0]!.name = 'mutated'
|
||||
models[0]!.name = 'mutated'
|
||||
provider.name = 'source mutated'
|
||||
model.name = 'source mutated'
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'catalog', name: 'Catalog Provider' }])
|
||||
await expect(ctx.llm.listModels('catalog')).resolves.toEqual([{
|
||||
provider: 'catalog', id: 'fast', name: 'source mutated', description: 'Low latency',
|
||||
}])
|
||||
})
|
||||
|
||||
it('defaults adapters to their route name and an empty advisory model list', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['plain'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ id: 1, name: 'Name' }, 'non-string id'],
|
||||
[{ id: 'other', name: 'Name' }, 'mismatched id'],
|
||||
[{ id: 'route', name: 1 }, 'non-string name'],
|
||||
[{ id: 'route', name: '' }, 'empty name'],
|
||||
] as const)('rejects invalid provider metadata atomically (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new CatalogAdapter(metadata as unknown as LlmProviderInfo, [])
|
||||
expect(() => ctx.llm.registerAdapter(['route'], adapter)).toThrow(expect.objectContaining({ code: 'INVALID_ADAPTER' }))
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ provider: 1, id: 'm', name: 'M' }, 'non-string provider'],
|
||||
[{ provider: 'other', id: 'm', name: 'M' }, 'mismatched provider'],
|
||||
[{ provider: 'route', id: 1, name: 'M' }, 'non-string id'],
|
||||
[{ provider: 'route', id: '', name: 'M' }, 'empty id'],
|
||||
[{ provider: 'route', id: 'm', name: 1 }, 'non-string name'],
|
||||
[{ provider: 'route', id: 'm', name: '' }, 'empty name'],
|
||||
[{ provider: 'route', id: 'm', name: 'M', description: 1 }, 'non-string description'],
|
||||
] as const)('rejects invalid model metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
|
||||
{ id: 'route', name: 'Route' },
|
||||
[metadata as unknown as LlmModelInfo],
|
||||
))
|
||||
await expect(ctx.llm.listModels('route')).rejects.toMatchObject({ code: 'INVALID_CATALOG' })
|
||||
})
|
||||
|
||||
it('rejects duplicate model ids in one provider catalog', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const model = { provider: 'route', id: 'same', name: 'Same' }
|
||||
ctx.llm.registerAdapter(['route'], new CatalogAdapter({ id: 'route', name: 'Route' }, [model, model]))
|
||||
await expect(ctx.llm.listModels('route')).rejects.toMatchObject({ code: 'INVALID_CATALOG' })
|
||||
})
|
||||
|
||||
it('lets llm/stream waterfall listeners wrap the underlying stream', async () => {
|
||||
@@ -301,11 +398,90 @@ describe('LlmService', () => {
|
||||
})
|
||||
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) chunks.push(chunk)
|
||||
for await (const chunk of ctx.llm.stream({ provider: 'test-model', model: 'dynamic-model', messages: [] })) chunks.push(chunk)
|
||||
expect(chunks).toHaveLength(4)
|
||||
expect(chunks[0]).toMatchObject({ index: 99 })
|
||||
})
|
||||
|
||||
it('resolves the provider after llm/stream listeners have had a chance to route it', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['routed'], adapter)
|
||||
ctx.on('llm/stream', (options, next) => {
|
||||
options.provider = 'routed'
|
||||
return next()
|
||||
})
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'initial', model: 'm', messages: [] })) { /* drain */ }
|
||||
expect(adapter.lastOptions?.provider).toBe('routed')
|
||||
})
|
||||
|
||||
it('keeps replay state when historical and target providers belong to the same adapter instance', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['historical', 'target'], adapter)
|
||||
const replayState = { private: 'state' }
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'target',
|
||||
model: 'new-model',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'old response' }],
|
||||
provenance: { provider: 'historical', model: 'old-model', replayState },
|
||||
}],
|
||||
})) { /* drain */ }
|
||||
|
||||
expect(adapter.lastOptions?.messages[0]?.provenance).toEqual({
|
||||
provider: 'historical', model: 'old-model', replayState,
|
||||
})
|
||||
})
|
||||
|
||||
it('strips replay state but preserves provenance when the target uses a different adapter instance', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['historical'], new RecordingAdapter(SCRIPT))
|
||||
const target = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['target'], target)
|
||||
|
||||
for await (const _chunk of ctx.llm.stream({
|
||||
provider: 'target',
|
||||
model: 'new-model',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'old response' }],
|
||||
provenance: { provider: 'historical', model: 'old-model', replayState: { private: 'state' } },
|
||||
}],
|
||||
})) { /* drain */ }
|
||||
|
||||
expect(target.lastOptions?.messages[0]?.provenance).toEqual({ provider: 'historical', model: 'old-model' })
|
||||
})
|
||||
|
||||
it('preserves immutability while stripping replay state from frozen requests', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['historical'], new RecordingAdapter(SCRIPT))
|
||||
const target = new RecordingAdapter(SCRIPT)
|
||||
ctx.llm.registerAdapter(['target'], target)
|
||||
const options = Object.freeze({
|
||||
provider: 'target',
|
||||
model: 'new-model',
|
||||
messages: [{
|
||||
role: 'assistant' as const,
|
||||
content: [{ type: 'text' as const, text: 'old response' }],
|
||||
provenance: { provider: 'historical', model: 'old-model', replayState: { private: 'state' } },
|
||||
}],
|
||||
})
|
||||
|
||||
for await (const _chunk of ctx.llm.stream(options)) { /* drain */ }
|
||||
|
||||
expect(target.lastOptions).not.toBe(options)
|
||||
expect(Object.isFrozen(target.lastOptions)).toBe(true)
|
||||
expect(target.lastOptions?.messages[0]?.provenance).toEqual({ provider: 'historical', model: 'old-model' })
|
||||
})
|
||||
|
||||
it('creates LlmError with a code for programmatic handling', () => {
|
||||
const err = new LlmError('something went wrong', 'CUSTOM_CODE')
|
||||
expect(err).toBeInstanceOf(Error)
|
||||
@@ -316,11 +492,12 @@ describe('LlmService', () => {
|
||||
|
||||
it('LlmError extends the shared HarnessError base', async () => {
|
||||
const { HarnessError, isHarnessError } = await import('@deepseek-ai/dsh-llm')
|
||||
const err = new LlmError('boom', 'AUTH', 401)
|
||||
const cause = new Error('root cause')
|
||||
const err = new LlmError('boom', 'AUTH', { cause })
|
||||
expect(err).toBeInstanceOf(HarnessError)
|
||||
expect(isHarnessError(err)).toBe(true)
|
||||
expect(err.code).toBe('AUTH')
|
||||
expect(err.status).toBe(401)
|
||||
expect(err.cause).toBe(cause)
|
||||
})
|
||||
|
||||
it('HarnessError carries a code, names itself by subclass, and chains cause', async () => {
|
||||
@@ -341,9 +518,9 @@ describe('LlmService', () => {
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'm1', name: 'm1' }])
|
||||
dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('rejects duplicate adapter registration with DUPLICATE_ADAPTER code', async () => {
|
||||
@@ -360,19 +537,30 @@ describe('LlmService', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects empty and internally duplicated provider registrations atomically', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new ScriptedAdapter(SCRIPT)
|
||||
|
||||
expect(() => ctx.llm.registerAdapter([], adapter)).toThrow(expect.objectContaining({ code: 'INVALID_ADAPTER' }))
|
||||
expect(() => ctx.llm.registerAdapter([''], adapter)).toThrow(expect.objectContaining({ code: 'INVALID_ADAPTER' }))
|
||||
expect(() => ctx.llm.registerAdapter(['first', 'first'], adapter)).toThrow(expect.objectContaining({ code: 'DUPLICATE_ADAPTER' }))
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('re-registers a model after its prior registration is disposed', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'm1', name: 'm1' }])
|
||||
dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
|
||||
// The duplicate check is not wedged: the same model registers cleanly again.
|
||||
const disposeAgain = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'm1', name: 'm1' }])
|
||||
disposeAgain()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user