refactor(llm): resolve model metadata together
This commit is contained in:
@@ -11,8 +11,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.resolveModelReasoning(provider: string, model: string, signal?: AbortSignal): Promise<LlmModelReasoningInfo | undefined>` Resolve ordered adapter-owned reasoning efforts and an optional deployment default for one exact route, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
@@ -21,9 +20,9 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
Reasoning effort is also an exact-route capability, but its identifiers are opaque adapter-owned strings rather than a core enum. `resolveModelReasoning()` validates and detaches the ordered display metadata; `undefined` means the model has no selectable effort. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. Invalid capability metadata fails with `INVALID_MODEL_REASONING`; an unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
|
||||
|
||||
### Events
|
||||
|
||||
@@ -33,7 +32,7 @@ Reasoning effort is also an exact-route capability, but its identifiers are opaq
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, `resolveModelContext()` when exact capacity is known, and `resolveModelReasoning()` when a model exposes selectable efforts; an asynchronous reasoning resolver must honor its optional cancellation signal. The defaults use the route id as its name, advertise no models, and return neither capacity nor reasoning metadata.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -10,9 +10,8 @@ import { Context, Service } from 'cordis'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmModelReasoningInfo,
|
||||
LlmResolvedModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
@@ -147,34 +146,20 @@ export abstract class LlmAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity for one model accepted by this adapter. Absence
|
||||
* means the adapter does not know the capacity, not that routing is invalid.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns provider-owned context metadata, or `undefined` when unavailable.
|
||||
* Resolve all metadata available for one exact model. This query is
|
||||
* independent of the advisory catalog and does not validate request routing.
|
||||
* @param provider - one provider route owned by this adapter.
|
||||
* @param model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; asynchronous
|
||||
* implementations must settle promptly after it aborts.
|
||||
* @returns provider/model identity plus any context and reasoning metadata.
|
||||
*/
|
||||
resolveModelContext(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve selectable reasoning efforts for one exact model. Absence means
|
||||
* the model has no selectable reasoning-effort capability.
|
||||
* @param _provider - one provider route owned by this adapter.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @param _signal - cancellation for this exact-model lookup; implementations
|
||||
* must settle promptly after it aborts.
|
||||
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
return Promise.resolve(undefined)
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({ provider, id: model, name: model })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -273,53 +258,59 @@ export class LlmService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve context capacity from the adapter that owns one exact route.
|
||||
* This query is independent of the advisory model catalog: an unlisted model
|
||||
* may return metadata, while `undefined` never rejects later routing.
|
||||
* Resolve and validate all metadata from the adapter that owns one exact
|
||||
* route. The result is detached from adapter-owned objects; catalog
|
||||
* membership remains advisory and does not control request routing.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @returns detached context metadata, or `undefined` when the adapter has none.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns exact model identity plus available context and reasoning metadata.
|
||||
*/
|
||||
async resolveModelContext(
|
||||
async resolveModelInfo(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
|
||||
if (context === undefined) return undefined
|
||||
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return this.resolveModelInfoFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelInfoFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
const provider = registration.provider.id
|
||||
const resolved = await registration.adapter.resolveModel(provider, model, signal)
|
||||
if (
|
||||
typeof resolved.provider !== 'string'
|
||||
|| resolved.provider !== provider
|
||||
|| typeof resolved.id !== 'string'
|
||||
|| resolved.id !== model
|
||||
|| typeof resolved.name !== 'string'
|
||||
|| resolved.name.length === 0
|
||||
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
|
||||
) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_INFO',
|
||||
)
|
||||
}
|
||||
const context = resolved.context
|
||||
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
|
||||
'INVALID_MODEL_CONTEXT',
|
||||
)
|
||||
}
|
||||
return { contextWindow: context.contextWindow }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve selectable reasoning efforts from the adapter that owns one exact
|
||||
* route. Metadata is validated and detached; an absent result means an
|
||||
* effort selector is unsupported for that model.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @param model - exact model id passed to the adapter.
|
||||
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
|
||||
* @returns detached reasoning metadata, or `undefined` when unsupported.
|
||||
*/
|
||||
async resolveModelReasoning(
|
||||
provider: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
return this.resolveModelReasoningFor(this.registration(provider), model, signal)
|
||||
}
|
||||
|
||||
private async resolveModelReasoningFor(
|
||||
registration: AdapterRegistration,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
const provider = registration.provider.id
|
||||
const reasoning = await registration.adapter.resolveModelReasoning(provider, model, signal)
|
||||
if (reasoning === undefined) return undefined
|
||||
const info: LlmResolvedModelInfo = {
|
||||
provider,
|
||||
id: model,
|
||||
name: resolved.name,
|
||||
...resolved.description === undefined ? {} : { description: resolved.description },
|
||||
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
|
||||
}
|
||||
const reasoning = resolved.reasoning
|
||||
if (reasoning === undefined) return info
|
||||
if (reasoning.efforts.length === 0) {
|
||||
throw new LlmError(
|
||||
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
|
||||
@@ -355,8 +346,11 @@ export class LlmService extends Service {
|
||||
)
|
||||
}
|
||||
return {
|
||||
efforts,
|
||||
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
|
||||
...info,
|
||||
reasoning: {
|
||||
efforts,
|
||||
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -379,7 +373,7 @@ export class LlmService extends Service {
|
||||
config: LlmCallConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmCallConfig> {
|
||||
const reasoning = await this.resolveModelReasoningFor(registration, config.model, signal)
|
||||
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
|
||||
const requested = config.reasoningEffort
|
||||
if (reasoning === undefined) {
|
||||
if (requested !== undefined) {
|
||||
@@ -520,7 +514,7 @@ export class LlmService extends Service {
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection remains fixed through asynchronous reasoning resolution
|
||||
* adapter selection remains fixed through asynchronous exact-model resolution
|
||||
* and dispatch. Selection, dispatch, and iteration failures retain their
|
||||
* original Error identity and are tagged in a call-local scope for narrow
|
||||
* agent-loop request recovery; middleware and nested-call failures remain
|
||||
|
||||
@@ -182,6 +182,14 @@ export interface LlmModelReasoningInfo {
|
||||
defaultEffort?: ReasoningEffortId
|
||||
}
|
||||
|
||||
/** Exact-route model metadata resolved by its owning adapter. */
|
||||
export interface LlmResolvedModelInfo extends LlmModelInfo {
|
||||
/** Provider-owned context capacity when known. */
|
||||
context?: LlmModelContext
|
||||
/** Adapter-owned selectable reasoning levels when exposed. */
|
||||
reasoning?: LlmModelReasoningInfo
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw streaming protocol emitted by adapters.
|
||||
* Block indexes correlate interleaved deltas, and `block-end` carries the
|
||||
|
||||
@@ -19,6 +19,7 @@ import type {
|
||||
LlmModelInfo,
|
||||
LlmModelReasoningInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
@@ -68,18 +69,17 @@ class CatalogAdapter extends ScriptedAdapter {
|
||||
return Promise.resolve(this.models)
|
||||
}
|
||||
|
||||
override resolveModelContext(
|
||||
_provider: string,
|
||||
override resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
return Promise.resolve(this.contexts[model])
|
||||
}
|
||||
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
model: string,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
return Promise.resolve(this.reasoning[model])
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
|
||||
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -667,8 +667,32 @@ describe('LlmService', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
|
||||
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
|
||||
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
|
||||
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
|
||||
provider: 'plain', id: 'unlisted', name: 'unlisted',
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
|
||||
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
|
||||
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
|
||||
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
|
||||
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
|
||||
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
|
||||
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
|
||||
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModel(): Promise<LlmResolvedModelInfo> {
|
||||
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
|
||||
}
|
||||
}(SCRIPT)
|
||||
ctx.llm.registerAdapter(['route'], adapter)
|
||||
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
|
||||
})
|
||||
|
||||
it('resolves detached model context independently of advisory catalog membership', async () => {
|
||||
@@ -681,11 +705,13 @@ describe('LlmService', () => {
|
||||
{ unlisted: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
source.contextWindow = 64_000
|
||||
expect(resolved).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
|
||||
expect(resolved.context).toEqual({ contextWindow: 32_000 })
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
|
||||
provider: 'route', id: 'other', name: 'other',
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
|
||||
@@ -705,10 +731,10 @@ describe('LlmService', () => {
|
||||
{ model: source },
|
||||
))
|
||||
|
||||
const resolved = await ctx.llm.resolveModelReasoning('route', 'model')
|
||||
expect(resolved).toEqual(source)
|
||||
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
|
||||
expect(resolved.reasoning).toEqual(source)
|
||||
source.efforts[0]!.name = 'mutated'
|
||||
expect(resolved?.efforts[0]?.name).toBe('Standard')
|
||||
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
|
||||
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
|
||||
provider: 'route',
|
||||
model: 'model',
|
||||
@@ -734,7 +760,7 @@ describe('LlmService', () => {
|
||||
{},
|
||||
{ model: metadata as unknown as LlmModelReasoningInfo },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelReasoning('route', 'model'))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
|
||||
})
|
||||
|
||||
@@ -764,13 +790,16 @@ describe('LlmService', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const adapter = new class extends RecordingAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
): Promise<LlmModelReasoningInfo> {
|
||||
return Promise.resolve({
|
||||
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
|
||||
const reasoning: LlmModelReasoningInfo = {
|
||||
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
|
||||
defaultEffort: ReasoningEffortId('standard'),
|
||||
}
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning,
|
||||
})
|
||||
}
|
||||
}(SCRIPT)
|
||||
@@ -799,19 +828,24 @@ describe('LlmService', () => {
|
||||
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
|
||||
})
|
||||
|
||||
it('pins one adapter registration across asynchronous reasoning resolution and dispatch', async () => {
|
||||
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
|
||||
const first = new class extends RecordingAdapter {
|
||||
override resolveModelReasoning(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
override async resolveModel(
|
||||
provider: string,
|
||||
model: string,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo> {
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return reasoning.promise
|
||||
return {
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
reasoning: await reasoning.promise,
|
||||
}
|
||||
}
|
||||
}(SCRIPT)
|
||||
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
|
||||
@@ -869,18 +903,18 @@ describe('LlmService', () => {
|
||||
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
|
||||
})
|
||||
|
||||
it('passes cancellation through reasoning capability resolution', async () => {
|
||||
it('passes cancellation through exact-model resolution', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const started = Promise.withResolvers<undefined>()
|
||||
const adapter = new class extends ScriptedAdapter {
|
||||
override resolveModelReasoning(
|
||||
override resolveModel(
|
||||
_provider: string,
|
||||
_model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<LlmModelReasoningInfo | undefined> {
|
||||
): Promise<LlmResolvedModelInfo> {
|
||||
started.resolve(undefined)
|
||||
return new Promise((_resolve, reject) => {
|
||||
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
|
||||
if (signal === undefined) {
|
||||
reject(new Error('missing reasoning signal'))
|
||||
return
|
||||
@@ -918,7 +952,7 @@ describe('LlmService', () => {
|
||||
[],
|
||||
{ model: { contextWindow } },
|
||||
))
|
||||
await expect(ctx.llm.resolveModelContext('route', 'model'))
|
||||
await expect(ctx.llm.resolveModelInfo('route', 'model'))
|
||||
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
|
||||
},
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user