feat: implement bounded LLM request recovery
This commit is contained in:
@@ -13,7 +13,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`.
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
@@ -21,12 +21,12 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
|---|---|---|
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) |
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call for caching, logging, or routing |
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -47,8 +47,9 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) matches its frozen serializable `failure.code`. The payload may also retain validated status, `Retry-After`, and branded provider request id facts; policy remains outside the error.
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
|
||||
- `QUOTA_EXCEEDED_CODE` — the non-transient provider-neutral code for exhausted account quota, balance, credits, budget, or usage limits. `isQuotaExceededError(detail)` keeps those failures distinct from request-rate limits.
|
||||
|
||||
### Real adapters
|
||||
|
||||
@@ -64,7 +65,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure.
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
|
||||
@@ -5,10 +5,10 @@
|
||||
*/
|
||||
|
||||
import { HarnessError } from './error.ts'
|
||||
import type { StreamChunk } from './types.ts'
|
||||
import type { LlmFailure, StreamChunk } from './types.ts'
|
||||
|
||||
/** Errors proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakSet<Error>
|
||||
/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakMap<Error, LlmFailure>
|
||||
|
||||
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
|
||||
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
|
||||
@@ -47,10 +47,70 @@ export function markLlmAdapterFailure(
|
||||
const error = value instanceof Error
|
||||
? value as Error & { code?: string }
|
||||
: new HarnessError(String(value), 'UNKNOWN', { cause: value })
|
||||
failures.add(error)
|
||||
const carried = error instanceof HarnessError ? ownFailureSnapshot(error) : undefined
|
||||
const failure = carried !== undefined && carried.code === error.code ? carried : Object.freeze({
|
||||
message: errorMessage(error),
|
||||
code: harnessErrorCode(error),
|
||||
})
|
||||
failures.set(error, failure)
|
||||
return error
|
||||
}
|
||||
|
||||
/** Snapshot an own data property without invoking an SDK-defined accessor. */
|
||||
function ownFailureSnapshot(error: Error): LlmFailure | undefined {
|
||||
try {
|
||||
const descriptor = Object.getOwnPropertyDescriptor(error, 'failure')
|
||||
return descriptor !== undefined && 'value' in descriptor
|
||||
? failureSnapshot(descriptor.value)
|
||||
: undefined
|
||||
} catch (_sdkPropertyTrap) {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate and detach an arbitrary serializable failure payload. */
|
||||
function failureSnapshot(value: unknown): LlmFailure | undefined {
|
||||
if (typeof value !== 'object' || value === null) return undefined
|
||||
try {
|
||||
const candidate = value as Partial<LlmFailure>
|
||||
const message = candidate.message
|
||||
const code = candidate.code
|
||||
const status = candidate.status
|
||||
const retryAfterMs = candidate.retryAfterMs
|
||||
const requestId = candidate.requestId
|
||||
if (typeof message !== 'string' || message.length === 0
|
||||
|| typeof code !== 'string' || code.length === 0
|
||||
|| (status !== undefined && (!Number.isInteger(status) || status < 100 || status > 599))
|
||||
|| (retryAfterMs !== undefined && (!Number.isFinite(retryAfterMs) || retryAfterMs <= 0))
|
||||
|| (requestId !== undefined && (typeof requestId !== 'string' || requestId.length === 0))) return undefined
|
||||
return Object.freeze({
|
||||
message,
|
||||
code,
|
||||
...status === undefined ? {} : { status },
|
||||
...retryAfterMs === undefined ? {} : { retryAfterMs },
|
||||
...requestId === undefined ? {} : { requestId },
|
||||
})
|
||||
} catch (_sdkFailureGetter) {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Read an SDK error message without letting an accessor replace the primary failure. */
|
||||
function errorMessage(error: Error): string {
|
||||
try {
|
||||
const message: unknown = error.message
|
||||
if (typeof message === 'string' && message.length > 0) return message
|
||||
} catch (_sdkMessageGetter) {
|
||||
// The fallback below preserves a serializable failure beside the original Error.
|
||||
}
|
||||
return 'LLM adapter failed'
|
||||
}
|
||||
|
||||
/** Trust only Harness-owned codes; third-party SDK codes are not our taxonomy. */
|
||||
function harnessErrorCode(error: Error): string {
|
||||
return error instanceof HarnessError ? error.code : 'UNKNOWN'
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a failure came from final adapter dispatch, iterator construction,
|
||||
* or iteration for the call represented by the exact returned stream handle.
|
||||
@@ -65,3 +125,18 @@ export function isLlmAdapterFailure(
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error && failures !== undefined && failures.has(value)
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieve normalized provider facts only for an Error tagged by this exact
|
||||
* model call's final adapter boundary.
|
||||
* @param stream - the exact stream returned to the consumer.
|
||||
* @param value - the caught failure.
|
||||
* @returns the immutable facts for that call, or `undefined` for middleware, nested, or consumer failures.
|
||||
*/
|
||||
export function llmFailureOf(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
value: unknown,
|
||||
): LlmFailure | undefined {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error ? failures?.get(value) : undefined
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
/**
|
||||
* dsh-llm's owned branded id: `CallId` (tool-call correlation).
|
||||
* dsh-llm's owned branded ids: tool-call correlation and provider request
|
||||
* diagnostics.
|
||||
*
|
||||
* The `Branded<B>` primitive itself lives in `@deepseek-ai/dsh-brand` (a
|
||||
* zero-dependency type-only package) so every owner of a cross-boundary id can
|
||||
@@ -25,3 +26,15 @@ export type CallId = Branded<'CallId'>
|
||||
export function CallId(id: string): CallId {
|
||||
return id as CallId
|
||||
}
|
||||
|
||||
/** Provider-issued request identifier retained for diagnostics across package boundaries. */
|
||||
export type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
|
||||
/**
|
||||
* Brand a provider-issued request identifier.
|
||||
* @param id - the opaque provider-issued string.
|
||||
* @returns the same string, branded; no validation is performed.
|
||||
*/
|
||||
export function ProviderRequestId(id: string): ProviderRequestId {
|
||||
return id as ProviderRequestId
|
||||
}
|
||||
|
||||
@@ -24,6 +24,9 @@ export class HarnessError extends Error {
|
||||
/** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */
|
||||
export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED'
|
||||
|
||||
/** Canonical provider-neutral code for an exhausted account quota or balance. */
|
||||
export const QUOTA_EXCEEDED_CODE = 'QUOTA'
|
||||
|
||||
/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */
|
||||
const STRUCTURED_CONTEXT_OVERFLOW = new RegExp(
|
||||
String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]`
|
||||
@@ -62,6 +65,19 @@ export function isContextWindowExceededError(detail: string): boolean {
|
||||
|| EXCEEDS_MODEL_CONTEXT.test(detail)
|
||||
}
|
||||
|
||||
/**
|
||||
* Recognize provider wording that identifies an exhausted account quota rather
|
||||
* than a transient request-rate limit.
|
||||
* @param detail - provider error code/type/message text joined into one string.
|
||||
* @returns true only for terminal quota, balance, credit, budget, or usage-limit wording.
|
||||
*/
|
||||
export function isQuotaExceededError(detail: string): boolean {
|
||||
return /\binsufficient[\s_-]+(?:quota|balance|credits?)\b/i.test(detail)
|
||||
|| /\b(?:quota|usage[\s_-]+limit)[\s_-]+(?:exceeded|exhausted|reached)\b/i.test(detail)
|
||||
|| /\b(?:balance|credits?)[\s_-]+(?:exhausted|depleted)\b/i.test(detail)
|
||||
|| /\bout[\s_-]+of[\s_-]+(?:credits?|budget)\b/i.test(detail)
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams).
|
||||
* @param value - the caught value (`unknown` in catch clauses).
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
|
||||
import type { GenerateOptions, LlmFailure, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
@@ -21,7 +22,7 @@ export * from './types.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export { isLlmAdapterFailure } from './adapter-failure.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -44,14 +45,53 @@ declare module 'cordis' {
|
||||
}
|
||||
}
|
||||
|
||||
/** Structured provider facts and cause accepted by {@link LlmError}. */
|
||||
export interface LlmErrorOptions extends ErrorOptions {
|
||||
/** Valid HTTP status observed at the provider boundary. */
|
||||
status?: number
|
||||
/** Positive finite provider-requested delay in milliseconds. */
|
||||
retryAfterMs?: number
|
||||
/** Non-empty opaque provider request id. */
|
||||
requestId?: ProviderRequestId
|
||||
}
|
||||
|
||||
/**
|
||||
* Typed error for LLM-related failures. Extends {@link HarnessError}, so the
|
||||
* `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy.
|
||||
*/
|
||||
export class LlmError extends HarnessError {
|
||||
constructor(message: string, code: string, options?: ErrorOptions) {
|
||||
/** Serializable facts retained beside this live Error. */
|
||||
readonly failure: LlmFailure
|
||||
|
||||
/**
|
||||
* @param message - non-empty human-readable failure summary.
|
||||
* @param code - non-empty stable provider-neutral machine code.
|
||||
* @param options - optional cause and validated serializable provider facts.
|
||||
*/
|
||||
constructor(message: string, code: string, options?: LlmErrorOptions) {
|
||||
if (typeof message !== 'string' || message.length === 0) throw new Error('LlmError message must be a non-empty string')
|
||||
if (typeof code !== 'string' || code.length === 0) throw new Error('LlmError code must be a non-empty string')
|
||||
if (options?.status !== undefined
|
||||
&& (!Number.isInteger(options.status) || options.status < 100 || options.status > 599)) {
|
||||
throw new Error('LlmError status must be an integer from 100 through 599')
|
||||
}
|
||||
if (options?.retryAfterMs !== undefined
|
||||
&& (!Number.isFinite(options.retryAfterMs) || options.retryAfterMs <= 0)) {
|
||||
throw new Error('LlmError retryAfterMs must be a positive finite number')
|
||||
}
|
||||
if (options?.requestId !== undefined
|
||||
&& (typeof options.requestId !== 'string' || options.requestId.length === 0)) {
|
||||
throw new Error('LlmError requestId must be a non-empty string')
|
||||
}
|
||||
super(message, code, options)
|
||||
this.name = 'LlmError'
|
||||
this.failure = Object.freeze({
|
||||
message,
|
||||
code,
|
||||
...options?.status === undefined ? {} : { status: options.status },
|
||||
...options?.retryAfterMs === undefined ? {} : { retryAfterMs: options.retryAfterMs },
|
||||
...options?.requestId === undefined ? {} : { requestId: options.requestId },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -262,7 +302,7 @@ export class LlmService extends Service {
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakSet<Error>()
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
|
||||
@@ -5,7 +5,21 @@
|
||||
*/
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { CallId } from './brand.ts'
|
||||
import type { CallId, ProviderRequestId } from './brand.ts'
|
||||
|
||||
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
|
||||
export interface LlmFailure {
|
||||
/** Human-readable provider or transport failure. */
|
||||
readonly message: string
|
||||
/** Stable provider-neutral machine-routing code. */
|
||||
readonly code: string
|
||||
/** HTTP status observed at the provider boundary, when available. */
|
||||
readonly status?: number
|
||||
/** Provider-requested delay in milliseconds, when valid and available. */
|
||||
readonly retryAfterMs?: number
|
||||
/** Opaque provider-issued request identifier for diagnostics. */
|
||||
readonly requestId?: ProviderRequestId
|
||||
}
|
||||
|
||||
/** Plain text visible to the end user. */
|
||||
export interface TextBlock {
|
||||
@@ -98,8 +112,8 @@ export interface FinishReasonMap {
|
||||
'stop': { kind: 'stop' }
|
||||
'tool-calls': { kind: 'tool-calls' }
|
||||
'max-tokens': { kind: 'max-tokens' }
|
||||
'aborted': { kind: 'aborted' }
|
||||
'error': { kind: 'error'; message: string; code?: string }
|
||||
'aborted': { kind: 'aborted'; failure: LlmFailure }
|
||||
'error': { kind: 'error'; failure: LlmFailure }
|
||||
}
|
||||
|
||||
/** Any known finish reason, derived from {@link FinishReasonMap}; switch on `kind` and fall through unknowns (merge-extensible). */
|
||||
|
||||
@@ -41,7 +41,10 @@ const chunkArb: fc.Arbitrary<StreamChunk> = indexArb.chain(index => fc.oneof(
|
||||
fc.constant<StreamChunk>({ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }),
|
||||
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'stop' } }),
|
||||
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'tool-calls' } }),
|
||||
fc.string().map((message): StreamChunk => ({ type: 'finish', reason: { kind: 'error', message } })),
|
||||
fc.string({ minLength: 1 }).map((message): StreamChunk => ({
|
||||
type: 'finish',
|
||||
reason: { kind: 'error', failure: { message, code: 'UNKNOWN' } },
|
||||
})),
|
||||
))
|
||||
|
||||
/** A stream is an arbitrary list of chunks (we do NOT force a terminal finish). */
|
||||
|
||||
@@ -4,9 +4,12 @@ import LlmService, {
|
||||
GenerateOptions,
|
||||
HarnessError,
|
||||
isContextWindowExceededError,
|
||||
isQuotaExceededError,
|
||||
isLlmAdapterFailure,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
llmFailureOf,
|
||||
ProviderRequestId,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
@@ -80,6 +83,17 @@ describe('LlmService', () => {
|
||||
expect(isContextWindowExceededError('context window size must be positive')).toBe(false)
|
||||
})
|
||||
|
||||
it('distinguishes exhausted account quota from transient rate limiting', () => {
|
||||
for (const detail of [
|
||||
'insufficient_quota',
|
||||
'account balance depleted',
|
||||
'usage-limit-exceeded',
|
||||
'out of credits',
|
||||
]) expect(isQuotaExceededError(detail)).toBe(true)
|
||||
expect(isQuotaExceededError('HTTP 429: rate limit reached')).toBe(false)
|
||||
expect(isQuotaExceededError('quota resets in one minute')).toBe(false)
|
||||
})
|
||||
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -168,6 +182,151 @@ describe('LlmService', () => {
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
expect(llmFailureOf(stream, caught)).toEqual({
|
||||
message: `${boundary} failed`,
|
||||
code: 'BOUNDARY_FAILED',
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps structured provider facts beside a frozen third-party Error', async () => {
|
||||
const original = new LlmError('provider busy', 'RATE_LIMIT', {
|
||||
status: 429,
|
||||
retryAfterMs: 1_500,
|
||||
requestId: ProviderRequestId('req-7'),
|
||||
})
|
||||
Object.freeze(original)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(llmFailureOf(stream, caught)).toEqual({
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
retryAfterMs: 1_500,
|
||||
requestId: ProviderRequestId('req-7'),
|
||||
})
|
||||
})
|
||||
|
||||
it('does not trust retry facts carried by an unknown third-party Error', async () => {
|
||||
const carried = { message: 'busy', code: 'SERVER', status: 503 }
|
||||
const original = Object.assign(new Error('busy'), { failure: carried })
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
const facts = llmFailureOf(stream, original)
|
||||
carried.status = 500
|
||||
|
||||
expect(facts).toEqual({ message: 'busy', code: 'UNKNOWN' })
|
||||
expect(Object.isFrozen(facts)).toBe(true)
|
||||
expect(facts).not.toBe(carried)
|
||||
})
|
||||
|
||||
it('keeps an unknown SDK Error exact without trusting its private code or accessors', async () => {
|
||||
const original = Object.assign(new Error('socket closed'), { code: 'ECONNRESET' })
|
||||
Object.defineProperty(original, 'failure', {
|
||||
get() { throw new Error('SDK failure accessor must not run') },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
|
||||
expect(original.code).toBe('ECONNRESET')
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'socket closed', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('keeps an SDK Error exact when its message accessor is hostile', async () => {
|
||||
const original = Object.defineProperty(new Error(), 'message', {
|
||||
get() { throw new Error('SDK message accessor trap') },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'LLM adapter failed', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('falls back safely when SDK objects trap failure inspection or expose malformed facts', async () => {
|
||||
const propertyTrap = new Proxy(new HarnessError('descriptor trapped', 'SERVER'), {
|
||||
getOwnPropertyDescriptor(target, property) {
|
||||
if (property === 'failure') throw new Error('SDK descriptor trap')
|
||||
return Reflect.getOwnPropertyDescriptor(target, property)
|
||||
},
|
||||
})
|
||||
const throwingFacts = Object.create(null) as Record<string, unknown>
|
||||
Object.defineProperty(throwingFacts, 'message', {
|
||||
get() { throw new Error('SDK fact getter trap') },
|
||||
})
|
||||
const carrying = (message: string, failure: unknown): HarnessError => Object.defineProperty(
|
||||
new HarnessError(message, 'SERVER'),
|
||||
'failure',
|
||||
{ value: failure },
|
||||
)
|
||||
const factGetter = carrying('fact getter failed', throwingFacts)
|
||||
const malformed = carrying('malformed facts', { message: 'provider busy', code: 'SERVER', requestId: 1 })
|
||||
const primitive = carrying('primitive facts', 1)
|
||||
const nullFacts = carrying('null facts', null)
|
||||
const mismatched = carrying('mismatched facts', { message: 'busy', code: 'RATE_LIMIT' })
|
||||
|
||||
for (const [original, expectedMessage] of [
|
||||
[propertyTrap, 'descriptor trapped'],
|
||||
[factGetter, 'fact getter failed'],
|
||||
[malformed, 'malformed facts'],
|
||||
[primitive, 'primitive facts'],
|
||||
[nullFacts, 'null facts'],
|
||||
[mismatched, 'mismatched facts'],
|
||||
] as const) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: expectedMessage, code: 'SERVER' })
|
||||
}
|
||||
})
|
||||
|
||||
it('retains a stable code from a HarnessError without requiring LlmError facts', async () => {
|
||||
const original = new HarnessError('stable adapter failure', 'ADAPTER_STABLE')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({
|
||||
message: 'stable adapter failure',
|
||||
code: 'ADAPTER_STABLE',
|
||||
})
|
||||
expect(llmFailureOf(stream, 'not an Error')).toBeUndefined()
|
||||
expect(llmFailureOf({ [Symbol.asyncIterator]: () => stream[Symbol.asyncIterator]() }, original)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps a nested adapter failure scoped to the nested model call', async () => {
|
||||
@@ -586,6 +745,15 @@ describe('LlmService', () => {
|
||||
expect(err.code).toBe('CUSTOM_CODE')
|
||||
})
|
||||
|
||||
it('rejects non-serializable structured failure facts at construction', () => {
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { status: 42 })).toThrow(/status/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { retryAfterMs: Number.NaN })).toThrow(/retryAfterMs/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: ProviderRequestId('') })).toThrow(/requestId/)
|
||||
expect(() => new LlmError(1 as never, 'RATE_LIMIT')).toThrow(/message/)
|
||||
expect(() => new LlmError('busy', 1 as never)).toThrow(/code/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: 1 as never })).toThrow(/requestId/)
|
||||
})
|
||||
|
||||
it('LlmError extends the shared HarnessError base', async () => {
|
||||
const { HarnessError, isHarnessError } = await import('@deepseek-ai/dsh-llm')
|
||||
const cause = new Error('root cause')
|
||||
|
||||
Reference in New Issue
Block a user