Merge remote-tracking branch 'origin/master' into worktree/routed-model-compaction-policy
# Conflicts: # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md # docs/cordis-catalog/events.md # docs/cordis-catalog/services.md # docs/event-producer-consumer.md # examples/headless-agent/tests/harness.ts # examples/repl-agent/cordis.yml # packages/compact/compact-basic/README.md # packages/compact/compact-basic/src/index.ts # packages/compact/compact-basic/tests/compact-basic.spec.ts # packages/compact/compact-basic/tests/loader-composition.spec.ts # packages/cordis/tool-cordis/src/api-catalog.ts # packages/llm/README.md # packages/llm/llm-deepseek/src/adapter.ts # packages/llm/llm-pi-ai/src/adapter.ts # packages/llm/llm/README.md # packages/llm/llm/src/index.ts # scripts/gen-cordis-catalog.ts # website/zh-CN/api/harness/events.md # website/zh-CN/api/harness/llm.md # website/zh-CN/api/harness/token-meter.md # website/zh-CN/guide/config.md
This commit is contained in:
@@ -6,7 +6,8 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|
||||
|---|---|---|
|
||||
| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` |
|
||||
| `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | Bounded transient request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
|
||||
|
||||
The interface lives at `llm/llm/`; adapters and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The same route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the interface or consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
|
||||
@@ -16,6 +16,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
name: DeepSeek V4 Flash
|
||||
@@ -33,6 +34,8 @@ The plugin registers the single provider route `deepseek`. A request selects it
|
||||
|
||||
`thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral.
|
||||
|
||||
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; agent-level retry is a separate plugin policy.
|
||||
|
||||
## App attribution
|
||||
|
||||
Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode.
|
||||
@@ -46,11 +49,11 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
|
||||
## Errors
|
||||
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: <REASON>}` chunks.
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites run against a local `node:http` mock SSE server (no network). Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
|
||||
Unit suites run against a local `node:http` mock SSE server (no network), including structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
@@ -30,6 +31,7 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmModelContext,
|
||||
@@ -13,6 +13,7 @@ import type {
|
||||
LlmProviderInfo,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { serializeRequest } from './serialize.ts'
|
||||
import type { RequestDefaults } from './serialize.ts'
|
||||
import { parseSse } from './sse.ts'
|
||||
@@ -41,6 +42,27 @@ export interface DeepSeekAdapterOptions {
|
||||
defaults?: RequestDefaults
|
||||
/** Advisory models exposed to discovery consumers; requests remain unrestricted. */
|
||||
models?: readonly DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
}
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT'
|
||||
|
||||
function providerRetryAfterMs(value: string | null): number | undefined {
|
||||
if (value === null) return undefined
|
||||
if (/^\d+$/.test(value)) {
|
||||
const delay = Number(value) * 1_000
|
||||
return Number.isFinite(delay) && delay > 0 ? delay : undefined
|
||||
}
|
||||
const delay = Date.parse(value) - Date.now()
|
||||
return Number.isFinite(delay) && delay > 0 ? delay : undefined
|
||||
}
|
||||
|
||||
function requestId(headers: Headers): ReturnType<typeof ProviderRequestId> | undefined {
|
||||
const value = headers.get('x-request-id') ?? headers.get('x-deepseek-request-id')
|
||||
return value === null || value.length === 0 ? undefined : ProviderRequestId(value)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -51,9 +73,10 @@ export interface DeepSeekAdapterOptions {
|
||||
*/
|
||||
export function httpErrorCode(status: number, error?: WireError['error']): string {
|
||||
if (status === 401 || status === 403) return 'AUTH'
|
||||
const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ')
|
||||
if (isQuotaExceededError(detail)) return QUOTA_EXCEEDED_CODE
|
||||
if (status === 429) return 'RATE_LIMIT'
|
||||
if (status === 400) {
|
||||
const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ')
|
||||
if (isContextWindowExceededError(detail)) return CONTEXT_WINDOW_EXCEEDED_CODE
|
||||
return 'INVALID_REQUEST'
|
||||
}
|
||||
@@ -65,13 +88,22 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin
|
||||
* The first real `LlmAdapter`. One instance serves every model name it was
|
||||
* registered under (the harness model name IS the wire model name).
|
||||
*
|
||||
* Abort: `options.signal` is handed to fetch — both the initial request and
|
||||
* the body stream reject on abort, which surfaces to the loop as a rejected
|
||||
* step (the loop already contains step errors).
|
||||
* One stable signal reaches both initial fetch and body reads. Caller aborts
|
||||
* map to `ABORTED`; the configured per-read idle watchdog maps to `TIMEOUT`.
|
||||
*/
|
||||
export class DeepSeekAdapter extends LlmAdapter {
|
||||
private readonly streamIdleTimeoutMs: number
|
||||
|
||||
constructor(private readonly options: DeepSeekAdapterOptions) {
|
||||
super()
|
||||
this.streamIdleTimeoutMs = options.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
|
||||
if (!Number.isFinite(this.streamIdleTimeoutMs)
|
||||
|| this.streamIdleTimeoutMs <= 0
|
||||
|| this.streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(
|
||||
`llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
override providerInfo(provider: string): LlmProviderInfo {
|
||||
@@ -96,24 +128,85 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
? consumer.signal
|
||||
: AbortSignal.any([options.signal, consumer.signal])
|
||||
using watchdog = idleWatchdog(upstream, this.streamIdleTimeoutMs, STREAM_IDLE_TIMEOUT_CODE)
|
||||
const iterator = this.request(options, watchdog.signal)[Symbol.asyncIterator]()
|
||||
let exhausted = false
|
||||
try {
|
||||
while (true) {
|
||||
const result = await watchdog.next(iterator)
|
||||
if (result.done) {
|
||||
exhausted = true
|
||||
return
|
||||
}
|
||||
yield result.value
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
if (timeoutOf(watchdog.signal, STREAM_IDLE_TIMEOUT_CODE) !== undefined) {
|
||||
throw new LlmError(
|
||||
`DeepSeek stream idle timeout after ${this.streamIdleTimeoutMs}ms`,
|
||||
'TIMEOUT',
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
if (options.signal?.aborted) {
|
||||
throw new LlmError('DeepSeek request aborted by caller', 'ABORTED', { cause: error })
|
||||
}
|
||||
if (error instanceof LlmError) throw error
|
||||
throw new LlmError(`DeepSeek API stream from ${this.options.baseURL} failed`, 'TRANSPORT', { cause: error })
|
||||
} finally {
|
||||
consumer.abort('DeepSeek stream consumer stopped')
|
||||
if (!exhausted && iterator.return !== undefined) {
|
||||
try {
|
||||
await iterator.return()
|
||||
} catch (_abortedTransportTeardown) {
|
||||
// The consumer controller already owns termination; a return-time abort cannot add a second outcome.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async * request(options: GenerateOptions, signal: AbortSignal): AsyncIterable<StreamChunk> {
|
||||
const body = serializeRequest(options, this.options.defaults ?? {})
|
||||
// Prepared outside the try so the TRANSPORT label below covers exactly the
|
||||
// transport boundary, never a serialization failure.
|
||||
const payload = JSON.stringify(body)
|
||||
const headers = {
|
||||
'authorization': `Bearer ${this.options.apiKey}`,
|
||||
'content-type': 'application/json',
|
||||
'accept': 'text/event-stream',
|
||||
...attributionHeaders(),
|
||||
...options.sessionId !== undefined
|
||||
? { 'x-deepseek-harness-session-id': String(options.sessionId) }
|
||||
: {},
|
||||
}
|
||||
|
||||
// TODO(http): adopt the Cordis HTTP service when shared transport configuration
|
||||
// outweighs its additional runtime dependencies.
|
||||
const response = await fetch(`${this.options.baseURL}/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'authorization': `Bearer ${this.options.apiKey}`,
|
||||
'content-type': 'application/json',
|
||||
'accept': 'text/event-stream',
|
||||
...attributionHeaders(),
|
||||
...options.sessionId !== undefined
|
||||
? { 'x-deepseek-harness-session-id': String(options.sessionId) }
|
||||
: {},
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
...options.signal ? { signal: options.signal } : {},
|
||||
})
|
||||
let response: Response
|
||||
try {
|
||||
response = await fetch(`${this.options.baseURL}/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers,
|
||||
body: payload,
|
||||
signal,
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
// The outer stream distinguishes caller cancellation and watchdog expiry.
|
||||
if (signal.aborted) throw error
|
||||
// fetch wraps every transport failure (DNS, refused connection, TLS,
|
||||
// proxy) in a bare `TypeError: fetch failed` whose actionable detail
|
||||
// lives on `cause`. Wrapping with the endpoint and chaining the cause
|
||||
// lets `errorChain` render the full diagnosis at every reporting seam.
|
||||
throw new LlmError(
|
||||
`DeepSeek API request to ${this.options.baseURL} failed`,
|
||||
'TRANSPORT',
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
let message = `DeepSeek API error (HTTP ${response.status})`
|
||||
@@ -126,7 +219,13 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
// Only swallow error-body parsing: the HTTP status still identifies the
|
||||
// failure, so malformed gateway JSON must not mask it.
|
||||
}
|
||||
throw new LlmError(message, httpErrorCode(response.status, providerError))
|
||||
const delay = providerRetryAfterMs(response.headers.get('retry-after'))
|
||||
const id = requestId(response.headers)
|
||||
throw new LlmError(message, httpErrorCode(response.status, providerError), {
|
||||
status: response.status,
|
||||
...delay === undefined ? {} : { providerRetryAfterMs: delay },
|
||||
...id === undefined ? {} : { requestId: id },
|
||||
})
|
||||
}
|
||||
if (!response.body) {
|
||||
throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE')
|
||||
|
||||
@@ -8,7 +8,8 @@
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { DeepSeekAdapter } from './adapter.ts'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
|
||||
import type { DeepSeekCatalogModel } from './adapter.ts'
|
||||
|
||||
export { DeepSeekAdapter } from './adapter.ts'
|
||||
@@ -41,6 +42,8 @@ export interface Config {
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
/** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
streamIdleTimeoutMs?: number
|
||||
}
|
||||
|
||||
const catalogModel: z<DeepSeekCatalogModel> = z.object({
|
||||
@@ -56,6 +59,7 @@ export const Config: z<Config> = z.object({
|
||||
thinking: z.union(['enabled', 'disabled']),
|
||||
reasoningEffort: z.union(['high', 'max']),
|
||||
models: z.array(catalogModel).default(DEFAULT_MODELS),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
})
|
||||
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
@@ -100,5 +104,6 @@ export function apply(ctx: Context, config: Config): void {
|
||||
reasoningEffort: config.reasoningEffort,
|
||||
},
|
||||
models: resolveModels(config.models),
|
||||
streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -35,7 +35,10 @@ export function mapFinishReason(reason: string): FinishReason {
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
default:
|
||||
// content_filter, insufficient_system_resource, future additions.
|
||||
return { kind: 'error', message: `model stopped: ${reason}`, code: reason.toUpperCase() }
|
||||
return {
|
||||
kind: 'error',
|
||||
failure: { message: `model stopped: ${reason}`, code: reason.toUpperCase() },
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -2,7 +2,15 @@ import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, {
|
||||
CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
errorChain,
|
||||
LlmError,
|
||||
ProviderRequestId,
|
||||
QUOTA_EXCEEDED_CODE,
|
||||
userAgent,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { DeepSeekAdapter } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
@@ -12,7 +20,7 @@ import { assemble } from './assemble.ts'
|
||||
/** One scripted behavior for the next request the mock server receives. */
|
||||
type Behavior =
|
||||
| { kind: 'sse'; events: string[]; delayMs?: number }
|
||||
| { kind: 'http-error'; status: number; body: string; contentType?: string }
|
||||
| { kind: 'http-error'; status: number; body: string; contentType?: string; headers?: Record<string, string> }
|
||||
| { kind: 'close-early'; events: string[] }
|
||||
|
||||
interface MockServer {
|
||||
@@ -30,6 +38,7 @@ const servers: Server[] = []
|
||||
afterEach(async () => {
|
||||
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
|
||||
vi.unstubAllEnvs()
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
/** Local chat-completions stand-in: replays scripted behaviors per request. */
|
||||
@@ -48,7 +57,10 @@ async function mockServer(script: Behavior[]): Promise<MockServer> {
|
||||
return
|
||||
}
|
||||
if (behavior.kind === 'http-error') {
|
||||
response.writeHead(behavior.status, { 'content-type': behavior.contentType ?? 'application/json' })
|
||||
response.writeHead(behavior.status, {
|
||||
'content-type': behavior.contentType ?? 'application/json',
|
||||
...behavior.headers,
|
||||
})
|
||||
response.end(behavior.body)
|
||||
return
|
||||
}
|
||||
@@ -202,6 +214,84 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
expect(code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
})
|
||||
|
||||
it('retains status, Retry-After seconds, and provider request id as structured facts', async () => {
|
||||
const server = await mockServer([{
|
||||
kind: 'http-error',
|
||||
status: 429,
|
||||
body: JSON.stringify({ error: { message: 'slow down' } }),
|
||||
headers: { 'retry-after': '2', 'x-request-id': 'req-429' },
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
let thrown: unknown
|
||||
try {
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(LlmError)
|
||||
expect((thrown as LlmError).failure).toEqual({
|
||||
message: 'slow down',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 2_000,
|
||||
requestId: ProviderRequestId('req-429'),
|
||||
})
|
||||
})
|
||||
|
||||
it('parses a future Retry-After HTTP date and the DeepSeek request-id fallback', async () => {
|
||||
const now = 1_800_000_000_000
|
||||
const dateNow = vi.spyOn(Date, 'now').mockReturnValue(now)
|
||||
try {
|
||||
const server = await mockServer([{
|
||||
kind: 'http-error',
|
||||
status: 503,
|
||||
body: JSON.stringify({ error: { message: 'come back later' } }),
|
||||
headers: {
|
||||
'retry-after': new Date(now + 3_000).toUTCString(),
|
||||
'x-deepseek-request-id': 'deepseek-503',
|
||||
},
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toMatchObject({
|
||||
failure: {
|
||||
message: 'come back later',
|
||||
code: 'SERVER',
|
||||
status: 503,
|
||||
providerRetryAfterMs: 3_000,
|
||||
requestId: ProviderRequestId('deepseek-503'),
|
||||
},
|
||||
})
|
||||
} finally {
|
||||
dateNow.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('omits zero, non-finite, invalid, and past Retry-After values', async () => {
|
||||
const values = [
|
||||
'0',
|
||||
'9'.repeat(400),
|
||||
'not-a-date',
|
||||
new Date(0).toUTCString(),
|
||||
]
|
||||
for (const value of values) {
|
||||
const server = await mockServer([{
|
||||
kind: 'http-error',
|
||||
status: 429,
|
||||
body: JSON.stringify({ error: { message: 'retry later' } }),
|
||||
headers: { 'retry-after': value },
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
let thrown: LlmError | undefined
|
||||
try {
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof LlmError) thrown = error
|
||||
}
|
||||
expect(thrown?.failure).toEqual({ message: 'retry later', code: 'RATE_LIMIT', status: 429 })
|
||||
}
|
||||
})
|
||||
|
||||
it('classifies only context-capacity HTTP 400 details as context overflow', () => {
|
||||
expect(httpErrorCode(400, { message: 'request too large for model context' }))
|
||||
.toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
@@ -210,6 +300,12 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413')
|
||||
})
|
||||
|
||||
it('distinguishes terminal quota exhaustion from transient HTTP 429 throttling', () => {
|
||||
expect(httpErrorCode(429, { code: 'insufficient_quota', message: 'account credits exhausted' }))
|
||||
.toBe(QUOTA_EXCEEDED_CODE)
|
||||
expect(httpErrorCode(429, { message: 'request rate limit exceeded' })).toBe('RATE_LIMIT')
|
||||
})
|
||||
|
||||
it('keeps the status-line message for JSON error bodies without a message', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }])
|
||||
const ctx = await harness(server.url)
|
||||
@@ -228,6 +324,40 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
expect(httpErrorCode(418)).toBe('HTTP_418')
|
||||
})
|
||||
|
||||
it('wraps a transport failure in TRANSPORT with the fetch cause chain in the message', async () => {
|
||||
// Port 1 is reserved/unbound: fetch rejects with `TypeError: fetch failed`
|
||||
// whose actionable detail (ECONNREFUSED) lives on `cause`.
|
||||
const ctx = await harness('http://127.0.0.1:1')
|
||||
let caught: unknown
|
||||
try {
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
expect(caught).toBeInstanceOf(LlmError)
|
||||
const llmError = caught as LlmError
|
||||
expect(llmError.code).toBe('TRANSPORT')
|
||||
expect(llmError.message).toContain('http://127.0.0.1:1')
|
||||
expect(llmError.cause).toBeInstanceOf(TypeError)
|
||||
// The chain renderer reaches the transport diagnosis through the cause.
|
||||
expect(errorChain(llmError)).toMatch(/ECONNREFUSED|EADDRNOTAVAIL|bad port/)
|
||||
})
|
||||
|
||||
it('classifies an aborted request without losing the transport rejection', async () => {
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
const ctx = await harness('http://127.0.0.1:1')
|
||||
let caught: unknown
|
||||
try {
|
||||
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [], signal: controller.signal })
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
expect(caught).toBeInstanceOf(LlmError)
|
||||
expect(caught).toMatchObject({ code: 'ABORTED' })
|
||||
expect((caught as LlmError).cause).toMatchObject({ name: 'AbortError' })
|
||||
})
|
||||
|
||||
it('throws EMPTY_RESPONSE when the response has no body', async () => {
|
||||
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue(
|
||||
@@ -243,14 +373,20 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects with STREAM_CLOSED when the server drops mid-stream', async () => {
|
||||
it('classifies an abrupt body close as TRANSPORT and retains its cause', async () => {
|
||||
const server = await mockServer([{
|
||||
kind: 'close-early',
|
||||
events: ['{"choices":[{"delta":{"content":"par"}}]}'],
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/terminated|socket|without \[DONE\]/)
|
||||
let caught: unknown
|
||||
try {
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
expect(caught).toMatchObject({ code: 'TRANSPORT' })
|
||||
expect(errorChain(caught)).toMatch(/terminated|socket|without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('aborts mid-stream via the request signal', async () => {
|
||||
@@ -272,7 +408,76 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
})()
|
||||
|
||||
setTimeout(() => { controller.abort() }, 30)
|
||||
await expect(pending).rejects.toThrow()
|
||||
await expect(pending).rejects.toMatchObject({ code: 'ABORTED' })
|
||||
})
|
||||
|
||||
it('maps connection failures to TRANSPORT without losing the cause', async () => {
|
||||
const cause = new TypeError('connection refused')
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockRejectedValue(cause)
|
||||
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' })
|
||||
try {
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
|
||||
}
|
||||
await expect(drain()).rejects.toMatchObject({ code: 'TRANSPORT', cause })
|
||||
} finally {
|
||||
fetchSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('renders a non-Error transport rejection without losing its cause', async () => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation(() => {
|
||||
const failed = Promise.withResolvers<Response>()
|
||||
failed.reject('offline')
|
||||
return failed.promise
|
||||
})
|
||||
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' })
|
||||
try {
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
|
||||
}
|
||||
await expect(drain()).rejects.toMatchObject({
|
||||
message: 'DeepSeek API request to https://example.invalid failed',
|
||||
code: 'TRANSPORT',
|
||||
cause: 'offline',
|
||||
})
|
||||
} finally {
|
||||
fetchSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('aborts the underlying body when the stream stays idle past its watchdog', async () => {
|
||||
vi.useFakeTimers()
|
||||
let stopped = false
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation((_input, init) => {
|
||||
const signal = init?.signal
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
signal?.addEventListener('abort', () => {
|
||||
stopped = true
|
||||
controller.error(signal.reason)
|
||||
}, { once: true })
|
||||
},
|
||||
})
|
||||
return Promise.resolve(new Response(body, { status: 200 }))
|
||||
})
|
||||
const adapter = new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'https://example.invalid',
|
||||
streamIdleTimeoutMs: 100,
|
||||
})
|
||||
try {
|
||||
const drain = (async () => {
|
||||
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
|
||||
})()
|
||||
const rejected = expect(drain).rejects.toMatchObject({ code: 'TIMEOUT' })
|
||||
await vi.advanceTimersByTimeAsync(0)
|
||||
await vi.advanceTimersByTimeAsync(100)
|
||||
await rejected
|
||||
expect(stopped).toBe(true)
|
||||
} finally {
|
||||
fetchSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
@@ -445,4 +650,30 @@ describe('plugin registration and config', () => {
|
||||
expect(adapter).toBeInstanceOf(DeepSeekAdapter)
|
||||
await expect(adapter.listModels('deepseek')).resolves.toEqual([])
|
||||
})
|
||||
|
||||
it('rejects invalid idle watchdog bounds for direct and plugin composition', async () => {
|
||||
expect(() => new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
streamIdleTimeoutMs: Number.POSITIVE_INFINITY,
|
||||
})).toThrow(/streamIdleTimeoutMs.*positive finite/)
|
||||
expect(() => new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
|
||||
})).toThrow(/streamIdleTimeoutMs.*no greater/)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
streamIdleTimeoutMs: 0,
|
||||
})).rejects.toThrow(/streamIdleTimeoutMs/)
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
|
||||
})).rejects.toThrow(/streamIdleTimeoutMs/)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -232,8 +232,7 @@ describe('mapFinishReason', () => {
|
||||
(wire) => {
|
||||
expect(mapFinishReason(wire)).toEqual({
|
||||
kind: 'error',
|
||||
message: `model stopped: ${wire}`,
|
||||
code: wire.toUpperCase(),
|
||||
failure: { message: `model stopped: ${wire}`, code: wire.toUpperCase() },
|
||||
})
|
||||
},
|
||||
)
|
||||
|
||||
@@ -19,6 +19,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ Configure credentials and deployment-specific transport settings per provider. O
|
||||
reasoning: high
|
||||
- provider: anthropic
|
||||
apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
maxRetries: 2
|
||||
streamIdleTimeoutMs: 300000
|
||||
- provider: openrouter
|
||||
apiKey: !!js process.env.OPENROUTER_API_KEY
|
||||
headers:
|
||||
@@ -30,7 +30,9 @@ Each provider name must exist in pi-ai's installed catalog and may appear only o
|
||||
|
||||
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelContext(provider, model)` performs the same exact descriptor lookup and returns its context window, keeping capacity metadata on the route-owning adapter rather than a consuming plugin.
|
||||
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `maxRetries`, and `maxRetryDelayMs`. They map to pi-ai's common stream options. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
## Provider/model routing and replay
|
||||
|
||||
@@ -43,7 +45,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
|
||||
## Vocabulary differences
|
||||
|
||||
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks. Provider-specific error text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
|
||||
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
|
||||
|
||||
@@ -57,7 +59,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`.
|
||||
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -95,3 +97,4 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes.
|
||||
- **Retry policy is not an adapter option** — SDK retries are disabled so durable agent steps and `llm/retry` events own every visible attempt; direct `ctx.llm.stream()` calls remain single-attempt.
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
@@ -32,6 +33,7 @@
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm-deepseek": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,7 +16,9 @@ import type {
|
||||
} from '@earendil-works/pi-ai'
|
||||
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelContext, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { PiAiProviderProfile } from './config.ts'
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveProfiles } from './config.ts'
|
||||
import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
|
||||
import { toPiContext } from './context.ts'
|
||||
import { toStreamChunks } from './stream.ts'
|
||||
|
||||
@@ -48,8 +50,8 @@ function profileOptions(profile: PiAiProviderProfile): SimpleStreamOptions {
|
||||
...profile.transport === undefined ? {} : { transport: profile.transport },
|
||||
...profile.timeoutMs === undefined ? {} : { timeoutMs: profile.timeoutMs },
|
||||
...profile.websocketConnectTimeoutMs === undefined ? {} : { websocketConnectTimeoutMs: profile.websocketConnectTimeoutMs },
|
||||
...profile.maxRetries === undefined ? {} : { maxRetries: profile.maxRetries },
|
||||
...profile.maxRetryDelayMs === undefined ? {} : { maxRetryDelayMs: profile.maxRetryDelayMs },
|
||||
// The agent recovery layer owns visible attempts; one adapter call is one SDK attempt.
|
||||
maxRetries: 0,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -68,11 +70,11 @@ function requestHeaders(headers: Readonly<Record<string, string>> | undefined):
|
||||
* request, so models need not be registered during the Cordis lifecycle.
|
||||
*/
|
||||
export class PiAiAdapter extends LlmAdapter {
|
||||
private readonly profiles: ReadonlyMap<string, PiAiProviderProfile>
|
||||
private readonly profiles: ReadonlyMap<string, ResolvedPiAiProviderProfile>
|
||||
|
||||
constructor(options: PiAiAdapterOptions) {
|
||||
super()
|
||||
this.profiles = new Map(options.profiles.map(profile => [profile.provider, profile]))
|
||||
this.profiles = new Map(resolveProfiles(options.profiles).map(profile => [profile.provider, profile]))
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
@@ -113,12 +115,12 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
}
|
||||
const model = resolveModel(profile, options.model)
|
||||
|
||||
// Pi-ai has no iterator-return cancellation hook. Chain an internal signal
|
||||
// and abort it when this generator exits so early consumers stop the HTTP stream.
|
||||
const controller = new AbortController()
|
||||
const onCallerAbort = (): void => { controller.abort(options.signal?.reason) }
|
||||
if (options.signal?.aborted) controller.abort(options.signal.reason)
|
||||
else options.signal?.addEventListener('abort', onCallerAbort, { once: true })
|
||||
const consumer = new AbortController()
|
||||
const upstream = options.signal === undefined
|
||||
? consumer.signal
|
||||
: AbortSignal.any([options.signal, consumer.signal])
|
||||
const streamIdleTimeoutMs = profile.streamIdleTimeoutMs
|
||||
using watchdog = idleWatchdog(upstream, streamIdleTimeoutMs, 'LLM_STREAM_IDLE_TIMEOUT')
|
||||
|
||||
try {
|
||||
const events = streamSimple(model, toPiContext(options), {
|
||||
@@ -126,15 +128,44 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
...options.temperature === undefined ? {} : { temperature: options.temperature },
|
||||
...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },
|
||||
...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) },
|
||||
signal: controller.signal,
|
||||
signal: watchdog.signal,
|
||||
// Profile headers are deployment-owned; attribution names are
|
||||
// Harness-owned and therefore win collisions.
|
||||
headers: requestHeaders(profile.headers),
|
||||
})
|
||||
yield* toStreamChunks(events, model.contextWindow)
|
||||
const iterator = toStreamChunks(events, model.contextWindow)[Symbol.asyncIterator]()
|
||||
let exhausted = false
|
||||
try {
|
||||
while (true) {
|
||||
const result = await watchdog.next(iterator)
|
||||
const timeout = timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT')
|
||||
if (timeout !== undefined) throw timeout
|
||||
if (result.done) {
|
||||
exhausted = true
|
||||
return
|
||||
}
|
||||
yield result.value
|
||||
}
|
||||
} finally {
|
||||
if (!exhausted) {
|
||||
consumer.abort('pi-ai stream consumer stopped')
|
||||
try {
|
||||
await iterator.return(undefined)
|
||||
} catch (_abortedSdkTeardown) {
|
||||
// The stable signal already owns SDK termination; return-time abort cannot add an outcome.
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
if (timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT') !== undefined) {
|
||||
throw new LlmError(`pi-ai stream idle timeout after ${streamIdleTimeoutMs}ms`, 'TIMEOUT', { cause: error })
|
||||
}
|
||||
if (options.signal?.aborted) {
|
||||
throw new LlmError('pi-ai request aborted by caller', 'ABORTED', { cause: error })
|
||||
}
|
||||
throw error
|
||||
} finally {
|
||||
options.signal?.removeEventListener('abort', onCallerAbort)
|
||||
controller.abort('consumer stopped streaming')
|
||||
consumer.abort('pi-ai stream consumer stopped')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,10 @@
|
||||
import { getProviders } from '@earendil-works/pi-ai'
|
||||
import type { CacheRetention, ThinkingBudgets, ThinkingLevel, Transport } from '@earendil-works/pi-ai'
|
||||
import z from 'schemastery'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
|
||||
/** Configuration for one pi-ai provider route. */
|
||||
export interface PiAiProviderProfile {
|
||||
@@ -30,10 +34,14 @@ export interface PiAiProviderProfile {
|
||||
timeoutMs?: number
|
||||
/** WebSocket connection timeout in milliseconds. */
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Provider SDK retry count. */
|
||||
maxRetries?: number
|
||||
/** Maximum provider-requested retry delay in milliseconds. */
|
||||
maxRetryDelayMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
}
|
||||
|
||||
/** Validated profile with every adapter-owned default resolved. */
|
||||
export interface ResolvedPiAiProviderProfile extends PiAiProviderProfile {
|
||||
/** Positive finite provider-idle interval after defaulting. */
|
||||
streamIdleTimeoutMs: number
|
||||
}
|
||||
|
||||
/** Plugin configuration: the non-empty provider profiles this instance owns. */
|
||||
@@ -60,8 +68,7 @@ const profile = z.object({
|
||||
transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']),
|
||||
timeoutMs: z.natural(),
|
||||
websocketConnectTimeoutMs: z.natural(),
|
||||
maxRetries: z.natural(),
|
||||
maxRetryDelayMs: z.natural(),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
})
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
@@ -75,11 +82,18 @@ export const Config: z<Config> = z.object({
|
||||
* @param profiles - configured provider profiles.
|
||||
* @returns validated profiles in configuration order.
|
||||
*/
|
||||
export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiProviderProfile[] {
|
||||
export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): ResolvedPiAiProviderProfile[] {
|
||||
if (profiles.length === 0) throw new Error('llm-pi-ai: providers must contain at least one profile')
|
||||
const supported = new Set<string>(getProviders())
|
||||
const seen = new Set<string>()
|
||||
return profiles.map((source) => {
|
||||
const legacy = source as PiAiProviderProfile & {
|
||||
maxRetries?: unknown
|
||||
maxRetryDelayMs?: unknown
|
||||
}
|
||||
if ('maxRetries' in legacy || 'maxRetryDelayMs' in legacy) {
|
||||
throw new Error('llm-pi-ai: maxRetries and maxRetryDelayMs were removed; compose agent recovery with dsh-llm-retry')
|
||||
}
|
||||
if (source.provider.length === 0) throw new Error('llm-pi-ai: provider names must be non-empty')
|
||||
if (!supported.has(source.provider)) throw new Error(`llm-pi-ai: unknown pi-ai provider "${source.provider}"`)
|
||||
if (seen.has(source.provider)) throw new Error(`llm-pi-ai: duplicate provider profile "${source.provider}"`)
|
||||
@@ -89,9 +103,18 @@ export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiP
|
||||
if (source.baseURL !== undefined && source.baseURL.length === 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${source.provider}" has an empty baseURL`)
|
||||
}
|
||||
const streamIdleTimeoutMs = source.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
|
||||
if (!Number.isFinite(streamIdleTimeoutMs)
|
||||
|| streamIdleTimeoutMs <= 0
|
||||
|| streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(
|
||||
`llm-pi-ai: provider "${source.provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
seen.add(source.provider)
|
||||
return {
|
||||
...source,
|
||||
streamIdleTimeoutMs,
|
||||
...source.headers === undefined ? {} : { headers: { ...source.headers } },
|
||||
...source.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...source.thinkingBudgets } },
|
||||
}
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
* @module dsh-llm-pi-ai/stream
|
||||
*/
|
||||
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { isContextOverflow } from '@earendil-works/pi-ai'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
|
||||
@@ -30,9 +30,15 @@ export function mapUsage(usage: PiUsage): TokenUsage {
|
||||
|
||||
function classifyPiAiError(message: string): string {
|
||||
if (/\b(?:401|403)\b/.test(message)) return 'AUTH'
|
||||
if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE
|
||||
if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT'
|
||||
if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST'
|
||||
if (/\b5\d\d\b/.test(message)) return 'SERVER'
|
||||
if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT'
|
||||
if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)
|
||||
|| /\b(?:other side closed|HTTP2 request did not get a response|WebSocket closed unexpectedly)\b/i.test(message)) {
|
||||
return 'TRANSPORT'
|
||||
}
|
||||
return 'PI_AI_ERROR'
|
||||
}
|
||||
|
||||
@@ -52,8 +58,10 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
|
||||
if (piAiOverflow || harnessOverflow) {
|
||||
return {
|
||||
kind: 'error',
|
||||
message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
failure: {
|
||||
message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,10 +69,13 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
|
||||
case 'stop': return { kind: 'stop' }
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
case 'toolUse': return { kind: 'tool-calls' }
|
||||
case 'aborted': return { kind: 'aborted' }
|
||||
case 'aborted': return {
|
||||
kind: 'aborted',
|
||||
failure: { message: message.errorMessage ?? 'pi-ai stream aborted', code: 'ABORTED' },
|
||||
}
|
||||
case 'error': {
|
||||
const text = message.errorMessage ?? 'pi-ai stream error'
|
||||
return { kind: 'error', message: text, code: classifyPiAiError(text) }
|
||||
return { kind: 'error', failure: { message: text, code: classifyPiAiError(text) } }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { getModels } from '@earendil-works/pi-ai'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
|
||||
@@ -14,6 +15,8 @@ interface MockServer {
|
||||
paths: string[]
|
||||
requests: unknown[]
|
||||
headers: IncomingMessage['headers'][]
|
||||
readonly closedResponses: number
|
||||
responseClosed: Promise<void>
|
||||
}
|
||||
|
||||
const servers: Server[] = []
|
||||
@@ -23,11 +26,23 @@ afterEach(async () => {
|
||||
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
|
||||
})
|
||||
|
||||
async function mockServer(script: { status?: number; events?: string[]; body?: string; delayMs?: number }[]): Promise<MockServer> {
|
||||
async function mockServer(script: {
|
||||
status?: number
|
||||
events?: string[]
|
||||
body?: string
|
||||
delayMs?: number
|
||||
headers?: Record<string, string>
|
||||
}[]): Promise<MockServer> {
|
||||
const paths: string[] = []
|
||||
const requests: unknown[] = []
|
||||
const headers: IncomingMessage['headers'][] = []
|
||||
let closedResponses = 0
|
||||
const responseClosed = Promise.withResolvers<undefined>()
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
response.on('close', () => {
|
||||
closedResponses += 1
|
||||
responseClosed.resolve(undefined)
|
||||
})
|
||||
let body = ''
|
||||
request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') })
|
||||
request.on('end', () => {
|
||||
@@ -36,7 +51,7 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s
|
||||
headers.push(request.headers)
|
||||
const behavior = script.shift() ?? { status: 500, body: 'script exhausted' }
|
||||
if (behavior.status !== undefined && behavior.status !== 200) {
|
||||
response.writeHead(behavior.status, { 'content-type': 'application/json' })
|
||||
response.writeHead(behavior.status, { 'content-type': 'application/json', ...behavior.headers })
|
||||
response.end(behavior.body ?? '{}')
|
||||
return
|
||||
}
|
||||
@@ -56,7 +71,14 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('no port')
|
||||
return { url: `http://127.0.0.1:${address.port}`, paths, requests, headers }
|
||||
return {
|
||||
url: `http://127.0.0.1:${address.port}`,
|
||||
paths,
|
||||
requests,
|
||||
headers,
|
||||
responseClosed: responseClosed.promise,
|
||||
get closedResponses() { return closedResponses },
|
||||
}
|
||||
}
|
||||
|
||||
const textEvents = [
|
||||
@@ -107,8 +129,7 @@ describe('PiAiAdapter provider routing', () => {
|
||||
transport: 'sse',
|
||||
timeoutMs: 5000,
|
||||
websocketConnectTimeoutMs: 3000,
|
||||
maxRetries: 0,
|
||||
maxRetryDelayMs: 10,
|
||||
streamIdleTimeoutMs: 10_000,
|
||||
thinkingBudgets: { high: 2048 },
|
||||
})
|
||||
await assemble(ctx, {
|
||||
@@ -161,13 +182,35 @@ describe('PiAiAdapter provider routing', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1`, maxRetries: 0 }],
|
||||
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }],
|
||||
})
|
||||
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
|
||||
expect(result.finish.kind).toBe('error')
|
||||
expect(server.paths).toEqual(['/v1/responses'])
|
||||
})
|
||||
|
||||
it('forces one wire request for an SDK-retryable provider failure', async () => {
|
||||
const server = await mockServer([
|
||||
{
|
||||
status: 429,
|
||||
headers: { 'retry-after-ms': '1' },
|
||||
body: JSON.stringify({ error: { message: 'retryable provider failure' } }),
|
||||
},
|
||||
{ status: 500, body: JSON.stringify({ error: { message: 'hidden SDK retry' } }) },
|
||||
{ status: 500, body: JSON.stringify({ error: { message: 'second hidden SDK retry' } }) },
|
||||
])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }],
|
||||
})
|
||||
|
||||
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
|
||||
|
||||
expect(result.finish).toMatchObject({ kind: 'error' })
|
||||
expect(server.paths).toEqual(['/v1/responses'])
|
||||
})
|
||||
|
||||
it('uses OpenAI Responses against an Azure project v1 path with its API key header', async () => {
|
||||
const server = await mockServer([{ status: 401, body: JSON.stringify({ error: { message: 'expected mock failure' } }) }])
|
||||
const ctx = new Context()
|
||||
@@ -178,7 +221,6 @@ describe('PiAiAdapter provider routing', () => {
|
||||
apiKey: 'test-key',
|
||||
baseURL: `${server.url}/api/projects/openai/openai/v1`,
|
||||
headers: { 'api-key': 'test-key', Authorization: '' },
|
||||
maxRetries: 0,
|
||||
}],
|
||||
})
|
||||
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-5.5', messages: [] })
|
||||
@@ -195,9 +237,10 @@ describe('PiAiAdapter provider routing', () => {
|
||||
[500, 'SERVER'],
|
||||
] as const)('maps HTTP %s failures to %s', async (status, code) => {
|
||||
const server = await mockServer([{ status, body: JSON.stringify({ error: { message: `provider ${status}` } }) }])
|
||||
const ctx = await harness(server.url, { maxRetries: 0 })
|
||||
const ctx = await harness(server.url)
|
||||
const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.finish).toMatchObject({ kind: 'error', code })
|
||||
expect(result.finish).toMatchObject({ kind: 'error', failure: { code } })
|
||||
expect(server.paths).toEqual(['/chat/completions'])
|
||||
})
|
||||
|
||||
it('uses the resolved catalog context window for usage-based overflow detection', async () => {
|
||||
@@ -218,10 +261,29 @@ describe('PiAiAdapter provider routing', () => {
|
||||
|
||||
expect(result.finish).toEqual({
|
||||
kind: 'error',
|
||||
message: `pi-ai detected context overflow for model "${model.id}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
failure: {
|
||||
message: `pi-ai detected context overflow for model "${model.id}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('stops the SDK request when the adapter idle watchdog expires', async () => {
|
||||
const server = await mockServer([{ events: textEvents, delayMs: 200 }])
|
||||
const ctx = await harness(server.url, { streamIdleTimeoutMs: 20 })
|
||||
|
||||
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toMatchObject({ code: 'TIMEOUT' })
|
||||
await Promise.race([
|
||||
server.responseClosed,
|
||||
new Promise<never>((_resolve, reject) => {
|
||||
setTimeout(() => { reject(new Error('SDK request did not close after idle timeout')) }, 100)
|
||||
}),
|
||||
])
|
||||
|
||||
expect(server.paths).toEqual(['/chat/completions'])
|
||||
expect(server.closedResponses).toBe(1)
|
||||
})
|
||||
})
|
||||
|
||||
describe('provider profile lifecycle', () => {
|
||||
@@ -283,16 +345,31 @@ describe('provider profile lifecycle', () => {
|
||||
expect(() => resolveProfiles([{ provider: 'openai', baseURL: '' }])).toThrow(/empty baseURL/)
|
||||
})
|
||||
|
||||
it('rejects negative or fractional stream tunables at schema validation', () => {
|
||||
it.each(['maxRetries', 'maxRetryDelayMs'] as const)(
|
||||
'rejects removed profile field %s instead of silently restoring hidden SDK retries',
|
||||
async (field) => {
|
||||
const legacy = { provider: 'openai', [field]: 2 }
|
||||
expect(() => resolveProfiles([legacy as never])).toThrow(/removed.*agent recovery/i)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmPiAi, { providers: [legacy as never] }))
|
||||
.rejects.toThrow(/removed.*agent recovery/i)
|
||||
},
|
||||
)
|
||||
|
||||
it('rejects invalid stream tunables at plugin load', async () => {
|
||||
const invalid = [
|
||||
{ timeoutMs: -1 },
|
||||
{ websocketConnectTimeoutMs: -1 },
|
||||
{ maxRetries: -1 },
|
||||
{ maxRetries: 0.5 },
|
||||
{ maxRetryDelayMs: -1 },
|
||||
{ streamIdleTimeoutMs: 0 },
|
||||
{ streamIdleTimeoutMs: Number.NaN },
|
||||
{ streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 },
|
||||
]
|
||||
for (const entry of invalid) {
|
||||
expect(() => new LlmPiAi.Config({ providers: [{ provider: 'openai', ...entry }] })).toThrow()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmPiAi, { providers: [{ provider: 'openai', ...entry }] }))
|
||||
.rejects.toThrow()
|
||||
}
|
||||
})
|
||||
|
||||
@@ -308,11 +385,59 @@ describe('provider profile lifecycle', () => {
|
||||
})()).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
expect(new LlmError('x', 'X')).toBeInstanceOf(Error)
|
||||
})
|
||||
|
||||
it('validates direct-constructor profiles at the embedding boundary', () => {
|
||||
expect(() => new PiAiAdapter({
|
||||
profiles: [{ provider: 'openai', streamIdleTimeoutMs: 0 }],
|
||||
})).toThrow(/streamIdleTimeoutMs.*positive finite/)
|
||||
expect(() => new PiAiAdapter({
|
||||
profiles: [{ provider: 'openai', streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 }],
|
||||
})).toThrow(/streamIdleTimeoutMs.*no greater/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('abort wiring', () => {
|
||||
it('preserves an unknown pre-dispatch adapter Error exactly', async () => {
|
||||
const original = new Error('SDK context conversion exploded')
|
||||
const message = Object.defineProperty({}, 'role', {
|
||||
get() { throw original },
|
||||
})
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({
|
||||
provider: 'deepseek',
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [message as never],
|
||||
})) { /* drain */ }
|
||||
}
|
||||
|
||||
await expect(drain()).rejects.toBe(original)
|
||||
})
|
||||
|
||||
it('lets a concurrent caller abort classify a pre-dispatch adapter failure', async () => {
|
||||
const controller = new AbortController()
|
||||
const original = new Error('conversion lost its caller')
|
||||
const message = Object.defineProperty({}, 'role', {
|
||||
get() {
|
||||
controller.abort('caller cancelled during conversion')
|
||||
throw original
|
||||
},
|
||||
})
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({
|
||||
provider: 'deepseek',
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [message as never],
|
||||
signal: controller.signal,
|
||||
})) { /* drain */ }
|
||||
}
|
||||
|
||||
await expect(drain()).rejects.toMatchObject({ code: 'ABORTED', cause: original })
|
||||
})
|
||||
|
||||
it('resolves catalog endpoints without an override before honoring pre-abort', async () => {
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key', maxRetries: 0 }] })
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
|
||||
const controller = new AbortController()
|
||||
controller.abort('already stopped')
|
||||
const chunks = []
|
||||
|
||||
@@ -485,20 +485,32 @@ describe('toStreamChunks', () => {
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 0 } },
|
||||
{ type: 'finish', reason: { kind: 'error', message: 'boom', code: 'PI_AI_ERROR' } },
|
||||
{ type: 'finish', reason: { kind: 'error', failure: { message: 'boom', code: 'PI_AI_ERROR' } } },
|
||||
])
|
||||
})
|
||||
|
||||
it('maps aborted error events to aborted finish', async () => {
|
||||
const error = assistant({ stopReason: 'aborted' })
|
||||
const chunks = await collect(toStreamChunks(feed({ type: 'error', reason: 'aborted', error })))
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'aborted' } })
|
||||
expect(chunks.at(-1)).toEqual({
|
||||
type: 'finish',
|
||||
reason: { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects a stream that ends without done or error', async () => {
|
||||
await expect(collect(toStreamChunks(feed({ type: 'start', partial: assistant() }))))
|
||||
.rejects.toThrow(/without done\/error/)
|
||||
})
|
||||
|
||||
it('preserves an unknown SDK iterator Error exactly', async () => {
|
||||
const original = Object.assign(new Error('SDK transport exploded'), { code: 'ECONNRESET' })
|
||||
async function* failedSdkStream(): AsyncGenerator<AssistantMessageEvent> {
|
||||
throw original
|
||||
}
|
||||
|
||||
await expect(collect(toStreamChunks(failedSdkStream()))).rejects.toBe(original)
|
||||
})
|
||||
})
|
||||
|
||||
describe('mapStopReason / mapUsage', () => {
|
||||
@@ -506,46 +518,65 @@ describe('mapStopReason / mapUsage', () => {
|
||||
['stop', { kind: 'stop' }],
|
||||
['length', { kind: 'max-tokens' }],
|
||||
['toolUse', { kind: 'tool-calls' }],
|
||||
['aborted', { kind: 'aborted' }],
|
||||
['aborted', { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } }],
|
||||
] as const)('maps %s', (stopReason, expected) => {
|
||||
expect(mapStopReason(assistant({ stopReason }))).toEqual(expected)
|
||||
})
|
||||
|
||||
it('defaults the error message when pi-ai omits it', () => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'error' })))
|
||||
.toEqual({ kind: 'error', message: 'pi-ai stream error', code: 'PI_AI_ERROR' })
|
||||
.toEqual({ kind: 'error', failure: { message: 'pi-ai stream error', code: 'PI_AI_ERROR' } })
|
||||
})
|
||||
|
||||
it('maps routable HTTP-ish error messages to stable codes', () => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 401: bad key' })))
|
||||
.toMatchObject({ kind: 'error', code: 'AUTH' })
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'AUTH' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: rate limit' })))
|
||||
.toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: insufficient_quota' })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'OpenAI API error (429): You exceeded your current quota, please check your plan and billing details.',
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' })))
|
||||
.toMatchObject({ kind: 'error', code: 'SERVER' })
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'SERVER' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'provider timed out' })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'TIMEOUT' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'ECONNRESET socket closed' })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: input exceeds the model context window limit',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: request too large for model context',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value',
|
||||
}))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' })
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } })
|
||||
})
|
||||
|
||||
it.each([
|
||||
'other side closed',
|
||||
'HTTP2 request did not get a response',
|
||||
'WebSocket closed unexpectedly',
|
||||
])('maps pi-ai transport wording %j', (errorMessage) => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } })
|
||||
})
|
||||
|
||||
it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => {
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'prompt is too long: 213462 tokens > 200000 maximum',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'ThrottlingException: Too many tokens, rate limit reached',
|
||||
}))).toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } })
|
||||
})
|
||||
|
||||
it('uses the resolved context window for silent and length-stop overflows', () => {
|
||||
@@ -553,15 +584,17 @@ describe('mapStopReason / mapUsage', () => {
|
||||
expect(mapStopReason(silent)).toEqual({ kind: 'stop' })
|
||||
expect(mapStopReason(silent, 100)).toEqual({
|
||||
kind: 'error',
|
||||
message: 'pi-ai detected context overflow for model "deepseek-v4-flash"',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
failure: {
|
||||
message: 'pi-ai detected context overflow for model "deepseek-v4-flash"',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
},
|
||||
})
|
||||
|
||||
const truncated = assistant({ stopReason: 'length', usage: usage(80, 0, 19) })
|
||||
expect(mapStopReason(truncated)).toEqual({ kind: 'max-tokens' })
|
||||
expect(mapStopReason(truncated, 100)).toMatchObject({
|
||||
kind: 'error',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE },
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ function textOf(result: AssembledResult): string {
|
||||
|
||||
function expectFinish(result: AssembledResult, expected: 'stop' | 'tool-calls'): void {
|
||||
if (result.finish.kind === 'error') {
|
||||
throw new Error(`provider request failed (${result.finish.code ?? 'unknown'}): ${result.finish.message}`)
|
||||
throw new Error(`provider request failed (${result.finish.failure.code}): ${result.finish.failure.message}`)
|
||||
}
|
||||
expect(result.finish.kind).toBe(expected)
|
||||
}
|
||||
|
||||
35
packages/llm/llm-pi-ai/tests/sdk-options.spec.ts
Normal file
35
packages/llm/llm-pi-ai/tests/sdk-options.spec.ts
Normal file
@@ -0,0 +1,35 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const streamSimple = vi.hoisted(() => vi.fn())
|
||||
|
||||
vi.mock('@earendil-works/pi-ai', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@earendil-works/pi-ai')>()
|
||||
return { ...actual, streamSimple }
|
||||
})
|
||||
|
||||
import { PiAiAdapter } from '../src/adapter.ts'
|
||||
|
||||
afterEach(() => { streamSimple.mockReset() })
|
||||
|
||||
describe('pi-ai SDK retry boundary', () => {
|
||||
it('pins one SDK attempt even when the installed provider currently defaults to zero retries', async () => {
|
||||
const failure = new Error('mock SDK boundary')
|
||||
streamSimple.mockReturnValue({
|
||||
async * [Symbol.asyncIterator](): AsyncGenerator<never> {
|
||||
throw failure
|
||||
},
|
||||
})
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai', apiKey: 'test-key' }] })
|
||||
const drain = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({
|
||||
provider: 'openai',
|
||||
model: 'gpt-4.1',
|
||||
messages: [],
|
||||
})) { /* drain */ }
|
||||
}
|
||||
|
||||
await expect(drain()).rejects.toBe(failure)
|
||||
expect(streamSimple).toHaveBeenCalledOnce()
|
||||
expect(streamSimple.mock.calls[0]?.[2]).toMatchObject({ maxRetries: 0 })
|
||||
})
|
||||
})
|
||||
@@ -19,6 +19,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
39
packages/llm/llm-retry/README.md
Normal file
39
packages/llm/llm-retry/README.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# `@deepseek-ai/dsh-llm-retry`
|
||||
|
||||
Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step.
|
||||
|
||||
The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
|
||||
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward.
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
config:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
```
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Transient request recovery
|
||||
|
||||
#### What the model sees
|
||||
|
||||
No retry event, delay, or failure prose is model-visible. After a retry, the next numbered step reconstructs the same explicit provider/model request from durable session history; failed chunks never enter derived messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Each retry is a new provider request and may repeat input-token billing. The finite budget caps attempts; `llm/retry` itself contributes no tokens.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface status event does not change cache identity.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Agent steps are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
|
||||
- **Finite plugin budgets add** — this policy counts only configured transient codes; context-overflow compaction counts only its own code. A future policy with overlapping codes must document and test registration-order behavior.
|
||||
- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation.
|
||||
47
packages/llm/llm-retry/package.json
Normal file
47
packages/llm/llm-retry/package.json
Normal file
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-retry",
|
||||
"description": "Bounded transient LLM request retry policy for the DeepSeek Harness",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cordisjs/plugin-include": "workspace:^",
|
||||
"@cordisjs/plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-sqlite": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
213
packages/llm/llm-retry/src/index.ts
Normal file
213
packages/llm/llm-retry/src/index.ts
Normal file
@@ -0,0 +1,213 @@
|
||||
/**
|
||||
* Bounded transient model-request retry policy on the agent loop's closed-step
|
||||
* recovery seam. Each scheduled retry is durable before its cancellable wait.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-retry
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { Agent, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import type {} from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
|
||||
declare module '@deepseek-ai/dsh-session' {
|
||||
interface SessionEventMap {
|
||||
/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
|
||||
'llm/retry': {
|
||||
turn: number
|
||||
step: number
|
||||
retry: number
|
||||
maxRetries: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const name = 'llm-retry'
|
||||
export const inject = ['agents']
|
||||
|
||||
const DEFAULT_MAX_TRANSIENT_RETRIES = 2
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
const DEFAULT_MAX_DELAY_MS = 10_000
|
||||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze(['RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
|
||||
|
||||
/** Deployment-owned limits and classification for transient request recovery. */
|
||||
export interface Config {
|
||||
/** Maximum transient retries after the first request (default 2). */
|
||||
maxTransientRetries?: number
|
||||
/** Initial local exponential-backoff delay in milliseconds (default 500). */
|
||||
initialDelayMs?: number
|
||||
/** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
|
||||
maxDelayMs?: number
|
||||
/** Symmetric random multiplier range around one (default 0.1). */
|
||||
jitterRatio?: number
|
||||
/** Stable failure codes eligible for this policy. */
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
export const Config: z<Config> = z.object({
|
||||
maxTransientRetries: z.number().step(1).min(0).default(DEFAULT_MAX_TRANSIENT_RETRIES),
|
||||
initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS),
|
||||
maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS),
|
||||
jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO),
|
||||
retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]),
|
||||
})
|
||||
|
||||
interface ResolvedConfig {
|
||||
readonly maxTransientRetries: number
|
||||
readonly initialDelayMs: number
|
||||
readonly maxDelayMs: number
|
||||
readonly jitterRatio: number
|
||||
readonly retryableCodes: ReadonlySet<string>
|
||||
}
|
||||
|
||||
function resolveConfig(config: Config): ResolvedConfig {
|
||||
const maxTransientRetries = config.maxTransientRetries ?? DEFAULT_MAX_TRANSIENT_RETRIES
|
||||
const initialDelayMs = config.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS
|
||||
const maxDelayMs = config.maxDelayMs ?? DEFAULT_MAX_DELAY_MS
|
||||
const jitterRatio = config.jitterRatio ?? DEFAULT_JITTER_RATIO
|
||||
const codes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES]
|
||||
|
||||
if (!Number.isInteger(maxTransientRetries) || maxTransientRetries < 0) {
|
||||
throw new Error('llm-retry: maxTransientRetries must be a non-negative integer')
|
||||
}
|
||||
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (initialDelayMs > maxDelayMs) {
|
||||
throw new Error('llm-retry: initialDelayMs must be less than or equal to maxDelayMs')
|
||||
}
|
||||
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) {
|
||||
throw new Error('llm-retry: jitterRatio must be between 0 and 1')
|
||||
}
|
||||
if (codes.length === 0) {
|
||||
throw new Error('llm-retry: retryableCodes must not be empty')
|
||||
}
|
||||
if (codes.some(code => code.length === 0)) {
|
||||
throw new Error('llm-retry: retryableCodes must contain only non-empty strings')
|
||||
}
|
||||
if (new Set(codes).size !== codes.length) {
|
||||
throw new Error('llm-retry: retryableCodes must not contain duplicates')
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
maxTransientRetries,
|
||||
initialDelayMs,
|
||||
maxDelayMs,
|
||||
jitterRatio,
|
||||
retryableCodes: new Set(codes),
|
||||
})
|
||||
}
|
||||
|
||||
/** Non-serializable seams used to make timing policy deterministic in tests. */
|
||||
export interface RetryInternals {
|
||||
/** Random sample in the inclusive zero-to-one range used for jitter. */
|
||||
random?: () => number
|
||||
}
|
||||
|
||||
function localDelay(config: ResolvedConfig, retry: number, random: () => number): number {
|
||||
const exponent = Math.min(retry - 1, 1024)
|
||||
const exponential = Math.min(config.initialDelayMs * 2 ** exponent, config.maxDelayMs)
|
||||
const jitter = 1 - config.jitterRatio + 2 * config.jitterRatio * random()
|
||||
return Math.min(exponential * jitter, config.maxDelayMs)
|
||||
}
|
||||
|
||||
function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean> {
|
||||
if (signal.aborted) return Promise.resolve(false)
|
||||
return new Promise((resolve) => {
|
||||
const timer = setTimeout(() => {
|
||||
signal.removeEventListener('abort', onAbort)
|
||||
resolve(true)
|
||||
}, delayMs)
|
||||
function onAbort(): void {
|
||||
clearTimeout(timer)
|
||||
resolve(false)
|
||||
}
|
||||
signal.addEventListener('abort', onAbort, { once: true })
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Install bounded transient request recovery.
|
||||
* @param ctx - plugin context that owns the listener and active waits.
|
||||
* @param config - retry budget, delay bounds, jitter, and eligible codes.
|
||||
* @param internals - non-serializable deterministic seams for tests.
|
||||
*/
|
||||
export function apply(ctx: Context, config: Config = {}, internals: RetryInternals = {}): void {
|
||||
const resolved = resolveConfig(config)
|
||||
const random = internals.random ?? Math.random
|
||||
const lifetime = new AbortController()
|
||||
const active = new Set<Promise<RequestErrorDecision>>()
|
||||
|
||||
async function backoff(
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
failure: LlmFailure,
|
||||
retry: number,
|
||||
delayMs: number,
|
||||
signal: AbortSignal,
|
||||
): Promise<RequestErrorDecision> {
|
||||
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
|
||||
if (fusedSignal.aborted) return { action: 'fail' }
|
||||
agent.session.append('llm/retry', {
|
||||
turn,
|
||||
step,
|
||||
retry,
|
||||
maxRetries: resolved.maxTransientRetries,
|
||||
delayMs,
|
||||
failure,
|
||||
})
|
||||
if (!await cancellableDelay(delayMs, fusedSignal)) return { action: 'fail' }
|
||||
return { action: 'retry' }
|
||||
}
|
||||
|
||||
const disposeListener = ctx.on('agent/request-error', (
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
_error: RequestError,
|
||||
failure: LlmFailure,
|
||||
priorFailures: readonly LlmFailure[],
|
||||
signal: AbortSignal,
|
||||
next: () => Promise<RequestErrorDecision>,
|
||||
) => {
|
||||
// A waterfall may have captured this callback before its registration was
|
||||
// removed. Lifetime cancellation must prevent that stale callback from
|
||||
// entering a downstream policy after disposal.
|
||||
if (lifetime.signal.aborted) return Promise.resolve<RequestErrorDecision>({ action: 'fail' })
|
||||
if (!resolved.retryableCodes.has(failure.code)) return next()
|
||||
const priorTransientFailures = priorFailures.filter(item => resolved.retryableCodes.has(item.code)).length
|
||||
if (priorTransientFailures >= resolved.maxTransientRetries) return next()
|
||||
|
||||
const retry = priorTransientFailures + 1
|
||||
let delayMs: number
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& Number.isFinite(failure.providerRetryAfterMs)
|
||||
&& failure.providerRetryAfterMs > 0) {
|
||||
if (failure.providerRetryAfterMs > resolved.maxDelayMs) return next()
|
||||
delayMs = failure.providerRetryAfterMs
|
||||
} else {
|
||||
delayMs = localDelay(resolved, retry, random)
|
||||
}
|
||||
|
||||
const tracked = backoff(agent, turn, step, failure, retry, delayMs, signal)
|
||||
.finally(() => active.delete(tracked))
|
||||
active.add(tracked)
|
||||
return tracked
|
||||
})
|
||||
|
||||
ctx.effect(() => async () => {
|
||||
disposeListener()
|
||||
lifetime.abort(new Error('llm-retry plugin disposed'))
|
||||
await Promise.allSettled([...active])
|
||||
}, 'llm-retry: abort and drain backoffs')
|
||||
}
|
||||
124
packages/llm/llm-retry/tests/loader-composition.spec.ts
Normal file
124
packages/llm/llm-retry/tests/loader-composition.spec.ts
Normal file
@@ -0,0 +1,124 @@
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import Include from '@cordisjs/plugin-include'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import LlmService, { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import * as retry from '../src/index.ts'
|
||||
|
||||
let root: string | undefined
|
||||
let context: Context | undefined
|
||||
|
||||
class TransientOnceAdapter extends LlmAdapter {
|
||||
requests = 0
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests += 1
|
||||
if (this.requests === 1) throw new LlmError('temporary outage', 'SERVER')
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'text-delta', index: 0, text: 'recovered' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
}
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await context?.fiber.dispose()
|
||||
context = undefined
|
||||
if (root !== undefined) await rm(root, { recursive: true, force: true })
|
||||
root = undefined
|
||||
})
|
||||
|
||||
async function loadYaml(lines: readonly string[]): Promise<Context> {
|
||||
root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-loader-'))
|
||||
const configPath = join(root, 'cordis.yml')
|
||||
await writeFile(configPath, [...lines, ''].join('\n'))
|
||||
|
||||
context = new Context()
|
||||
context.baseUrl = pathToFileURL(root).href + '/'
|
||||
await context.plugin(Loader)
|
||||
context.loader.builtins.include = Include
|
||||
const modules = new Map<string, unknown>([
|
||||
['@deepseek-ai/dsh-llm', LlmService],
|
||||
['@deepseek-ai/dsh-session', SessionStore],
|
||||
['@deepseek-ai/dsh-system-prompt', SystemPrompt],
|
||||
['@deepseek-ai/dsh-tools', ToolRegistry],
|
||||
['@deepseek-ai/dsh-agent', AgentRegistry],
|
||||
['@deepseek-ai/dsh-llm-retry', retry],
|
||||
['@deepseek-ai/dsh-agent-loop', AgentLoop],
|
||||
])
|
||||
context.loader.internal = {
|
||||
version: 'v2',
|
||||
async import(specifier: string) {
|
||||
if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`)
|
||||
return modules.get(specifier)
|
||||
},
|
||||
} as unknown as NonNullable<typeof context.loader.internal>
|
||||
await context.loader.create({
|
||||
name: 'cordis:include',
|
||||
config: { path: pathToFileURL(configPath).href },
|
||||
})
|
||||
await context.loader.await()
|
||||
return context
|
||||
}
|
||||
|
||||
describe('real Loader composition', () => {
|
||||
it('loads the flat policy and records recovery through the shipping loop', async () => {
|
||||
const loaded = await loadYaml([
|
||||
"- name: '@deepseek-ai/dsh-llm'",
|
||||
"- name: '@deepseek-ai/dsh-session'",
|
||||
"- name: '@deepseek-ai/dsh-system-prompt'",
|
||||
"- name: '@deepseek-ai/dsh-tools'",
|
||||
"- name: '@deepseek-ai/dsh-agent'",
|
||||
"- name: '@deepseek-ai/dsh-llm-retry'",
|
||||
' config:',
|
||||
' maxTransientRetries: 1',
|
||||
' initialDelayMs: 1',
|
||||
' maxDelayMs: 1',
|
||||
' jitterRatio: 0',
|
||||
' retryableCodes: [RATE_LIMIT, SERVER]',
|
||||
"- name: '@deepseek-ai/dsh-agent-loop'",
|
||||
])
|
||||
|
||||
const unloaded = [...loaded.loader.entries()]
|
||||
.filter(entry => entry.fiber === undefined && !entry.disabled)
|
||||
.map(entry => entry.options.name)
|
||||
expect(unloaded).toEqual([])
|
||||
expect(loaded.agents).toBeInstanceOf(AgentRegistry)
|
||||
|
||||
const adapter = new TransientOnceAdapter()
|
||||
loaded.llm.registerAdapter(['mock'], adapter)
|
||||
const agent = loaded.agentLoop.create(SessionId('loader-retry'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(loaded, agent)
|
||||
agent.send([{ type: 'text', text: 'recover' }])
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toBe(2)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1)
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'recovered' }],
|
||||
})
|
||||
})
|
||||
})
|
||||
57
packages/llm/llm-retry/tests/persistence.spec.ts
Normal file
57
packages/llm/llm-retry/tests/persistence.spec.ts
Normal file
@@ -0,0 +1,57 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import SessionPersistenceSqlite from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import type {} from '../src/index.ts'
|
||||
|
||||
const dirs: string[] = []
|
||||
|
||||
afterEach(async () => {
|
||||
for (const dir of dirs.splice(0)) await rm(dir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
async function backend(kind: 'jsonl' | 'sqlite'): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
if (kind === 'jsonl') {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-jsonl-'))
|
||||
dirs.push(root)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
} else {
|
||||
await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
}
|
||||
return ctx
|
||||
}
|
||||
|
||||
describe.each(['jsonl', 'sqlite'] as const)('%s retry-event persistence', (kind) => {
|
||||
it('round-trips the event losslessly without adding a model message', async () => {
|
||||
const ctx = await backend(kind)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId(`retry-${kind}`))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const event = session.append('llm/retry', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 750,
|
||||
failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 },
|
||||
})
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'aborted', reason: 'cancelled in backoff' } })
|
||||
|
||||
expect(session.deriveMessages()).toEqual([])
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(session.id)
|
||||
|
||||
expect(loaded.events.find(item => item.type === 'llm/retry')).toEqual(event)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
453
packages/llm/llm-retry/tests/retry.spec.ts
Normal file
453
packages/llm/llm-retry/tests/retry.spec.ts
Normal file
@@ -0,0 +1,453 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Fiber } from 'cordis'
|
||||
import LlmService, { CallId, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import * as retry from '../src/index.ts'
|
||||
|
||||
type ScriptEntry = Error | Iterable<StreamChunk> | AsyncIterable<StreamChunk>
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
readonly requests: GenerateOptions[] = []
|
||||
|
||||
constructor(private readonly entries: ScriptEntry[]) {
|
||||
super()
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests.push(options)
|
||||
const entry = this.entries.shift()
|
||||
if (entry === undefined) throw new Error('retry test script exhausted')
|
||||
if (entry instanceof Error) throw entry
|
||||
yield* entry
|
||||
}
|
||||
}
|
||||
|
||||
async function* partialToolFailure(error: Error): AsyncGenerator<StreamChunk> {
|
||||
const id = CallId('discarded-call')
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'text-delta', index: 0, text: 'discarded partial output' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'discarded partial output' } }
|
||||
yield { type: 'block-start', index: 1, blockType: 'tool-call' }
|
||||
yield { type: 'tool-call-delta', index: 1, id, name: 'danger', argumentsDelta: '{}' }
|
||||
yield { type: 'block-end', index: 1, block: { type: 'tool-call', id, name: 'danger', arguments: '{}' } }
|
||||
throw error
|
||||
}
|
||||
|
||||
function textResponse(text: string): StreamChunk[] {
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
}
|
||||
|
||||
async function harness(
|
||||
adapter: LlmAdapter,
|
||||
config: retry.Config = {},
|
||||
beforeRetry?: (ctx: Context) => void,
|
||||
internals: retry.RetryInternals = {},
|
||||
): Promise<{ ctx: Context; retryFiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
beforeRetry?.(ctx)
|
||||
const resolvedConfig = Object.assign({
|
||||
maxTransientRetries: 2,
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0,
|
||||
}, config)
|
||||
const retryFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
retry.apply(inner, resolvedConfig, internals)
|
||||
}, { inject: retry.inject }))
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
return { ctx, retryFiber }
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
function waitForRetry(ctx: Context, agent: Agent, retryNumber: number): Promise<Extract<SessionEvent, { type: 'llm/retry' }>> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('session/event', (session, event) => {
|
||||
if (session === agent.session && event.type === 'llm/retry' && event.data.retry === retryNumber) {
|
||||
dispose()
|
||||
resolve(event)
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
let context: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers()
|
||||
await context?.fiber.dispose()
|
||||
context = undefined
|
||||
})
|
||||
|
||||
describe('bounded transient retry policy', () => {
|
||||
it('records the scheduled delay before opening a fresh request attempt', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('busy', 'RATE_LIMIT', { status: 429 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-success'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const scheduled = new Promise<Extract<(typeof agent.session.events)[number], { type: 'llm/retry' }>>((resolve) => {
|
||||
const dispose = context?.on('session/event', (session, event) => {
|
||||
if (session === agent.session && event.type === 'llm/retry') {
|
||||
dispose?.()
|
||||
resolve(event)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
const event = await scheduled
|
||||
|
||||
expect(event.data).toEqual({
|
||||
turn: 1,
|
||||
step: 1,
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 500,
|
||||
failure: { message: 'busy', code: 'RATE_LIMIT', status: 429 },
|
||||
})
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
await vi.advanceTimersByTimeAsync(499)
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(agent.session.events.filter(item => item.type === 'step/start').map(item => item.data.step))
|
||||
.toEqual([1, 2])
|
||||
expect(agent.session.deriveMessages().at(-1)).toEqual({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'done' }],
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
})
|
||||
})
|
||||
|
||||
it('leaves partial failed chunks on their step without committing a message or tool side effect', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
partialToolFailure(new LlmError('stream interrupted', 'TRANSPORT')),
|
||||
textResponse('recovered'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
let toolExecutions = 0
|
||||
context.tools.register(defineTool({
|
||||
name: 'danger',
|
||||
description: 'must not run for a failed provider attempt',
|
||||
parameters: {},
|
||||
async execute() {
|
||||
toolExecutions += 1
|
||||
return [{ type: 'text', text: 'unexpected' }]
|
||||
},
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-partial'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(500)
|
||||
await idle
|
||||
|
||||
const failedChunks = agent.session.events.filter(event =>
|
||||
event.type === 'assistant/chunk' && event.data.step === 1,
|
||||
)
|
||||
expect(failedChunks).toHaveLength(6)
|
||||
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
|
||||
.toEqual([2])
|
||||
expect(agent.session.events.some(event => event.type === 'tool/call')).toBe(false)
|
||||
expect(toolExecutions).toBe(0)
|
||||
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'recovered' }],
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
})
|
||||
})
|
||||
|
||||
it('applies bounded exponential jitter and stops after the configured budget', async () => {
|
||||
vi.useFakeTimers()
|
||||
const samples = [0, 1]
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('busy one', 'SERVER'),
|
||||
new LlmError('busy two', 'SERVER'),
|
||||
new LlmError('busy three', 'SERVER'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, { jitterRatio: 0.1 }, undefined, {
|
||||
random: () => samples.shift() ?? 0.5,
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-exhausted'), { provider: 'mock', model: 'mock' })
|
||||
const first = waitForRetry(context, agent, 1)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
expect((await first).data.delayMs).toBe(450)
|
||||
|
||||
const second = waitForRetry(context, agent, 2)
|
||||
await vi.advanceTimersByTimeAsync(450)
|
||||
expect((await second).data.delayMs).toBe(1_100)
|
||||
|
||||
const idle = waitForIdle(context, agent)
|
||||
await vi.advanceTimersByTimeAsync(1_100)
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(3)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(2)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'error', failure: { message: 'busy three', code: 'SERVER' } } },
|
||||
})
|
||||
})
|
||||
|
||||
it('uses a bounded provider Retry-After verbatim and delegates an over-cap instruction', async () => {
|
||||
vi.useFakeTimers()
|
||||
const accepted = new ScriptedAdapter([
|
||||
new LlmError('wait', 'RATE_LIMIT', { providerRetryAfterMs: 2_000 }),
|
||||
textResponse('done'),
|
||||
])
|
||||
;({ ctx: context } = await harness(accepted, { jitterRatio: 1 }))
|
||||
const acceptedAgent = context.agentLoop.create(SessionId('retry-after-accepted'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, acceptedAgent, 1)
|
||||
acceptedAgent.send([{ type: 'text', text: 'go' }])
|
||||
expect((await scheduled).data.delayMs).toBe(2_000)
|
||||
const acceptedIdle = waitForIdle(context, acceptedAgent)
|
||||
await vi.advanceTimersByTimeAsync(2_000)
|
||||
await acceptedIdle
|
||||
expect(accepted.requests).toHaveLength(2)
|
||||
|
||||
await context.fiber.dispose()
|
||||
const rejected = new ScriptedAdapter([
|
||||
new LlmError('wait too long', 'RATE_LIMIT', { providerRetryAfterMs: 10_001 }),
|
||||
])
|
||||
;({ ctx: context } = await harness(rejected))
|
||||
const rejectedAgent = context.agentLoop.create(SessionId('retry-after-rejected'), { provider: 'mock', model: 'mock' })
|
||||
const rejectedIdle = waitForIdle(context, rejectedAgent)
|
||||
rejectedAgent.send([{ type: 'text', text: 'go' }])
|
||||
await rejectedIdle
|
||||
expect(rejected.requests).toHaveLength(1)
|
||||
expect(rejectedAgent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
})
|
||||
|
||||
it('delegates non-transient failures without scheduling a timer', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-auth'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await idle
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('aborts and drains a captured backoff before plugin disposal completes', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'TRANSPORT'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
const mounted = await harness(adapter)
|
||||
context = mounted.ctx
|
||||
const agent = context.agentLoop.create(SessionId('retry-hmr'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
await mounted.retryFiber.dispose()
|
||||
await idle
|
||||
await vi.advanceTimersByTimeAsync(60_000)
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.filter(event => event.type === 'step/start')).toHaveLength(1)
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('does not make plugin disposal wait for a delegated recovery policy', async () => {
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
const mounted = await harness(adapter)
|
||||
context = mounted.ctx
|
||||
const downstream = Promise.withResolvers<RequestErrorDecision>()
|
||||
const entered = Promise.withResolvers<undefined>()
|
||||
context.on('agent/request-error', () => {
|
||||
entered.resolve(undefined)
|
||||
return downstream.promise
|
||||
})
|
||||
const agent = context.agentLoop.create(SessionId('retry-delegated-disposal'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await entered.promise
|
||||
|
||||
const disposing = mounted.retryFiber.dispose()
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
const outcome = await Promise.race([
|
||||
disposing.then(() => 'disposed' as const),
|
||||
new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }),
|
||||
])
|
||||
if (timer !== undefined) clearTimeout(timer)
|
||||
downstream.resolve({ action: 'fail' })
|
||||
await disposing
|
||||
await idle
|
||||
|
||||
expect(outcome).toBe('disposed')
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('fails a captured callback after disposal without entering downstream policy', async () => {
|
||||
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
|
||||
const captured = Promise.withResolvers<undefined>()
|
||||
let invokeCaptured: (() => Promise<void>) | undefined
|
||||
const mounted = await harness(adapter, {}, (ctx) => {
|
||||
ctx.on('agent/request-error', (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
return new Promise<RequestErrorDecision>((resolve) => {
|
||||
invokeCaptured = async () => { resolve(await next()) }
|
||||
captured.resolve(undefined)
|
||||
})
|
||||
})
|
||||
})
|
||||
context = mounted.ctx
|
||||
let downstreamCalls = 0
|
||||
context.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
downstreamCalls += 1
|
||||
return next()
|
||||
})
|
||||
const agent = context.agentLoop.create(SessionId('retry-captured-disposal'), {
|
||||
provider: 'mock',
|
||||
model: 'mock',
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await captured.promise
|
||||
|
||||
await mounted.retryFiber.dispose()
|
||||
if (invokeCaptured === undefined) throw new Error('request-error waterfall did not capture retry callback')
|
||||
await invokeCaptured()
|
||||
await idle
|
||||
|
||||
expect(downstreamCalls).toBe(0)
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('lets turn cancellation win during backoff without opening another step', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'TIMEOUT'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-cancel'), { provider: 'mock', model: 'mock' })
|
||||
const scheduled = waitForRetry(context, agent, 1)
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await scheduled
|
||||
const idle = waitForIdle(context, agent)
|
||||
agent.cancel('user cancelled during retry')
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'aborted', reason: 'user cancelled during retry' } },
|
||||
})
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it('lets an earlier recovery listener cancel before retry policy runs', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'SERVER'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter, {}, (ctx) => {
|
||||
ctx.on('agent/request-error', async (agent, _turn, _step, _error, _failure, _history, _signal, next) => {
|
||||
agent.cancel('cancelled by earlier recovery policy')
|
||||
return next()
|
||||
})
|
||||
}))
|
||||
const agent = context.agentLoop.create(SessionId('retry-pre-cancel'), { provider: 'mock', model: 'mock' })
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'aborted', reason: 'cancelled by earlier recovery policy' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('handles synchronous cancellation from the retry status event', async () => {
|
||||
vi.useFakeTimers()
|
||||
const adapter = new ScriptedAdapter([
|
||||
new LlmError('temporary', 'SERVER'),
|
||||
textResponse('must not run'),
|
||||
])
|
||||
;({ ctx: context } = await harness(adapter))
|
||||
const agent = context.agentLoop.create(SessionId('retry-event-cancel'), { provider: 'mock', model: 'mock' })
|
||||
context.on('session/event', (session, event) => {
|
||||
if (session === agent.session && event.type === 'llm/retry') agent.cancel('cancelled by retry observer')
|
||||
})
|
||||
const idle = waitForIdle(context, agent)
|
||||
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
await idle
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1)
|
||||
expect(vi.getTimerCount()).toBe(0)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ maxTransientRetries: -1 }, /maxTransientRetries/],
|
||||
[{ maxTransientRetries: 1.5 }, /maxTransientRetries/],
|
||||
[{ initialDelayMs: 0 }, /initialDelayMs/],
|
||||
[{ maxDelayMs: Number.POSITIVE_INFINITY }, /maxDelayMs/],
|
||||
[{ initialDelayMs: MAX_TIMER_DELAY_MS + 1 }, /initialDelayMs/],
|
||||
[{ maxDelayMs: MAX_TIMER_DELAY_MS + 1 }, /maxDelayMs/],
|
||||
[{ initialDelayMs: 20, maxDelayMs: 10 }, /less than or equal/],
|
||||
[{ jitterRatio: 1.1 }, /jitterRatio/],
|
||||
[{ retryableCodes: [] }, /must not be empty/],
|
||||
[{ retryableCodes: ['SERVER', 'SERVER'] }, /duplicates/],
|
||||
[{ retryableCodes: [''] }, /non-empty strings/],
|
||||
] as const)('fails direct composition for invalid config %#', (config, message) => {
|
||||
expect(() => { retry.apply(new Context(), config as retry.Config) }).toThrow(message)
|
||||
})
|
||||
})
|
||||
33
packages/llm/llm-retry/tsconfig.json
Normal file
33
packages/llm/llm-retry/tsconfig.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -14,7 +14,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`.
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
@@ -24,12 +24,12 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
|---|---|---|
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) |
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call for caching, logging, or routing |
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -50,8 +50,10 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) matches its frozen serializable `failure.code`. The payload may also retain validated status, `Retry-After`, and branded provider request id facts; policy remains outside the error.
|
||||
- `errorChain(value)` — renders a thrown value with its full `cause` chain and AggregateError members for diagnostic surfaces (UI notices, logger lines, durable `turn/end` messages), so transport wrappers like undici's `TypeError: fetch failed` surface the underlying `ECONNREFUSED`/DNS/TLS detail instead of masking it. Rendering only — route on `code`, never by parsing the result.
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
|
||||
- `QUOTA_EXCEEDED_CODE` — the non-transient provider-neutral code for exhausted account quota, balance, credits, budget, or usage limits. `isQuotaExceededError(detail)` keeps those failures distinct from request-rate limits.
|
||||
|
||||
### Real adapters
|
||||
|
||||
@@ -67,7 +69,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure.
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
|
||||
@@ -5,10 +5,10 @@
|
||||
*/
|
||||
|
||||
import { HarnessError } from './error.ts'
|
||||
import type { StreamChunk } from './types.ts'
|
||||
import type { LlmFailure, StreamChunk } from './types.ts'
|
||||
|
||||
/** Errors proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakSet<Error>
|
||||
/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakMap<Error, LlmFailure>
|
||||
|
||||
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
|
||||
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
|
||||
@@ -47,10 +47,71 @@ export function markLlmAdapterFailure(
|
||||
const error = value instanceof Error
|
||||
? value as Error & { code?: string }
|
||||
: new HarnessError(String(value), 'UNKNOWN', { cause: value })
|
||||
failures.add(error)
|
||||
const carried = error instanceof HarnessError ? ownFailureSnapshot(error) : undefined
|
||||
const failure = carried !== undefined && carried.code === error.code ? carried : Object.freeze({
|
||||
message: errorMessage(error),
|
||||
code: harnessErrorCode(error),
|
||||
})
|
||||
failures.set(error, failure)
|
||||
return error
|
||||
}
|
||||
|
||||
/** Snapshot an own data property without invoking an SDK-defined accessor. */
|
||||
function ownFailureSnapshot(error: Error): LlmFailure | undefined {
|
||||
try {
|
||||
const descriptor = Object.getOwnPropertyDescriptor(error, 'failure')
|
||||
return descriptor !== undefined && 'value' in descriptor
|
||||
? failureSnapshot(descriptor.value)
|
||||
: undefined
|
||||
} catch (_sdkPropertyTrap) {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate and detach an arbitrary serializable failure payload. */
|
||||
function failureSnapshot(value: unknown): LlmFailure | undefined {
|
||||
if (typeof value !== 'object' || value === null) return undefined
|
||||
try {
|
||||
const candidate = value as Partial<LlmFailure>
|
||||
const message = candidate.message
|
||||
const code = candidate.code
|
||||
const status = candidate.status
|
||||
const providerRetryAfterMs = candidate.providerRetryAfterMs
|
||||
const requestId = candidate.requestId
|
||||
if (typeof message !== 'string' || message.length === 0
|
||||
|| typeof code !== 'string' || code.length === 0
|
||||
|| (status !== undefined && (!Number.isInteger(status) || status < 100 || status > 599))
|
||||
|| (providerRetryAfterMs !== undefined
|
||||
&& (!Number.isFinite(providerRetryAfterMs) || providerRetryAfterMs <= 0))
|
||||
|| (requestId !== undefined && (typeof requestId !== 'string' || requestId.length === 0))) return undefined
|
||||
return Object.freeze({
|
||||
message,
|
||||
code,
|
||||
...status === undefined ? {} : { status },
|
||||
...providerRetryAfterMs === undefined ? {} : { providerRetryAfterMs },
|
||||
...requestId === undefined ? {} : { requestId },
|
||||
})
|
||||
} catch (_sdkFailureGetter) {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Read an SDK error message without letting an accessor replace the primary failure. */
|
||||
function errorMessage(error: Error): string {
|
||||
try {
|
||||
const message: unknown = error.message
|
||||
if (typeof message === 'string' && message.length > 0) return message
|
||||
} catch (_sdkMessageGetter) {
|
||||
// The fallback below preserves a serializable failure beside the original Error.
|
||||
}
|
||||
return 'LLM adapter failed'
|
||||
}
|
||||
|
||||
/** Trust only Harness-owned codes; third-party SDK codes are not our taxonomy. */
|
||||
function harnessErrorCode(error: Error): string {
|
||||
return error instanceof HarnessError ? error.code : 'UNKNOWN'
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a failure came from final adapter dispatch, iterator construction,
|
||||
* or iteration for the call represented by the exact returned stream handle.
|
||||
@@ -65,3 +126,18 @@ export function isLlmAdapterFailure(
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error && failures !== undefined && failures.has(value)
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieve normalized provider facts only for an Error tagged by this exact
|
||||
* model call's final adapter boundary.
|
||||
* @param stream - the exact stream returned to the consumer.
|
||||
* @param value - the caught failure.
|
||||
* @returns the immutable facts for that call, or `undefined` for middleware, nested, or consumer failures.
|
||||
*/
|
||||
export function llmFailureOf(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
value: unknown,
|
||||
): LlmFailure | undefined {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error ? failures?.get(value) : undefined
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
/**
|
||||
* dsh-llm's owned branded id: `CallId` (tool-call correlation).
|
||||
* dsh-llm's owned branded ids: tool-call correlation and provider request
|
||||
* diagnostics.
|
||||
*
|
||||
* The `Branded<B>` primitive itself lives in `@deepseek-ai/dsh-brand` (a
|
||||
* zero-dependency type-only package) so every owner of a cross-boundary id can
|
||||
@@ -25,3 +26,15 @@ export type CallId = Branded<'CallId'>
|
||||
export function CallId(id: string): CallId {
|
||||
return id as CallId
|
||||
}
|
||||
|
||||
/** Provider-issued request identifier retained for diagnostics across package boundaries. */
|
||||
export type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
|
||||
/**
|
||||
* Brand a provider-issued request identifier.
|
||||
* @param id - the opaque provider-issued string.
|
||||
* @returns the same string, branded; no validation is performed.
|
||||
*/
|
||||
export function ProviderRequestId(id: string): ProviderRequestId {
|
||||
return id as ProviderRequestId
|
||||
}
|
||||
|
||||
@@ -24,6 +24,9 @@ export class HarnessError extends Error {
|
||||
/** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */
|
||||
export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED'
|
||||
|
||||
/** Canonical provider-neutral code for an exhausted account quota or balance. */
|
||||
export const QUOTA_EXCEEDED_CODE = 'QUOTA'
|
||||
|
||||
/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */
|
||||
const STRUCTURED_CONTEXT_OVERFLOW = new RegExp(
|
||||
String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]`
|
||||
@@ -62,6 +65,65 @@ export function isContextWindowExceededError(detail: string): boolean {
|
||||
|| EXCEEDS_MODEL_CONTEXT.test(detail)
|
||||
}
|
||||
|
||||
/**
|
||||
* Recognize provider wording that identifies an exhausted account quota rather
|
||||
* than a transient request-rate limit.
|
||||
* @param detail - provider error code/type/message text joined into one string.
|
||||
* @returns true only for terminal quota, balance, credit, budget, or usage-limit wording.
|
||||
*/
|
||||
export function isQuotaExceededError(detail: string): boolean {
|
||||
return /\binsufficient[\s_-]+(?:quota|balance|credits?)\b/i.test(detail)
|
||||
|| /\b(?:quota|usage[\s_-]+limit)[\s_-]+(?:exceeded|exhausted|reached)\b/i.test(detail)
|
||||
|| /\bexceed(?:ed|s)?[\s_-]+(?:(?:your|the)[\s_-]+)?(?:current[\s_-]+)?quota\b/i.test(detail)
|
||||
|| /\b(?:balance|credits?)[\s_-]+(?:exhausted|depleted)\b/i.test(detail)
|
||||
|| /\bout[\s_-]+of[\s_-]+(?:credits?|budget)\b/i.test(detail)
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a thrown value with its full `cause` chain and AggregateError
|
||||
* members, so transport wrappers like undici's `TypeError: fetch failed`
|
||||
* surface the underlying failure instead of masking it. Diagnostic-surface
|
||||
* rendering only (messages, notices, logs) — never parse the result; route on
|
||||
* {@link HarnessError.code}.
|
||||
* @param value - the caught value (`unknown` in catch clauses).
|
||||
* @returns the outermost message first, each cause appended with `: ` (skipped
|
||||
* when it repeats the wrapper message verbatim), and AggregateError members
|
||||
* bracketed and `; `-joined.
|
||||
*/
|
||||
export function errorChain(value: unknown): string {
|
||||
// Tracks the active recursion path (entries removed on exit), so only true
|
||||
// cycles are flagged and a diamond-shared cause still renders in full.
|
||||
const path = new Set<unknown>()
|
||||
const render = (current: unknown): string => {
|
||||
if (path.has(current)) return '<circular cause>'
|
||||
path.add(current)
|
||||
try {
|
||||
if (!(current instanceof Error)) return String(current)
|
||||
const message = current.message === '' ? current.name : current.message
|
||||
const members = current instanceof AggregateError && current.errors.length > 0
|
||||
? ` [${current.errors.map(render).join('; ')}]`
|
||||
: ''
|
||||
const causeText = current.cause === undefined || current.cause === null
|
||||
? ''
|
||||
: render(current.cause)
|
||||
// Wrappers like `new HarnessError(String(value), code, { cause: value })`
|
||||
// repeat their cause verbatim; rendering it again would only add noise.
|
||||
const cause = causeText === '' || causeText === message ? '' : `: ${causeText}`
|
||||
return `${message}${members}${cause}`
|
||||
} catch {
|
||||
// Only hostile coercion or hostile accessors (a throwing toString /
|
||||
// Symbol.toPrimitive on a non-Error, or a throwing message/name/cause/
|
||||
// errors getter on an Error subclass): this renderer feeds UI notices
|
||||
// and logs, so nothing may escape. Inner frames catch their own throws,
|
||||
// so only the hostile node collapses, not the whole chain.
|
||||
return '<unrenderable value>'
|
||||
} finally {
|
||||
path.delete(current)
|
||||
}
|
||||
}
|
||||
return render(value)
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams).
|
||||
* @param value - the caught value (`unknown` in catch clauses).
|
||||
|
||||
@@ -9,12 +9,14 @@
|
||||
import { Context, Service } from 'cordis'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
Message,
|
||||
StreamChunk,
|
||||
} from './types.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
@@ -28,7 +30,7 @@ export * from './types.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export { isLlmAdapterFailure } from './adapter-failure.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -51,14 +53,53 @@ declare module 'cordis' {
|
||||
}
|
||||
}
|
||||
|
||||
/** Structured provider facts and cause accepted by {@link LlmError}. */
|
||||
export interface LlmErrorOptions extends ErrorOptions {
|
||||
/** Valid HTTP status observed at the provider boundary. */
|
||||
status?: number
|
||||
/** Positive finite provider-requested delay in milliseconds. */
|
||||
providerRetryAfterMs?: number
|
||||
/** Non-empty opaque provider request id. */
|
||||
requestId?: ProviderRequestId
|
||||
}
|
||||
|
||||
/**
|
||||
* Typed error for LLM-related failures. Extends {@link HarnessError}, so the
|
||||
* `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy.
|
||||
*/
|
||||
export class LlmError extends HarnessError {
|
||||
constructor(message: string, code: string, options?: ErrorOptions) {
|
||||
/** Serializable facts retained beside this live Error. */
|
||||
readonly failure: LlmFailure
|
||||
|
||||
/**
|
||||
* @param message - non-empty human-readable failure summary.
|
||||
* @param code - non-empty stable provider-neutral machine code.
|
||||
* @param options - optional cause and validated serializable provider facts.
|
||||
*/
|
||||
constructor(message: string, code: string, options?: LlmErrorOptions) {
|
||||
if (typeof message !== 'string' || message.length === 0) throw new Error('LlmError message must be a non-empty string')
|
||||
if (typeof code !== 'string' || code.length === 0) throw new Error('LlmError code must be a non-empty string')
|
||||
if (options?.status !== undefined
|
||||
&& (!Number.isInteger(options.status) || options.status < 100 || options.status > 599)) {
|
||||
throw new Error('LlmError status must be an integer from 100 through 599')
|
||||
}
|
||||
if (options?.providerRetryAfterMs !== undefined
|
||||
&& (!Number.isFinite(options.providerRetryAfterMs) || options.providerRetryAfterMs <= 0)) {
|
||||
throw new Error('LlmError providerRetryAfterMs must be a positive finite number')
|
||||
}
|
||||
if (options?.requestId !== undefined
|
||||
&& (typeof options.requestId !== 'string' || options.requestId.length === 0)) {
|
||||
throw new Error('LlmError requestId must be a non-empty string')
|
||||
}
|
||||
super(message, code, options)
|
||||
this.name = 'LlmError'
|
||||
this.failure = Object.freeze({
|
||||
message,
|
||||
code,
|
||||
...options?.status === undefined ? {} : { status: options.status },
|
||||
...options?.providerRetryAfterMs === undefined ? {} : { providerRetryAfterMs: options.providerRetryAfterMs },
|
||||
...options?.requestId === undefined ? {} : { requestId: options.requestId },
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -306,7 +347,7 @@ export class LlmService extends Service {
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakSet<Error>()
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
|
||||
@@ -5,7 +5,21 @@
|
||||
*/
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { CallId } from './brand.ts'
|
||||
import type { CallId, ProviderRequestId } from './brand.ts'
|
||||
|
||||
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
|
||||
export interface LlmFailure {
|
||||
/** Human-readable provider or transport failure. */
|
||||
readonly message: string
|
||||
/** Stable provider-neutral machine-routing code. */
|
||||
readonly code: string
|
||||
/** HTTP status observed at the provider boundary, when available. */
|
||||
readonly status?: number
|
||||
/** Provider-requested delay in milliseconds, when valid and available. */
|
||||
readonly providerRetryAfterMs?: number
|
||||
/** Opaque provider-issued request identifier for diagnostics. */
|
||||
readonly requestId?: ProviderRequestId
|
||||
}
|
||||
|
||||
/** Plain text visible to the end user. */
|
||||
export interface TextBlock {
|
||||
@@ -98,8 +112,8 @@ export interface FinishReasonMap {
|
||||
'stop': { kind: 'stop' }
|
||||
'tool-calls': { kind: 'tool-calls' }
|
||||
'max-tokens': { kind: 'max-tokens' }
|
||||
'aborted': { kind: 'aborted' }
|
||||
'error': { kind: 'error'; message: string; code?: string }
|
||||
'aborted': { kind: 'aborted'; failure: LlmFailure }
|
||||
'error': { kind: 'error'; failure: LlmFailure }
|
||||
}
|
||||
|
||||
/** Any known finish reason, derived from {@link FinishReasonMap}; switch on `kind` and fall through unknowns (merge-extensible). */
|
||||
|
||||
@@ -41,7 +41,10 @@ const chunkArb: fc.Arbitrary<StreamChunk> = indexArb.chain(index => fc.oneof(
|
||||
fc.constant<StreamChunk>({ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }),
|
||||
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'stop' } }),
|
||||
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'tool-calls' } }),
|
||||
fc.string().map((message): StreamChunk => ({ type: 'finish', reason: { kind: 'error', message } })),
|
||||
fc.string({ minLength: 1 }).map((message): StreamChunk => ({
|
||||
type: 'finish',
|
||||
reason: { kind: 'error', failure: { message, code: 'UNKNOWN' } },
|
||||
})),
|
||||
))
|
||||
|
||||
/** A stream is an arbitrary list of chunks (we do NOT force a terminal finish). */
|
||||
|
||||
@@ -1,12 +1,16 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, {
|
||||
errorChain,
|
||||
GenerateOptions,
|
||||
HarnessError,
|
||||
isContextWindowExceededError,
|
||||
isQuotaExceededError,
|
||||
isLlmAdapterFailure,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
llmFailureOf,
|
||||
ProviderRequestId,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
@@ -88,6 +92,62 @@ describe('LlmService', () => {
|
||||
expect(isContextWindowExceededError('context window size must be positive')).toBe(false)
|
||||
})
|
||||
|
||||
it('distinguishes exhausted account quota from transient rate limiting', () => {
|
||||
for (const detail of [
|
||||
'insufficient_quota',
|
||||
'account balance depleted',
|
||||
'usage-limit-exceeded',
|
||||
'out of credits',
|
||||
'OpenAI API error (429): You exceeded your current quota, please check your plan and billing details.',
|
||||
]) expect(isQuotaExceededError(detail)).toBe(true)
|
||||
expect(isQuotaExceededError('HTTP 429: rate limit reached')).toBe(false)
|
||||
expect(isQuotaExceededError('quota resets in one minute')).toBe(false)
|
||||
})
|
||||
|
||||
it('errorChain renders the full cause chain of a wrapped transport failure', () => {
|
||||
const chain = new TypeError('fetch failed', { cause: new Error('connect ECONNREFUSED 127.0.0.1:443') })
|
||||
expect(errorChain(chain)).toBe('fetch failed: connect ECONNREFUSED 127.0.0.1:443')
|
||||
})
|
||||
|
||||
it('errorChain renders AggregateError members (Happy Eyeballs multi-address failures)', () => {
|
||||
const aggregate = new AggregateError(
|
||||
[new Error('connect ECONNREFUSED ::1:443'), new Error('connect ECONNREFUSED 127.0.0.1:443')],
|
||||
'',
|
||||
)
|
||||
const wrapped = new TypeError('fetch failed', { cause: aggregate })
|
||||
expect(errorChain(wrapped)).toBe(
|
||||
'fetch failed: AggregateError [connect ECONNREFUSED ::1:443; connect ECONNREFUSED 127.0.0.1:443]',
|
||||
)
|
||||
})
|
||||
|
||||
it('errorChain survives non-Error values, hostile coercion, and circular causes', () => {
|
||||
expect(errorChain('plain string')).toBe('plain string')
|
||||
expect(errorChain({ toString: () => { throw new Error('hostile') } })).toBe('<unrenderable value>')
|
||||
const circular = new Error('outer')
|
||||
circular.cause = circular
|
||||
expect(errorChain(circular)).toBe('outer: <circular cause>')
|
||||
// A hostile accessor collapses only its own node, not the whole chain.
|
||||
const hostileNode = new Error('node')
|
||||
Object.defineProperty(hostileNode, 'message', { get() { throw new Error('hostile getter') } })
|
||||
expect(errorChain(new Error('outer', { cause: hostileNode }))).toBe('outer: <unrenderable value>')
|
||||
// A diamond-shared (non-cyclic) cause renders in full on both paths.
|
||||
const shared = new Error('shared')
|
||||
const diamond = new AggregateError([new Error('a', { cause: shared }), new Error('b', { cause: shared })], 'agg')
|
||||
expect(errorChain(diamond)).toBe('agg [a: shared; b: shared]')
|
||||
})
|
||||
|
||||
it('errorChain falls back to the error name, skips empty aggregates, and stops at null causes', () => {
|
||||
expect(errorChain(new TypeError('', { cause: null }))).toBe('TypeError')
|
||||
expect(errorChain(new AggregateError([], 'all failed'))).toBe('all failed')
|
||||
})
|
||||
|
||||
it('errorChain collapses a cause that repeats the wrapper message verbatim', () => {
|
||||
// The `new HarnessError(String(value), code, { cause: value })` normalization
|
||||
// pattern repeats its cause; rendering it twice would only add noise.
|
||||
const wrapped = new HarnessError('boom', 'UNKNOWN', { cause: 'boom' })
|
||||
expect(errorChain(wrapped)).toBe('boom')
|
||||
})
|
||||
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -176,6 +236,151 @@ describe('LlmService', () => {
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
expect(llmFailureOf(stream, caught)).toEqual({
|
||||
message: `${boundary} failed`,
|
||||
code: 'BOUNDARY_FAILED',
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps structured provider facts beside a frozen third-party Error', async () => {
|
||||
const original = new LlmError('provider busy', 'RATE_LIMIT', {
|
||||
status: 429,
|
||||
providerRetryAfterMs: 1_500,
|
||||
requestId: ProviderRequestId('req-7'),
|
||||
})
|
||||
Object.freeze(original)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(llmFailureOf(stream, caught)).toEqual({
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 1_500,
|
||||
requestId: ProviderRequestId('req-7'),
|
||||
})
|
||||
})
|
||||
|
||||
it('does not trust retry facts carried by an unknown third-party Error', async () => {
|
||||
const carried = { message: 'busy', code: 'SERVER', status: 503 }
|
||||
const original = Object.assign(new Error('busy'), { failure: carried })
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
const facts = llmFailureOf(stream, original)
|
||||
carried.status = 500
|
||||
|
||||
expect(facts).toEqual({ message: 'busy', code: 'UNKNOWN' })
|
||||
expect(Object.isFrozen(facts)).toBe(true)
|
||||
expect(facts).not.toBe(carried)
|
||||
})
|
||||
|
||||
it('keeps an unknown SDK Error exact without trusting its private code or accessors', async () => {
|
||||
const original = Object.assign(new Error('socket closed'), { code: 'ECONNRESET' })
|
||||
Object.defineProperty(original, 'failure', {
|
||||
get() { throw new Error('SDK failure accessor must not run') },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
|
||||
expect(original.code).toBe('ECONNRESET')
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'socket closed', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('keeps an SDK Error exact when its message accessor is hostile', async () => {
|
||||
const original = Object.defineProperty(new Error(), 'message', {
|
||||
get() { throw new Error('SDK message accessor trap') },
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: 'LLM adapter failed', code: 'UNKNOWN' })
|
||||
})
|
||||
|
||||
it('falls back safely when SDK objects trap failure inspection or expose malformed facts', async () => {
|
||||
const propertyTrap = new Proxy(new HarnessError('descriptor trapped', 'SERVER'), {
|
||||
getOwnPropertyDescriptor(target, property) {
|
||||
if (property === 'failure') throw new Error('SDK descriptor trap')
|
||||
return Reflect.getOwnPropertyDescriptor(target, property)
|
||||
},
|
||||
})
|
||||
const throwingFacts = Object.create(null) as Record<string, unknown>
|
||||
Object.defineProperty(throwingFacts, 'message', {
|
||||
get() { throw new Error('SDK fact getter trap') },
|
||||
})
|
||||
const carrying = (message: string, failure: unknown): HarnessError => Object.defineProperty(
|
||||
new HarnessError(message, 'SERVER'),
|
||||
'failure',
|
||||
{ value: failure },
|
||||
)
|
||||
const factGetter = carrying('fact getter failed', throwingFacts)
|
||||
const malformed = carrying('malformed facts', { message: 'provider busy', code: 'SERVER', requestId: 1 })
|
||||
const primitive = carrying('primitive facts', 1)
|
||||
const nullFacts = carrying('null facts', null)
|
||||
const mismatched = carrying('mismatched facts', { message: 'busy', code: 'RATE_LIMIT' })
|
||||
|
||||
for (const [original, expectedMessage] of [
|
||||
[propertyTrap, 'descriptor trapped'],
|
||||
[factGetter, 'fact getter failed'],
|
||||
[malformed, 'malformed facts'],
|
||||
[primitive, 'primitive facts'],
|
||||
[nullFacts, 'null facts'],
|
||||
[mismatched, 'mismatched facts'],
|
||||
] as const) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({ message: expectedMessage, code: 'SERVER' })
|
||||
}
|
||||
})
|
||||
|
||||
it('retains a stable code from a HarnessError without requiring LlmError facts', async () => {
|
||||
const original = new HarnessError('stable adapter failure', 'ADAPTER_STABLE')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
|
||||
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
|
||||
|
||||
await expect((async () => {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
})()).rejects.toBe(original)
|
||||
expect(llmFailureOf(stream, original)).toEqual({
|
||||
message: 'stable adapter failure',
|
||||
code: 'ADAPTER_STABLE',
|
||||
})
|
||||
expect(llmFailureOf(stream, 'not an Error')).toBeUndefined()
|
||||
expect(llmFailureOf({ [Symbol.asyncIterator]: () => stream[Symbol.asyncIterator]() }, original)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps a nested adapter failure scoped to the nested model call', async () => {
|
||||
@@ -628,6 +833,16 @@ describe('LlmService', () => {
|
||||
expect(err.code).toBe('CUSTOM_CODE')
|
||||
})
|
||||
|
||||
it('rejects non-serializable structured failure facts at construction', () => {
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { status: 42 })).toThrow(/status/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { providerRetryAfterMs: Number.NaN }))
|
||||
.toThrow(/providerRetryAfterMs/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: ProviderRequestId('') })).toThrow(/requestId/)
|
||||
expect(() => new LlmError(1 as never, 'RATE_LIMIT')).toThrow(/message/)
|
||||
expect(() => new LlmError('busy', 1 as never)).toThrow(/code/)
|
||||
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: 1 as never })).toThrow(/requestId/)
|
||||
})
|
||||
|
||||
it('LlmError extends the shared HarnessError base', async () => {
|
||||
const { HarnessError, isHarnessError } = await import('@deepseek-ai/dsh-llm')
|
||||
const cause = new Error('root cause')
|
||||
|
||||
Reference in New Issue
Block a user