feat: implement bounded LLM request recovery

This commit is contained in:
Tianyi Cui
2026-07-20 03:34:19 +08:00
parent 7cf966fc0e
commit 3b0b0cefeb
115 changed files with 3311 additions and 366 deletions

View File

@@ -6,7 +6,8 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|---|---|---|
| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` |
| `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` |
| `llm-retry/` | Bounded transient request retry policy | (listens to `agent/request-error`) |
| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) |
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
The interface lives at `llm/llm/`; adapters and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership.
The interface lives at `llm/llm/`; adapters, retry policy, and reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership.

View File

@@ -16,6 +16,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com
thinking: enabled # optional; provider default is enabled
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
models: # optional; defaults to V4 Flash and V4 Pro
- id: deepseek-v4-flash
name: DeepSeek V4 Flash
@@ -29,6 +30,8 @@ The plugin registers the single provider route `deepseek`. A request selects it
`thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral.
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; agent-level retry is a separate plugin policy.
## App attribution
Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode.
@@ -42,11 +45,11 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
## Errors
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: <REASON>}` chunks.
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. Connection failures are `TRANSPORT`; protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks.
## Testing
Unit suites run against a local `node:http` mock SSE server (no network). Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
Unit suites run against a local `node:http` mock SSE server (no network), including structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback.
## Model Experience

View File

@@ -23,6 +23,7 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-timeout": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
@@ -30,6 +31,7 @@
},
"devDependencies": {
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-timeout": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -5,8 +5,9 @@
* @module dsh-llm-deepseek/adapter
*/
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
import { serializeRequest } from './serialize.ts'
import type { RequestDefaults } from './serialize.ts'
import { parseSse } from './sse.ts'
@@ -33,6 +34,31 @@ export interface DeepSeekAdapterOptions {
defaults?: RequestDefaults
/** Advisory models exposed to discovery consumers; requests remain unrestricted. */
models?: readonly DeepSeekCatalogModel[]
/** Maximum provider idle time while one stream read is outstanding. */
streamIdleTimeoutMs?: number
}
/** Default maximum idle interval while an adapter stream read is outstanding. */
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT'
function retryAfterMs(value: string | null): number | undefined {
if (value === null) return undefined
if (/^\d+$/.test(value)) {
const delay = Number(value) * 1_000
return Number.isFinite(delay) && delay > 0 ? delay : undefined
}
const delay = Date.parse(value) - Date.now()
return Number.isFinite(delay) && delay > 0 ? delay : undefined
}
function requestId(headers: Headers): ReturnType<typeof ProviderRequestId> | undefined {
const value = headers.get('x-request-id') ?? headers.get('x-deepseek-request-id')
return value === null || value.length === 0 ? undefined : ProviderRequestId(value)
}
function errorMessage(value: unknown): string {
return value instanceof Error ? value.message : String(value)
}
/**
@@ -43,9 +69,10 @@ export interface DeepSeekAdapterOptions {
*/
export function httpErrorCode(status: number, error?: WireError['error']): string {
if (status === 401 || status === 403) return 'AUTH'
const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ')
if (isQuotaExceededError(detail)) return QUOTA_EXCEEDED_CODE
if (status === 429) return 'RATE_LIMIT'
if (status === 400) {
const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ')
if (isContextWindowExceededError(detail)) return CONTEXT_WINDOW_EXCEEDED_CODE
return 'INVALID_REQUEST'
}
@@ -57,13 +84,22 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin
* The first real `LlmAdapter`. One instance serves every model name it was
* registered under (the harness model name IS the wire model name).
*
* Abort: `options.signal` is handed to fetch — both the initial request and
* the body stream reject on abort, which surfaces to the loop as a rejected
* step (the loop already contains step errors).
* One stable signal reaches both initial fetch and body reads. Caller aborts
* map to `ABORTED`; the configured per-read idle watchdog maps to `TIMEOUT`.
*/
export class DeepSeekAdapter extends LlmAdapter {
private readonly streamIdleTimeoutMs: number
constructor(private readonly options: DeepSeekAdapterOptions) {
super()
this.streamIdleTimeoutMs = options.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
if (!Number.isFinite(this.streamIdleTimeoutMs)
|| this.streamIdleTimeoutMs <= 0
|| this.streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) {
throw new Error(
`llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
)
}
}
override providerInfo(provider: string): LlmProviderInfo {
@@ -80,6 +116,48 @@ export class DeepSeekAdapter extends LlmAdapter {
}
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
const consumer = new AbortController()
const upstream = options.signal === undefined
? consumer.signal
: AbortSignal.any([options.signal, consumer.signal])
using watchdog = idleWatchdog(upstream, this.streamIdleTimeoutMs, STREAM_IDLE_TIMEOUT_CODE)
const iterator = this.request(options, watchdog.signal)[Symbol.asyncIterator]()
let exhausted = false
try {
while (true) {
const result = await watchdog.next(iterator)
if (result.done) {
exhausted = true
return
}
yield result.value
}
} catch (error: unknown) {
if (timeoutOf(watchdog.signal, STREAM_IDLE_TIMEOUT_CODE) !== undefined) {
throw new LlmError(
`DeepSeek stream idle timeout after ${this.streamIdleTimeoutMs}ms`,
'TIMEOUT',
{ cause: error },
)
}
if (options.signal?.aborted) {
throw new LlmError('DeepSeek request aborted by caller', 'ABORTED', { cause: error })
}
if (error instanceof LlmError) throw error
throw new LlmError(`DeepSeek transport failed: ${errorMessage(error)}`, 'TRANSPORT', { cause: error })
} finally {
consumer.abort('DeepSeek stream consumer stopped')
if (!exhausted && iterator.return !== undefined) {
try {
await iterator.return()
} catch (_abortedTransportTeardown) {
// The consumer controller already owns termination; a return-time abort cannot add a second outcome.
}
}
}
}
private async * request(options: GenerateOptions, signal: AbortSignal): AsyncIterable<StreamChunk> {
const body = serializeRequest(options, this.options.defaults ?? {})
// TODO(http): adopt the Cordis HTTP service when shared transport configuration
@@ -96,7 +174,7 @@ export class DeepSeekAdapter extends LlmAdapter {
: {},
},
body: JSON.stringify(body),
...options.signal ? { signal: options.signal } : {},
signal,
})
if (!response.ok) {
@@ -110,7 +188,13 @@ export class DeepSeekAdapter extends LlmAdapter {
// Only swallow error-body parsing: the HTTP status still identifies the
// failure, so malformed gateway JSON must not mask it.
}
throw new LlmError(message, httpErrorCode(response.status, providerError))
const delay = retryAfterMs(response.headers.get('retry-after'))
const id = requestId(response.headers)
throw new LlmError(message, httpErrorCode(response.status, providerError), {
status: response.status,
...delay === undefined ? {} : { retryAfterMs: delay },
...id === undefined ? {} : { requestId: id },
})
}
if (!response.body) {
throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE')

View File

@@ -8,7 +8,8 @@
import type { Context } from 'cordis'
import z from 'schemastery'
import type {} from '@deepseek-ai/dsh-llm'
import { DeepSeekAdapter } from './adapter.ts'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
import type { DeepSeekCatalogModel } from './adapter.ts'
export { DeepSeekAdapter } from './adapter.ts'
@@ -41,6 +42,8 @@ export interface Config {
reasoningEffort?: 'high' | 'max'
/** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
models?: DeepSeekCatalogModel[]
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
streamIdleTimeoutMs?: number
}
const catalogModel: z<DeepSeekCatalogModel> = z.object({
@@ -55,6 +58,7 @@ export const Config: z<Config> = z.object({
thinking: z.union(['enabled', 'disabled']),
reasoningEffort: z.union(['high', 'max']),
models: z.array(catalogModel).default(DEFAULT_MODELS),
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
})
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
@@ -92,5 +96,6 @@ export function apply(ctx: Context, config: Config): void {
reasoningEffort: config.reasoningEffort,
},
models: resolveModels(config.models),
streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS,
}))
}

View File

@@ -35,7 +35,10 @@ export function mapFinishReason(reason: string): FinishReason {
case 'length': return { kind: 'max-tokens' }
default:
// content_filter, insufficient_system_resource, future additions.
return { kind: 'error', message: `model stopped: ${reason}`, code: reason.toUpperCase() }
return {
kind: 'error',
failure: { message: `model stopped: ${reason}`, code: reason.toUpperCase() },
}
}
}

View File

@@ -2,7 +2,14 @@ import { createServer } from 'node:http'
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm'
import LlmService, {
CONTEXT_WINDOW_EXCEEDED_CODE,
LlmError,
ProviderRequestId,
QUOTA_EXCEEDED_CODE,
userAgent,
} from '@deepseek-ai/dsh-llm'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
import { SessionId } from '@deepseek-ai/dsh-session'
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
import { DeepSeekAdapter } from '@deepseek-ai/dsh-llm-deepseek'
@@ -12,7 +19,7 @@ import { assemble } from './assemble.ts'
/** One scripted behavior for the next request the mock server receives. */
type Behavior =
| { kind: 'sse'; events: string[]; delayMs?: number }
| { kind: 'http-error'; status: number; body: string; contentType?: string }
| { kind: 'http-error'; status: number; body: string; contentType?: string; headers?: Record<string, string> }
| { kind: 'close-early'; events: string[] }
interface MockServer {
@@ -30,6 +37,7 @@ const servers: Server[] = []
afterEach(async () => {
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
vi.unstubAllEnvs()
vi.useRealTimers()
})
/** Local chat-completions stand-in: replays scripted behaviors per request. */
@@ -48,7 +56,10 @@ async function mockServer(script: Behavior[]): Promise<MockServer> {
return
}
if (behavior.kind === 'http-error') {
response.writeHead(behavior.status, { 'content-type': behavior.contentType ?? 'application/json' })
response.writeHead(behavior.status, {
'content-type': behavior.contentType ?? 'application/json',
...behavior.headers,
})
response.end(behavior.body)
return
}
@@ -202,6 +213,84 @@ describe('DeepSeekAdapter against a mock server', () => {
expect(code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
})
it('retains status, Retry-After seconds, and provider request id as structured facts', async () => {
const server = await mockServer([{
kind: 'http-error',
status: 429,
body: JSON.stringify({ error: { message: 'slow down' } }),
headers: { 'retry-after': '2', 'x-request-id': 'req-429' },
}])
const ctx = await harness(server.url)
let thrown: unknown
try {
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
} catch (error: unknown) {
thrown = error
}
expect(thrown).toBeInstanceOf(LlmError)
expect((thrown as LlmError).failure).toEqual({
message: 'slow down',
code: 'RATE_LIMIT',
status: 429,
retryAfterMs: 2_000,
requestId: ProviderRequestId('req-429'),
})
})
it('parses a future Retry-After HTTP date and the DeepSeek request-id fallback', async () => {
const now = 1_800_000_000_000
const dateNow = vi.spyOn(Date, 'now').mockReturnValue(now)
try {
const server = await mockServer([{
kind: 'http-error',
status: 503,
body: JSON.stringify({ error: { message: 'come back later' } }),
headers: {
'retry-after': new Date(now + 3_000).toUTCString(),
'x-deepseek-request-id': 'deepseek-503',
},
}])
const ctx = await harness(server.url)
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
.rejects.toMatchObject({
failure: {
message: 'come back later',
code: 'SERVER',
status: 503,
retryAfterMs: 3_000,
requestId: ProviderRequestId('deepseek-503'),
},
})
} finally {
dateNow.mockRestore()
}
})
it('omits zero, non-finite, invalid, and past Retry-After values', async () => {
const values = [
'0',
'9'.repeat(400),
'not-a-date',
new Date(0).toUTCString(),
]
for (const value of values) {
const server = await mockServer([{
kind: 'http-error',
status: 429,
body: JSON.stringify({ error: { message: 'retry later' } }),
headers: { 'retry-after': value },
}])
const ctx = await harness(server.url)
let thrown: LlmError | undefined
try {
await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
} catch (error: unknown) {
if (error instanceof LlmError) thrown = error
}
expect(thrown?.failure).toEqual({ message: 'retry later', code: 'RATE_LIMIT', status: 429 })
}
})
it('classifies only context-capacity HTTP 400 details as context overflow', () => {
expect(httpErrorCode(400, { message: 'request too large for model context' }))
.toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
@@ -210,6 +299,12 @@ describe('DeepSeekAdapter against a mock server', () => {
expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413')
})
it('distinguishes terminal quota exhaustion from transient HTTP 429 throttling', () => {
expect(httpErrorCode(429, { code: 'insufficient_quota', message: 'account credits exhausted' }))
.toBe(QUOTA_EXCEEDED_CODE)
expect(httpErrorCode(429, { message: 'request rate limit exceeded' })).toBe('RATE_LIMIT')
})
it('keeps the status-line message for JSON error bodies without a message', async () => {
const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }])
const ctx = await harness(server.url)
@@ -272,7 +367,76 @@ describe('DeepSeekAdapter against a mock server', () => {
})()
setTimeout(() => { controller.abort() }, 30)
await expect(pending).rejects.toThrow()
await expect(pending).rejects.toMatchObject({ code: 'ABORTED' })
})
it('maps connection failures to TRANSPORT without losing the cause', async () => {
const cause = new TypeError('connection refused')
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockRejectedValue(cause)
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' })
try {
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
}
await expect(drain()).rejects.toMatchObject({ code: 'TRANSPORT', cause })
} finally {
fetchSpy.mockRestore()
}
})
it('renders a non-Error transport rejection without losing its cause', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation(() => {
const failed = Promise.withResolvers<Response>()
failed.reject('offline')
return failed.promise
})
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' })
try {
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
}
await expect(drain()).rejects.toMatchObject({
message: 'DeepSeek transport failed: offline',
code: 'TRANSPORT',
cause: 'offline',
})
} finally {
fetchSpy.mockRestore()
}
})
it('aborts the underlying body when the stream stays idle past its watchdog', async () => {
vi.useFakeTimers()
let stopped = false
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation((_input, init) => {
const signal = init?.signal
const body = new ReadableStream<Uint8Array>({
start(controller) {
signal?.addEventListener('abort', () => {
stopped = true
controller.error(signal.reason)
}, { once: true })
},
})
return Promise.resolve(new Response(body, { status: 200 }))
})
const adapter = new DeepSeekAdapter({
apiKey: 'k',
baseURL: 'https://example.invalid',
streamIdleTimeoutMs: 100,
})
try {
const drain = (async () => {
for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ }
})()
const rejected = expect(drain).rejects.toMatchObject({ code: 'TIMEOUT' })
await vi.advanceTimersByTimeAsync(0)
await vi.advanceTimersByTimeAsync(100)
await rejected
expect(stopped).toBe(true)
} finally {
fetchSpy.mockRestore()
}
})
})
@@ -419,4 +583,30 @@ describe('plugin registration and config', () => {
expect(adapter).toBeInstanceOf(DeepSeekAdapter)
await expect(adapter.listModels('deepseek')).resolves.toEqual([])
})
it('rejects invalid idle watchdog bounds for direct and plugin composition', async () => {
expect(() => new DeepSeekAdapter({
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
streamIdleTimeoutMs: Number.POSITIVE_INFINITY,
})).toThrow(/streamIdleTimeoutMs.*positive finite/)
expect(() => new DeepSeekAdapter({
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
})).toThrow(/streamIdleTimeoutMs.*no greater/)
const ctx = new Context()
await ctx.plugin(LlmService)
await expect(ctx.plugin(LlmDeepSeek, {
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
streamIdleTimeoutMs: 0,
})).rejects.toThrow(/streamIdleTimeoutMs/)
await expect(ctx.plugin(LlmDeepSeek, {
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
})).rejects.toThrow(/streamIdleTimeoutMs/)
})
})

View File

@@ -232,8 +232,7 @@ describe('mapFinishReason', () => {
(wire) => {
expect(mapFinishReason(wire)).toEqual({
kind: 'error',
message: `model stopped: ${wire}`,
code: wire.toUpperCase(),
failure: { message: `model stopped: ${wire}`, code: wire.toUpperCase() },
})
},
)

View File

@@ -19,6 +19,9 @@
},
{
"path": "../../llm/llm"
},
{
"path": "../../util/timeout"
}
]
}

View File

@@ -19,7 +19,7 @@ Configure credentials and deployment-specific transport settings per provider. O
reasoning: high
- provider: anthropic
apiKey: !!js process.env.ANTHROPIC_API_KEY
maxRetries: 2
streamIdleTimeoutMs: 300000
- provider: openrouter
apiKey: !!js process.env.OPENROUTER_API_KEY
headers:
@@ -30,7 +30,9 @@ Each provider name must exist in pi-ai's installed catalog and may appear only o
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry.
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `maxRetries`, and `maxRetryDelayMs`. They map to pi-ai's common stream options. Harness app attribution wins a conflicting configured header name.
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
## Provider/model routing and replay
@@ -43,7 +45,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
## Vocabulary differences
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks. Provider-specific error text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
@@ -57,7 +59,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
## Testing
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`.
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`.
## Model Experience
@@ -95,3 +97,4 @@ Recorded response content appends to the next request and does not invalidate it
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes.
- **Retry policy is not an adapter option** — SDK retries are disabled so durable agent steps and `llm/retry` events own every visible attempt; direct `ctx.llm.stream()` calls remain single-attempt.

View File

@@ -23,6 +23,7 @@
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-timeout": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
@@ -32,6 +33,7 @@
"devDependencies": {
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-llm-deepseek": "workspace:^",
"@deepseek-ai/dsh-timeout": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -16,7 +16,9 @@ import type {
} from '@earendil-works/pi-ai'
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { PiAiProviderProfile } from './config.ts'
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
import { resolveProfiles } from './config.ts'
import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
import { toPiContext } from './context.ts'
import { toStreamChunks } from './stream.ts'
@@ -48,8 +50,8 @@ function profileOptions(profile: PiAiProviderProfile): SimpleStreamOptions {
...profile.transport === undefined ? {} : { transport: profile.transport },
...profile.timeoutMs === undefined ? {} : { timeoutMs: profile.timeoutMs },
...profile.websocketConnectTimeoutMs === undefined ? {} : { websocketConnectTimeoutMs: profile.websocketConnectTimeoutMs },
...profile.maxRetries === undefined ? {} : { maxRetries: profile.maxRetries },
...profile.maxRetryDelayMs === undefined ? {} : { maxRetryDelayMs: profile.maxRetryDelayMs },
// The agent recovery layer owns visible attempts; one adapter call is one SDK attempt.
maxRetries: 0,
}
}
@@ -68,11 +70,11 @@ function requestHeaders(headers: Readonly<Record<string, string>> | undefined):
* request, so models need not be registered during the Cordis lifecycle.
*/
export class PiAiAdapter extends LlmAdapter {
private readonly profiles: ReadonlyMap<string, PiAiProviderProfile>
private readonly profiles: ReadonlyMap<string, ResolvedPiAiProviderProfile>
constructor(options: PiAiAdapterOptions) {
super()
this.profiles = new Map(options.profiles.map(profile => [profile.provider, profile]))
this.profiles = new Map(resolveProfiles(options.profiles).map(profile => [profile.provider, profile]))
}
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
@@ -97,12 +99,12 @@ export class PiAiAdapter extends LlmAdapter {
}
const model = resolveModel(profile, options.model)
// Pi-ai has no iterator-return cancellation hook. Chain an internal signal
// and abort it when this generator exits so early consumers stop the HTTP stream.
const controller = new AbortController()
const onCallerAbort = (): void => { controller.abort(options.signal?.reason) }
if (options.signal?.aborted) controller.abort(options.signal.reason)
else options.signal?.addEventListener('abort', onCallerAbort, { once: true })
const consumer = new AbortController()
const upstream = options.signal === undefined
? consumer.signal
: AbortSignal.any([options.signal, consumer.signal])
const streamIdleTimeoutMs = profile.streamIdleTimeoutMs
using watchdog = idleWatchdog(upstream, streamIdleTimeoutMs, 'LLM_STREAM_IDLE_TIMEOUT')
try {
const events = streamSimple(model, toPiContext(options), {
@@ -110,15 +112,44 @@ export class PiAiAdapter extends LlmAdapter {
...options.temperature === undefined ? {} : { temperature: options.temperature },
...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },
...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) },
signal: controller.signal,
signal: watchdog.signal,
// Profile headers are deployment-owned; attribution names are
// Harness-owned and therefore win collisions.
headers: requestHeaders(profile.headers),
})
yield* toStreamChunks(events, model.contextWindow)
const iterator = toStreamChunks(events, model.contextWindow)[Symbol.asyncIterator]()
let exhausted = false
try {
while (true) {
const result = await watchdog.next(iterator)
const timeout = timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT')
if (timeout !== undefined) throw timeout
if (result.done) {
exhausted = true
return
}
yield result.value
}
} finally {
if (!exhausted) {
consumer.abort('pi-ai stream consumer stopped')
try {
await iterator.return(undefined)
} catch (_abortedSdkTeardown) {
// The stable signal already owns SDK termination; return-time abort cannot add an outcome.
}
}
}
} catch (error: unknown) {
if (timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT') !== undefined) {
throw new LlmError(`pi-ai stream idle timeout after ${streamIdleTimeoutMs}ms`, 'TIMEOUT', { cause: error })
}
if (options.signal?.aborted) {
throw new LlmError('pi-ai request aborted by caller', 'ABORTED', { cause: error })
}
throw error
} finally {
options.signal?.removeEventListener('abort', onCallerAbort)
controller.abort('consumer stopped streaming')
consumer.abort('pi-ai stream consumer stopped')
}
}
}

View File

@@ -7,6 +7,10 @@
import { getProviders } from '@earendil-works/pi-ai'
import type { CacheRetention, ThinkingBudgets, ThinkingLevel, Transport } from '@earendil-works/pi-ai'
import z from 'schemastery'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
/** Default maximum idle interval while an adapter stream read is outstanding. */
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
/** Configuration for one pi-ai provider route. */
export interface PiAiProviderProfile {
@@ -30,10 +34,14 @@ export interface PiAiProviderProfile {
timeoutMs?: number
/** WebSocket connection timeout in milliseconds. */
websocketConnectTimeoutMs?: number
/** Provider SDK retry count. */
maxRetries?: number
/** Maximum provider-requested retry delay in milliseconds. */
maxRetryDelayMs?: number
/** Maximum provider idle time while one stream read is outstanding. */
streamIdleTimeoutMs?: number
}
/** Validated profile with every adapter-owned default resolved. */
export interface ResolvedPiAiProviderProfile extends PiAiProviderProfile {
/** Positive finite provider-idle interval after defaulting. */
streamIdleTimeoutMs: number
}
/** Plugin configuration: the non-empty provider profiles this instance owns. */
@@ -60,8 +68,7 @@ const profile = z.object({
transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']),
timeoutMs: z.natural(),
websocketConnectTimeoutMs: z.natural(),
maxRetries: z.natural(),
maxRetryDelayMs: z.natural(),
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
})
/** Runtime schema for {@link Config}. */
@@ -75,11 +82,18 @@ export const Config: z<Config> = z.object({
* @param profiles - configured provider profiles.
* @returns validated profiles in configuration order.
*/
export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiProviderProfile[] {
export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): ResolvedPiAiProviderProfile[] {
if (profiles.length === 0) throw new Error('llm-pi-ai: providers must contain at least one profile')
const supported = new Set<string>(getProviders())
const seen = new Set<string>()
return profiles.map((source) => {
const legacy = source as PiAiProviderProfile & {
maxRetries?: unknown
maxRetryDelayMs?: unknown
}
if ('maxRetries' in legacy || 'maxRetryDelayMs' in legacy) {
throw new Error('llm-pi-ai: maxRetries and maxRetryDelayMs were removed; compose agent recovery with dsh-llm-retry')
}
if (source.provider.length === 0) throw new Error('llm-pi-ai: provider names must be non-empty')
if (!supported.has(source.provider)) throw new Error(`llm-pi-ai: unknown pi-ai provider "${source.provider}"`)
if (seen.has(source.provider)) throw new Error(`llm-pi-ai: duplicate provider profile "${source.provider}"`)
@@ -89,9 +103,18 @@ export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiP
if (source.baseURL !== undefined && source.baseURL.length === 0) {
throw new Error(`llm-pi-ai: provider "${source.provider}" has an empty baseURL`)
}
const streamIdleTimeoutMs = source.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
if (!Number.isFinite(streamIdleTimeoutMs)
|| streamIdleTimeoutMs <= 0
|| streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) {
throw new Error(
`llm-pi-ai: provider "${source.provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
)
}
seen.add(source.provider)
return {
...source,
streamIdleTimeoutMs,
...source.headers === undefined ? {} : { headers: { ...source.headers } },
...source.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...source.thinkingBudgets } },
}

View File

@@ -8,7 +8,7 @@
* @module dsh-llm-pi-ai/stream
*/
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm'
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
import { isContextOverflow } from '@earendil-works/pi-ai'
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
@@ -30,9 +30,12 @@ export function mapUsage(usage: PiUsage): TokenUsage {
function classifyPiAiError(message: string): string {
if (/\b(?:401|403)\b/.test(message)) return 'AUTH'
if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE
if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT'
if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST'
if (/\b5\d\d\b/.test(message)) return 'SERVER'
if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT'
if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)) return 'TRANSPORT'
return 'PI_AI_ERROR'
}
@@ -52,8 +55,10 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
if (piAiOverflow || harnessOverflow) {
return {
kind: 'error',
message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`,
code: CONTEXT_WINDOW_EXCEEDED_CODE,
failure: {
message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`,
code: CONTEXT_WINDOW_EXCEEDED_CODE,
},
}
}
@@ -61,10 +66,13 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number)
case 'stop': return { kind: 'stop' }
case 'length': return { kind: 'max-tokens' }
case 'toolUse': return { kind: 'tool-calls' }
case 'aborted': return { kind: 'aborted' }
case 'aborted': return {
kind: 'aborted',
failure: { message: message.errorMessage ?? 'pi-ai stream aborted', code: 'ABORTED' },
}
case 'error': {
const text = message.errorMessage ?? 'pi-ai stream error'
return { kind: 'error', message: text, code: classifyPiAiError(text) }
return { kind: 'error', failure: { message: text, code: classifyPiAiError(text) } }
}
}
}

View File

@@ -6,6 +6,7 @@ import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
import { getModels } from '@earendil-works/pi-ai'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
import { resolveProfiles } from '../src/config.ts'
import { assemble } from './assemble.ts'
@@ -14,6 +15,8 @@ interface MockServer {
paths: string[]
requests: unknown[]
headers: IncomingMessage['headers'][]
readonly closedResponses: number
responseClosed: Promise<void>
}
const servers: Server[] = []
@@ -23,11 +26,23 @@ afterEach(async () => {
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
})
async function mockServer(script: { status?: number; events?: string[]; body?: string; delayMs?: number }[]): Promise<MockServer> {
async function mockServer(script: {
status?: number
events?: string[]
body?: string
delayMs?: number
headers?: Record<string, string>
}[]): Promise<MockServer> {
const paths: string[] = []
const requests: unknown[] = []
const headers: IncomingMessage['headers'][] = []
let closedResponses = 0
const responseClosed = Promise.withResolvers<undefined>()
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
response.on('close', () => {
closedResponses += 1
responseClosed.resolve(undefined)
})
let body = ''
request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') })
request.on('end', () => {
@@ -36,7 +51,7 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s
headers.push(request.headers)
const behavior = script.shift() ?? { status: 500, body: 'script exhausted' }
if (behavior.status !== undefined && behavior.status !== 200) {
response.writeHead(behavior.status, { 'content-type': 'application/json' })
response.writeHead(behavior.status, { 'content-type': 'application/json', ...behavior.headers })
response.end(behavior.body ?? '{}')
return
}
@@ -56,7 +71,14 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
const address = server.address()
if (address === null || typeof address === 'string') throw new Error('no port')
return { url: `http://127.0.0.1:${address.port}`, paths, requests, headers }
return {
url: `http://127.0.0.1:${address.port}`,
paths,
requests,
headers,
responseClosed: responseClosed.promise,
get closedResponses() { return closedResponses },
}
}
const textEvents = [
@@ -107,8 +129,7 @@ describe('PiAiAdapter provider routing', () => {
transport: 'sse',
timeoutMs: 5000,
websocketConnectTimeoutMs: 3000,
maxRetries: 0,
maxRetryDelayMs: 10,
streamIdleTimeoutMs: 10_000,
thinkingBudgets: { high: 2048 },
})
await assemble(ctx, {
@@ -161,13 +182,35 @@ describe('PiAiAdapter provider routing', () => {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(LlmPiAi, {
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1`, maxRetries: 0 }],
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }],
})
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
expect(result.finish.kind).toBe('error')
expect(server.paths).toEqual(['/v1/responses'])
})
it('forces one wire request for an SDK-retryable provider failure', async () => {
const server = await mockServer([
{
status: 429,
headers: { 'retry-after-ms': '1' },
body: JSON.stringify({ error: { message: 'retryable provider failure' } }),
},
{ status: 500, body: JSON.stringify({ error: { message: 'hidden SDK retry' } }) },
{ status: 500, body: JSON.stringify({ error: { message: 'second hidden SDK retry' } }) },
])
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(LlmPiAi, {
providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }],
})
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] })
expect(result.finish).toMatchObject({ kind: 'error' })
expect(server.paths).toEqual(['/v1/responses'])
})
it('uses OpenAI Responses against an Azure project v1 path with its API key header', async () => {
const server = await mockServer([{ status: 401, body: JSON.stringify({ error: { message: 'expected mock failure' } }) }])
const ctx = new Context()
@@ -178,7 +221,6 @@ describe('PiAiAdapter provider routing', () => {
apiKey: 'test-key',
baseURL: `${server.url}/api/projects/openai/openai/v1`,
headers: { 'api-key': 'test-key', Authorization: '' },
maxRetries: 0,
}],
})
const result = await assemble(ctx, { provider: 'openai', model: 'gpt-5.5', messages: [] })
@@ -195,9 +237,10 @@ describe('PiAiAdapter provider routing', () => {
[500, 'SERVER'],
] as const)('maps HTTP %s failures to %s', async (status, code) => {
const server = await mockServer([{ status, body: JSON.stringify({ error: { message: `provider ${status}` } }) }])
const ctx = await harness(server.url, { maxRetries: 0 })
const ctx = await harness(server.url)
const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
expect(result.finish).toMatchObject({ kind: 'error', code })
expect(result.finish).toMatchObject({ kind: 'error', failure: { code } })
expect(server.paths).toEqual(['/chat/completions'])
})
it('uses the resolved catalog context window for usage-based overflow detection', async () => {
@@ -218,10 +261,29 @@ describe('PiAiAdapter provider routing', () => {
expect(result.finish).toEqual({
kind: 'error',
message: `pi-ai detected context overflow for model "${model.id}"`,
code: CONTEXT_WINDOW_EXCEEDED_CODE,
failure: {
message: `pi-ai detected context overflow for model "${model.id}"`,
code: CONTEXT_WINDOW_EXCEEDED_CODE,
},
})
})
it('stops the SDK request when the adapter idle watchdog expires', async () => {
const server = await mockServer([{ events: textEvents, delayMs: 200 }])
const ctx = await harness(server.url, { streamIdleTimeoutMs: 20 })
await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }))
.rejects.toMatchObject({ code: 'TIMEOUT' })
await Promise.race([
server.responseClosed,
new Promise<never>((_resolve, reject) => {
setTimeout(() => { reject(new Error('SDK request did not close after idle timeout')) }, 100)
}),
])
expect(server.paths).toEqual(['/chat/completions'])
expect(server.closedResponses).toBe(1)
})
})
describe('provider profile lifecycle', () => {
@@ -280,16 +342,31 @@ describe('provider profile lifecycle', () => {
expect(() => resolveProfiles([{ provider: 'openai', baseURL: '' }])).toThrow(/empty baseURL/)
})
it('rejects negative or fractional stream tunables at schema validation', () => {
it.each(['maxRetries', 'maxRetryDelayMs'] as const)(
'rejects removed profile field %s instead of silently restoring hidden SDK retries',
async (field) => {
const legacy = { provider: 'openai', [field]: 2 }
expect(() => resolveProfiles([legacy as never])).toThrow(/removed.*agent recovery/i)
const ctx = new Context()
await ctx.plugin(LlmService)
await expect(ctx.plugin(LlmPiAi, { providers: [legacy as never] }))
.rejects.toThrow(/removed.*agent recovery/i)
},
)
it('rejects invalid stream tunables at plugin load', async () => {
const invalid = [
{ timeoutMs: -1 },
{ websocketConnectTimeoutMs: -1 },
{ maxRetries: -1 },
{ maxRetries: 0.5 },
{ maxRetryDelayMs: -1 },
{ streamIdleTimeoutMs: 0 },
{ streamIdleTimeoutMs: Number.NaN },
{ streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 },
]
for (const entry of invalid) {
expect(() => new LlmPiAi.Config({ providers: [{ provider: 'openai', ...entry }] })).toThrow()
const ctx = new Context()
await ctx.plugin(LlmService)
await expect(ctx.plugin(LlmPiAi, { providers: [{ provider: 'openai', ...entry }] }))
.rejects.toThrow()
}
})
@@ -301,11 +378,59 @@ describe('provider profile lifecycle', () => {
})()).rejects.toMatchObject({ code: 'NO_ADAPTER' })
expect(new LlmError('x', 'X')).toBeInstanceOf(Error)
})
it('validates direct-constructor profiles at the embedding boundary', () => {
expect(() => new PiAiAdapter({
profiles: [{ provider: 'openai', streamIdleTimeoutMs: 0 }],
})).toThrow(/streamIdleTimeoutMs.*positive finite/)
expect(() => new PiAiAdapter({
profiles: [{ provider: 'openai', streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 }],
})).toThrow(/streamIdleTimeoutMs.*no greater/)
})
})
describe('abort wiring', () => {
it('preserves an unknown pre-dispatch adapter Error exactly', async () => {
const original = new Error('SDK context conversion exploded')
const message = Object.defineProperty({}, 'role', {
get() { throw original },
})
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({
provider: 'deepseek',
model: 'deepseek-v4-flash',
messages: [message as never],
})) { /* drain */ }
}
await expect(drain()).rejects.toBe(original)
})
it('lets a concurrent caller abort classify a pre-dispatch adapter failure', async () => {
const controller = new AbortController()
const original = new Error('conversion lost its caller')
const message = Object.defineProperty({}, 'role', {
get() {
controller.abort('caller cancelled during conversion')
throw original
},
})
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({
provider: 'deepseek',
model: 'deepseek-v4-flash',
messages: [message as never],
signal: controller.signal,
})) { /* drain */ }
}
await expect(drain()).rejects.toMatchObject({ code: 'ABORTED', cause: original })
})
it('resolves catalog endpoints without an override before honoring pre-abort', async () => {
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key', maxRetries: 0 }] })
const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] })
const controller = new AbortController()
controller.abort('already stopped')
const chunks = []

View File

@@ -485,20 +485,32 @@ describe('toStreamChunks', () => {
)))
expect(chunks).toEqual([
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 0 } },
{ type: 'finish', reason: { kind: 'error', message: 'boom', code: 'PI_AI_ERROR' } },
{ type: 'finish', reason: { kind: 'error', failure: { message: 'boom', code: 'PI_AI_ERROR' } } },
])
})
it('maps aborted error events to aborted finish', async () => {
const error = assistant({ stopReason: 'aborted' })
const chunks = await collect(toStreamChunks(feed({ type: 'error', reason: 'aborted', error })))
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'aborted' } })
expect(chunks.at(-1)).toEqual({
type: 'finish',
reason: { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } },
})
})
it('rejects a stream that ends without done or error', async () => {
await expect(collect(toStreamChunks(feed({ type: 'start', partial: assistant() }))))
.rejects.toThrow(/without done\/error/)
})
it('preserves an unknown SDK iterator Error exactly', async () => {
const original = Object.assign(new Error('SDK transport exploded'), { code: 'ECONNRESET' })
async function* failedSdkStream(): AsyncGenerator<AssistantMessageEvent> {
throw original
}
await expect(collect(toStreamChunks(failedSdkStream()))).rejects.toBe(original)
})
})
describe('mapStopReason / mapUsage', () => {
@@ -506,46 +518,52 @@ describe('mapStopReason / mapUsage', () => {
['stop', { kind: 'stop' }],
['length', { kind: 'max-tokens' }],
['toolUse', { kind: 'tool-calls' }],
['aborted', { kind: 'aborted' }],
['aborted', { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } }],
] as const)('maps %s', (stopReason, expected) => {
expect(mapStopReason(assistant({ stopReason }))).toEqual(expected)
})
it('defaults the error message when pi-ai omits it', () => {
expect(mapStopReason(assistant({ stopReason: 'error' })))
.toEqual({ kind: 'error', message: 'pi-ai stream error', code: 'PI_AI_ERROR' })
.toEqual({ kind: 'error', failure: { message: 'pi-ai stream error', code: 'PI_AI_ERROR' } })
})
it('maps routable HTTP-ish error messages to stable codes', () => {
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 401: bad key' })))
.toMatchObject({ kind: 'error', code: 'AUTH' })
.toMatchObject({ kind: 'error', failure: { code: 'AUTH' } })
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: rate limit' })))
.toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
.toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } })
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: insufficient_quota' })))
.toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } })
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' })))
.toMatchObject({ kind: 'error', code: 'SERVER' })
.toMatchObject({ kind: 'error', failure: { code: 'SERVER' } })
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'provider timed out' })))
.toMatchObject({ kind: 'error', failure: { code: 'TIMEOUT' } })
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'ECONNRESET socket closed' })))
.toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } })
expect(mapStopReason(assistant({
stopReason: 'error',
errorMessage: 'HTTP 400: input exceeds the model context window limit',
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
expect(mapStopReason(assistant({
stopReason: 'error',
errorMessage: 'HTTP 400: request too large for model context',
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
expect(mapStopReason(assistant({
stopReason: 'error',
errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value',
}))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' })
}))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } })
})
it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => {
expect(mapStopReason(assistant({
stopReason: 'error',
errorMessage: 'prompt is too long: 213462 tokens > 200000 maximum',
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
}))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } })
expect(mapStopReason(assistant({
stopReason: 'error',
errorMessage: 'ThrottlingException: Too many tokens, rate limit reached',
}))).toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
}))).toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } })
})
it('uses the resolved context window for silent and length-stop overflows', () => {
@@ -553,15 +571,17 @@ describe('mapStopReason / mapUsage', () => {
expect(mapStopReason(silent)).toEqual({ kind: 'stop' })
expect(mapStopReason(silent, 100)).toEqual({
kind: 'error',
message: 'pi-ai detected context overflow for model "deepseek-v4-flash"',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
failure: {
message: 'pi-ai detected context overflow for model "deepseek-v4-flash"',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
},
})
const truncated = assistant({ stopReason: 'length', usage: usage(80, 0, 19) })
expect(mapStopReason(truncated)).toEqual({ kind: 'max-tokens' })
expect(mapStopReason(truncated, 100)).toMatchObject({
kind: 'error',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE },
})
})

View File

@@ -70,7 +70,7 @@ function textOf(result: AssembledResult): string {
function expectFinish(result: AssembledResult, expected: 'stop' | 'tool-calls'): void {
if (result.finish.kind === 'error') {
throw new Error(`provider request failed (${result.finish.code ?? 'unknown'}): ${result.finish.message}`)
throw new Error(`provider request failed (${result.finish.failure.code}): ${result.finish.failure.message}`)
}
expect(result.finish.kind).toBe(expected)
}

View File

@@ -0,0 +1,35 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
const streamSimple = vi.hoisted(() => vi.fn())
vi.mock('@earendil-works/pi-ai', async (importOriginal) => {
const actual = await importOriginal<typeof import('@earendil-works/pi-ai')>()
return { ...actual, streamSimple }
})
import { PiAiAdapter } from '../src/adapter.ts'
afterEach(() => { streamSimple.mockReset() })
describe('pi-ai SDK retry boundary', () => {
it('pins one SDK attempt even when the installed provider currently defaults to zero retries', async () => {
const failure = new Error('mock SDK boundary')
streamSimple.mockReturnValue({
async * [Symbol.asyncIterator](): AsyncGenerator<never> {
throw failure
},
})
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai', apiKey: 'test-key' }] })
const drain = async (): Promise<void> => {
for await (const _chunk of adapter.stream({
provider: 'openai',
model: 'gpt-4.1',
messages: [],
})) { /* drain */ }
}
await expect(drain()).rejects.toBe(failure)
expect(streamSimple).toHaveBeenCalledOnce()
expect(streamSimple.mock.calls[0]?.[2]).toMatchObject({ maxRetries: 0 })
})
})

View File

@@ -19,6 +19,9 @@
},
{
"path": "../../llm/llm"
},
{
"path": "../../util/timeout"
}
]
}

View File

@@ -0,0 +1,39 @@
# `@deepseek-ai/dsh-llm-retry`
Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step.
The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid provider `retryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward.
```yaml
- name: '@deepseek-ai/dsh-llm-retry'
config:
maxTransientRetries: 2
initialDelayMs: 500
maxDelayMs: 10000
jitterRatio: 0.1
retryableCodes: [RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
```
## Model Experience
### Transient request recovery
#### What the model sees
No retry event, delay, or failure prose is model-visible. After a retry, the next numbered step reconstructs the same explicit provider/model request from durable session history; failed chunks never enter derived messages.
#### Token effect
Each retry is a new provider request and may repeat input-token billing. The finite budget caps attempts; `llm/retry` itself contributes no tokens.
#### KV Cache effect
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface status event does not change cache identity.
## Known Limitations and Deferred Work
- **Agent steps are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
- **Finite plugin budgets add** — this policy counts only configured transient codes; context-overflow compaction counts only its own code. A future policy with overlapping codes must document and test registration-order behavior.
- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation.

View File

@@ -0,0 +1,47 @@
{
"name": "@deepseek-ai/dsh-llm-retry",
"description": "Bounded transient LLM request retry policy for the DeepSeek Harness",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-agent": "^0.0.1",
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-timeout": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@cordisjs/plugin-include": "workspace:^",
"@cordisjs/plugin-loader": "workspace:^",
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-agent-loop": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
"@deepseek-ai/dsh-session-persistence-sqlite": "workspace:^",
"@deepseek-ai/dsh-system-prompt": "workspace:^",
"@deepseek-ai/dsh-timeout": "workspace:^",
"@deepseek-ai/dsh-tools": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,211 @@
/**
* Bounded transient model-request retry policy on the agent loop's closed-step
* recovery seam. Each scheduled retry is durable before its cancellable wait.
*
* @module @deepseek-ai/dsh-llm-retry
*/
import type { Context } from 'cordis'
import z from 'schemastery'
import type { Agent, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
import type {} from '@deepseek-ai/dsh-session'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
declare module '@deepseek-ai/dsh-session' {
interface SessionEventMap {
/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
'llm/retry': {
turn: number
step: number
retry: number
maxRetries: number
delayMs: number
failure: LlmFailure
}
}
}
export const name = 'llm-retry'
export const inject = ['agents']
const DEFAULT_MAX_TRANSIENT_RETRIES = 2
const DEFAULT_INITIAL_DELAY_MS = 500
const DEFAULT_MAX_DELAY_MS = 10_000
const DEFAULT_JITTER_RATIO = 0.1
const DEFAULT_RETRYABLE_CODES = Object.freeze(['RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
/** Deployment-owned limits and classification for transient request recovery. */
export interface Config {
/** Maximum transient retries after the first request (default 2). */
maxTransientRetries?: number
/** Initial local exponential-backoff delay in milliseconds (default 500). */
initialDelayMs?: number
/** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
maxDelayMs?: number
/** Symmetric random multiplier range around one (default 0.1). */
jitterRatio?: number
/** Stable failure codes eligible for this policy. */
retryableCodes?: string[]
}
/** Runtime schema for {@link Config}. */
export const Config: z<Config> = z.object({
maxTransientRetries: z.number().step(1).min(0).default(DEFAULT_MAX_TRANSIENT_RETRIES),
initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS),
maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS),
jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO),
retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]),
})
interface ResolvedConfig {
readonly maxTransientRetries: number
readonly initialDelayMs: number
readonly maxDelayMs: number
readonly jitterRatio: number
readonly retryableCodes: ReadonlySet<string>
}
function resolveConfig(config: Config): ResolvedConfig {
const maxTransientRetries = config.maxTransientRetries ?? DEFAULT_MAX_TRANSIENT_RETRIES
const initialDelayMs = config.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS
const maxDelayMs = config.maxDelayMs ?? DEFAULT_MAX_DELAY_MS
const jitterRatio = config.jitterRatio ?? DEFAULT_JITTER_RATIO
const codes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES]
if (!Number.isInteger(maxTransientRetries) || maxTransientRetries < 0) {
throw new Error('llm-retry: maxTransientRetries must be a non-negative integer')
}
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) {
throw new Error(`llm-retry: initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
}
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) {
throw new Error(`llm-retry: maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
}
if (initialDelayMs > maxDelayMs) {
throw new Error('llm-retry: initialDelayMs must be less than or equal to maxDelayMs')
}
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) {
throw new Error('llm-retry: jitterRatio must be between 0 and 1')
}
if (codes.length === 0) {
throw new Error('llm-retry: retryableCodes must not be empty')
}
if (codes.some(code => code.length === 0)) {
throw new Error('llm-retry: retryableCodes must contain only non-empty strings')
}
if (new Set(codes).size !== codes.length) {
throw new Error('llm-retry: retryableCodes must not contain duplicates')
}
return Object.freeze({
maxTransientRetries,
initialDelayMs,
maxDelayMs,
jitterRatio,
retryableCodes: new Set(codes),
})
}
/** Non-serializable seams used to make timing policy deterministic in tests. */
export interface RetryInternals {
/** Random sample in the inclusive zero-to-one range used for jitter. */
random?: () => number
}
function localDelay(config: ResolvedConfig, retry: number, random: () => number): number {
const exponent = Math.min(retry - 1, 1024)
const exponential = Math.min(config.initialDelayMs * 2 ** exponent, config.maxDelayMs)
const jitter = 1 - config.jitterRatio + 2 * config.jitterRatio * random()
return Math.min(exponential * jitter, config.maxDelayMs)
}
function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean> {
if (signal.aborted) return Promise.resolve(false)
return new Promise((resolve) => {
const timer = setTimeout(() => {
signal.removeEventListener('abort', onAbort)
resolve(true)
}, delayMs)
function onAbort(): void {
clearTimeout(timer)
resolve(false)
}
signal.addEventListener('abort', onAbort, { once: true })
})
}
/**
* Install bounded transient request recovery.
* @param ctx - plugin context that owns the listener and active waits.
* @param config - retry budget, delay bounds, jitter, and eligible codes.
* @param internals - non-serializable deterministic seams for tests.
*/
export function apply(ctx: Context, config: Config = {}, internals: RetryInternals = {}): void {
const resolved = resolveConfig(config)
const random = internals.random ?? Math.random
const lifetime = new AbortController()
const active = new Set<Promise<RequestErrorDecision>>()
async function backoff(
agent: Agent,
turn: number,
step: number,
failure: LlmFailure,
retry: number,
delayMs: number,
signal: AbortSignal,
): Promise<RequestErrorDecision> {
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
if (fusedSignal.aborted) return { action: 'fail' }
agent.session.append('llm/retry', {
turn,
step,
retry,
maxRetries: resolved.maxTransientRetries,
delayMs,
failure,
})
if (!await cancellableDelay(delayMs, fusedSignal)) return { action: 'fail' }
return { action: 'retry' }
}
const disposeListener = ctx.on('agent/request-error', (
agent: Agent,
turn: number,
step: number,
_error: RequestError,
failure: LlmFailure,
priorFailures: readonly LlmFailure[],
signal: AbortSignal,
next: () => Promise<RequestErrorDecision>,
) => {
// A waterfall may have captured this callback before its registration was
// removed. Lifetime cancellation must prevent that stale callback from
// entering a downstream policy after disposal.
if (lifetime.signal.aborted) return Promise.resolve<RequestErrorDecision>({ action: 'fail' })
if (!resolved.retryableCodes.has(failure.code)) return next()
const priorTransientFailures = priorFailures.filter(item => resolved.retryableCodes.has(item.code)).length
if (priorTransientFailures >= resolved.maxTransientRetries) return next()
const retry = priorTransientFailures + 1
let delayMs: number
if (failure.retryAfterMs !== undefined && Number.isFinite(failure.retryAfterMs) && failure.retryAfterMs > 0) {
if (failure.retryAfterMs > resolved.maxDelayMs) return next()
delayMs = failure.retryAfterMs
} else {
delayMs = localDelay(resolved, retry, random)
}
const tracked = backoff(agent, turn, step, failure, retry, delayMs, signal)
.finally(() => active.delete(tracked))
active.add(tracked)
return tracked
})
ctx.effect(() => async () => {
disposeListener()
lifetime.abort(new Error('llm-retry plugin disposed'))
await Promise.allSettled([...active])
}, 'llm-retry: abort and drain backoffs')
}

View File

@@ -0,0 +1,124 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { pathToFileURL } from 'node:url'
import { afterEach, describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import Loader from '@cordisjs/plugin-loader'
import Include from '@cordisjs/plugin-include'
import AgentRegistry from '@deepseek-ai/dsh-agent'
import type { Agent } from '@deepseek-ai/dsh-agent'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
import LlmService, { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry from '@deepseek-ai/dsh-tools'
import * as retry from '../src/index.ts'
let root: string | undefined
let context: Context | undefined
class TransientOnceAdapter extends LlmAdapter {
requests = 0
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
this.requests += 1
if (this.requests === 1) throw new LlmError('temporary outage', 'SERVER')
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'text-delta', index: 0, text: 'recovered' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } }
yield { type: 'finish', reason: { kind: 'stop' } }
}
}
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
return new Promise((resolve) => {
const dispose = ctx.on('agent/status', (subject, status) => {
if (subject === agent && status === 'idle') {
dispose()
resolve()
}
})
})
}
afterEach(async () => {
await context?.fiber.dispose()
context = undefined
if (root !== undefined) await rm(root, { recursive: true, force: true })
root = undefined
})
async function loadYaml(lines: readonly string[]): Promise<Context> {
root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-loader-'))
const configPath = join(root, 'cordis.yml')
await writeFile(configPath, [...lines, ''].join('\n'))
context = new Context()
context.baseUrl = pathToFileURL(root).href + '/'
await context.plugin(Loader)
context.loader.builtins.include = Include
const modules = new Map<string, unknown>([
['@deepseek-ai/dsh-llm', LlmService],
['@deepseek-ai/dsh-session', SessionStore],
['@deepseek-ai/dsh-system-prompt', SystemPrompt],
['@deepseek-ai/dsh-tools', ToolRegistry],
['@deepseek-ai/dsh-agent', AgentRegistry],
['@deepseek-ai/dsh-llm-retry', retry],
['@deepseek-ai/dsh-agent-loop', AgentLoop],
])
context.loader.internal = {
version: 'v2',
async import(specifier: string) {
if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`)
return modules.get(specifier)
},
} as unknown as NonNullable<typeof context.loader.internal>
await context.loader.create({
name: 'cordis:include',
config: { path: pathToFileURL(configPath).href },
})
await context.loader.await()
return context
}
describe('real Loader composition', () => {
it('loads the flat policy and records recovery through the shipping loop', async () => {
const loaded = await loadYaml([
"- name: '@deepseek-ai/dsh-llm'",
"- name: '@deepseek-ai/dsh-session'",
"- name: '@deepseek-ai/dsh-system-prompt'",
"- name: '@deepseek-ai/dsh-tools'",
"- name: '@deepseek-ai/dsh-agent'",
"- name: '@deepseek-ai/dsh-llm-retry'",
' config:',
' maxTransientRetries: 1',
' initialDelayMs: 1',
' maxDelayMs: 1',
' jitterRatio: 0',
' retryableCodes: [RATE_LIMIT, SERVER]',
"- name: '@deepseek-ai/dsh-agent-loop'",
])
const unloaded = [...loaded.loader.entries()]
.filter(entry => entry.fiber === undefined && !entry.disabled)
.map(entry => entry.options.name)
expect(unloaded).toEqual([])
expect(loaded.agents).toBeInstanceOf(AgentRegistry)
const adapter = new TransientOnceAdapter()
loaded.llm.registerAdapter(['mock'], adapter)
const agent = loaded.agentLoop.create(SessionId('loader-retry'), { provider: 'mock', model: 'mock' })
const idle = waitForIdle(loaded, agent)
agent.send([{ type: 'text', text: 'recover' }])
await idle
expect(adapter.requests).toBe(2)
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1)
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
role: 'assistant',
content: [{ type: 'text', text: 'recovered' }],
})
})
})

View File

@@ -0,0 +1,57 @@
import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
import SessionPersistenceSqlite from '@deepseek-ai/dsh-session-persistence-sqlite'
import type {} from '../src/index.ts'
const dirs: string[] = []
afterEach(async () => {
for (const dir of dirs.splice(0)) await rm(dir, { recursive: true, force: true })
})
async function backend(kind: 'jsonl' | 'sqlite'): Promise<Context> {
const ctx = new Context()
await ctx.plugin(SessionStore)
if (kind === 'jsonl') {
const root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-jsonl-'))
dirs.push(root)
await ctx.plugin(SessionPersistenceJsonl, { root })
} else {
await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
}
return ctx
}
describe.each(['jsonl', 'sqlite'] as const)('%s retry-event persistence', (kind) => {
it('round-trips the event losslessly without adding a model message', async () => {
const ctx = await backend(kind)
try {
const session = ctx.sessions.create(SessionId(`retry-${kind}`))
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('step/start', { turn: 1, step: 1 })
session.append('step/end', { turn: 1, step: 1 })
const event = session.append('llm/retry', {
turn: 1,
step: 1,
retry: 1,
maxRetries: 2,
delayMs: 750,
failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 },
})
session.append('turn/end', { turn: 1, reason: { kind: 'aborted', reason: 'cancelled in backoff' } })
expect(session.deriveMessages()).toEqual([])
await ctx.sessions.flush(session)
const loaded = await ctx.sessionPersistence.load(session.id)
expect(loaded.events.find(item => item.type === 'llm/retry')).toEqual(event)
} finally {
await ctx.fiber.dispose()
}
})
})

View File

@@ -0,0 +1,453 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import type { Fiber } from 'cordis'
import LlmService, { CallId, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
import AgentRegistry from '@deepseek-ai/dsh-agent'
import type { Agent, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
import * as retry from '../src/index.ts'
type ScriptEntry = Error | Iterable<StreamChunk> | AsyncIterable<StreamChunk>
class ScriptedAdapter extends LlmAdapter {
readonly requests: GenerateOptions[] = []
constructor(private readonly entries: ScriptEntry[]) {
super()
}
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
this.requests.push(options)
const entry = this.entries.shift()
if (entry === undefined) throw new Error('retry test script exhausted')
if (entry instanceof Error) throw entry
yield* entry
}
}
async function* partialToolFailure(error: Error): AsyncGenerator<StreamChunk> {
const id = CallId('discarded-call')
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'text-delta', index: 0, text: 'discarded partial output' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'discarded partial output' } }
yield { type: 'block-start', index: 1, blockType: 'tool-call' }
yield { type: 'tool-call-delta', index: 1, id, name: 'danger', argumentsDelta: '{}' }
yield { type: 'block-end', index: 1, block: { type: 'tool-call', id, name: 'danger', arguments: '{}' } }
throw error
}
function textResponse(text: string): StreamChunk[] {
return [
{ type: 'block-start', index: 0, blockType: 'text' },
{ type: 'text-delta', index: 0, text },
{ type: 'block-end', index: 0, block: { type: 'text', text } },
{ type: 'finish', reason: { kind: 'stop' } },
]
}
async function harness(
adapter: LlmAdapter,
config: retry.Config = {},
beforeRetry?: (ctx: Context) => void,
internals: retry.RetryInternals = {},
): Promise<{ ctx: Context; retryFiber: Fiber }> {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await ctx.plugin(AgentRegistry)
beforeRetry?.(ctx)
const resolvedConfig = Object.assign({
maxTransientRetries: 2,
initialDelayMs: 500,
maxDelayMs: 10_000,
jitterRatio: 0,
}, config)
const retryFiber = await ctx.plugin(Object.assign((inner: Context) => {
retry.apply(inner, resolvedConfig, internals)
}, { inject: retry.inject }))
await ctx.plugin(AgentLoop, { agents: [] })
ctx.llm.registerAdapter(['mock'], adapter)
return { ctx, retryFiber }
}
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
return new Promise((resolve) => {
const dispose = ctx.on('agent/status', (subject, status) => {
if (subject === agent && status === 'idle') {
dispose()
resolve()
}
})
})
}
function waitForRetry(ctx: Context, agent: Agent, retryNumber: number): Promise<Extract<SessionEvent, { type: 'llm/retry' }>> {
return new Promise((resolve) => {
const dispose = ctx.on('session/event', (session, event) => {
if (session === agent.session && event.type === 'llm/retry' && event.data.retry === retryNumber) {
dispose()
resolve(event)
}
})
})
}
let context: Context | undefined
afterEach(async () => {
vi.useRealTimers()
await context?.fiber.dispose()
context = undefined
})
describe('bounded transient retry policy', () => {
it('records the scheduled delay before opening a fresh request attempt', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
new LlmError('busy', 'RATE_LIMIT', { status: 429 }),
textResponse('done'),
])
;({ ctx: context } = await harness(adapter))
const agent = context.agentLoop.create(SessionId('retry-success'), {
provider: 'mock',
model: 'mock',
})
const scheduled = new Promise<Extract<(typeof agent.session.events)[number], { type: 'llm/retry' }>>((resolve) => {
const dispose = context?.on('session/event', (session, event) => {
if (session === agent.session && event.type === 'llm/retry') {
dispose?.()
resolve(event)
}
})
})
agent.send([{ type: 'text', text: 'go' }])
const event = await scheduled
expect(event.data).toEqual({
turn: 1,
step: 1,
retry: 1,
maxRetries: 2,
delayMs: 500,
failure: { message: 'busy', code: 'RATE_LIMIT', status: 429 },
})
expect(adapter.requests).toHaveLength(1)
await vi.advanceTimersByTimeAsync(499)
expect(adapter.requests).toHaveLength(1)
const idle = waitForIdle(context, agent)
await vi.advanceTimersByTimeAsync(1)
await idle
expect(adapter.requests).toHaveLength(2)
expect(agent.session.events.filter(item => item.type === 'step/start').map(item => item.data.step))
.toEqual([1, 2])
expect(agent.session.deriveMessages().at(-1)).toEqual({
role: 'assistant',
content: [{ type: 'text', text: 'done' }],
provenance: { provider: 'mock', model: 'mock' },
})
})
it('leaves partial failed chunks on their step without committing a message or tool side effect', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
partialToolFailure(new LlmError('stream interrupted', 'TRANSPORT')),
textResponse('recovered'),
])
;({ ctx: context } = await harness(adapter))
let toolExecutions = 0
context.tools.register(defineTool({
name: 'danger',
description: 'must not run for a failed provider attempt',
parameters: {},
async execute() {
toolExecutions += 1
return [{ type: 'text', text: 'unexpected' }]
},
}))
const agent = context.agentLoop.create(SessionId('retry-partial'), { provider: 'mock', model: 'mock' })
const scheduled = waitForRetry(context, agent, 1)
agent.send([{ type: 'text', text: 'go' }])
await scheduled
const idle = waitForIdle(context, agent)
await vi.advanceTimersByTimeAsync(500)
await idle
const failedChunks = agent.session.events.filter(event =>
event.type === 'assistant/chunk' && event.data.step === 1,
)
expect(failedChunks).toHaveLength(6)
expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step))
.toEqual([2])
expect(agent.session.events.some(event => event.type === 'tool/call')).toBe(false)
expect(toolExecutions).toBe(0)
expect(agent.session.deriveMessages().at(-1)).toMatchObject({
role: 'assistant',
content: [{ type: 'text', text: 'recovered' }],
provenance: { provider: 'mock', model: 'mock' },
})
})
it('applies bounded exponential jitter and stops after the configured budget', async () => {
vi.useFakeTimers()
const samples = [0, 1]
const adapter = new ScriptedAdapter([
new LlmError('busy one', 'SERVER'),
new LlmError('busy two', 'SERVER'),
new LlmError('busy three', 'SERVER'),
])
;({ ctx: context } = await harness(adapter, { jitterRatio: 0.1 }, undefined, {
random: () => samples.shift() ?? 0.5,
}))
const agent = context.agentLoop.create(SessionId('retry-exhausted'), { provider: 'mock', model: 'mock' })
const first = waitForRetry(context, agent, 1)
agent.send([{ type: 'text', text: 'go' }])
expect((await first).data.delayMs).toBe(450)
const second = waitForRetry(context, agent, 2)
await vi.advanceTimersByTimeAsync(450)
expect((await second).data.delayMs).toBe(1_100)
const idle = waitForIdle(context, agent)
await vi.advanceTimersByTimeAsync(1_100)
await idle
expect(adapter.requests).toHaveLength(3)
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(2)
expect(agent.session.events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'error', failure: { message: 'busy three', code: 'SERVER' } } },
})
})
it('uses a bounded provider Retry-After verbatim and delegates an over-cap instruction', async () => {
vi.useFakeTimers()
const accepted = new ScriptedAdapter([
new LlmError('wait', 'RATE_LIMIT', { retryAfterMs: 2_000 }),
textResponse('done'),
])
;({ ctx: context } = await harness(accepted, { jitterRatio: 1 }))
const acceptedAgent = context.agentLoop.create(SessionId('retry-after-accepted'), { provider: 'mock', model: 'mock' })
const scheduled = waitForRetry(context, acceptedAgent, 1)
acceptedAgent.send([{ type: 'text', text: 'go' }])
expect((await scheduled).data.delayMs).toBe(2_000)
const acceptedIdle = waitForIdle(context, acceptedAgent)
await vi.advanceTimersByTimeAsync(2_000)
await acceptedIdle
expect(accepted.requests).toHaveLength(2)
await context.fiber.dispose()
const rejected = new ScriptedAdapter([
new LlmError('wait too long', 'RATE_LIMIT', { retryAfterMs: 10_001 }),
])
;({ ctx: context } = await harness(rejected))
const rejectedAgent = context.agentLoop.create(SessionId('retry-after-rejected'), { provider: 'mock', model: 'mock' })
const rejectedIdle = waitForIdle(context, rejectedAgent)
rejectedAgent.send([{ type: 'text', text: 'go' }])
await rejectedIdle
expect(rejected.requests).toHaveLength(1)
expect(rejectedAgent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
})
it('delegates non-transient failures without scheduling a timer', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
;({ ctx: context } = await harness(adapter))
const agent = context.agentLoop.create(SessionId('retry-auth'), { provider: 'mock', model: 'mock' })
const idle = waitForIdle(context, agent)
agent.send([{ type: 'text', text: 'go' }])
await idle
expect(adapter.requests).toHaveLength(1)
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
expect(vi.getTimerCount()).toBe(0)
})
it('aborts and drains a captured backoff before plugin disposal completes', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
new LlmError('temporary', 'TRANSPORT'),
textResponse('must not run'),
])
const mounted = await harness(adapter)
context = mounted.ctx
const agent = context.agentLoop.create(SessionId('retry-hmr'), { provider: 'mock', model: 'mock' })
const scheduled = waitForRetry(context, agent, 1)
agent.send([{ type: 'text', text: 'go' }])
await scheduled
const idle = waitForIdle(context, agent)
await mounted.retryFiber.dispose()
await idle
await vi.advanceTimersByTimeAsync(60_000)
expect(adapter.requests).toHaveLength(1)
expect(agent.session.events.filter(event => event.type === 'step/start')).toHaveLength(1)
expect(vi.getTimerCount()).toBe(0)
})
it('does not make plugin disposal wait for a delegated recovery policy', async () => {
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
const mounted = await harness(adapter)
context = mounted.ctx
const downstream = Promise.withResolvers<RequestErrorDecision>()
const entered = Promise.withResolvers<undefined>()
context.on('agent/request-error', () => {
entered.resolve(undefined)
return downstream.promise
})
const agent = context.agentLoop.create(SessionId('retry-delegated-disposal'), {
provider: 'mock',
model: 'mock',
})
const idle = waitForIdle(context, agent)
agent.send([{ type: 'text', text: 'go' }])
await entered.promise
const disposing = mounted.retryFiber.dispose()
let timer: ReturnType<typeof setTimeout> | undefined
const outcome = await Promise.race([
disposing.then(() => 'disposed' as const),
new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }),
])
if (timer !== undefined) clearTimeout(timer)
downstream.resolve({ action: 'fail' })
await disposing
await idle
expect(outcome).toBe('disposed')
expect(adapter.requests).toHaveLength(1)
})
it('fails a captured callback after disposal without entering downstream policy', async () => {
const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')])
const captured = Promise.withResolvers<undefined>()
let invokeCaptured: (() => Promise<void>) | undefined
const mounted = await harness(adapter, {}, (ctx) => {
ctx.on('agent/request-error', (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
return new Promise<RequestErrorDecision>((resolve) => {
invokeCaptured = async () => { resolve(await next()) }
captured.resolve(undefined)
})
})
})
context = mounted.ctx
let downstreamCalls = 0
context.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => {
downstreamCalls += 1
return next()
})
const agent = context.agentLoop.create(SessionId('retry-captured-disposal'), {
provider: 'mock',
model: 'mock',
})
const idle = waitForIdle(context, agent)
agent.send([{ type: 'text', text: 'go' }])
await captured.promise
await mounted.retryFiber.dispose()
if (invokeCaptured === undefined) throw new Error('request-error waterfall did not capture retry callback')
await invokeCaptured()
await idle
expect(downstreamCalls).toBe(0)
expect(adapter.requests).toHaveLength(1)
})
it('lets turn cancellation win during backoff without opening another step', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
new LlmError('temporary', 'TIMEOUT'),
textResponse('must not run'),
])
;({ ctx: context } = await harness(adapter))
const agent = context.agentLoop.create(SessionId('retry-cancel'), { provider: 'mock', model: 'mock' })
const scheduled = waitForRetry(context, agent, 1)
agent.send([{ type: 'text', text: 'go' }])
await scheduled
const idle = waitForIdle(context, agent)
agent.cancel('user cancelled during retry')
await idle
expect(adapter.requests).toHaveLength(1)
expect(agent.session.events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'aborted', reason: 'user cancelled during retry' } },
})
expect(vi.getTimerCount()).toBe(0)
})
it('lets an earlier recovery listener cancel before retry policy runs', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
new LlmError('temporary', 'SERVER'),
textResponse('must not run'),
])
;({ ctx: context } = await harness(adapter, {}, (ctx) => {
ctx.on('agent/request-error', async (agent, _turn, _step, _error, _failure, _history, _signal, next) => {
agent.cancel('cancelled by earlier recovery policy')
return next()
})
}))
const agent = context.agentLoop.create(SessionId('retry-pre-cancel'), { provider: 'mock', model: 'mock' })
const idle = waitForIdle(context, agent)
agent.send([{ type: 'text', text: 'go' }])
await idle
expect(adapter.requests).toHaveLength(1)
expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false)
expect(agent.session.events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'aborted', reason: 'cancelled by earlier recovery policy' } },
})
})
it('handles synchronous cancellation from the retry status event', async () => {
vi.useFakeTimers()
const adapter = new ScriptedAdapter([
new LlmError('temporary', 'SERVER'),
textResponse('must not run'),
])
;({ ctx: context } = await harness(adapter))
const agent = context.agentLoop.create(SessionId('retry-event-cancel'), { provider: 'mock', model: 'mock' })
context.on('session/event', (session, event) => {
if (session === agent.session && event.type === 'llm/retry') agent.cancel('cancelled by retry observer')
})
const idle = waitForIdle(context, agent)
agent.send([{ type: 'text', text: 'go' }])
await idle
expect(adapter.requests).toHaveLength(1)
expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1)
expect(vi.getTimerCount()).toBe(0)
})
it.each([
[{ maxTransientRetries: -1 }, /maxTransientRetries/],
[{ maxTransientRetries: 1.5 }, /maxTransientRetries/],
[{ initialDelayMs: 0 }, /initialDelayMs/],
[{ maxDelayMs: Number.POSITIVE_INFINITY }, /maxDelayMs/],
[{ initialDelayMs: MAX_TIMER_DELAY_MS + 1 }, /initialDelayMs/],
[{ maxDelayMs: MAX_TIMER_DELAY_MS + 1 }, /maxDelayMs/],
[{ initialDelayMs: 20, maxDelayMs: 10 }, /less than or equal/],
[{ jitterRatio: 1.1 }, /jitterRatio/],
[{ retryableCodes: [] }, /must not be empty/],
[{ retryableCodes: ['SERVER', 'SERVER'] }, /duplicates/],
[{ retryableCodes: [''] }, /non-empty strings/],
] as const)('fails direct composition for invalid config %#', (config, message) => {
expect(() => { retry.apply(new Context(), config as retry.Config) }).toThrow(message)
})
})

View File

@@ -0,0 +1,33 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../../../vendor/schemastery"
},
{
"path": "../../llm/llm"
},
{
"path": "../../core/session"
},
{
"path": "../../core/agent"
},
{
"path": "../../util/timeout"
}
]
}

View File

@@ -13,7 +13,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
@@ -21,12 +21,12 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
| Event | Mode | Purpose |
|---|---|---|
| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) |
| `llm/stream` | waterfall | Intercept/wrap every streaming model call for caching, logging, or routing |
### Extension points
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)
@@ -47,8 +47,9 @@ Every product adapter sends application identity on provider HTTP requests. `att
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract.
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) matches its frozen serializable `failure.code`. The payload may also retain validated status, `Retry-After`, and branded provider request id facts; policy remains outside the error.
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
- `QUOTA_EXCEEDED_CODE` — the non-transient provider-neutral code for exhausted account quota, balance, credits, budget, or usage limits. `isQuotaExceededError(detail)` keeps those failures distinct from request-rate limits.
### Real adapters
@@ -64,7 +65,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
## Known Limitations and Deferred Work
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure.
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine.
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)).
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.

View File

@@ -5,10 +5,10 @@
*/
import { HarnessError } from './error.ts'
import type { StreamChunk } from './types.ts'
import type { LlmFailure, StreamChunk } from './types.ts'
/** Errors proven to originate in one model call's final adapter boundary. */
export type AdapterFailureScope = WeakSet<Error>
/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */
export type AdapterFailureScope = WeakMap<Error, LlmFailure>
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
@@ -47,10 +47,70 @@ export function markLlmAdapterFailure(
const error = value instanceof Error
? value as Error & { code?: string }
: new HarnessError(String(value), 'UNKNOWN', { cause: value })
failures.add(error)
const carried = error instanceof HarnessError ? ownFailureSnapshot(error) : undefined
const failure = carried !== undefined && carried.code === error.code ? carried : Object.freeze({
message: errorMessage(error),
code: harnessErrorCode(error),
})
failures.set(error, failure)
return error
}
/** Snapshot an own data property without invoking an SDK-defined accessor. */
function ownFailureSnapshot(error: Error): LlmFailure | undefined {
try {
const descriptor = Object.getOwnPropertyDescriptor(error, 'failure')
return descriptor !== undefined && 'value' in descriptor
? failureSnapshot(descriptor.value)
: undefined
} catch (_sdkPropertyTrap) {
return undefined
}
}
/** Validate and detach an arbitrary serializable failure payload. */
function failureSnapshot(value: unknown): LlmFailure | undefined {
if (typeof value !== 'object' || value === null) return undefined
try {
const candidate = value as Partial<LlmFailure>
const message = candidate.message
const code = candidate.code
const status = candidate.status
const retryAfterMs = candidate.retryAfterMs
const requestId = candidate.requestId
if (typeof message !== 'string' || message.length === 0
|| typeof code !== 'string' || code.length === 0
|| (status !== undefined && (!Number.isInteger(status) || status < 100 || status > 599))
|| (retryAfterMs !== undefined && (!Number.isFinite(retryAfterMs) || retryAfterMs <= 0))
|| (requestId !== undefined && (typeof requestId !== 'string' || requestId.length === 0))) return undefined
return Object.freeze({
message,
code,
...status === undefined ? {} : { status },
...retryAfterMs === undefined ? {} : { retryAfterMs },
...requestId === undefined ? {} : { requestId },
})
} catch (_sdkFailureGetter) {
return undefined
}
}
/** Read an SDK error message without letting an accessor replace the primary failure. */
function errorMessage(error: Error): string {
try {
const message: unknown = error.message
if (typeof message === 'string' && message.length > 0) return message
} catch (_sdkMessageGetter) {
// The fallback below preserves a serializable failure beside the original Error.
}
return 'LLM adapter failed'
}
/** Trust only Harness-owned codes; third-party SDK codes are not our taxonomy. */
function harnessErrorCode(error: Error): string {
return error instanceof HarnessError ? error.code : 'UNKNOWN'
}
/**
* Whether a failure came from final adapter dispatch, iterator construction,
* or iteration for the call represented by the exact returned stream handle.
@@ -65,3 +125,18 @@ export function isLlmAdapterFailure(
const failures = adapterFailureScopes.get(stream)
return value instanceof Error && failures !== undefined && failures.has(value)
}
/**
* Retrieve normalized provider facts only for an Error tagged by this exact
* model call's final adapter boundary.
* @param stream - the exact stream returned to the consumer.
* @param value - the caught failure.
* @returns the immutable facts for that call, or `undefined` for middleware, nested, or consumer failures.
*/
export function llmFailureOf(
stream: AsyncIterable<StreamChunk>,
value: unknown,
): LlmFailure | undefined {
const failures = adapterFailureScopes.get(stream)
return value instanceof Error ? failures?.get(value) : undefined
}

View File

@@ -1,5 +1,6 @@
/**
* dsh-llm's owned branded id: `CallId` (tool-call correlation).
* dsh-llm's owned branded ids: tool-call correlation and provider request
* diagnostics.
*
* The `Branded<B>` primitive itself lives in `@deepseek-ai/dsh-brand` (a
* zero-dependency type-only package) so every owner of a cross-boundary id can
@@ -25,3 +26,15 @@ export type CallId = Branded<'CallId'>
export function CallId(id: string): CallId {
return id as CallId
}
/** Provider-issued request identifier retained for diagnostics across package boundaries. */
export type ProviderRequestId = Branded<'ProviderRequestId'>
/**
* Brand a provider-issued request identifier.
* @param id - the opaque provider-issued string.
* @returns the same string, branded; no validation is performed.
*/
export function ProviderRequestId(id: string): ProviderRequestId {
return id as ProviderRequestId
}

View File

@@ -24,6 +24,9 @@ export class HarnessError extends Error {
/** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */
export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED'
/** Canonical provider-neutral code for an exhausted account quota or balance. */
export const QUOTA_EXCEEDED_CODE = 'QUOTA'
/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */
const STRUCTURED_CONTEXT_OVERFLOW = new RegExp(
String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]`
@@ -62,6 +65,19 @@ export function isContextWindowExceededError(detail: string): boolean {
|| EXCEEDS_MODEL_CONTEXT.test(detail)
}
/**
* Recognize provider wording that identifies an exhausted account quota rather
* than a transient request-rate limit.
* @param detail - provider error code/type/message text joined into one string.
* @returns true only for terminal quota, balance, credit, budget, or usage-limit wording.
*/
export function isQuotaExceededError(detail: string): boolean {
return /\binsufficient[\s_-]+(?:quota|balance|credits?)\b/i.test(detail)
|| /\b(?:quota|usage[\s_-]+limit)[\s_-]+(?:exceeded|exhausted|reached)\b/i.test(detail)
|| /\b(?:balance|credits?)[\s_-]+(?:exhausted|depleted)\b/i.test(detail)
|| /\bout[\s_-]+of[\s_-]+(?:credits?|budget)\b/i.test(detail)
}
/**
* Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams).
* @param value - the caught value (`unknown` in catch clauses).

View File

@@ -7,7 +7,8 @@
*/
import { Context, Service } from 'cordis'
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
import type { GenerateOptions, LlmFailure, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
import type { ProviderRequestId } from './brand.ts'
import { deepFreeze } from './call-config.ts'
import { HarnessError } from './error.ts'
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
@@ -21,7 +22,7 @@ export * from './types.ts'
export { BlockAssembler } from './assembler.ts'
export { callConfigEquals, deepFreeze } from './call-config.ts'
export type { LlmCallConfig } from './call-config.ts'
export { isLlmAdapterFailure } from './adapter-failure.ts'
export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts'
declare module 'cordis' {
interface Context {
@@ -44,14 +45,53 @@ declare module 'cordis' {
}
}
/** Structured provider facts and cause accepted by {@link LlmError}. */
export interface LlmErrorOptions extends ErrorOptions {
/** Valid HTTP status observed at the provider boundary. */
status?: number
/** Positive finite provider-requested delay in milliseconds. */
retryAfterMs?: number
/** Non-empty opaque provider request id. */
requestId?: ProviderRequestId
}
/**
* Typed error for LLM-related failures. Extends {@link HarnessError}, so the
* `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy.
*/
export class LlmError extends HarnessError {
constructor(message: string, code: string, options?: ErrorOptions) {
/** Serializable facts retained beside this live Error. */
readonly failure: LlmFailure
/**
* @param message - non-empty human-readable failure summary.
* @param code - non-empty stable provider-neutral machine code.
* @param options - optional cause and validated serializable provider facts.
*/
constructor(message: string, code: string, options?: LlmErrorOptions) {
if (typeof message !== 'string' || message.length === 0) throw new Error('LlmError message must be a non-empty string')
if (typeof code !== 'string' || code.length === 0) throw new Error('LlmError code must be a non-empty string')
if (options?.status !== undefined
&& (!Number.isInteger(options.status) || options.status < 100 || options.status > 599)) {
throw new Error('LlmError status must be an integer from 100 through 599')
}
if (options?.retryAfterMs !== undefined
&& (!Number.isFinite(options.retryAfterMs) || options.retryAfterMs <= 0)) {
throw new Error('LlmError retryAfterMs must be a positive finite number')
}
if (options?.requestId !== undefined
&& (typeof options.requestId !== 'string' || options.requestId.length === 0)) {
throw new Error('LlmError requestId must be a non-empty string')
}
super(message, code, options)
this.name = 'LlmError'
this.failure = Object.freeze({
message,
code,
...options?.status === undefined ? {} : { status: options.status },
...options?.retryAfterMs === undefined ? {} : { retryAfterMs: options.retryAfterMs },
...options?.requestId === undefined ? {} : { requestId: options.requestId },
})
}
}
@@ -262,7 +302,7 @@ export class LlmService extends Service {
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
*/
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
const failures: AdapterFailureScope = new WeakSet<Error>()
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
return bindAdapterFailureScope(stream, failures)
}

View File

@@ -5,7 +5,21 @@
*/
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId } from './brand.ts'
import type { CallId, ProviderRequestId } from './brand.ts'
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
export interface LlmFailure {
/** Human-readable provider or transport failure. */
readonly message: string
/** Stable provider-neutral machine-routing code. */
readonly code: string
/** HTTP status observed at the provider boundary, when available. */
readonly status?: number
/** Provider-requested delay in milliseconds, when valid and available. */
readonly retryAfterMs?: number
/** Opaque provider-issued request identifier for diagnostics. */
readonly requestId?: ProviderRequestId
}
/** Plain text visible to the end user. */
export interface TextBlock {
@@ -98,8 +112,8 @@ export interface FinishReasonMap {
'stop': { kind: 'stop' }
'tool-calls': { kind: 'tool-calls' }
'max-tokens': { kind: 'max-tokens' }
'aborted': { kind: 'aborted' }
'error': { kind: 'error'; message: string; code?: string }
'aborted': { kind: 'aborted'; failure: LlmFailure }
'error': { kind: 'error'; failure: LlmFailure }
}
/** Any known finish reason, derived from {@link FinishReasonMap}; switch on `kind` and fall through unknowns (merge-extensible). */

View File

@@ -41,7 +41,10 @@ const chunkArb: fc.Arbitrary<StreamChunk> = indexArb.chain(index => fc.oneof(
fc.constant<StreamChunk>({ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }),
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'stop' } }),
fc.constant<StreamChunk>({ type: 'finish', reason: { kind: 'tool-calls' } }),
fc.string().map((message): StreamChunk => ({ type: 'finish', reason: { kind: 'error', message } })),
fc.string({ minLength: 1 }).map((message): StreamChunk => ({
type: 'finish',
reason: { kind: 'error', failure: { message, code: 'UNKNOWN' } },
})),
))
/** A stream is an arbitrary list of chunks (we do NOT force a terminal finish). */

View File

@@ -4,9 +4,12 @@ import LlmService, {
GenerateOptions,
HarnessError,
isContextWindowExceededError,
isQuotaExceededError,
isLlmAdapterFailure,
LlmAdapter,
LlmError,
llmFailureOf,
ProviderRequestId,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
@@ -80,6 +83,17 @@ describe('LlmService', () => {
expect(isContextWindowExceededError('context window size must be positive')).toBe(false)
})
it('distinguishes exhausted account quota from transient rate limiting', () => {
for (const detail of [
'insufficient_quota',
'account balance depleted',
'usage-limit-exceeded',
'out of credits',
]) expect(isQuotaExceededError(detail)).toBe(true)
expect(isQuotaExceededError('HTTP 429: rate limit reached')).toBe(false)
expect(isQuotaExceededError('quota resets in one minute')).toBe(false)
})
it('routes stream() to the registered adapter', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
@@ -168,6 +182,151 @@ describe('LlmService', () => {
expect(caught).toBe(original)
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
expect(llmFailureOf(stream, caught)).toEqual({
message: `${boundary} failed`,
code: 'BOUNDARY_FAILED',
})
})
it('keeps structured provider facts beside a frozen third-party Error', async () => {
const original = new LlmError('provider busy', 'RATE_LIMIT', {
status: 429,
retryAfterMs: 1_500,
requestId: ProviderRequestId('req-7'),
})
Object.freeze(original)
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
let caught: unknown
try {
for await (const _chunk of stream) { /* drain */ }
} catch (error: unknown) {
caught = error
}
expect(caught).toBe(original)
expect(llmFailureOf(stream, caught)).toEqual({
message: 'provider busy',
code: 'RATE_LIMIT',
status: 429,
retryAfterMs: 1_500,
requestId: ProviderRequestId('req-7'),
})
})
it('does not trust retry facts carried by an unknown third-party Error', async () => {
const carried = { message: 'busy', code: 'SERVER', status: 503 }
const original = Object.assign(new Error('busy'), { failure: carried })
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toBe(original)
const facts = llmFailureOf(stream, original)
carried.status = 500
expect(facts).toEqual({ message: 'busy', code: 'UNKNOWN' })
expect(Object.isFrozen(facts)).toBe(true)
expect(facts).not.toBe(carried)
})
it('keeps an unknown SDK Error exact without trusting its private code or accessors', async () => {
const original = Object.assign(new Error('socket closed'), { code: 'ECONNRESET' })
Object.defineProperty(original, 'failure', {
get() { throw new Error('SDK failure accessor must not run') },
})
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toBe(original)
expect(original.code).toBe('ECONNRESET')
expect(llmFailureOf(stream, original)).toEqual({ message: 'socket closed', code: 'UNKNOWN' })
})
it('keeps an SDK Error exact when its message accessor is hostile', async () => {
const original = Object.defineProperty(new Error(), 'message', {
get() { throw new Error('SDK message accessor trap') },
})
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toBe(original)
expect(llmFailureOf(stream, original)).toEqual({ message: 'LLM adapter failed', code: 'UNKNOWN' })
})
it('falls back safely when SDK objects trap failure inspection or expose malformed facts', async () => {
const propertyTrap = new Proxy(new HarnessError('descriptor trapped', 'SERVER'), {
getOwnPropertyDescriptor(target, property) {
if (property === 'failure') throw new Error('SDK descriptor trap')
return Reflect.getOwnPropertyDescriptor(target, property)
},
})
const throwingFacts = Object.create(null) as Record<string, unknown>
Object.defineProperty(throwingFacts, 'message', {
get() { throw new Error('SDK fact getter trap') },
})
const carrying = (message: string, failure: unknown): HarnessError => Object.defineProperty(
new HarnessError(message, 'SERVER'),
'failure',
{ value: failure },
)
const factGetter = carrying('fact getter failed', throwingFacts)
const malformed = carrying('malformed facts', { message: 'provider busy', code: 'SERVER', requestId: 1 })
const primitive = carrying('primitive facts', 1)
const nullFacts = carrying('null facts', null)
const mismatched = carrying('mismatched facts', { message: 'busy', code: 'RATE_LIMIT' })
for (const [original, expectedMessage] of [
[propertyTrap, 'descriptor trapped'],
[factGetter, 'fact getter failed'],
[malformed, 'malformed facts'],
[primitive, 'primitive facts'],
[nullFacts, 'null facts'],
[mismatched, 'mismatched facts'],
] as const) {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toBe(original)
expect(llmFailureOf(stream, original)).toEqual({ message: expectedMessage, code: 'SERVER' })
}
})
it('retains a stable code from a HarnessError without requiring LlmError facts', async () => {
const original = new HarnessError('stable adapter failure', 'ADAPTER_STABLE')
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original))
const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] })
await expect((async () => {
for await (const _chunk of stream) { /* drain */ }
})()).rejects.toBe(original)
expect(llmFailureOf(stream, original)).toEqual({
message: 'stable adapter failure',
code: 'ADAPTER_STABLE',
})
expect(llmFailureOf(stream, 'not an Error')).toBeUndefined()
expect(llmFailureOf({ [Symbol.asyncIterator]: () => stream[Symbol.asyncIterator]() }, original)).toBeUndefined()
})
it('keeps a nested adapter failure scoped to the nested model call', async () => {
@@ -586,6 +745,15 @@ describe('LlmService', () => {
expect(err.code).toBe('CUSTOM_CODE')
})
it('rejects non-serializable structured failure facts at construction', () => {
expect(() => new LlmError('busy', 'RATE_LIMIT', { status: 42 })).toThrow(/status/)
expect(() => new LlmError('busy', 'RATE_LIMIT', { retryAfterMs: Number.NaN })).toThrow(/retryAfterMs/)
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: ProviderRequestId('') })).toThrow(/requestId/)
expect(() => new LlmError(1 as never, 'RATE_LIMIT')).toThrow(/message/)
expect(() => new LlmError('busy', 1 as never)).toThrow(/code/)
expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: 1 as never })).toThrow(/requestId/)
})
it('LlmError extends the shared HarnessError base', async () => {
const { HarnessError, isHarnessError } = await import('@deepseek-ai/dsh-llm')
const cause = new Error('root cause')