feat: implement bounded LLM request recovery

2026-07-20 03:34:19 +08:00
parent 7cf966fc0e
commit 3b0b0cefeb
115 changed files with 3311 additions and 366 deletions
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -85,7 +85,7 @@ forever:
      agent/request (config only) -> log request/header -> llm/stream (frozen)
      on final adapter-path or terminal in-band failure:
        'step/end'
-        agent/request-error(original error, consecutive retry attempt, signal)
+        agent/request-error(original error, failure facts, immutable prior failures, signal)
        retry in the next numbered step or preserve the original error
      otherwise:
        'assistant/chunk'
@@ -110,11 +110,11 @@ Each step assembles ordered prompt sections, tool schemas, and `{{name}}` variab

 Tool-time context—including async `agent.inject()` notices and post-tool `additionalContexts`—settles, then follows recorded results. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering before signal closure. Leftovers become queued input. Terminal `agent/turn-stop` runs after continuation and steering folding, stays authoritative through turn close and flush, and discards later steering but preserves queued prompts.

-`dsh-compact-basic` handles pressure and canonical overflow at checkpoints; retry requires a balanced surface replacement ([decision](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)).
+`dsh-compact-basic` handles pressure/overflow; `dsh-llm-retry` handles bounded transient backoff. Independent budgets compose on `agent/request-error` ([recovery decision](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)).

 ### Failure Boundaries

-The turn is the containment boundary. Final adapter-path and terminal in-band failures close the step before `agent/request-error`; retry opens a numbered step; otherwise, the provider error survives. Attempts reset on success.
+The turn is the containment boundary. Adapter failures close the step, entering `agent/request-error` with the exact `Error`, `LlmFailure`, and retry history. Retry opens a numbered step; success clears history; exhaustion stores the failure on `turn/end`. Failed chunks commit no message or tool.

 Other failures use `agent/error`. Cancellation and disposal beat recovery; undispatched model tool calls receive synthetic `tool/call` and `ABORTED` result pairs before `turn/end`. `cancel()` clears queues and aborts active work; disposal awaits quiescence before unregistering.

@@ -142,7 +142,7 @@ Durability is a plugin concern. Persistence backends buffer synchronous `session

 Messages contain typed blocks (`text`, `reasoning`, `tool-call`, `tool-result`) derived from merge-extensible `ContentBlockMap`; the same pattern types `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`. New block types coordinate adapters, UI bridges, compaction pricing, token metering, and persistence as one repo-wide contract; replay measurement types live in [token-meter.md](core-data-structures/token-meter.md).

-Streaming uses raw chunks (`block-start` through `finish`) and `BlockAssembler`. The loop logs and assembles chunks, storing provider/model provenance plus replay state. An `LlmAdapter` implements `stream()`, registers provider routes, and may expose selector metadata; it resolves and validates model ids. Replay state reaches targets only when both routes map to one adapter instance, which owns validation and conversion. The contract lives in [llm-streaming.md](core-data-structures/llm-streaming.md).
+Streaming uses raw chunks and `BlockAssembler`. One `LlmAdapter.stream()` is one provider attempt; adapters report facts, while recovery policy lives on `agent/request-error`. The loop logs chunks and successful provenance/replay state. Remote adapters stop stalled transport with per-read idle watchdogs. Replay state reaches targets only when routes share an adapter instance ([contract](core-data-structures/llm-streaming.md)).

 ## Extension And Composition

--- a/docs/config-catalog.md
+++ b/docs/config-catalog.md
@@ -27,7 +27,7 @@ export interface AcpConfig {

 Depends on: `Stream` (`@agentclientprotocol/sdk`)

-Source: [`packages/ui/acp/src/index.ts:206`](../packages/ui/acp/src/index.ts)
+Source: [`packages/ui/acp/src/index.ts:207`](../packages/ui/acp/src/index.ts)

 ## `@deepseek-ai/dsh-acp-demo`

@@ -66,6 +66,8 @@ export interface Config {
  toolBash?: NonNullable<agentCore.Config['toolBash']>
  /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */
  toolTasks?: NonNullable<agentCore.Config['toolTasks']>
+  /** Bounded transient model-request retry policy forwarded through agent-core. */
+  llmRetry?: NonNullable<agentCore.Config['llmRetry']>
 }
 ```

@@ -142,6 +144,8 @@ export interface Config {
  toolBash?: toolBash.Config
  /** Generic background-task controls; set false to keep the task service without model-facing task tools. */
  toolTasks?: toolTasks.Config | false
+  /** Bounded transient model-request retry policy. */
+  llmRetry?: llmRetry.Config
 }

 /** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */
@@ -157,9 +161,9 @@ export interface SkillConfig {
 }
 ```

-Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts)
+Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`llmRetry`](../packages/llm/llm-retry/src/index.ts) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts)

-Source: [`packages/examples/agent-spine-demo/src/index.ts:59`](../packages/examples/agent-spine-demo/src/index.ts)
+Source: [`packages/examples/agent-spine-demo/src/index.ts:60`](../packages/examples/agent-spine-demo/src/index.ts)

 ## `@deepseek-ai/dsh-bash-local`

@@ -237,6 +241,8 @@ export interface Config {
  toolBash?: NonNullable<agentCore.Config['toolBash']>
  /** Generic background-task control-tool config forwarded through agent-spine-demo. */
  toolTasks?: NonNullable<agentCore.Config['toolTasks']>
+  /** Bounded transient model-request retry policy forwarded through agent-spine-demo. */
+  llmRetry?: NonNullable<agentCore.Config['llmRetry']>
  /** Controls automatic AGENTS.md/CLAUDE.md loading; configure a byte budget or set `false`. */
  workspaceContext: agentCore.Config['workspaceContext']
 }
@@ -427,6 +433,8 @@ export interface Config {
  reasoningEffort?: 'high' | 'max'
  /** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
  models?: DeepSeekCatalogModel[]
+  /** Maximum provider idle time while one stream read is outstanding (default five minutes). */
+  streamIdleTimeoutMs?: number
 }

 /** One optional model entry advertised by the hand-written adapter. */
@@ -440,7 +448,7 @@ export interface DeepSeekCatalogModel {
 }
 ```

-Source: [`packages/llm/llm-deepseek/src/index.ts:33`](../packages/llm/llm-deepseek/src/index.ts)
+Source: [`packages/llm/llm-deepseek/src/index.ts:34`](../packages/llm/llm-deepseek/src/index.ts)

 ## `@deepseek-ai/dsh-llm-pi-ai`

@@ -475,16 +483,14 @@ export interface PiAiProviderProfile {
  timeoutMs?: number
  /** WebSocket connection timeout in milliseconds. */
  websocketConnectTimeoutMs?: number
-  /** Provider SDK retry count. */
-  maxRetries?: number
-  /** Maximum provider-requested retry delay in milliseconds. */
-  maxRetryDelayMs?: number
+  /** Maximum provider idle time while one stream read is outstanding. */
+  streamIdleTimeoutMs?: number
 }
 ```

 Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `ThinkingLevel` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)

-Source: [`packages/llm/llm-pi-ai/src/config.ts:40`](../packages/llm/llm-pi-ai/src/config.ts)
+Source: [`packages/llm/llm-pi-ai/src/config.ts:48`](../packages/llm/llm-pi-ai/src/config.ts)

 ## `@deepseek-ai/dsh-llm-replay`

@@ -530,6 +536,28 @@ export interface ReplayModelConfig {

 Source: [`packages/support/llm-replay/src/index.ts:375`](../packages/support/llm-replay/src/index.ts)

+## `@deepseek-ai/dsh-llm-retry`
+
+Requires: `agents`
+
+```ts config-catalog
+/** Deployment-owned limits and classification for transient request recovery. */
+export interface Config {
+  /** Maximum transient retries after the first request (default 2). */
+  maxTransientRetries?: number
+  /** Initial local exponential-backoff delay in milliseconds (default 500). */
+  initialDelayMs?: number
+  /** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
+  maxDelayMs?: number
+  /** Symmetric random multiplier range around one (default 0.1). */
+  jitterRatio?: number
+  /** Stable failure codes eligible for this policy. */
+  retryableCodes?: string[]
+}
+```
+
+Source: [`packages/llm/llm-retry/src/index.ts:39`](../packages/llm/llm-retry/src/index.ts)
+
 ## `@deepseek-ai/dsh-mcp-client`

 Requires: `tools`
@@ -822,7 +850,7 @@ export interface Config {
 }
 ```

-Source: [`packages/ui/stdio/src/index.ts:33`](../packages/ui/stdio/src/index.ts)
+Source: [`packages/ui/stdio/src/index.ts:34`](../packages/ui/stdio/src/index.ts)

 ## `@deepseek-ai/dsh-stdio-demo`

@@ -864,6 +892,8 @@ export interface Config {
  toolBash?: NonNullable<agentCore.Config['toolBash']>
  /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */
  toolTasks?: NonNullable<agentCore.Config['toolTasks']>
+  /** Bounded transient model-request retry policy forwarded through agent-core. */
+  llmRetry?: NonNullable<agentCore.Config['llmRetry']>
  /**
   * If set, the pre-created agent RESUMES this persisted session id instead of
   * starting fresh. Sourced from an env var in the leaf `cordis.yml`
@@ -1266,7 +1296,7 @@ export interface TuiConfig {
 }
 ```

-Source: [`packages/ui/tui/src/index.ts:100`](../packages/ui/tui/src/index.ts)
+Source: [`packages/ui/tui/src/index.ts:101`](../packages/ui/tui/src/index.ts)

 ## `@deepseek-ai/dsh-user-approval`

--- a/docs/cordis-catalog/events.md
+++ b/docs/cordis-catalog/events.md
@@ -75,7 +75,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w

 Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)

-Source: [`packages/core/agent/src/types.ts:311`](../../packages/core/agent/src/types.ts)
+Source: [`packages/core/agent/src/types.ts:312`](../../packages/core/agent/src/types.ts)

 ### `agent/post-step` — serial

@@ -201,17 +201,18 @@ Recover a model-request failure after its failed step has closed. `retry` opens
 * @param turn - the open turn number.
 * @param step - the failed step number.
 * @param error - the original model-request failure.
- * @param retryAttempt - zero-based number of prior recovery retries.
+ * @param failure - serializable facts normalized at the final adapter boundary.
+ * @param priorFailures - immutable failures that already authorized another request in this consecutive sequence.
 * @param signal - the turn abort signal.
 * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
 * @mode waterfall
 */
-'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise<RequestErrorDecision>): Promise<RequestErrorDecision>
+'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], signal: AbortSignal, next: () => Promise<RequestErrorDecision>): Promise<RequestErrorDecision>
 ```

-Types: [Agent](../core-data-structures/core.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)
+Types: [Agent](../core-data-structures/core.md) · [LlmFailure](../core-data-structures/llm-streaming.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)

-Source: [`packages/core/agent/src/types.ts:278`](../../packages/core/agent/src/types.ts)
+Source: [`packages/core/agent/src/types.ts:279`](../../packages/core/agent/src/types.ts)

 ### `agent/session-prefix` — waterfall

@@ -322,7 +323,7 @@ Override whether the turn continues. The default continues after tool calls or s

 Types: [Agent](../core-data-structures/core.md) · [ContinuationDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)

-Source: [`packages/core/agent/src/types.ts:288`](../../packages/core/agent/src/types.ts)
+Source: [`packages/core/agent/src/types.ts:289`](../../packages/core/agent/src/types.ts)

 ### `agent/turn-stop` — serial

@@ -343,7 +344,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a

 Types: [Agent](../core-data-structures/core.md) · [ContinuationStop](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)

-Source: [`packages/core/agent/src/types.ts:298`](../../packages/core/agent/src/types.ts)
+Source: [`packages/core/agent/src/types.ts:299`](../../packages/core/agent/src/types.ts)

 ## `agent-loop/*`

@@ -473,7 +474,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t

 Types: [GenerateOptions](../core-data-structures/core.md) · [LlmService](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)

-Source: [`packages/llm/llm/src/index.ts:43`](../../packages/llm/llm/src/index.ts)
+Source: [`packages/llm/llm/src/index.ts:44`](../../packages/llm/llm/src/index.ts)

 ## `session/*`

--- a/docs/cordis-catalog/services.md
+++ b/docs/cordis-catalog/services.md
@@ -525,7 +525,7 @@ stream(options: GenerateOptions): AsyncIterable<StreamChunk>

 Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)

-Source: [`packages/llm/llm/src/index.ts:97`](../../packages/llm/llm/src/index.ts)
+Source: [`packages/llm/llm/src/index.ts:137`](../../packages/llm/llm/src/index.ts)

 ## `ctx.permission` — `PermissionService`

--- a/docs/core-data-structures/core.md
+++ b/docs/core-data-structures/core.md
@@ -226,6 +226,22 @@ interface GenerateOptions {

 Why a model response stopped is a merge-extensible reason:

+```ts type-equiv
+/** Serializable provider-boundary facts; policy decides whether they are retryable. */
+interface LlmFailure {
+  /** Human-readable provider or transport failure. */
+  readonly message: string
+  /** Stable provider-neutral machine-routing code. */
+  readonly code: string
+  /** HTTP status observed at the provider boundary, when available. */
+  readonly status?: number
+  /** Provider-requested delay in milliseconds, when valid and available. */
+  readonly retryAfterMs?: number
+  /** Opaque provider-issued request identifier for diagnostics. */
+  readonly requestId?: ProviderRequestId
+}
+```
+
 ```ts type-equiv
 /**
 * Why a model response stopped.
@@ -235,8 +251,8 @@ interface FinishReasonMap {
  'stop': { kind: 'stop' }
  'tool-calls': { kind: 'tool-calls' }
  'max-tokens': { kind: 'max-tokens' }
-  'aborted': { kind: 'aborted' }
-  'error': { kind: 'error'; message: string; code?: string }
+  'aborted': { kind: 'aborted'; failure: LlmFailure }
+  'error': { kind: 'error'; failure: LlmFailure }
 }
 ```

@@ -445,14 +461,14 @@ type ContinuationDecision =
  | { action: 'continue'; reason?: { content: ContentBlock[]; source: MessageSource } }
 ```

-`agent/request-error` receives the original `RequestError`, whose optional provider-neutral `code` supports stable routing without message parsing:
+`agent/request-error` receives the exact original `RequestError` beside its immutable `LlmFailure`, an immutable list of failures that already authorized another request in the consecutive sequence, the turn signal, and `next()`. Recovery plugins route on `failure.code`, not the live error's message; each policy counts only its own codes, and a successful request clears the history:

 ```ts type-equiv
 /** Model-request failure with an optional machine-routable provider code. */
 type RequestError = Error & { code?: string }
 ```

-It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` preserves that error:
+It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` retains the structured failure on `turn/end`:

 ```ts type-equiv
 /** Failed-request recovery decision; `retry` opens another numbered step while listeners delegate by calling `next()`. */
--- a/docs/core-data-structures/llm-streaming.md
+++ b/docs/core-data-structures/llm-streaming.md
@@ -31,18 +31,38 @@ type StreamChunk =
  }
 ```

+Every thrown or in-band final-adapter failure normalizes to one serializable provider-neutral payload. `retryAfterMs` is a validated positive delay observed at the provider boundary, not a retry decision; `ProviderRequestId` is an opaque branded string for diagnostics.
+
+```ts type-equiv
+/** Serializable provider-boundary facts; policy decides whether they are retryable. */
+interface LlmFailure {
+  /** Human-readable provider or transport failure. */
+  readonly message: string
+  /** Stable provider-neutral machine-routing code. */
+  readonly code: string
+  /** HTTP status observed at the provider boundary, when available. */
+  readonly status?: number
+  /** Provider-requested delay in milliseconds, when valid and available. */
+  readonly retryAfterMs?: number
+  /** Opaque provider-issued request identifier for diagnostics. */
+  readonly requestId?: ProviderRequestId
+}
+```
+
 ## The adapter contract

 Every adapter MUST obey these, and every consumer may rely on them:

 - **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering.
 - **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`.
- **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop closes the failed step and offers either form to `agent/request-error`; absent recovery it becomes a turn error, and no normal completed assistant message is logged for that request.
+- **Two sanctioned error paths, one fact shape.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted', failure}` (provider in-band errors, for adapters that can't throw mid-stream). `LlmError.failure` carries the same `LlmFailure`. The final adapter boundary preserves the exact thrown `Error` object and associates immutable facts with that call; the agent loop closes the failed step and offers the error, facts, and immutable prior-retried facts to `agent/request-error`. Absent recovery the structured failure becomes the turn error, and no normal assistant message or tool side effect is committed for that attempt.
+- **One adapter call is one provider attempt.** Adapters disable library retries. Agent-level recovery opens another durable numbered step; direct `ctx.llm.stream()` callers remain single-attempt.
+- **Provider stalls are bounded at the transport.** Both shipping remote adapters expose positive finite `streamIdleTimeoutMs` with a five-minute default. The watchdog arms only while iterator `next()` is outstanding, uses one stable signal for the whole request, maps its own expiry to `TIMEOUT`, and keeps an earlier caller abort as `ABORTED`.
 - **Context overflow has one canonical code.** Both DeepSeek adapters classify explicit provider detail through `isContextWindowExceededError()` and surface `CONTEXT_WINDOW_EXCEEDED`, whether the failure arrives as a thrown HTTP `LlmError` or an in-band finish error. Consumers route on the code, never provider text.
 - **Every provider HTTP request carries the app-attribution header.** Adapters send `attributionHeaders()` (below) - the `User-Agent` baseline - and prove it with a wire-level test (mock server asserting the received header, or the library's header hook for a library-backed adapter).
 - **Replay state is adapter-owned.** A successful `finish` may carry lossless-JSON state needed to reconstruct a native provider response. The loop stores it with the assembled assistant message unless an `agent/step-result` listener rewrote the content. On a later request, `LlmService` passes the state only when the historical provider and target provider are currently registered to the exact same adapter instance. That adapter validates the state and owns any cross-model or cross-provider conversion; other adapters receive the provider-neutral content and provenance without the private state.

-This contract was pinned down by two deliberately independent implementations: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (a generic multi-provider adapter through `@earendil-works/pi-ai`). The library-backed adapter cannot throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not.
+This contract is pinned down by two deliberately independent implementations: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (a generic multi-provider adapter through `@earendil-works/pi-ai`). The library-backed adapter exercises the finish-chunk error path, while transport-boundary tests prove each idle watchdog stops its actual request.

 ## `AppIdentity` — app attribution

--- a/docs/core-data-structures/session.md
+++ b/docs/core-data-structures/session.md
@@ -435,7 +435,7 @@ declare class Session {
 - `context/message` → a user-role message at its chronological position. The default `envelope` is `context`, which wraps content as `<context source="…">…</context>`; `envelope: 'raw'` uses caller-owned framing verbatim. Optional JSON `meta` remains in the event log and is never rendered.
 - `steering/message` → a user-role message wrapped in `<steering source="…">…</steering>` at its chronological position.

-Everything else (`turn/*`, `step/*`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data.
+Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`, with normalized `LlmFailure` facts for a final model-request failure and message/code for other live errors. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data.

 ## Live-session fork API

@@ -479,9 +479,13 @@ interface TurnEndReasonMap {
   * The turn failed: a step threw or the model reported a failure. `step` is the
   * step number the failure occurred on (the operational error's location — the
   * single durable record of an in-turn failure; live diagnostics also fire via
-   * `agent/error`). `code` is the error's code when one was attached.
+   * `agent/error`). Final model-request failures retain their normalized facts
+   * as one `failure`; other turn failures retain their live Error message/code.
   */
-  error: { kind: 'error'; step: number; message: string; code?: string }
+  error: { kind: 'error'; step: number } & (
+    | { failure: LlmFailure; message?: never; code?: never }
+    | { message: string; code?: string; failure?: never }
+  )
  disposed: { kind: 'disposed' }
  /** At least one step reached its output-token ceiling, even if a plugin continued the turn. */
  'max-tokens': { kind: 'max-tokens' }
--- a/docs/event-producer-consumer.md
+++ b/docs/event-producer-consumer.md
@@ -10,24 +10,24 @@ This matrix shows which packages dispatch each harness-owned event and which pac
 | `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:362`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) |
 | `agent/created` | `emit` | [`packages/core/agent/src/types.ts:147`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) |
 | `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:156`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) |
-| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:311`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`tui`](../packages/ui/tui) |
+| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:312`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`tui`](../packages/ui/tui) |
 | `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) |
 | `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:204`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`time-context`](../packages/context/time-context), [`user-approval`](../packages/ui/user-approval) |
 | `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:214`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) |
 | `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:175`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
 | `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:226`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp) |
-| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:278`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic) |
+| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:279`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic), [`llm-retry`](../packages/llm/llm-retry) |
 | `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:241`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill), [`workspace-context`](../packages/context/workspace-context) |
 | `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:188`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`stdio`](../packages/ui/stdio) |
 | `agent/status` | `emit` | [`packages/core/agent/src/types.ts:165`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) |
 | `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:252`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
-| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:288`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
-| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:298`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) |
+| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:289`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
+| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:299`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) |
 | `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:31`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/ui/acp) |
 | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:61`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
 | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:70`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) |
 | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:53`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
-| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:43`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) |
+| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:44`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) |
 | `session/created` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence) |
 | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`session-persistence`](../packages/session-persistence/session-persistence) |
 | `session/event` | `emit` | [`packages/core/session/src/index.ts:69`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/ui/acp), [`cli-demo`](../packages/examples/cli-demo), [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio`](../packages/ui/stdio), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`workspace-context`](../packages/context/workspace-context) |
--- a/docs/module-graph.md
+++ b/docs/module-graph.md
@@ -18,6 +18,7 @@ flowchart TD
    pkg_llm["llm"]
    pkg_llm_deepseek["llm-deepseek"]
    pkg_llm_pi_ai["llm-pi-ai"]
+    pkg_llm_retry["llm-retry"]
    pkg_token_meter["token-meter"]
  end
  subgraph group_core["packages/core"]
@@ -157,7 +158,9 @@ flowchart TD
  pkg_scripts --> pkg_app_boot
  pkg_telemetry --> pkg_brand
  pkg_llm_deepseek --> pkg_llm
+  pkg_llm_deepseek --> pkg_timeout
  pkg_llm_pi_ai --> pkg_llm
+  pkg_llm_pi_ai --> pkg_timeout
  pkg_session --> pkg_brand
  pkg_session --> pkg_llm
  pkg_session --> pkg_scope
@@ -196,6 +199,10 @@ flowchart TD
  pkg_llm_replay --> pkg_session
  pkg_sandbox_local --> pkg_llm
  pkg_sandbox_local --> pkg_sandbox
+  pkg_llm_retry --> pkg_agent
+  pkg_llm_retry --> pkg_llm
+  pkg_llm_retry --> pkg_session
+  pkg_llm_retry --> pkg_timeout
  pkg_bash_local --> pkg_bash
  pkg_bash_local --> pkg_timeout
  pkg_compact_basic --> pkg_agent
@@ -318,6 +325,7 @@ flowchart TD
  pkg_acp --> pkg_agent
  pkg_acp --> pkg_bash
  pkg_acp --> pkg_llm
+  pkg_acp --> pkg_llm_retry
  pkg_acp --> pkg_permission
  pkg_acp --> pkg_sandbox
  pkg_acp --> pkg_session
@@ -380,11 +388,13 @@ flowchart TD
  pkg_stdio --> pkg_agent
  pkg_stdio --> pkg_agent_loop
  pkg_stdio --> pkg_llm
+  pkg_stdio --> pkg_llm_retry
  pkg_stdio --> pkg_session
  pkg_stdio --> pkg_user_interaction
  pkg_tui --> pkg_agent
  pkg_tui --> pkg_agent_loop
  pkg_tui --> pkg_llm
+  pkg_tui --> pkg_llm_retry
  pkg_tui --> pkg_session
  pkg_tui --> pkg_tools
  pkg_tui --> pkg_user_interaction
@@ -393,6 +403,7 @@ flowchart TD
  pkg_agent_spine_demo --> pkg_home
  pkg_agent_spine_demo --> pkg_invariants
  pkg_agent_spine_demo --> pkg_llm
+  pkg_agent_spine_demo --> pkg_llm_retry
  pkg_agent_spine_demo --> pkg_session
  pkg_agent_spine_demo --> pkg_skill
  pkg_agent_spine_demo --> pkg_skill_local
@@ -466,8 +477,8 @@ flowchart TD
 | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) |
 | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) |
 | [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand) |
-| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) |
-| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) |
+| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm), [`timeout`](../packages/util/timeout) |
+| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm), [`timeout`](../packages/util/timeout) |
 | [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
 | [`system-prompt`](../packages/core/system-prompt) | `core` | [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
 | [`fs`](../packages/fs/fs) | `fs` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm) |
@@ -488,6 +499,7 @@ flowchart TD
 | [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`session`](../packages/core/session) |
 | [`llm-replay`](../packages/support/llm-replay) | `support` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
 | [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) |
+| [`llm-retry`](../packages/llm/llm-retry) | `llm` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
 | [`bash-local`](../packages/bash/bash-local) | `bash` | [`bash`](../packages/bash/bash), [`timeout`](../packages/util/timeout) |
 | [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter) |
 | [`spill-local`](../packages/spill/spill-local) | `spill` | [`spill`](../packages/spill/spill) |
@@ -517,7 +529,7 @@ flowchart TD
 | [`tool-cordis`](../packages/cordis/tool-cordis) | `cordis` | [`scope`](../packages/core/scope), [`tools`](../packages/core/tools) |
 | [`hooks-codex`](../packages/hooks/hooks-codex) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`tools`](../packages/core/tools) |
 | [`agent-loop-testkit`](../packages/support/agent-loop-testkit) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) |
-| [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`permission`](../packages/ui/permission), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`user-interaction`](../packages/ui/user-interaction) |
+| [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`permission`](../packages/ui/permission), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`user-interaction`](../packages/ui/user-interaction) |
 | [`tool-ask-user`](../packages/ui/tool-ask-user) | `ui` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) |
 | [`workspace-context`](../packages/context/workspace-context) | `context` | [`agent`](../packages/core/agent), [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`paths`](../packages/util/paths), [`session`](../packages/core/session), [`tools`](../packages/core/tools) |
 | [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) |
@@ -529,9 +541,9 @@ flowchart TD
 | [`tool-subagent`](../packages/subagent/tool-subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools) |
 | [`hooks-claude`](../packages/hooks/hooks-claude) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) |
 | [`jsonrpc`](../packages/ui/jsonrpc) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`llm-deepseek`](../packages/llm/llm-deepseek), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent) |
-| [`stdio`](../packages/ui/stdio) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`user-interaction`](../packages/ui/user-interaction) |
-| [`tui`](../packages/ui/tui) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) |
-| [`agent-spine-demo`](../packages/examples/agent-spine-demo) | `examples` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`home`](../packages/util/home), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`skill`](../packages/skill/skill), [`skill-local`](../packages/skill/skill-local), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tool-bash`](../packages/bash/tool-bash), [`tool-skill`](../packages/skill/tool-skill), [`tool-tasks`](../packages/tasks/tool-tasks), [`tools`](../packages/core/tools), [`workspace-context`](../packages/context/workspace-context) |
+| [`stdio`](../packages/ui/stdio) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`user-interaction`](../packages/ui/user-interaction) |
+| [`tui`](../packages/ui/tui) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) |
+| [`agent-spine-demo`](../packages/examples/agent-spine-demo) | `examples` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`home`](../packages/util/home), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`skill`](../packages/skill/skill), [`skill-local`](../packages/skill/skill-local), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tool-bash`](../packages/bash/tool-bash), [`tool-skill`](../packages/skill/tool-skill), [`tool-tasks`](../packages/tasks/tool-tasks), [`tools`](../packages/core/tools), [`workspace-context`](../packages/context/workspace-context) |
 | [`workflow-workerthread`](../packages/workflow/workflow-workerthread) | `workflow` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools), [`workflow`](../packages/workflow/workflow) |
 | [`subagent-fork`](../packages/subagent/subagent-fork) | `subagent` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) |
 | [`subagent-spawn`](../packages/subagent/subagent-spawn) | `subagent` | [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) |
--- a/docs/persistence-catalog.md
+++ b/docs/persistence-catalog.md
@@ -79,7 +79,7 @@ export type SessionEvent<T extends SessionEventType = SessionEventType> = {
 }[T]
 ```

-Sources: [`packages/core/session/src/types.ts:255`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:262`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:292`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:324`](../packages/core/session/src/types.ts)
+Sources: [`packages/core/session/src/types.ts:259`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:266`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:296`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:328`](../packages/core/session/src/types.ts)

 ## Events

@@ -151,7 +151,7 @@ Source: [`packages/ui/user-approval/src/index.ts:68`](../packages/ui/user-approv

 Types: [StreamChunk](core-data-structures/llm-streaming.md)

-Source: [`packages/core/session/src/types.ts:219`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:223`](../packages/core/session/src/types.ts)

 #### `assistant/message` — surface

@@ -167,7 +167,7 @@ Source: [`packages/core/session/src/types.ts:219`](../packages/core/session/src/

 Types: [ContentBlock](core-data-structures/core.md) · [TokenUsage](core-data-structures/llm-streaming.md)

-Source: [`packages/core/session/src/types.ts:226`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:230`](../packages/core/session/src/types.ts)

 ### `bash/*`

@@ -258,7 +258,7 @@ Source: [`packages/compact/compact/src/types.ts:22`](../packages/compact/compact

 Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:212`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:216`](../packages/core/session/src/types.ts)

 ### `hook/*`

@@ -306,6 +306,24 @@ Source: [`packages/hooks/hook-protocol/src/types.ts:19`](../packages/hooks/hook-

 Source: [`packages/hooks/hook-protocol/src/types.ts:31`](../packages/hooks/hook-protocol/src/types.ts)

+### `llm/*`
+
+#### `llm/retry` — log-only
+
+```ts persistence-catalog
+/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
+'llm/retry': {
+  turn: number
+  step: number
+  retry: number
+  maxRetries: number
+  delayMs: number
+  failure: LlmFailure
+}
+```
+
+Source: [`packages/llm/llm-retry/src/index.ts:18`](../packages/llm/llm-retry/src/index.ts)
+
 ### `permission/*`

 #### `permission/preset` — log-only
@@ -336,7 +354,7 @@ Source: [`packages/ui/permission/src/index.ts:33`](../packages/ui/permission/src

 Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:204`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:208`](../packages/core/session/src/types.ts)

 ### `request/*`

@@ -350,7 +368,7 @@ Source: [`packages/core/session/src/types.ts:204`](../packages/core/session/src/
 'request/header': { header: EpochHeader; reason: RequestHeaderReason }
 ```

-Source: [`packages/core/session/src/types.ts:251`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:255`](../packages/core/session/src/types.ts)

 ### `steering/*`

@@ -363,7 +381,7 @@ Source: [`packages/core/session/src/types.ts:251`](../packages/core/session/src/

 Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:244`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:248`](../packages/core/session/src/types.ts)

 ### `step/*`

@@ -374,7 +392,7 @@ Source: [`packages/core/session/src/types.ts:244`](../packages/core/session/src/
 'step/end': { turn: number; step: number }
 ```

-Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:201`](../packages/core/session/src/types.ts)

 #### `step/start` — log-only

@@ -383,7 +401,7 @@ Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/
 'step/start': { turn: number; step: number }
 ```

-Source: [`packages/core/session/src/types.ts:195`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:199`](../packages/core/session/src/types.ts)

 ### `todo/*`

@@ -396,7 +414,7 @@ Source: [`packages/core/session/src/types.ts:195`](../packages/core/session/src/

 Types: [TodoItem](core-data-structures/session.md)

-Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:250`](../packages/core/session/src/types.ts)

 ### `tool/*`

@@ -413,7 +431,7 @@ Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/

 Types: [CallId](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:232`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:236`](../packages/core/session/src/types.ts)

 #### `tool/code-dispatch` — log-only

@@ -457,7 +475,7 @@ Source: [`packages/core/tools/src/code-mode.ts:34`](../packages/core/tools/src/c

 Types: [CallId](core-data-structures/core.md) · [ContentBlock](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:242`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/types.ts)

 ### `turn/*`

@@ -474,7 +492,7 @@ Source: [`packages/core/session/src/types.ts:242`](../packages/core/session/src/

 Types: [TurnEndReason](core-data-structures/session.md)

-Source: [`packages/core/session/src/types.ts:193`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/types.ts)

 #### `turn/start` — log-only

@@ -490,7 +508,7 @@ Source: [`packages/core/session/src/types.ts:193`](../packages/core/session/src/

 Types: [TurnTrigger](core-data-structures/session.md)

-Source: [`packages/core/session/src/types.ts:187`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:191`](../packages/core/session/src/types.ts)

 ### `user/*`

@@ -503,4 +521,4 @@ Source: [`packages/core/session/src/types.ts:187`](../packages/core/session/src/

 Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md)

-Source: [`packages/core/session/src/types.ts:199`](../packages/core/session/src/types.ts)
+Source: [`packages/core/session/src/types.ts:203`](../packages/core/session/src/types.ts)
--- a/docs/testing.md
+++ b/docs/testing.md
@@ -17,6 +17,8 @@ We are DeepSeek — do not ration real-API tests. A no-key test proves plumbing;

 Mock only the genuinely expensive or non-deterministic boundary (the LLM adapter, the network, the clock); keep everything downstream real. A hand-rolled stand-in proves the bridge moves bytes, not that the shipping tool behaves as asserted — the two drift while the test stays green. Example: bridge tool-call tests run the scripted mock MODEL but the real tool + real executor (`makeBridgeHarness({ withBash: true })` plugs `dsh-bash-local` + `dsh-tool-bash` and runs an actual `echo`).

+Recovery tests separate pre/post-chunk failures by step and prove failed chunks derive no message or tool side effect. Cover exhaustion, cancellation, policy composition, persistence, status, wire counts, transport-closing idle timeouts, and shipping Loader composition.
+
 ## Verify the world, not the self-report

 An e2e assertion re-runs the command or re-reads the file externally; a keyword probe on the agent's own output lets a cheating agent pass. Assert untouched files are byte-identical. e2e tests own their resources: create the harness in the test, dispose in `afterEach` (even on failure/retry/timeout); shared fixtures live in a plain `tests/harness.ts`, never another `*.e2e.ts` (importing a spec re-registers its `describe` and duplicates real API calls).