fix(llm): preserve serving retry policy

This commit is contained in:
Turtle
2026-07-25 13:30:34 +08:00
parent a623ed5183
commit 015ba14bae
27 changed files with 278 additions and 82 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-24-provider-retry-policies.md: 0ff304715aa5e1bfaf296e3f53e1da28d8b4042d
2026-07-24-provider-retry-policies.zh.md: 3b1d8f12a5fb37424e8b964f7bbb2be278263056
2026-07-24-provider-retry-policies.md: 0b9456eb1563bfcbfa06d93403fd68124ed1f707
2026-07-24-provider-retry-policies.zh.md: d3ef90ec739eec7300b7d0bb526cb5de958ed93f

View File

@@ -12,7 +12,7 @@ Provider policy must follow the request that actually failed, including a route
## Decision
Each concrete adapter accepts an optional `retryPolicy` inside its provider configuration. The adapter validates and resolves the policy, and `ctx.llm` captures it when that exact provider route registers. `@deepseek-ai/dsh-llm-retry` reads the registered policy for the provider whose step failed. A provider without `retryPolicy` uses the normal defaults.
Each concrete adapter accepts an optional `retryPolicy` inside its provider configuration. The adapter validates and resolves the policy, and `ctx.llm` captures it when that exact provider route registers. When a call enters its final adapter boundary, `ctx.llm` binds the serving registration's immutable policy to that call; the agent loop passes it to closed-step recovery even if the route is disposed or replaced while the request is in flight. `@deepseek-ai/dsh-llm-retry` combines that call-local policy with the failed step's durable provider identity. A call that never reaches a final adapter has no serving policy and delegates. A provider without `retryPolicy` uses the normal defaults.
```yaml
providers:
@@ -34,7 +34,7 @@ providers:
jitterRatio: 0.2
```
The listener selects the policy from the durable `request/header` in force when the failed step closed, excluding later recovery mutations. Normal mode retains the bounded transient behavior: it retries configured codes up to `maxRetries`, counts retries scheduled by the same provider policy in the current consecutive failure sequence, and otherwise delegates.
The listener reads the provider from the durable `request/header` in force when the failed step closed, excluding later recovery mutations, but never re-resolves policy from the mutable provider registry. Normal mode retains the bounded transient behavior: it retries configured codes up to `maxRetries`, counts retries scheduled by the same provider policy in the current consecutive failure sequence, and otherwise delegates.
Always mode asks downstream recovery first so a specialized policy such as context-overflow compaction can make progress. A downstream retry wins. A downstream failure decision or thrown recovery error falls back to an unbounded retry of the same provider request; the thrown error is logged. Success, turn cancellation, and plugin disposal are the only termination paths.
@@ -56,10 +56,10 @@ Each scheduled retry appends a non-surface `llm/retry` event with the failed pro
## Verification
Adapter tests validate nested policies at provider load and prove registration captures configured and default policies. Unit and real-Loader composition tests select policies from the failed request's provider, exercise always mode beyond the normal budget, pin jitter and delay caps, prove downstream recovery ordering, prove cancellation interrupts stalled downstream recovery, and prove cancellation and disposal stop active backoff waits. Request-level coverage compares the complete messages of failed and retried attempts and rejects both provider error text and discarded partial output. JSONL and SQLite tests round-trip an always event without `Infinity`; invariant tests bind its provider to the request header and its retry number to the active provider policy; ACP and TUI tests render finite and infinite limits.
Adapter tests validate nested policies at provider load, prove registration captures configured and default policies, and retain the serving policy across in-flight route replacement. Unit and real-Loader composition tests select policies from the failed request's serving registration, exercise always mode beyond the normal budget, pin jitter and delay caps, prove downstream recovery ordering, prove cancellation interrupts stalled downstream recovery, and prove cancellation and disposal stop active backoff waits. Request-level coverage compares the complete messages of failed and retried attempts and rejects both provider error text and discarded partial output. JSONL and SQLite tests round-trip an always event without `Infinity`; invariant tests bind its provider to the request header and its retry number to the active provider policy; ACP and TUI tests render finite and infinite limits.
## Consequences
Normal mode remains a finite default, while an explicit always policy can spend unbounded requests and time on permanent authentication, quota, invalid-request, protocol, or context failures. Operators must pair always mode with a cancellable caller and provider-specific cost controls. Retry state stays observable and durable without becoming model-visible, and exact-provider selection keeps one provider's exceptional policy from changing another provider's recovery behavior.
Normal mode remains a finite default, while an explicit always policy can spend unbounded requests and time on permanent authentication, quota, invalid-request, protocol, or context failures. Operators must pair always mode with a cancellable caller and provider-specific cost controls. Retry state stays observable and durable without becoming model-visible, and serving-registration capture prevents adapter lifecycle changes from retroactively changing an in-flight request's recovery contract.
This decision extends the closed-step recovery, single visible adapter attempt, structured failure, and durable status design in [bounded recovery for transient LLM request failures](../architecture/2026-06-21-bounded-llm-request-recovery.md).

View File

@@ -12,7 +12,7 @@ Status: implemented
## 决策
每个具体适配器都在其提供方配置中接受可选的 `retryPolicy`。适配器负责校验并解析策略,`ctx.llm` 则在该精确提供方路由注册时捕获策略。`@deepseek-ai/dsh-llm-retry` 读取失败步骤对应提供方的已注册策略。未配置 `retryPolicy` 的提供方使用 normal 默认值。
每个具体适配器都在其提供方配置中接受可选的 `retryPolicy`。适配器负责校验并解析策略,`ctx.llm` 则在该精确提供方路由注册时捕获策略。当调用进入最终适配器边界时,`ctx.llm` 会把实际提供服务的注册项所持不可变策略绑定到该调用;即使路由在请求进行期间被 dispose 或替换agent loop 仍会把该策略传给已关闭步骤恢复。`@deepseek-ai/dsh-llm-retry` 会把绑定到该调用的策略与失败步骤的持久提供方标识结合起来。未到达最终适配器的调用没有实际提供服务的策略,因而会委托后续处理。未配置 `retryPolicy` 的提供方使用 normal 默认值。
```yaml
providers:
@@ -34,7 +34,7 @@ providers:
jitterRatio: 0.2
```
监听器根据失败步骤关闭时生效的持久 `request/header` 选择策略后续恢复产生的改动不参与选择。normal 模式保留有界瞬态错误处理行为:它重试配置的错误代码,次数不超过 `maxRetries`;在当前连续失败序列中,同一提供方策略安排的重试都计入次数;其他情况委托后续处理。
监听器失败步骤关闭时生效的持久 `request/header` 读取提供方,后续恢复产生的改动不参与选择,但绝不会从可变的提供方注册表重新解析策略。normal 模式保留有界瞬态错误处理行为:它重试配置的错误代码,次数不超过 `maxRetries`;在当前连续失败序列中,同一提供方策略安排的重试都计入次数;其他情况委托后续处理。
always 模式先请求下游恢复使上下文溢出压缩compaction之类的专用策略有机会取得进展。下游若决定重试则以该决定为准。下游若决定失败或恢复过程抛出错误则回退为无界重试同一提供方请求抛出的错误会写入日志。成功、轮次取消和插件 dispose资源释放是仅有的终止路径。
@@ -56,10 +56,10 @@ always 模式先请求下游恢复使上下文溢出压缩compaction
## 验证
适配器测试会在提供方加载时校验嵌套策略,证明注册流程会捕获已配置策略和默认策略。单元测试与真实 Loader 组合测试根据失败请求的提供方选择策略、验证 always 模式可越过 normal 预算、固定抖动和延迟上限、证明下游恢复顺序、证明取消会中断停滞的下游恢复,并证明取消与 dispose 会停止正在进行的退避等待。请求级覆盖会比较失败尝试与重试尝试的完整消息并排除提供方错误文本和丢弃的部分输出。JSONL 与 SQLite 测试会往返读写不含 `Infinity` 的 always 事件不变式测试会将事件中的提供方绑定到请求头并将重试编号绑定到活跃的提供方策略ACP 与 TUI 测试会分别渲染有限和无限上限。
适配器测试会在提供方加载时校验嵌套策略,证明注册流程会捕获已配置策略和默认策略,并证明请求进行期间替换路由后仍会保留实际提供服务的策略。单元测试与真实 Loader 组合测试根据失败请求实际使用的注册项选择策略、验证 always 模式可越过 normal 预算、固定抖动和延迟上限、证明下游恢复顺序、证明取消会中断停滞的下游恢复,并证明取消与 dispose 会停止正在进行的退避等待。请求级覆盖会比较失败尝试与重试尝试的完整消息并排除提供方错误文本和丢弃的部分输出。JSONL 与 SQLite 测试会往返读写不含 `Infinity` 的 always 事件不变式测试会将事件中的提供方绑定到请求头并将重试编号绑定到活跃的提供方策略ACP 与 TUI 测试会分别渲染有限和无限上限。
## 后果
normal 模式仍是有限的默认策略;显式的 always 策略可能在永久性的身份验证、配额、无效请求、协议或上下文错误上耗费无限次请求和无限时间。运维方必须为 always 模式配备可取消的调用方和针对提供方的成本控制。重试状态保持可观察且持久,但不会对模型可见;精确提供方选择也能避免某个提供方的例外策略改变其他提供方的恢复行为
normal 模式仍是有限的默认策略;显式的 always 策略可能在永久性的身份验证、配额、无效请求、协议或上下文错误上耗费无限次请求和无限时间。运维方必须为 always 模式配备可取消的调用方和针对提供方的成本控制。重试状态保持可观察且持久,但不会对模型可见;捕获实际提供服务的注册项,也能防止适配器生命周期变化反过来改变进行中请求的恢复契约
本决策扩展了[瞬态 LLM大语言模型请求失败的有界恢复](../architecture/2026-06-21-bounded-llm-request-recovery.md)中确定的已关闭 step 恢复、单次可见适配器尝试、结构化失败与持久状态设计。