Merge updated origin/master into feat/tui-master-port

This commit is contained in:
Tianyi Cui
2026-07-27 23:55:53 +08:00
118 changed files with 2540 additions and 848 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md
2026-06-21-bounded-llm-request-recovery.md: 9c9d8a02595b988535158c9aa0ec43d6f1fa0c89
2026-06-21-bounded-llm-request-recovery.zh.md: bb9e430eaf87789452fd4cc89085d7d635f54ed1
2026-06-21-bounded-llm-request-recovery.md: 83d47e3a7d91bbcd2ceaf7b11cf13316142eb3ed
2026-06-21-bounded-llm-request-recovery.zh.md: 00dcbad3d1023ad33a22297bfe938b94bce839d4

View File

@@ -4,6 +4,8 @@ Status: implemented
English | [中文](2026-06-21-bounded-llm-request-recovery.zh.md)
The [per-provider request retry policy](../feature/2026-07-24-provider-retry-policies.md) extends this foundation with exact-provider configuration and an explicit unbounded mode. This note continues to own structured failure facts, the closed-step recovery boundary, normal mode's transient defaults, visible single attempts, and durable retry status.
## Problem
`dsh-llm` can report provider failures either by throwing during adapter dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary tags thrown failures so `dsh-agent-loop` can distinguish them from middleware and result-processing defects, and the loop normalizes both delivery forms into `agent/request-error` after closing the failed step. An unhandled failure is terminal; a handling listener repairs policy-owned state, returns `{ kind: 'retry' }`, and stops waterfall delegation. The [retry-action decision](../simplification/2026-07-27-request-error-retry-action.md) owns this return contract.
@@ -16,7 +18,7 @@ The prior boundary left three narrower gaps.
- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with an `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log.
- A recovered failure has no durable status fact. The failed step and chunks remain reconstructable, but an observer cannot tell whether the agent is deliberately backing off, for how long, or why. A long silent wait looks like a stalled loop.
The goal is bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer.
The default policy provides bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer.
## Decision
@@ -50,31 +52,19 @@ The shared transient-code set is intentionally small: adapter mappings for `RATE
`@deepseek-ai/dsh-llm-retry` is a function plugin that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow.
The `agent/request-error` seam carries only the current `LlmFailure`; the loop owns no retry policy or attempt history. Each recovery plugin keeps a private per-agent counter for its own handled failures and clears it at terminal `agent/settled`. Alternating transient and context-overflow failures therefore consume the `dsh-llm-retry` and compact-basic budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies.
The `agent/request-error` seam carries the current `LlmFailure`, an immutable list of prior failures that authorized retry turns in the consecutive recovery sequence, and the serving registration's immutable retry policy. The loop transports but does not interpret that policy, owns the consecutive failure history, and clears it after a successful model request. Normal `dsh-llm-retry` policy counts durable retry records scheduled by the same exact-provider policy, while `dsh-compact-basic` keeps its own context-overflow budget. Alternating transient and context-overflow failures therefore consume their owning finite budgets independently; the maximum request count is one plus the sum of the loaded finite budgets.
The plugin resolves and validates this deployment configuration at load:
```ts ignore-check
interface Config {
maxTransientRetries?: number
initialDelayMs?: number
maxDelayMs?: number
jitterRatio?: number
retryableCodes?: string[]
}
```
The defaults are two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the five transient codes above (`RATE_LIMIT`, `SERVER`, `TIMEOUT`, `TRANSPORT`, and `EMPTY_RESPONSE`). The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). Two retries mean at most three provider requests when no other recovery policy applies. `maxTransientRetries` is a non-negative integer, delays are positive finite numbers with `initialDelayMs <= maxDelayMs`, `jitterRatio` is in `[0, 1]`, and codes are non-empty and unique. These are Cordis config fields rather than hidden constants so deployments can choose different cost and latency budgets.
The [provider-policy decision](../feature/2026-07-24-provider-retry-policies.md) owns the current configuration shape. Provider adapters register their nested `retryPolicy`; omission uses normal defaults: two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the five transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47).
For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid `providerRetryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered.
The plugin owns a lifetime `AbortController` and tracks every active backoff callback. Each wait fuses the waterfall's turn signal with that lifetime signal. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; a captured callback whose lifetime signal aborts returns without retrying or entering the rest of its captured waterfall. This makes HMR disposal quiescent even though Cordis has already captured the listener.
The plugin owns a lifetime `AbortController` and tracks every active recovery callback, including delegated waterfall work and backoff. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; abort wins over a late delegated retry decision, and a captured callback can neither retry nor enter the rest of its waterfall after disposal. This makes HMR disposal quiescent even though Cordis has already captured the listener.
Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, one-based transient retry number, configured maximum, scheduled delay, and `LlmFailure`. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection.
Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, provider, policy mode, complete resolved-policy key, provider-policy retry number, mode-specific finite maximum when present, scheduled delay, and `LlmFailure`. The key sorts the code set and separates retry histories when a provider route is replaced by a behaviorally different same-mode policy. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection.
The listener calls `next()` for a non-transient code, an exhausted policy budget, or an over-cap provider delay. This preserves composition with context-overflow recovery and later policy plugins. For an owned failure it records and awaits the delay, then returns `{ kind: 'retry' }` without delegating. Turn cancellation and plugin disposal end the wait without returning a retry; the loop's cancellation/disposal checks remain authoritative.
The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same bounded policy. Library consumers retain explicit plugin composition: omitting the plugin leaves request failures terminal.
The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same provider-routed policy. Library consumers retain explicit plugin composition: omitting the plugin leaves request failures terminal.
### Make one layer own visible attempts
@@ -101,7 +91,7 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
- Automatic provider or model failover. Requests already select one explicit provider and model, and the provider registry deliberately has one adapter owner per provider.
- Retrying or continuing after a successful terminal finish, or splicing chunks from two attempts into one assistant message.
- Repairing malformed tool arguments, refusals, content filters, or other semantic model output.
- Unbounded retries, unattended retry-until-cancelled behavior, circuit breakers, shared provider health, or cross-agent retry budgets.
- Circuit breakers, shared provider health, or cross-agent retry budgets.
- Changing `llm/stream` into a response lifecycle or adding convenience generation APIs without a production consumer.
## Alternatives considered
@@ -110,7 +100,7 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
- **Add response start, interrupted, discarded, failed, and committed events to `dsh-llm`** — rejected because the agent log already separates raw chunks, successful messages, and numbered attempts. A second state machine would duplicate ownership without enabling the bounded same-route retry.
- **Add logical routes, capability matrices, and failover selection** — rejected because current requests already name provider and model explicitly, one adapter owns each provider, and no current consumer requires automatic fallback or can prove semantic compatibility.
- **Put `retryable` or `failover` on `LlmFailure`** — rejected because adapters report facts while deployment policy decides action. The same 429 may be retried in an interactive bundle and rejected in a cost-capped batch.
- **Retry forever while the caller remains active** — rejected because it gives one request unbounded cost and latency. Visible status makes bounded waiting understandable; it does not make an unlimited budget safe.
- **Retry forever while the caller remains active** — the [per-provider policy](../feature/2026-07-24-provider-retry-policies.md) supersedes this rejection for explicit `always` entries while retaining bounded normal mode as the default.
- **Log retry status only through the process logger** — rejected because process logs do not reconstruct session behavior and cannot drive replayed UI state.
- **Keep only flat codes** — rejected because retry delay and provider request id are structured provider facts, and HTTP status is necessary for diagnosis when different wire failures share one stable code.
@@ -120,11 +110,11 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
- An adapter-thrown `Error` reaches `agent/request-error` as the exact same object while its sidecar `LlmFailure` reaches the adjacent argument; tests retain the existing identity assertion for extensible and frozen third-party errors.
- DeepSeek and pi-ai adapter tests cover representative 400, 401/403, 429, 5xx, connection, malformed/truncated stream, timeout, abort, retry-after seconds/date, request-id, and unknown-SDK-error paths without recovery policy parsing message text.
- Pi-ai pins the SDK option to zero retries and performs one observed wire attempt for a retryable provider response; separate tests make removing either boundary fail.
- `agent/request-error` carries only current failure facts; each plugin clears its private per-agent counter at terminal idle, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets.
- `dsh-llm-retry` validates every config field at Loader startup, delegates all ineligible paths with `next()`, and makes at most `maxTransientRetries + 1` provider requests when no other policy applies.
- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, makes no retry request after disposal, and leaves no timer or promise alive.
- `agent/request-error` carries current failure facts, immutable prior-retried failure facts, and the serving registration's immutable retry policy; a success clears the history, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets.
- Each provider adapter validates its nested retry policy at Loader startup, and `ctx.llm` captures it with the route; normal mode delegates ineligible paths and makes at most `maxRetries + 1` provider requests when no other policy applies.
- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive.
- Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random seams, and abort during backoff.
- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new step, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery.
- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new turn, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery.
- The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry has distinct provenance.
- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI retraction plus scheduled-retry rendering. Keyless snapshots cover scheduling, cancellation, success, and exhaustion; ACP automation snapshots confirm that a discarded attempt stays off the wire while the recovered reply is emitted.
- Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it.
@@ -132,12 +122,12 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
## Consequences
- Every transient recovery attempt is visible as a closed failed turn plus `llm/retry`, and the bounded policy prevents hidden SDK retries from multiplying cost. A retry can still duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk.
- Every retry attempt is visible as a closed failed turn plus `llm/retry`, and adapter-level single-attempt behavior prevents hidden SDK retries from multiplying policy decisions. A retry can still duplicate provider billing even when no chunk arrived; normal mode limits that risk, while explicit always mode accepts it until cancellation or success.
- Provider SDKs may hide status or retry headers. Those adapters retain the stable facts they expose and otherwise use a coarse code rather than letting recovery policy parse fragile text.
- Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work.
- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots pin the transition.
- Adapter-local idle enforcement stops stalled transports without counting consumer think time. Contract tests at each transport boundary guard against SDK drift.
- Multiple recovery plugins add their finite budgets. Their classifiers remain disjoint here; an overlapping classifier would be registration-order policy and must be documented and tested by the plugins that introduce it.
- Multiple normal recovery plugins add their finite budgets. Always mode delegates first and then supplies an unbounded fallback; overlapping classifiers remain registration-order policy and must be documented and tested by the plugins that introduce them.
## Related

View File

@@ -4,6 +4,8 @@ Status: implemented
[English](2026-06-21-bounded-llm-request-recovery.md) | 中文
[按提供方配置的请求重试策略](../feature/2026-07-24-provider-retry-policies.md)在此基础上增加了确切提供方配置与显式无界 mode。本说明继续负责结构化失败事实、已关闭步骤的恢复边界、normal mode 的暂时性默认值、可见的单次尝试和持久重试状态。
## 问题
`dsh-llm` 可能在适配器分发或迭代时抛出异常,也可能以 `finish { kind: 'error' | 'aborted' }` 结束,以这两种形式报告提供方失败。最终适配器边界会标记抛出的失败,使 `dsh-agent-loop` 能将其与中间件和结果处理缺陷区分开。循环关闭失败步骤后,会把两种交付形式统一规范化为 `agent/request-error`。未被处理的失败是终态;处理失败的监听器修复策略自有状态,返回 `{ kind: 'retry' }`,并停止 waterfall 委托。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.md)规定这一返回契约。
@@ -16,7 +18,7 @@ Status: implemented
- 重试的归属因适配器而异。手写 DeepSeek 适配器只尝试一次,pi-ai profile 则可以启用库内部的不透明重试。如果把隐藏的传输重试与 `agent/request-error` 监听器结合,尝试次数会成倍增加,中间失败也不会记入会话日志。
- 恢复后的失败没有持久状态事实。失败的步骤和分片仍可重建,但观察者无法得知 agent(智能体)是否在有意退避、将等待多久,以及等待原因。长时间的静默等待看起来与循环停滞无异。
本决策的目标是从同一个显式提供方/模型请求的暂时性失败中进行有界恢复。提供方或模型故障转移、响应拼接和语义输出修复都属于其他问题,目前没有消费方。
默认策略的目标是从同一个显式提供方/模型请求的暂时性失败中进行有界恢复。提供方或模型故障转移、响应拼接和语义输出修复都属于其他问题,目前没有消费方。
## 决策
@@ -50,31 +52,19 @@ agent loop(智能体循环)会保留 `RequestError` 作为该精确的错误
`@deepseek-ai/dsh-llm-retry` 是监听 `agent/request-error` 的函数插件。它不引入服务或新的循环分支;agent-loop 包仅会更改通过现有失败步骤恢复控制流携带的数据。
`agent/request-error` seam 只携带当前 `LlmFailure`;循环不拥有重试策略或尝试历史。每个恢复插件为自身处理的失败维护一个逐 agent 的私有计数器,并在终态 `agent/settled` 时清零。因此,暂时性失败与上下文溢出交替出现时,`dsh-llm-retry` 与 compact-basic 的预算独立消耗;最大请求数等于 1 加上已加载恢复策略的有限预算总和。
`agent/request-error` seam 携带当前 `LlmFailure`、在连续恢复序列中授权重试轮次的不可变先前失败列表,以及实际服务注册所对应的不可变重试策略。循环只传递而不解释该策略;它拥有连续失败历史,并在模型请求成功后清除。`dsh-llm-retry` 的 normal 策略统计由同一项确切提供方策略安排的持久重试记录,`dsh-compact-basic` 则维护自己的上下文溢出预算。因此,暂时性失败与上下文溢出交替出现时,会各自独立消耗其有限预算;最大请求数等于 1 加上所有已加载有限预算之和。
该插件在加载时解析并验证以下部署配置:
```ts ignore-check
interface Config {
maxTransientRetries?: number
initialDelayMs?: number
maxDelayMs?: number
jitterRatio?: number
retryableCodes?: string[]
}
```
默认值为两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code(`RATE_LIMIT`、`SERVER`、`TIMEOUT`、`TRANSPORT` 和 `EMPTY_RESPONSE`)。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。在没有其他恢复策略时,两次重试表示最多发起三次提供方请求。`maxTransientRetries` 是非负整数,延迟是正的有限数且满足 `initialDelayMs <= maxDelayMs`,`jitterRatio` 位于 `[0, 1]`,code 非空且不重复。这些都是 Cordis 配置字段,而不是隐藏常量,使部署能够选择不同的成本与延迟预算。
当前配置形状由[提供方策略决策](../feature/2026-07-24-provider-retry-policies.md)规定。提供方适配器会注册嵌套的 `retryPolicy`;省略时使用 normal 默认值:两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。
对于预算未耗尽的合格失败,从 1 开始的暂时性重试计数使用有界指数退避。有效的 `providerRetryAfterMs` 只有在不超过 `maxDelayMs` 时才会取代指数退避;提供方延迟更长时,系统会委托给下一监听器,而不会违反提供方指令提前重试。本地退避乘以 `[1 - jitterRatio, 1 + jitterRatio]` 内的注入随机因子,并将最终值限制到 `maxDelayMs`;提供方延迟不加抖动。
插件拥有一个全生命期 `AbortController`,并跟踪每个活跃的退避回调。每次等待都会融合 waterfall(瀑布式事件)的轮次信号与该生命期信号。effect 清理会先注销监听器,再中止并等待活跃回调;被捕获回调的生命期信号中止时,回调会直接返回,不重试,也不进入其捕获 waterfall 的剩余部分。尽管 Cordis 已捕获该监听器,此设计仍能使 HMR(热模块替换)释放达到完全停稳。
插件拥有一个全生命期 `AbortController`,并跟踪每个活跃的恢复回调,包括委托的 waterfall(瀑布式事件)工作与退避。effect 清理会先注销监听器,再中止并等待活跃回调;中止会胜过较晚到达的委托重试决策,被捕获的回调在插件释放后既不能重试,也不能进入其 waterfall 的剩余部分。尽管 Cordis 已捕获该监听器,此设计仍能使 HMR(热模块替换)释放达到完全停稳。
休眠前,`dsh-llm-retry` 会追加一条不进入表层的 `llm/retry` 会话事件,其中包含轮次、失败步骤、从 1 开始的暂时性重试编号、已配置上限、计划延迟和 `LlmFailure`。该插件拥有 `SessionEventMap` 声明合并;`dsh-session` 继续负责通用持久化,不会吸收可选策略的词汇。事件记录已安排的内容,而不是下一个请求已完成;延迟期间取消随后会在 `turn/end` 中可见。因为该事件的目的是表示运行状态,而不是收集跟踪数据,所以它仅与生产渲染器及回放/快照覆盖一起交付。
休眠前,`dsh-llm-retry` 会追加一条不进入表层的 `llm/retry` 会话事件,其中包含轮次、失败步骤、提供方、策略 mode、完整的解析策略 key、提供方策略重试编号、该 mode 存在时的有限上限、计划延迟和 `LlmFailure`。该 key 会对 code 集排序,并在提供方路由被行为不同但 mode 相同的策略替换时分隔重试历史。该插件拥有 `SessionEventMap` 声明合并;`dsh-session` 继续负责通用持久化,不会吸收可选策略的词汇。事件记录已安排的内容,而不是下一个请求已完成;延迟期间取消随后会在 `turn/end` 中可见。因为该事件的目的是表示运行状态,而不是收集跟踪数据,所以它仅与生产渲染器及回放/快照覆盖一起交付。
对非暂时性 code、耗尽的策略预算或超出上限的提供方延迟,监听器会调用 `next()`。这保留了与上下文溢出恢复及后续策略插件的组合能力。对自身处理的失败,它会记录并等待延迟,然后在不委托的情况下返回 `{ kind: 'retry' }`。轮次取消和插件释放会结束等待且不返回重试动作,此后仍以循环的取消/释放检查为准。
agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次性 CLI(命令行界面)和 ACP(Agent Client Protocol)示例组合使用同一有界策略。库消费方仍需显式组合插件:省略该插件时,请求失败保持终态。
agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次性 CLI(命令行界面)和 ACP(Agent Client Protocol)示例组合使用同一套按提供方路由的策略。库消费方仍需显式组合插件:省略该插件时,请求失败保持终态。
### 由单一层负责可见的尝试
@@ -101,7 +91,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
- 自动提供方或模型故障转移。请求已显式选择一个提供方和模型,提供方注册表也有意规定每个提供方只由一个适配器负责。
- 在成功的终止性 finish 后重试或继续,或将两次尝试的分片拼接成一条 assistant 消息。
- 修复格式错误的工具参数、拒答、内容过滤或其他语义模型输出。
- 无界重试、无人值守地持续重试直至取消、熔断器、共享提供方健康状态或跨 agent 重试预算。
- 熔断器、共享提供方健康状态或跨 agent 重试预算。
- 在没有生产消费方的情况下,把 `llm/stream` 改造成响应生命周期或增加便利的生成 API。
## 考虑过的替代方案
@@ -110,7 +100,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
- **向 `dsh-llm` 增加响应开始、中断、丢弃、失败和提交事件**:拒绝采用,因为 agent 日志已经分隔原始分片、成功消息和编号尝试。第二套状态机会重复归属关系,又不能支持有界的同路由重试。
- **增加逻辑路由、能力矩阵和故障转移选择**:拒绝采用,因为当前请求已经显式指定提供方和模型,每个提供方由一个适配器负责,而且没有当前消费方要求自动回退或能够证明语义兼容性。
- **把 `retryable` 或 `failover` 放在 `LlmFailure` 上**:拒绝采用,因为适配器报告事实,部署策略决定动作。同一个 429 可以在交互式组合包中重试,也可以在成本受限的批处理中被拒绝。
- **只要调用方仍处于活跃状态就无限重试**:拒绝采用,因为这会让一次请求产生无界成本和延迟。可见状态能使有界等待易于理解,却不能让无限预算变得安全。
- **只要调用方仍处于活跃状态就无限重试**:[按提供方配置的策略](../feature/2026-07-24-provider-retry-policies.md)对显式 `always` 配置项推翻了这项拒绝,同时保留有界的 normal mode 作为默认值。
- **只通过进程 logger 记录重试状态**:拒绝采用,因为进程日志无法重建会话行为,也不能驱动回放后的 UI 状态。
- **只保留扁平 code**:拒绝采用,因为重试延迟和提供方请求 id 是结构化的提供方事实,而当不同协议失败共用一个稳定 code 时,诊断还需要 HTTP 状态。
@@ -120,11 +110,11 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
- 适配器抛出的 `Error` 会以完全相同的对象抵达 `agent/request-error`,其伴随的 `LlmFailure` 则抵达相邻参数;测试保留针对可扩展及冻结第三方错误的现有对象标识断言。
- DeepSeek 和 pi-ai 适配器测试覆盖具有代表性的 400、401/403、429、5xx、连接、格式错误/截断流、超时、中止、Retry-After 秒数/日期、请求 id 和未知 SDK 错误路径,恢复策略无需解析消息文本。
- Pi 将 SDK 选项固定为零次重试,并针对可重试的提供方响应执行一次可观测的实际网络请求;独立测试确保移除任一边界都会失败。
- `agent/request-error` 只携带当前失败事实;每个插件在终态空闲时清零其逐 agent 私有计数器,暂时性失败/上下文溢出交替发生的集成测试证明两种策略只消耗各自的有限预算。
- `dsh-llm-retry` 在 Loader 启动时验证每个配置字段,使用 `next()` 委托所有不合格路径,而且在没有其他策略时最多发起 `maxTransientRetries + 1` 次提供方请求。
- 退避期间执行 HMR 的测试证明:释放过程会注销监听器、中止并等待其捕获的回调,释放后不发起重试请求,也不留下存活的定时器或 promise。
- `agent/request-error` 携带当前失败事实、不可变的先前已重试失败事实,以及实际服务注册所对应的不可变重试策略;成功会清除历史,暂时性失败/上下文溢出交替发生的集成测试证明两种策略只消耗各自的有限预算。
- 每个提供方适配器都在 Loader 启动时验证其嵌套重试策略,`ctx.llm` 则将该策略与路由一同捕获;normal mode 会委托不合格路径,而且在没有其他策略时最多发起 `maxRetries + 1` 次提供方请求。
- 退避期间执行 HMR 的测试证明:释放过程会注销监听器、中止并等待其捕获的回调,释放后不发出重试决策,也不留下存活的定时器或 promise。
- 纯单元测试覆盖暂时性 code 选择、指数退避和抖动边界、有效及超出上限的 `Retry-After`、耗尽的预算、确定性定时器/随机数 seam,以及退避期间中止。
- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在新步骤中重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compact-basic` 上下文溢出恢复的组合。
- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在新轮次中重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compact-basic` 上下文溢出恢复的组合。
- 部分分片集成测试证明:失败分片仍归属于失败步骤,该步骤不会提交 assistant 消息或工具副作用,成功的重试具有不同的来源信息。
- 插件拥有的不进入表层的 `llm/retry` 事件可在 JSONL 和 SQLite 往返后保留,被消息派生忽略,并驱动 TUI 撤回和计划重试渲染。无密钥快照覆盖调度、取消、成功和耗尽;ACP 自动化快照确认,被丢弃的尝试不会通过协议发出,而恢复后的回复会正常发出。
- 空闲看门狗测试证明:只有 `next()` 尚未完成时才会重新布防稳定信号;在消费方思考期间及 `finally` 中会解除布防;它与总调用 deadline 以及更早发生的调用方中止分开分类。适配器测试证明该信号会终止底层请求,而不只是与其脱离。
@@ -132,12 +122,12 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
## 后果
- 每次暂时性恢复尝试都以一个已关闭失败轮次加 `llm/retry` 的形式可见,有界策略还会防止隐藏的 SDK 重试成倍增加成本。即使没有分片到达,重试仍可能造成提供方重复计费;有限的尝试预算只能限制而无法消除此风险。
- 每次重试尝试都以一个已关闭失败轮次加 `llm/retry` 的形式可见,适配器级的单次尝试行为会防止隐藏的 SDK 重试成倍增加策略决策。即使没有分片到达,重试仍可能造成提供方重复计费;normal mode 会限制此风险,而显式 always mode 会接受它,直至取消或成功。
- 提供方 SDK 可能隐藏状态或重试标头。适配器会保留 SDK 公开的稳定事实,否则使用粗粒度 code,而不会让恢复策略解析脆弱的文本。
- 持久重试事件扩展了会话协议和 UI 状态机。事件与其消费方一同交付,可避免产生无人使用的遥测词汇;但以后更改 schema 仍需要同步完成持久化和回放工作。
- 清除失败步骤的实时分片可能会明显撤回输出。与把丢弃的文本或不完整工具 JSON 呈现为已提交历史相比,这是更好的选择;快照固定这一转换。
- 适配器局部的空闲强制机制可以终止停滞的传输,而不会计入消费方思考时间。每个传输边界的契约测试会防止 SDK 漂移。
- 多个恢复插件会叠加各自的有限预算。此处它们的分类器互不重叠;重叠的分类器会形成依赖注册顺序的策略,必须由引入它们的插件记录并测试。
- 多个 normal 恢复插件会叠加各自的有限预算。always mode 会先委托,再提供无界回退;重叠的分类器仍会形成依赖注册顺序的策略,必须由引入它们的插件记录并测试。
## 相关资料

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-14-provider-routed-llm-adapters.md: 98205d18d07752e0cdba86d7cba80368d45fd816
2026-07-14-provider-routed-llm-adapters.zh.md: c35225a86baf4c2d09732b5940abbc8046d365fb
2026-07-14-provider-routed-llm-adapters.md: 1bd9197667f6e49c5025c98b4a77500f78595c2b
2026-07-14-provider-routed-llm-adapters.zh.md: 4d57f2cb33ac296500a4a19771ea493621ff93f6

View File

@@ -28,7 +28,7 @@ A provider has exactly one adapter owner in a Cordis context. `dsh-llm-deepseek`
### Explicit pi-ai provider profiles
`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, and a Harness stream-idle timeout. Provider retry fields are deliberately absent: the adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` owns bounded agent-level recovery. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback.
`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, a Harness stream-idle timeout, and a provider-owned `retryPolicy`. The adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` executes the resolved policy at the agent failed-step seam. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback.
The plugin registers all configured provider names against one `PiAiAdapter` in one all-or-nothing call. A request uses its provider to select the matching profile and finds its model in `getModels(provider)` to obtain the catalog descriptor. An unknown provider fails at plugin load; an unknown model fails before network I/O with `UNKNOWN_MODEL`. The catalog object is never mutated. When a profile supplies `baseURL`, the adapter clones the selected descriptor and overrides only `baseUrl`, so a private endpoint can retain pi-ai's API, capabilities, compatibility flags, context limits, and reasoning map. The private endpoint must implement the selected provider's protocol, and the model id must still exist in the installed pi-ai catalog.

View File

@@ -28,7 +28,7 @@ Status: implemented
### 显式 pi-ai 提供方配置
`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一,并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称,以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时和 Harness 流空闲超时。配置中有意不提供重试字段:适配器强制将 pi-ai 的 `maxRetries` 设为零,使一次 `stream()` 调用只发起一次可见的提供方请求;有界的 agent 层恢复由 `dsh-llm-retry` 负责。凭据不设全局值:显式密钥仅对所属配置生效;未提供密钥时,pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置,不会回退到环境认证。
`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一,并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称,以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时、Harness 流空闲超时,以及由提供方拥有的 `retryPolicy`。适配器强制将 pi-ai 的 `maxRetries` 设为零,使一次 `stream()` 调用只发起一次可见的提供方请求;`dsh-llm-retry` 则在 agent 失败步骤 seam 上执行解析后的策略。凭据不设全局值:显式密钥仅对所属配置生效;未提供密钥时,pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置,不会回退到环境认证。
插件通过一次全有或全无调用,将所有已配置的提供方名称注册到同一个 `PiAiAdapter`。请求按 provider 选择对应配置,并在 `getModels(provider)` 中查找模型以取得目录描述符。未知提供方会在插件加载时失败;未知模型会在网络 I/O 前以 `UNKNOWN_MODEL` 失败。适配器不会修改目录对象。当配置提供 `baseURL` 时,适配器复制选中的描述符,仅覆盖 `baseUrl`,使私有端点保留 pi-ai 的 API、能力、兼容标志、上下文限制与推理映射。私有端点必须实现所选提供方的协议,模型 ID 也仍须存在于已安装的 pi-ai 目录中。