Merge branch 'codex/invariant-service-seam' into codex/invariant-package-registration-gate

This commit is contained in:
Tianyi Cui
2026-07-20 19:48:51 +08:00
391 changed files with 15415 additions and 8532 deletions

View File

@@ -10,7 +10,7 @@ The harness needs one internal language for messages that the loop, session log,
Own the vocabulary: messages are arrays of typed content blocks (`text`, `reasoning`, `tool-call`, `tool-result`), with the union derived from the merge-extensible `ContentBlockMap` so plugins add block types via declaration merging. The same merge-extensible-map pattern types every "stringly" field (`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming is a raw chunk protocol; `BlockAssembler` is the single shared assembly implementation. Adapters translate to provider wire formats — mapping cost lives in adapters, where it belongs.
In-session context injection (`context/message`, `steering/message`) renders as tagged user-role envelopes (the system-reminder pattern) rather than a new role, so adapters carry zero burden. Live-adapter validation confirms this rendering for current DeepSeek behavior; a future provider-specific mismatch belongs in that adapter rather than a new canonical role.
In-session context injection (`context/message`) and mid-turn steering (`steering/message`) originally rendered as tagged user-role envelopes (the system-reminder pattern) rather than a new role, so adapters carry zero burden. Both now project as plain user content with no wrapper; see [the injected-content-envelope Agent Note](../simplification/2026-07-20-unwrap-injected-content-envelopes.md). Live-adapter validation confirms this rendering for current DeepSeek behavior; a future provider-specific mismatch belongs in that adapter rather than a new canonical role.
## Alternatives considered

View File

@@ -12,7 +12,7 @@ Three seams: the queue-aware cancel, the `AgentHandle` disposer, and the bash ow
### 1. Queue-aware `Agent.cancel(reason?)`
A new `cancel()` verb on the `Agent` interface — the single public stop primitive. (It originally shipped alongside a narrower step-only `abort()`; that verb was later removed as unused, leaving `cancel()` the only public way to stop work.) It clears the inbox's queued + steering FIFOs, aborts the in-flight step if any, and drives a **turn-scoped cancellation marker** the driver loop checks at every turn-decision point — so a prompt that is queued-but-not-yet-started never runs, a cancel landing in the pre-step / continuation window drops the about-to-run turn (ending it `aborted`), and a later prompt cannot be batched into the cancelled turn. `whenIdle()` reaches post-cancel quiescence. ACP `session/cancel` maps to `cancel()`. The marker is armed ONLY when there is something to cancel, so an idle no-op cancel cannot strand the next prompt.
A new `cancel()` verb on the `Agent` interface — the single public stop primitive. (It originally shipped alongside a narrower step-only `abort()`; that verb was later removed as unused, leaving `cancel()` the only public way to stop work.) It clears the inbox's queued + steering FIFOs, aborts the in-flight step if any, and drives a **turn-scoped cancellation marker** the driver loop checks at every turn-decision point — so a prompt that is queued-but-not-yet-started never runs, a cancel landing in the pre-step / continuation window drops the about-to-run turn (ending it `aborted`), and a later accepted prompt remains an independent queued turn. `whenIdle()` reaches post-cancel quiescence. ACP `session/cancel` maps to `cancel()`. The marker is armed ONLY when there is something to cancel, so an idle no-op cancel cannot strand the next prompt.
### 2. `AgentHandle` async disposer
@@ -29,7 +29,7 @@ Background-task ownership moved from a `tool-bash` plugin-local `Map<string, Age
These invariants hold and are pinned by tests:
- ACP disconnect/session close leaves no registered agent AND no session-store entry for that session, even when `session/load` races teardown.
- `session/cancel` before a queued prompt starts prevents that prompt from running and cannot batch the next prompt into the cancelled turn.
- `session/cancel` before a queued prompt starts prevents that prompt from running; a later accepted prompt remains an independent queued turn.
- A `tool-bash` HMR reload does NOT make an existing background task readable or killable by a different session (ownership survives on the executor).
- Existing non-ACP demos still work without managing handles explicitly; config-created agents remain owned by the `AgentLoop` plugin fiber.

View File

@@ -67,3 +67,5 @@ Every surface-eligible event must carry `surfaceOp` or it would disappear from d
- **`packages/session-persistence/session-persistence`**: Abstract interface unchanged.
The surface is the foundation for future history manipulation. A compaction or tool-result-prune plugin appends one of the existing message-producing event types (a `user/message` carrying the summary, say) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed entries — the new event takes the range's place on the surface while the plugin's own trace events (e.g. `compaction/start`, `compaction/end`) stay off it. Replay preserves the decision deterministically.
A `tool/result` replacement may rewrite exactly one current `tool/result` and must preserve every data field except `content`. Session acceptance enforces this rule together with positional range and provenance validation, independent of optional diagnostic plugins.

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: f1a1868cd00007fb24efb21779dcc94c098b54e2
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: a4993de2830301610bb2a9b0d28e8bbdf0ed9c46
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: d470cceaff68229b3872d0ade93d5fabc2e10c3f
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: b7753fb226638b16b0f244b681cd2b9bcc9f25c2

View File

@@ -16,9 +16,9 @@ Successful calls are not the only pressure signal. A provider can reject a reque
`agent/pre-step` is narrowed to `(agent, turn, step, signal)`. It remains a generic serial checkpoint before `step/start`, but it carries no compaction-only prompt or prefix fields.
The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after assistant output, every dispatched or synthetic tool result, post-tool context, and steering are durable, but before `step/end`. This placement gives pressure policy the complete successful-call state without splitting an assistant tool call from its result. A listener failure is an ordinary turn failure; it never enters model-request recovery.
The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after assistant output, every dispatched or synthetic tool result, post-tool context, and steering are durable, but before `step/end`. This placement gives pressure policy the complete successful-call state without splitting an assistant tool call from its result. A propagated listener failure is an ordinary turn failure; it never enters model-request recovery. Compact-basic contains its expected operational failures as described below.
`dsh-compact-basic` reads the exact latest routed model from the durable request header only to establish that a completed route exists, then asks the singleton `ctx.tokenMeter` to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work; any durable non-empty model name uses the same estimator. Operational measurement or summarization failures warn and continue with full history.
`dsh-compact-basic` reads the exact latest routed model from the durable request header only to establish that a completed route exists, then asks the singleton `ctx.tokenMeter` to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work; any durable non-empty model name uses the same estimator. Operational measurement or summarization failures warn and continue from the latest durable surface: full history before any replacement, or the pruned surface if pruning already landed.
### Request recovery is limited to the final model boundary
@@ -32,17 +32,17 @@ If cancellation lands after assistant tool calls are durable but before all call
`CompactService.compactIfNeeded(agent, trigger, signal)` accepts `trigger: 'pressure' | 'context-overflow'`. The interface gains no estimation methods or token types; `ctx.tokenMeter` remains the reusable accounting owner.
For `pressure`, compact-basic applies the service-wide threshold and retained-tail policy to one unified `ctx.tokenMeter.measure()` result. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization provider/model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`.
For `pressure`, compact-basic applies the service-wide threshold and retained-tail policy to one unified `ctx.tokenMeter.measure()` result. Below pressure it returns without pruning. Once pressure qualifies, optional `ctx.toolResultPrune` rewrites oversized current results and compact-basic remeasures through the same meter; safe pressure skips the model call, while remaining pressure selects and summarizes from the pruned surface. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. The common defaults remain threshold ratio `0.8`, retained history `floor(contextWindow × 0.16)`, summarization provider/model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`.
For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It chooses the maximal tool-balanced head range while leaving the newest indivisible unit, then attempts exactly one shrinking compaction under the same signal. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` only when compaction succeeds and the generation increases. A backend returning a result without replacement cannot authorize retry.
For canonical overflow, compact-basic bypasses scalar pressure and the normal retained-token budget. It prunes first, then chooses the maximal tool-balanced head range while leaving the newest indivisible unit and attempts one shrinking summary compaction under the same signal when a range exists. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` whenever pruning or summarization increases it. This remains true when pruning lands before later summary work throws; cancellation still wins. A backend returning a result without replacement cannot authorize retry, while pruning-only progress can authorize a retry without a `CompactionResult`.
`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, a missing routed model, no safe range, no generation change, and recovery throws all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. Cancellation or disposal remains authoritative even if recovery work completes concurrently.
`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, a missing routed model, no safe range, no generation change, and recovery throws before any replacement all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. A recovery throw after generation advances authorizes retry from durable progress; cancellation or disposal remains authoritative even if recovery work completes concurrently.
The default summarizer resolves explicit configuration, then the latest logged route, then agent options. Because direct `llm/stream` middleware may reroute that auxiliary call, `compact/summary.{provider, model}` records the final mutable `GenerateOptions` target observed after dispatch rather than the pre-waterfall candidate.
## Testing
Unit tests cover final-adapter failure provenance and identity, closed-step retry numbering and reset, cancellation and disposal, post-step ordering, routed-envelope pressure, balanced overflow reduction, generation proof, caps, delegation, and auxiliary-call routing. Real-loop tests cover thrown and in-band overflow through compaction to a reconstructed retry request.
Unit tests cover final-adapter failure provenance and identity, closed-step retry numbering and reset, cancellation and disposal, post-step ordering, routed-envelope pressure, pressure-gated pruning, pruning-only relief, pruned-input summarization, balanced overflow reduction, durable prune progress before later failure, generation proof, caps, delegation, and auxiliary-call routing. Real-loop tests cover thrown and in-band overflow through pruning or summary compaction to a reconstructed retry request.
## Alternatives considered
@@ -54,8 +54,8 @@ Unit tests cover final-adapter failure provenance and identity, closed-step retr
## Consequences
Post-step pressure describes the completed routed request, including durable tool results and request-only prefix fields. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change.
Post-step pressure describes the completed routed request, including durable tool results and request-only prefix fields. Optional model-free pruning removes predictable tool-output bulk before summary selection and can independently create retry-worthy progress. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change.
The cost is one additional serial checkpoint on successful steps and adapter-maintained overflow classification. Provider wording and heuristic character density remain maintenance risks. Surface compaction still cannot repair an envelope that alone exceeds the window or split one indivisible oversized message/tool unit.
The cost is one additional serial checkpoint on successful steps and adapter-maintained overflow classification. Provider wording and heuristic character density remain maintenance risks. Surface compaction still cannot repair an envelope that alone exceeds the window, split an indivisible non-tool node, or repair a tool unit whose non-prunable remainder remains oversized. The optional pruner can repair an otherwise indivisible tool pair when removable text-bearing tool-result content is the bulk.
This Agent Note supersedes only the pre-step automatic-trigger portion of the [compaction capability-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md). The service split, standalone token meter, balanced range contract, log-recorded lock, summary replacement, and sole `summarize()` subclass hook remain unchanged.

View File

@@ -16,9 +16,9 @@ Status: implemented
`agent/pre-step` 收窄为 `(agent, turn, step, signal)`。它仍是 `step/start` 之前的通用串行检查点,但不再携带压缩专用的提示词或前缀字段。
循环在 assistant 输出、所有已分发或合成的工具结果、工具后上下文与 steering 都持久化之后、`step/end` 之前,触发等待式串行 `agent/post-step(agent, turn, step, signal)`。该位置让压力策略看到完整的成功调用状态,同时不会拆开 assistant 工具调用与其结果。监听器失败属于普通 turn 失败,绝不会进入模型请求恢复。
循环在 assistant 输出、所有已分发或合成的工具结果、工具后上下文与 steering 都持久化之后、`step/end` 之前,触发等待式串行 `agent/post-step(agent, turn, step, signal)`。该位置让压力策略看到完整的成功调用状态,同时不会拆开 assistant 工具调用与其结果。向外传播的监听器失败属于普通 turn 失败,绝不会进入模型请求恢复;compact-basic 会按下文所述在内部处理其预期的操作性失败。
`dsh-compact-basic` 从持久请求头读取精确的最新实际路由模型,只用它确认已经存在完整路由,随后让单例 `ctx.tokenMeter` 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断,因此不执行工作;任意持久记录的非空模型名都使用同一个估算器。操作性的计量或摘要失败会发出警告,并继续使用完整历史。
`dsh-compact-basic` 从持久请求头读取精确的最新实际路由模型,只用它确认已经存在完整路由,随后让单例 `ctx.tokenMeter` 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断,因此不执行工作;任意持久记录的非空模型名都使用同一个估算器。操作性的计量或摘要失败会发出警告,并从最新持久表层继续:任何替换发生前使用完整历史;若剪枝已经落盘,则使用已剪枝表层。
### 请求恢复只覆盖最终模型边界
@@ -32,17 +32,17 @@ Status: implemented
`CompactService.compactIfNeeded(agent, trigger, signal)` 接收 `trigger: 'pressure' | 'context-overflow'`。接口不增加估算方法或 token 类型;`ctx.tokenMeter` 继续作为可复用的核算所有者。
对于 `pressure`,compact-basic 把服务级阈值与保留尾部策略应用到一次统一的 `ctx.tokenMeter.measure()` 结果。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要提供方/模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`。
对于 `pressure`,compact-basic 把服务级阈值与保留尾部策略应用到一次统一的 `ctx.tokenMeter.measure()` 结果。低于压力时直接返回,不执行剪枝。压力达到条件后,可选的 `ctx.toolResultPrune` 会改写当前表层中过大的工具结果,compact-basic 再通过同一个 meter 重新计量;若压力恢复安全则跳过模型调用,否则从已剪枝表层选择范围并生成摘要。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史 `floor(contextWindow × 0.16)`、摘要提供方/模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`。
对于规范化溢出,compact-basic 绕过标量压力与普通保留 token 预算。它在保留最新不可分割单元的同时,选择最大的工具配对平衡头部范围,并在同一 signal 下只尝试一次缩小压缩。自动监听器先记录 `session.surface.replaceGeneration`,只有压缩成功且 generation 增加时才返回 `{ action: 'retry' }`。后端若只返回结果但没有替换表层,不能授权重试。
对于规范化溢出,compact-basic 绕过标量压力与普通保留 token 预算。它先执行剪枝,再在保留最新不可分割单元的同时选择最大的工具配对平衡头部范围;存在范围时,才在同一 signal 下尝试一次缩小摘要压缩。自动监听器先记录 `session.surface.replaceGeneration`,剪枝或摘要让 generation 增加时就返回 `{ action: 'retry' }`。即使剪枝先落盘而后续摘要工作抛错,这条规则仍然成立;取消依然优先。后端若只返回结果但没有替换表层,不能授权重试;只有剪枝取得进展时,即使没有 `CompactionResult` 也可以授权重试。
`maxOverflowRetries` 可选且默认为 `1`;`0` 只禁用溢出恢复,不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失路由模型、没有安全范围、generation 未变化,以及恢复抛错都会委托给下一个监听器。若没有后续恢复,循环报告原始提供方错误对象与代码。即使恢复工作并发完成,取消或销毁仍具有最终优先级。
`maxOverflowRetries` 可选且默认为 `1`;`0` 只禁用溢出恢复,不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失路由模型、没有安全范围、generation 未变化,以及在任何替换之前恢复抛错,都会委托给下一个监听器。若没有后续恢复,循环报告原始提供方错误对象与代码。generation 增加后的恢复抛错会基于持久进展授权重试;即使恢复工作并发完成,取消或销毁仍具有最终优先级。
默认摘要器依次解析显式配置、最近记录的路由与 agent options。因为直接 `llm/stream` 中间件可以重新路由该辅助调用,`compact/summary.{provider, model}` 记录分发后最终可变的 `GenerateOptions` 目标,而不是 waterfall 之前的候选值。
## 测试
单元测试覆盖最终适配器失败的来源与身份、已关闭 step 的重试编号与重置、取消与销毁、post-step 顺序、已路由信封压力、平衡溢出缩减、generation 证明、上限、委托与辅助调用路由。真实循环测试覆盖抛出式和带内溢出,并验证压缩后的重试请求从替换表层重建。
单元测试覆盖最终适配器失败的来源与身份、已关闭 step 的重试编号与重置、取消与销毁、post-step 顺序、已路由信封压力、压力门控剪枝、剪枝独立解除压力、从已剪枝输入生成摘要、平衡溢出缩减、后续失败前已落盘的剪枝进展、generation 证明、上限、委托与辅助调用路由。真实循环测试覆盖抛出式和带内溢出,并验证剪枝或摘要压缩后的重试请求从替换表层重建。
## 考虑过的替代方案
@@ -54,8 +54,8 @@ Status: implemented
## 后果
Post-step 压力描述已完成的路由请求,包括持久工具结果与仅请求前缀字段。当成功 usage 锚点不存在时,规范化溢出提供兜底路径。恢复有明确上限、以取消为准,并保持单调:只有模型可见的表层 generation 变化后才重试。
Post-step 压力描述已完成的路由请求,包括持久工具结果与仅请求前缀字段。可选的无模型剪枝会在选择摘要前移除可预测的工具输出体积,也能独立产生足以重试的进展。当成功 usage 锚点不存在时,规范化溢出提供兜底路径。恢复有明确上限、以取消为准,并保持单调:只有模型可见的表层 generation 变化后才重试。
代价是成功 step 增加一个串行检查点,并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况,也不能拆分单个不可分割的超大消息或工具单元。
代价是成功 step 增加一个串行检查点,并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况,也不能拆分不可分割的非工具节点,或修复非可剪枝剩余部分仍然过大的工具单元。若可移除的文本工具结果是主要体积,可选剪枝器仍可修复原本不可分割的工具配对。
本 Agent Note 只取代[压缩能力接缝 Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 中的 pre-step 自动触发部分。服务拆分、独立 token meter、平衡范围契约、日志记录锁、摘要替换与唯一 `summarize()` 子类 hook 均保持不变。