Merge remote-tracking branch 'origin/master' into worktree/semantic-session-checkpoints
# Conflicts: # docs/event-producer-consumer.md # docs/module-graph.md # examples/tui-agent/tests/tui-keyless-smoke.e2e.ts # packages/examples/acp-demo/package.json # packages/examples/cli-demo/tsconfig.json # packages/support/invariants/src/index.ts # packages/support/invariants/tests/invariants.spec.ts # pnpm-lock.yaml
This commit is contained in:
Yichen Jiang
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-21-compaction-summary-prefix-cache-reuse.md: 490eb57a5891bf9cd0799c5d49d25d4e9838041f
|
||||
2026-07-21-compaction-summary-prefix-cache-reuse.zh.md: 02412ff07e87e12c7e7de00b5c69e1282433f735
|
||||
@@ -0,0 +1,45 @@
|
||||
# Agent Note: The summarization call replays the conversation prefix for KV-cache reuse
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-21-compaction-summary-prefix-cache-reuse.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Automatic compaction fires mid-conversation, right after the loop has warmed the provider's KV cache with the last routed request (`system` + `tools` + `messagePrefix` + derived history). The default summarizer then issued a *separate* auxiliary request whose prefix shared nothing with that warm request: a bespoke summarizer `system` prompt followed by the older history flattened to a single rendered transcript string. A provider caches on the request's leading token sequence, so a first token that differs — a different system prompt — invalidates the entire cached prefix. Every compaction therefore paid full prompt-processing cost for the whole replayed history twice: once for the conversation request that tripped pressure, and again for the summarization call, defeating the cache exactly when the conversation is largest.
|
||||
|
||||
## Decision
|
||||
|
||||
The summarization directive moves from the **front** of the request (a fresh `system` prompt) to the **end** of the conversation (the final `user` message). The auxiliary call now reproduces the last routed request's prefix verbatim and appends one trailing instruction, so it is a genuine prefix-extension of the warm request and the provider reuses the cached tokens.
|
||||
|
||||
### `SummarizationInput` carries the replayed prefix, not a rendered string
|
||||
|
||||
`summarize()` (and the internal `summarizeWithLlm`) take a `SummarizationInput` — `{ system?, tools?, messages }` — instead of a flat transcript string. `region.ts` builds it from `session.requestHeader()` (the durable `system`, `tools`, and `messagePrefix`) plus the shadowed region mapped through `session.deriveEventMessage`, which yields byte-identical `Message` objects to what `deriveMessages()` folded into the routed request. `summarizeWithLlm` forwards `system` and `tools` onto `GenerateOptions` and sends `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`. `tools` ride along even though the summarizer never calls one: dropping them would shorten the token sequence and break alignment with the cached request.
|
||||
|
||||
### The instruction is a trailing user message
|
||||
|
||||
`COMPACTION_INSTRUCTION` opens "You are now acting as a compaction engine…" and directs the model to condense *the conversation ABOVE*. It keeps the prior checkpoint's structured headings and adds two rules the front-loaded system prompt did not need in its new position: do not mention the summarization request, and output only the checkpoint text without calling a tool. The shadowed region always ends on a tool-pairing-balanced boundary, so appending a `user` message after it is a valid message ordering for OpenAI-compatible and DeepSeek adapters.
|
||||
|
||||
### Cache reuse is best-effort, correctness is not
|
||||
|
||||
Auto-compaction always anchors at the surface head, so the shadowed region is the head of the routed request and the replayed prefix matches it exactly — the guaranteed-hit case. Manual mid-range `compactRegion` still replays the true prefix and stays correct, but forgoes reuse because its shadowed region is not the request head. A configured `summarizationProvider`/`summarizationModel` that differs from the conversation's route also forgoes reuse; that is the deployment's explicit trade-off, not a defect. Target resolution (configured override → latest routed header → agent options, else throw) is unchanged.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the summarizer system prompt but reuse the rest** — rejected: the system slot is the very first token region a provider caches on, so a distinct summarizer system prompt invalidates the whole prefix regardless of what follows. Only moving the directive off the front recovers the cache.
|
||||
- **Send only the shadowed region without the `system`/`tools`/`messagePrefix` head** — rejected: a shorter or differently-headed sequence still diverges from the cached request at the first token, so it caches no better while losing the framing the summary needs.
|
||||
- **Omit `tools` from the summarization request** (the model never calls one) — rejected: tool schemas are part of the cached token sequence; omitting them misaligns every following token and defeats reuse.
|
||||
- **A dedicated `assistant/chunk`-emitting summarization sub-session for snapshot replay** — out of scope here; the replay gap predates this change and is tracked in the [compaction-seam note](../feature/2026-06-18-compaction-capability-seam.md).
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`dsh-compact-basic`** owns `SummarizationInput`; the protected `summarize(input, agent, signal?)` hook signature changed (acceptable pre-release), and `region.ts` gained `buildSummarizationInput` folding `deriveEventMessage` over the shadowed seqs behind the header prefix.
|
||||
- **Dead render surface removed.** The old flattening path (`renderTranscript` / `renderContentBlocks` and its spec in `dsh-compact`) had no remaining consumer and was deleted with its export.
|
||||
- **README model experience** for `dsh-compact-basic` now documents the auxiliary request as the replayed prefix plus a trailing compaction-instruction message, and its KV-cache effect as reuse of the warm conversation prefix.
|
||||
- **The framed checkpoint output is unchanged**, so the landed `user/message` and every conversation-request snapshot are unaffected; only the auxiliary request's shape changed.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit:** `compact-basic.spec.ts` asserts the auxiliary call forwards `system`/`tools`/leading messages and appends the compaction instruction as the final message, and that `compactRegion` replays the latest routed header prefix. Existing content assertions read the summarizer input through the replayed messages rather than a transcript string.
|
||||
- **Loop:** `compact-loop-repro.spec.ts` classifies the summarization request by the compaction instruction in its trailing user message, and the overflow-recovery tests continue to pin conversation-vs-summary request counts across the real loop.
|
||||
- **Snapshot gap unchanged:** the summarization call still emits no `assistant/chunk` events, so it remains outside keyless replay; the pre-existing gap is owned by the [compaction-seam note](../feature/2026-06-18-compaction-capability-seam.md).
|
||||
@@ -0,0 +1,45 @@
|
||||
# Agent Note: 摘要调用回放对话前缀以复用 KV 缓存
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-21-compaction-summary-prefix-cache-reuse.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
自动压缩（compaction）在对话中途触发，恰好在循环用最后一个已路由请求（`system` + `tools` + `messagePrefix` + 派生历史）预热了提供方的 KV 缓存之后。随后默认摘要器发出一个*独立的*辅助请求，其前缀与那个已预热请求没有任何共享部分：一个专门的摘要器 `system` 提示词，后接被拍平成单个渲染后 transcript（文本记录）字符串的较早历史。提供方基于请求起始的 token 序列做缓存，因此第一个 token 只要不同（即一个不同的系统提示词），整个已缓存前缀就会失效。于是每次压缩都要为整段回放的历史付出两次完整的提示词处理成本：一次用于触发压力的对话请求，另一次用于摘要调用，恰好在对话最大时让缓存失去作用。
|
||||
|
||||
## Decision
|
||||
|
||||
摘要指令从请求的**前端**（一个全新的 `system` 提示词）移到对话的**末尾**（最后一条 `user` 消息）。辅助调用现在逐字复现最后一个已路由请求的前缀，并追加一条尾部指令，因此它是已预热请求的真正前缀扩展，提供方会复用已缓存的 token。
|
||||
|
||||
### `SummarizationInput` 携带回放的前缀，而非渲染后的字符串
|
||||
|
||||
`summarize()`（以及内部的 `summarizeWithLlm`）接受一个 `SummarizationInput`（`{ system?, tools?, messages }`）而不是一个扁平的 transcript 字符串。`region.ts` 用 `session.requestHeader()`（持久的 `system`、`tools` 和 `messagePrefix`）加上经 `session.deriveEventMessage` 映射的被遮蔽区域来构建它，后者产出与 `deriveMessages()` 折叠进已路由请求的内容字节级一致的 `Message` 对象。`summarizeWithLlm` 把 `system` 和 `tools` 转发到 `GenerateOptions`，并发送 `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`。`tools` 会一同带上，即便摘要器从不调用任何工具：丢弃它们会缩短 token 序列，破坏与已缓存请求的对齐。
|
||||
|
||||
### 指令是一条尾部 user 消息
|
||||
|
||||
`COMPACTION_INSTRUCTION` 以 "You are now acting as a compaction engine…" 开头，指示模型浓缩*上方的对话*。它保留先前检查点的结构化标题，并在其新位置上新增了两条前置系统提示词此前不需要的规则：不要提及摘要请求，以及只输出检查点文本而不调用任何工具。被遮蔽区域总是结束在工具配对平衡的边界上，因此在其后追加一条 `user` 消息，对 OpenAI 兼容适配器和 DeepSeek 适配器而言是合法的消息排序。
|
||||
|
||||
### 缓存复用是尽力而为，正确性不是
|
||||
|
||||
自动压缩总是锚定在表层头部，因此被遮蔽区域就是已路由请求的头部，回放的前缀与之完全匹配，这就是保证命中的情形。手动的中段 `compactRegion` 仍然回放真实的前缀并保持正确，但会放弃复用，因为它的被遮蔽区域不是请求头部。配置的 `summarizationProvider`/`summarizationModel` 若与对话的路由不同，也会放弃复用；这是部署方明确的权衡，而非缺陷。目标解析（配置的覆盖值 → 最新的已路由 header → agent（智能体）选项，否则抛出）保持不变。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **保留摘要器系统提示词但复用其余部分**——否决：system 槽位正是提供方最先做缓存的 token 区域，因此一个不同的摘要器系统提示词无论后面跟着什么都会使整个前缀失效。只有把指令移离前端才能恢复缓存。
|
||||
- **只发送被遮蔽区域而不带 `system`/`tools`/`messagePrefix` 头部**——否决：更短或头部不同的序列在第一个 token 处仍然与已缓存请求分叉，因此缓存效果并不更好，反而丢失了摘要所需的框架。
|
||||
- **从摘要请求中省略 `tools`**（模型从不调用任何工具）——否决：工具 schema 是已缓存 token 序列的一部分；省略它们会让后续每个 token 失去对齐，破坏复用。
|
||||
- **为快照回放专门建立一个发出 `assistant/chunk` 的摘要子会话**——此处超出范围；该回放缺口早于本次改动，记录在 [compaction-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 中。
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`dsh-compact-basic`** 拥有 `SummarizationInput`；受保护的 `summarize(input, agent, signal?)` 钩子签名发生变化（发布前可接受），并且 `region.ts` 新增了 `buildSummarizationInput`，它在 header 前缀之后对被遮蔽的 seq 折叠 `deriveEventMessage`。
|
||||
- **移除无用的渲染表面。** 旧的拍平路径（`renderTranscript` / `renderContentBlocks` 及其在 `dsh-compact` 中的 spec）已无消费方，连同其导出一并删除。
|
||||
- **README 的 Model Experience** 现在把 `dsh-compact-basic` 的辅助请求记述为回放的前缀加上一条尾部压缩指令消息，并把其 KV 缓存效果记述为复用已预热的对话前缀。
|
||||
- **带框架的检查点输出未改变**，因此落地的 `user/message` 和每个对话请求快照都不受影响；只有辅助请求的形状发生了变化。
|
||||
|
||||
## Testing
|
||||
|
||||
- **单元：** `compact-basic.spec.ts` 断言辅助调用转发 `system`/`tools`/前导消息，并把压缩指令作为最后一条消息追加，且 `compactRegion` 回放最新的已路由 header 前缀。现有的内容断言通过回放的消息而非 transcript 字符串来读取摘要器输入。
|
||||
- **循环：** `compact-loop-repro.spec.ts` 依据摘要请求尾部 user 消息中的压缩指令对其分类，溢出恢复测试则继续在真实循环中固定对话请求与摘要请求的数量。
|
||||
- **快照缺口未变：** 摘要调用仍然不发出 `assistant/chunk` 事件，因此它仍处于无密钥回放之外；这一既有缺口归 [compaction-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 所有。
|
||||
Reference in New Issue
Block a user