docs: rebalance prose cleanup and add trimming skill

packages/compact/compact-basic/README.md

@@ -8,12 +8,12 @@ This is the implementation tier of the compaction capability — see the [interf
 
 This backend owns the compaction policy:
 
-- **Estimation** — a configurable characters-per-token heuristic counts the current session prefix, derived history, and system prompt.
-- **Retention** — compact the oldest whole surface units while preserving a recent tail and tool-call/result pairing. Turn boundaries do not protect old steps inside a runaway turn. An indivisible unit larger than the budget remains out of scope.
-- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source.
-- **Summarization** — a direct `llm/stream` call uses the configured model and cap. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls.
+- **Estimation** — a configurable characters-per-token heuristic counts the current session prefix supplied to pre-step, derived history, and system prompt, matching the next request rather than stale logged prefix state.
+- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts. Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
+- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
+- **Summarization** — a direct `llm/stream` call uses the configured model and cap without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
 - **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
-- **Lifecycle** — `compactRegion()` records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step so the loop derives history once after mutation.
+- **Lifecycle** — `compactRegion()` records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step, outside an open step, so a tool-heavy turn remains compactable and the loop derives history once after mutation.
 - **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Recoverable failure records an error end and leaves the surface unchanged.
 
 `estimateContentTokens()` and `summarize()` are overridable hooks: a tokenizer-based or template-based backend can subclass `BasicCompactService` and override just those, reusing the retention walk and surface plumbing. `summarize()` returns the summary blocks together with the call envelope it actually used (`{ summary, model, maxTokens? }`) — the caller logs that envelope on the `compact/summary` provenance event, so an overriding backend reports its own envelope honestly.

packages/compact/compact-basic/src/index.ts

@@ -1,6 +1,8 @@
 /**
- * `BasicCompactService`: the first implementation of the `@deepseek-ai/dsh-compact` seam. It
- * owns the entire compaction strategy.
+ * Basic compaction backend. It estimates request pressure, retains a recent
+ * tool-balanced surface tail, summarizes the older head through a one-shot model
+ * call, and replaces that head with one checkpoint. Auto-compaction runs before
+ * every step so a growing turn can compact its earlier closed steps.
  * @module @deepseek-ai/dsh-compact-basic
  */
 
@@ -29,8 +31,8 @@ const SUMMARY_OPEN_TAG = '<compacted-summary>'
 const SUMMARY_CLOSE_TAG = '</compacted-summary>'
 
 /**
- * The summarization system prompt: instructs the model to condense the conversation into a
- * fixed, fully-populated structure rather than freeform bullets.
+ * Fixed summary structure for resumable checkpoints. A tagged prior checkpoint
+ * is merged with newer history instead of copied forward verbatim.
  */
 const SUMMARIZE_SYSTEM_PROMPT = [
   'You are a compaction engine for an AI coding assistant. Condense the conversation transcript into a structured checkpoint that lets another model resume the work with no loss of essential context.',
@@ -73,8 +75,8 @@ const CHECKPOINT_PREAMBLE =
   'This is an automatically generated checkpoint condensing an earlier span of the conversation to free up context. Treat the captured context as established background and build on it without restating it. Continue the task directly from the messages that follow, without acknowledging this checkpoint.'
 
 /**
- * Map a terminal `FinishReason` to the error a SUMMARIZATION must throw, or `undefined` for an
- * acceptable finish. `FinishReason` is merge-extensible.
+ * Map a terminal summary failure to an error. A max-token finish is rejected
+ * because committing an incomplete checkpoint would shadow the full history.
  */
 function finishError(finish: FinishReason): Error | undefined {
   switch (finish.kind) {
@@ -116,7 +118,8 @@ export class BasicCompactService extends CompactService {
     this.config = resolveConfig(config)
 
     if (this.config.auto) {
-      // Auto-compaction: delegate to compactIfNeeded before every step.
+      // Check before every step so a single growing turn can compact earlier closed steps.
+      // This serial pre-step seam mutates the surface outside the pending step.
       ctx.on('agent/pre-step', async (agent: Agent, _turn: number, _step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal) => {
         try {
           const result = await this.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)
@@ -223,8 +226,9 @@ export class BasicCompactService extends CompactService {
   }
 
   /**
-   * Summarize conversation text into content blocks via `ctx.llm.stream()` assembled through a
-   * `BlockAssembler`.
+   * Summarize through a direct one-shot `ctx.llm.stream()` call, not an agent
+   * step or `agent/request` dispatch. Failure finishes and truncated summaries
+   * reject; the signal is forwarded and only text reaches the checkpoint.
    *
    * @param text - plain-text rendering of the conversation region to condense.
    * @param agent - supplies the fallback model and the session id stamped on
@@ -274,10 +278,10 @@ export class BasicCompactService extends CompactService {
   // ---- Core API (implements the abstract contract) ----
 
   /**
-   * The sole token-pressure gate: estimate the NEXT request's pressure — the session prefix +
-   * the surface-derived history + the system prompt ({@link estimatePressure}) — and if it
-   * exceeds the threshold (`contextWindow * thresholdRatio`), compact the oldest surface nodes
-   * outside the `retainTokens` budget.
+   * The sole pressure gate: count the next request's prefix, derived history,
+   * and system prompt. Above threshold, retain a recent tool-balanced tail and
+   * compact the head, reconsolidating any prior automatic checkpoint. Returns
+   * `null` when no safe or necessary range exists.
    */
   override async compactIfNeeded(
     agent: Agent,
@@ -333,7 +337,7 @@ export class BasicCompactService extends CompactService {
     agent: Agent,
     signal?: AbortSignal,
   ): Promise<CompactionResult> {
-    // Resolve the range by surface POSITION, not numeric seq interval.
+    // Resolve by surface position: a newer replacement seq may occupy an older slot.
     const nodes = session.surface.nodes
     const startIdx = nodes.findIndex(n => n.seq === start)
     const endIdx = nodes.findIndex(n => n.seq === end)
@@ -343,8 +347,7 @@ export class BasicCompactService extends CompactService {
       throw new Error(`compactRegion: start seq ${start} (position ${startIdx}) is after end seq ${end} (position ${endIdx}) on the surface`)
     }
 
-    // The region must never split a step's assistant-message tool-calls from their tool/results
-    // (which would orphan one side and produce a transcript every provider rejects).
+    // Both range edges must preserve assistant tool-call/result pairing.
     const events = session.events
     if (!isToolPairingBalanced(nodes, events, start)) {
       throw new Error(`compactRegion: start seq ${start} is not a balanced boundary (would split a step's tool-call/result pair)`)

packages/compact/compact-basic/tests/compact-basic.spec.ts

@@ -203,7 +203,8 @@ function expectNoOrphanToolResults(messages: Message[]): void {
 
 describe('BasicCompactService step-alignment (never split a tool-call/result pair)', () => {
   it('compactIfNeeded rounds the retained boundary head-ward to keep a whole step (no orphaned tool-result)', async () => {
-    // 3 turns, each one step = { assistant(tool-call), tool/result }.
+    // Retain the recent tail while the older assistant/result pairs compact as
+    // whole units; no boundary may orphan a result.
     const svc = createTestService({ contextWindow: 280, thresholdRatio: 0.5, retainTokens: 55 })
     const session = toolTurnSession(3)
 
@@ -218,7 +219,8 @@ describe('BasicCompactService step-alignment (never split a tool-call/result pai
   })
 
   it('compactIfNeeded returns null when the only compactable region is an un-splittable single step', async () => {
-    // The surface is exactly one step: [assistant(tool-call), tool/result].
+    // The only candidate cut is inside one assistant/result pair; with no safe
+    // compactable prefix, decline rather than split it.
     const s = new Session(SessionId('one-step'))
     s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
     s.append('step/start', { turn: 1, step: 1 })
@@ -587,14 +589,16 @@ describe('BasicCompactService.compactIfNeeded', () => {
   })
 
   it('returns null when the whole surface fits the retain budget (over threshold by role/system overhead)', async () => {
-    // threshold = floor(480*0.1) = 48.
+    // Role overhead pushes the request above its 48-token threshold, but the
+    // raw four-node retention walk remains below retainTokens=45, so all fit.
     const svc = createTestService({ contextWindow: 480, thresholdRatio: 0.1, retainTokens: 45 })
     const session = multiTurnSession(2, 1)
     expect(await compactIfNeeded(svc, session, '', 'm', SIGNAL)).toBeNull()
   })
 
   it('compacts a runaway turn: its early CLOSED steps summarize while recent steps stay verbatim', async () => {
-    // The Regression that motivated dropping turn-protection.
+    // Completed early steps of the open turn remain eligible; protecting the
+    // whole turn would make a runaway turn impossible to compact.
     const svc = createTestService({ contextWindow: 800, thresholdRatio: 0.1, retainTokens: 25 })
     const s = new Session(SessionId('runaway'))
     // ONE open turn with 5 closed steps; each step is [asst(tool-call), result].
@@ -740,8 +744,8 @@ describe('BasicCompactService blocking (compaction in progress)', () => {
   })
 
   it('is not wedged by an orphaned compact/start from a prior (now-closed) turn', async () => {
-    // A crash mid-compaction left a compact/start with no compact/end; the turn it lived in was
-    // later closed (persistence repair appends turn/end).
+    // An orphaned start in a closed repaired turn is stale; only the current
+    // turn participates in the in-progress lock.
     const svc = createTestService()
     const s = new Session(SessionId('stale-lock'))
     s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
@@ -1218,7 +1222,8 @@ describe('BasicCompactService auto-compaction (agent/pre-step listener)', () =>
 
   it('summarization is interceptable at llm/stream (model routing for direct calls)', async () => {
     const { ctx, adapter } = await ctxWithModel('ROUTED SUMMARY', 'routed-model')
-    // One-shot summaries use llm/stream, not the loop's agent/request seam.
+    // One-shot summaries bypass agent/request but remain mutable at llm/stream;
+    // adapter selection happens after the waterfall rewrite.
     ctx.on('llm/stream', (options, next) => {
       options.model = 'routed-model'
       return next()
@@ -1472,16 +1477,13 @@ describe('BasicCompactService edge cases', () => {
     const svc = createTestService()
     const s = new Session(SessionId('empties'))
     s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
-    // Step 1: an empty-text user, an empty-reasoning assistant with NO tool-call
-    // (balanced: nothing to answer), and empty context/steering — all extract to
-    // nothing and are skipped.
     s.append('step/start', { turn: 1, step: 1 })
     s.append('user/message', { content: [{ type: 'text', text: '' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
     s.append('assistant/message', { turn: 1, step: 1, content: [{ type: 'reasoning', text: '' }] }, { surfaceOp: 'append' })
     s.append('context/message', { content: [], source: { kind: 'user' } }, { surfaceOp: 'append' })
     s.append('steering/message', { turn: 1, content: [{ type: 'text', text: '' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
     s.append('step/end', { turn: 1, step: 1 })
-    // Step 2: a tool exchange whose tool/result has empty content → empty extraction → skipped.
+    // Keep the log pairing-valid while the empty result covers the final message kind.
     s.append('step/start', { turn: 1, step: 2 })
     s.append('assistant/message', {
       turn: 1, step: 2,
@@ -1495,10 +1497,6 @@ describe('BasicCompactService edge cases', () => {
 
     const nodes = s.surface.nodes
     await compactRegion(svc, s, nodes[0]!.seq, nodes[nodes.length - 1]!.seq, 'm')
-    // Every empty-content message (user text, empty reasoning, empty-content
-    // tool/result, empty context, empty steering) extracted to nothing and was
-    // skipped — the only surviving line is the assistant's tool-call (which a
-    // balanced surface requires to answer the tool/result).
     expect(svc.summarizeCalls[0]!.text).toBe('Assistant: [tool-call: bash({})]')
   })
 
@@ -1546,31 +1544,26 @@ describe('BasicCompactService edge cases', () => {
 
 describe('BasicCompactService positional range (surface seqs are not monotonic after a replace)', () => {
   it('compacts a second region after the first replace lands a high-seq summary at the head position', async () => {
-    // A replace inserts the new summary node (a high seq) AT the shadowed range's surface
-    // position, so the surface becomes [highSeqSummary, …olderRetainedLowerSeqs].
+    // Replacement makes surface seqs non-monotonic. The next region is a
+    // positional span even when startSeq > endSeq.
     const svc = createTestService({ auto: false })
     const session = multiTurnSession(4, 1)
 
-    // First compaction: shadow the two oldest surface nodes.
+    // A replacement puts its high-seq summary at the surface head.
     const nodes0 = session.surface.nodes
     const first = await compactRegion(svc, session, nodes0[0]!.seq, nodes0[1]!.seq, 'm')
 
-    // The summary node now sits at the head with a seq HIGHER than the retained older nodes
-    // that follow it — the non-monotonic surface.
     const nodes1 = session.surface.nodes
     expect(nodes1[0]!.seq).toBeGreaterThanOrEqual(first.summarySeq)
     expect(nodes1[0]!.seq).toBeGreaterThan(nodes1[1]!.seq)
 
-    // Second compaction: shadow [summary(head) … turn-2's step end].
     const startSeq = nodes1[0]!.seq
     const endSeq = nodes1[2]!.seq
     expect(startSeq).toBeGreaterThan(endSeq)
     const second = await compactRegion(svc, session, startSeq, endSeq, 'm')
 
-    // Exactly the three nodes at surface positions [0..2] are shadowed, in
-    // surface order — the positional slice, regardless of their seq values.
+    // Selection follows surface positions, not sequence-number order.
     expect(second.shadowedSeqs).toEqual([nodes1[0]!.seq, nodes1[1]!.seq, nodes1[2]!.seq])
-    // The surface still derives cleanly: a new head replace node + the rest.
     const finalNodes = session.surface.nodes
     expect(finalNodes[0]!.seq).toBeGreaterThanOrEqual(second.summarySeq)
     expect(session.deriveMessages().length).toBe(finalNodes.length)
@@ -1580,20 +1573,15 @@ describe('BasicCompactService positional range (surface seqs are not monotonic a
     const svc = createTestService({ auto: false })
     const session = multiTurnSession(3, 1)
 
-    // First compaction shadows the oldest two surface nodes, landing a high-seq
-    // summary node at the head.
+    // Put a high-seq summary at the head; log order would place retained older nodes first.
     const n0 = session.surface.nodes
     await compactRegion(svc, session, n0[0]!.seq, n0[1]!.seq, 'm')
 
-    // Second compaction spans [head summary … turn-2's step end]. The head's seq
-    // is higher than the older retained nodes' seqs, so a log-seq-order walk
-    // would emit the older messages BEFORE the checkpoint.
     const n1 = session.surface.nodes
     svc.summarizeCalls = []
     await compactRegion(svc, session, n1[0]!.seq, n1[2]!.seq, 'm')
 
-    // The extracted transcript follows surface order: the checkpoint (head)
-    // first, then the older retained messages — matching deriveMessages().
+    // Extraction must match surface and `deriveMessages()` order.
     const { text } = svc.summarizeCalls[0]!
     const checkpointIdx = text.indexOf('compacted-summary')
     const olderIdx = text.indexOf('turn 2 user')

packages/compact/compact-basic/tests/compact-loop-repro.spec.ts

@@ -14,9 +14,10 @@ import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
 import type { SurfaceEvent } from '@deepseek-ai/dsh-session'
 
 /**
- * CBR-001 regression: a compaction checkpoint that the real loop lands is a free surface
- * boundary (it carries no tool-call/result pair), so it must be a valid region edge on BOTH
- * sides.
+ * CBR-001 regression through the real loop. A replacement checkpoint has a high
+ * log seq at the surface head and carries no tool pair, so both adjacent cuts
+ * must be safe and re-compacting that checkpoint alone must succeed. This pins
+ * surface-position semantics rather than raw-log scanning.
  */
 
 const TOKENS_PER_BLOCK = 10
@@ -117,9 +118,8 @@ describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', ()
       )
       expect(checkpoints.length).toBeGreaterThan(0)
 
-      // The loop fired compaction mid-flight, so each landed checkpoint sits at a high log seq
-      // beside the step it landed in, even though its surface position is the head of the range
-      // it shadowed.
+      // High log position does not make a text-only checkpoint mid-step; both
+      // its start and end cuts are balanced in surface order.
       const nodes = agent.session.surface.nodes
       for (const cp of checkpoints) {
         const node = nodes.find(n => n.seq === cp.seq)