fix(workspace-context): preserve UTF-8 render proof

packages/context/workspace-context/README.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/context/workspace-context/README.md
-README.md: dc8889cf99b1af691c72af621e78dfb93228ad92
-README.zh.md: ef1dd31ecd5b368aa3e92176edda5aa53268f7d8
+README.md: 0be9ea1af7205f39cf9479ed155c4ae9910041ca
+README.zh.md: d1fbb073d686bef1e2dd65f37fd5df689f592d50

packages/context/workspace-context/README.md

@@ -50,7 +50,7 @@ The plugin owns the complete `<system-reminder>` framing, and every injected `us
 
 Model-visible text contains no hidden state markers. Each baseline or dynamic context event instead carries a typed `workspace-instructions` source with a list of `{ action, scope, path, digest? }` changes; the complete startup or resume baseline also carries `baseline: true`. On every relevant tool touch, the plugin reconstructs loaded state from its visible session events and overlays a short in-memory pending window for context present on the immutable top-level `tools/result` but not yet appended by the loop. A matching durable `user/message` confirms the pending transition. If the owning `step/end` arrives before a matching context reaches the log, the plugin clears the pending transition and its version fast path so the next successful touch can load it again. Nested Code Mode results stage pending changes under the outer execution token for same-run duplicate suppression; the outer result rolls that state back and recommits only contexts that survived outer policy.
 
-An unchanged path and SHA-1 content digest is not injected again. A per-session, per-scope provider cache stores only `{ path, version, digest, trimmedDigest }`: when the provider's opaque `FsVersion` and the effective visible state both match, reconciliation skips the content read; a changed version triggers a bounded read and SHA-1 confirmation before any model-visible update. The `trimmedDigest` — SHA-1 over the whitespace-trimmed content — is the per-directory duplicate key, so an unchanged file can still be removed when an earlier candidate converges on its content. Resume works because SHA-1 state is persisted in the typed source, while an empty in-memory version cache merely causes one confirming read. Compaction re-arms a scope after its context event leaves the visible surface even when the cached version is unchanged. A removal is a tombstone, so a later candidate reappearance is loaded again. A model-visible change enters the source, pending state, and version cache only when its file-specific section retains at least one content byte, or when its original content is genuinely empty. Partial truncation records the complete-content digest once any content byte survives; truncation to zero remains eligible for a later touch, while a same-digest version refresh updates only the provider cache.
+An unchanged path and SHA-1 content digest is not injected again. A per-session, per-scope provider cache stores only `{ path, version, digest, trimmedDigest }`: when the provider's opaque `FsVersion` and the effective visible state both match, reconciliation skips the content read; a changed version triggers a bounded read and SHA-1 confirmation before any model-visible update. The `trimmedDigest` — SHA-1 over the whitespace-trimmed content — is the per-directory duplicate key, so an unchanged file can still be removed when an earlier candidate converges on its content. Resume works because SHA-1 state is persisted in the typed source, while an empty in-memory version cache merely causes one confirming read. Compaction re-arms a scope after its context event leaves the visible surface even when the cached version is unchanged. A removal is a tombstone, so a later candidate reappearance is loaded again. A model-visible change enters the source, pending state, and version cache only when its file-specific section retains at least one content byte, or when its original content is genuinely empty. Partial truncation records the complete-content digest once any content byte survives; truncation to zero remains eligible for a later touch, while a same-digest version refresh updates only the provider cache. A baseline may still publish its budget diagnostic with an empty change list. A dynamic batch with no committed change is not injected at all, and a later touch retries it.
 
 The initial baseline event itself is not rewritten. Its typed changes remain authoritative only while that event is in the visible session surface; the next successful filesystem touch re-adds an unchanged baseline scope after compaction, or appends its replacement or removal. The in-memory scope marker and provider-version cache only select and accelerate probes. A hot plugin remount retains a baseline only when its typed event remains visible, while rebuilding current scope and version tracking; otherwise it injects a current baseline. A resumed loop always recomposes the current baseline and also reconciles still-visible dynamic scopes before its first request. There is no file watcher, so an on-disk change becomes visible at the next successful `read`, `write`, or `edit` touch, or when a resumed loop prepares its baseline.
 

packages/context/workspace-context/README.zh.md

@@ -50,7 +50,7 @@ These instructions apply to work under `packages/app`. Use them as guidance when
 
 模型可见文本不含隐藏状态标记。每个基线或动态上下文事件改为携带带类型的 `workspace-instructions` 来源，其中包含 `{ action, scope, path, digest? }` 变更列表；完整的启动或恢复基线还会携带 `baseline: true`。每次相关工具 touch 时，插件会从可见会话事件重建已加载状态，并叠加一个短暂内存 pending 窗口，用于不可变顶层 `tools/result` 上存在但 loop 尚未追加的上下文。匹配的持久 `user/message` 会确认 pending 转换。如果所属 `step/end` 在匹配上下文进入日志之前到达，插件会清除 pending 转换及其版本快速路径，使下一次成功 touch 可以重新加载。嵌套 Code Mode 结果会在外层执行 token 下暂存 pending 变更，用于抑制同次运行中的重复项；外层结果会回滚该状态，再只重新提交经过外层策略的上下文。
 
-路径与 SHA-1 内容 digest 都未变时，不会重复注入。每会话、每 scope 提供方 cache 只存储 `{ path, version, digest, trimmedDigest }`：当提供方的不透明 `FsVersion` 与有效可见状态都匹配时，对账会跳过内容读取；版本改变会在任何模型可见更新之前触发有界读取与 SHA-1 确认。`trimmedDigest` 是针对去除空白后内容的 SHA-1，也是每目录重复 key，因此较早候选文件与某个未更改文件的内容收敛后，后者仍可被移除。恢复可行，因为 SHA-1 状态持久化在带类型的来源中，而空的内存版本 cache 只会导致一次确认读取。压缩（compaction）会在 scope 的上下文事件离开可见表层后重新启用它，即使缓存版本未变。移除是 tombstone，因此候选文件之后重新出现时会重新加载。模型可见变更只有在对应文件专属段落保留至少一个内容字节，或原始内容确实为空时，才会进入来源、pending 状态和版本 cache。只要任一内容字节保留下来，部分截断就会记录完整内容的 digest；截断到零字节则仍可在后续 touch 处理，而相同 digest 的版本刷新只更新提供方 cache。
+路径与 SHA-1 内容 digest 都未变时，不会重复注入。每会话、每 scope 提供方 cache 只存储 `{ path, version, digest, trimmedDigest }`：当提供方的不透明 `FsVersion` 与有效可见状态都匹配时，对账会跳过内容读取；版本改变会在任何模型可见更新之前触发有界读取与 SHA-1 确认。`trimmedDigest` 是针对去除空白后内容的 SHA-1，也是每目录重复 key，因此较早候选文件与某个未更改文件的内容收敛后，后者仍可被移除。恢复可行，因为 SHA-1 状态持久化在带类型的来源中，而空的内存版本 cache 只会导致一次确认读取。压缩（compaction）会在 scope 的上下文事件离开可见表层后重新启用它，即使缓存版本未变。移除是 tombstone，因此候选文件之后重新出现时会重新加载。模型可见变更只有在对应文件专属段落保留至少一个内容字节，或原始内容确实为空时，才会进入来源、pending 状态和版本 cache。只要任一内容字节保留下来，部分截断就会记录完整内容的 digest；截断到零字节则仍可在后续 touch 处理，而相同 digest 的版本刷新只更新提供方 cache。基线即使带空变更列表，仍可发布字节预算诊断。动态批次若没有可提交变更，则完全不注入，并在后续 touch 时重试。
 
 初始基线事件自身不会被改写。其带类型的变更仅在该事件仍位于可见会话表层时才是权威状态；下一次成功的文件系统 touch 会在压缩后重新添加未变的基线 scope，或追加其替换或移除。内存中的 scope 标记和提供方版本 cache 只负责选择探测对象并加速探测。插件热重挂只有在其带类型的事件仍然可见时才保留基线，同时会重建当前 scope 与版本跟踪状态；否则会注入当前基线。恢复的 loop 始终重新组合当前基线，并在第一个请求前对账仍可见的动态 scope。没有文件 watcher，因此磁盘变更会在下一次成功 `read`、`write` 或 `edit` touch 时可见，也会在恢复 loop 准备基线时可见。
 

packages/context/workspace-context/src/render.ts

@@ -69,18 +69,16 @@ function byteLength(value: string): number {
   return Buffer.byteLength(value, 'utf8')
 }
 
-function zeroContentTruncatedPaths(truncated: TruncatedInstruction[]): Set<string> {
-  return new Set(truncated
-    .filter(item => item.originalBytes > 0 && item.includedBytes === 0)
-    .map(item => item.displayPath))
-}
-
 function truncateUtf8(value: string, maxBytes: number): string {
-  let truncated = Buffer.from(value, 'utf8').subarray(0, Math.max(0, maxBytes)).toString('utf8')
-  while (byteLength(truncated) > maxBytes) {
-    truncated = truncated.slice(0, -1)
+  const bytes = Buffer.from(value, 'utf8')
+  if (bytes.length <= maxBytes) return value
+  let end = Math.max(0, Math.trunc(maxBytes))
+  // If the first excluded byte is a UTF-8 continuation byte, the budget cut
+  // through that code point. Back up to its lead byte and exclude it too.
+  while (end > 0 && (bytes.readUInt8(end) & 0xc0) === 0x80) {
+    end -= 1
   }
-  return truncated
+  return bytes.subarray(0, end).toString('utf8')
 }
 
 function escapeInstructionFrameBody(body: string): string {
@@ -199,12 +197,10 @@ export function renderInstructionChanges(
   }
   const rendered = renderInstructionContext(items.map(item => item.file), maxBytes, style)
   const represented = new Set(rendered.represented.map(file => file.absolutePath))
-  const contentOmitted = zeroContentTruncatedPaths(rendered.truncated)
   return {
     text: rendered.text,
     changes: items
-      .filter(item => represented.has(item.file.absolutePath)
-        && !contentOmitted.has(item.file.displayPath))
+      .filter(item => represented.has(item.file.absolutePath))
       .map(item => item.change),
   }
 }
@@ -294,21 +290,26 @@ function renderInstructionContext(
   /* v8 ignore next -- callers only reach this after a non-empty fullText was built. */
   if (mostSpecific === undefined) return { text: '', omitted: [], truncated: [], represented: [] }
   const omitted = files.slice(0, -1).map(file => ({ absolutePath: file.absolutePath, displayPath: file.displayPath }))
+  const originalBytes = byteLength(mostSpecific.content)
 
   for (const candidateStyle of [style, { ...style, intro: COMPACT_WORKSPACE_CONTEXT_INTRO }]) {
     const truncatedFile = truncateToFit(mostSpecific, [], maxBytes, omitted, candidateStyle)
+    const includedBytes = byteLength(truncatedFile.content)
     const truncated = [{
       displayPath: mostSpecific.displayPath,
-      originalBytes: byteLength(mostSpecific.content),
-      includedBytes: byteLength(truncatedFile.content),
+      originalBytes,
+      includedBytes,
     }]
     const text = buildInstructionText([truncatedFile], maxBytes, omitted, truncated, candidateStyle)
-    if (byteLength(text) <= maxBytes) return { text, omitted, truncated, represented: [mostSpecific] }
+    if (byteLength(text) <= maxBytes) {
+      const represented = includedBytes > 0 || originalBytes === 0 ? [mostSpecific] : []
+      return { text, omitted, truncated, represented }
+    }
   }
 
   const truncated = [{
     displayPath: mostSpecific.displayPath,
-    originalBytes: byteLength(mostSpecific.content),
+    originalBytes,
     includedBytes: 0,
   }]
   const compactNotice = escapeInstructionFrameBody(markerText(maxBytes, omitted, truncated))
@@ -316,7 +317,8 @@ function renderInstructionContext(
     [compactNotice, style.section(withTruncatedContent(mostSpecific, 0))].join('\n\n'),
   )
   if (byteLength(compactWithHeading) <= maxBytes) {
-    return { text: compactWithHeading, omitted, truncated, represented: [mostSpecific] }
+    const represented = originalBytes === 0 ? [mostSpecific] : []
+    return { text: compactWithHeading, omitted, truncated, represented }
   }
   const text = byteLength(compactNotice) <= maxBytes ? compactNotice : truncateUtf8(compactNotice, maxBytes)
   return { text, omitted, truncated, represented: [] }
@@ -334,8 +336,7 @@ export function renderWorkspaceInstructionSet(
   options: { maxBytes: number },
 ): RenderedInstructionSet {
   const { represented, ...rendered } = renderInstructionContext(files, options.maxBytes, BASELINE_RENDER_STYLE)
-  const contentOmitted = zeroContentTruncatedPaths(rendered.truncated)
-  return { rendered, included: represented.filter(file => !contentOmitted.has(file.displayPath)) }
+  return { rendered, included: represented }
 }
 
 /**

packages/context/workspace-context/tests/workspace-context.spec.ts

@@ -41,7 +41,7 @@ import {
   type InstructionVersionCache,
   type PendingInstructionChange,
 } from '../src/state.ts'
-import { candidateScopeKey, renderInstructionChanges } from '../src/render.ts'
+import { candidateScopeKey, renderInstructionChanges, renderWorkspaceInstructionSet } from '../src/render.ts'
 import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
 
 /** Per-candidate reconciliation scope key: directory paired with the file name. */
@@ -818,6 +818,15 @@ describe('workspace context rendering', () => {
     expect(Buffer.byteLength(rendered.text, 'utf8')).toBe(120)
   })
 
+  it('represents a genuinely empty instruction when its compact heading fits', () => {
+    const file = { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: '' }
+    const rendered = renderWorkspaceInstructionSet([file], { maxBytes: 117 })
+
+    expect(rendered.rendered.text).toContain('truncated pkg/AGENTS.md from 0 to 0 bytes')
+    expect(rendered.rendered.text).toContain('Instructions from: pkg/AGENTS.md')
+    expect(rendered.included).toEqual([file])
+  })
+
   it('truncates the compact notice itself when the render budget is smaller than the notice', () => {
     const rendered = renderWorkspaceContext([
       { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: 'x'.repeat(1000) },
@@ -881,6 +890,23 @@ describe('workspace context rendering', () => {
     expect(rendered.changes).toEqual([])
   })
 
+  it('does not commit a multibyte change when the budget cuts its first code point', () => {
+    const change = {
+      action: 'set' as const,
+      scope: sk('pkg', 'AGENTS.md'),
+      path: 'pkg/AGENTS.md',
+      digest: 'digest',
+    }
+    const rendered = renderInstructionChanges([{
+      change,
+      file: { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: '😀'.repeat(100) },
+    }], 366)
+
+    expect(rendered.text).not.toContain('<27>')
+    expect(rendered.text).not.toContain('😀')
+    expect(rendered.changes).toEqual([])
+  })
+
   it('keeps compact truncation notices within budget when a multibyte display path is cut', () => {
     const rendered = renderWorkspaceContext([
       { absolutePath: '/repo/路径/AGENTS.md', displayPath: '路径/AGENTS.md', content: 'x'.repeat(1000) },