Merge remote-tracking branch 'origin/master' into worktree/explicit-change-scope-report-20260727

.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.i18n.yaml

@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md
+2026-07-23-session-telemetry-otel-revival.md: a58598d8a956d47cb0cf6aa3e659f38314bc4b17
+2026-07-23-session-telemetry-otel-revival.zh.md: cc09717e349d5ae2ab5157bf46de30b1823c775f

.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md

@@ -0,0 +1,37 @@
+# Agent Note: Session telemetry seam with mandatory redaction and the OTel backend
+
+Status: implemented
+
+English | [中文](2026-07-23-session-telemetry-otel-revival.zh.md)
+
+## Problem
+
+Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the `session-telemetry-otlp-rfc` branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The capture-side design (backend contract, coordinator, handoff cursor, chunk projection) was sound and reviewed; the export-side stance was the blocker.
+
+## Decision
+
+`packages/telemetry/` revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them:
+
+- **`@deepseek-ai/dsh-session-telemetry`** — the seam. `TelemetryBackend` (`emit`/`flush?`/`shutdown`), the service-registered `Telemetry` form, and `TelemetryCoordinator` owning capture: adoption with cursor read-back, the per-append firehose (project → `structuredClone` → redact → `emit`, zero I/O), the fixed first-chunk-per-(turn, step) projection, the `agent/error` relay, and dispose-time `shutdown` records.
+- **The `telemetry/record` waterfall** — the delta over the branch version and the seam's redaction extension point. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermost `next()` is a pass-through, deployments mount their rules as listeners (stacking by transforming `next()`'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten.
+- **`@deepseek-ai/dsh-session-telemetry-otel`** — the reference backend: OTel JS SDK log pipeline (`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP exporter), configured verbatim through `exporter`/`processor` passthroughs. `exporter.url` is required and validated at load; unmounted or unconfigured, nothing leaves the process.
+
+The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly.
+
+## Alternatives considered
+
+**Implement the runtime-telemetry RFC's outbox (durable spool, per-sink cursors, at-least-once, a `readCommitted` persistence-seam method).** Deferred, not rejected: the SDK stance makes delivery semantics the reporting SDK's territory, and the OTel SDK's own batch pipeline is the honest default. The outbox is a pure additive layer (the `emit()` contract does not move); revive it when a deployment states a crash-loss requirement telemetry must satisfy.
+
+**No in-process redaction point, delegating to receiver-side collector processors.** Rejected — receiver-side redaction ships the secret first and scrubs it second. The waterfall puts an auditable, stackable scrubbing point before bytes leave the process; where the branch version (what PR #222 shipped) had no redaction point at all, every record now passes one.
+
+**A built-in conservative rule set as the waterfall's innermost `next()`.** Rejected: as an SDK we cannot know which patterns are secrets in a given deployment, a shipped list invites false confidence ("redaction is on") while catching only known shapes, and false positives would corrupt exported bodies for consumers who never asked. The seam owns the mechanism; the deployment owns the policy — the innermost `next()` is a pass-through, and rules mount as listeners.
+
+**Map onto OTel spans (GenAI semantic conventions) instead of logs.** Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve.
+
+**Full-log replay when no handoff cursor survived (re-export constructor seeds).** Shipped in the first revival round, then narrowed: adoption now replays from the session's construction boundary (`Session.firstLiveSeq`, the constructor-seed length — a fact the session already validated but did not expose; `header.seedLength` cannot serve, it is the durable fork-lineage value and a resumed session's constructor seed is its full stored log). A resumed session's history already shipped from the previous process under the same id, and a fork's inherited prefix already shipped in the parent's stream — re-exporting either re-billed every resume for its full history and doubled query-time counts on OTLP backends with no native ingest dedupe. Receivers stitch fork lineage via `session.parent_id` + `session.seed_length`. What the narrowing gives up, consistently with the at-most-once stance: a resume no longer backfills records the previous process failed to deliver (telemetry unmounted then, or queued at crash) — the full replay's only real benefit, bought at the common case's expense. A deployment that states a backfill requirement needs the deferred outbox above, not replay. The boundary also swallows the synthetic turn closers `SessionPersistence.load()` writes when repairing a crash-interrupted log (they sit below `firstLiveSeq` despite never existing in the previous process) — deliberate, not incidental: exporting a synthetic closer cannot complete the remote turn whose real tail records died in the crashed process's queue, it can only make an incomplete turn look closed. The wire stream stays faithful to what the crashed process actually shipped; receivers read a never-closed turn on a resumed stream as "the previous process died inside it" (the OTel README states the rule), and a later clean `shutdown` marker attests only to the resumed process's exit. Threading the pre-repair boundary through load/prepare so repairs export as live events would couple three packages to un-ship that signal.
+
+**Forwarding the seam's turn-boundary `flush()` hint to the OTel provider's `forceFlush()`.** Shipped in the first revival round, then removed after three review rounds each found a new silent-loss path in the same wrapper state: a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no `flush()` implemented, the batch processor is the only flusher, its `scheduledDelayMillis` (already deployment-tunable through the `processor` passthrough) governs export cadence, and `shutdown()`'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement `scheduledDelayMillis` cannot meet — and then by calling the retained `BatchLogRecordProcessor`'s own `forceFlush()`, never the provider's timeout-wrapped one.
+
+## Consequences
+
+A deployment adds one `cordis.yml` entry with an OTLP endpoint and gets its session stream in any OTel-compatible stack; removing the entry is the opt-out, with no residual state. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount `telemetry/record` listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.

.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md

@@ -0,0 +1,37 @@
+# Agent Note: Session telemetry seam with mandatory redaction and the OTel backend
+
+Status: implemented
+
+[English](2026-07-23-session-telemetry-otel-revival.md) | 中文
+
+## Problem
+
+每个想把 harness 会话接入可观测性体系的部署方都得手写一套会话日志消费端：订阅、生命周期交接、以及最难的脱敏——原始日志携带文件内容与命令输出，可能内嵌凭据。遥测 seam 和 OTel backend 曾在 `session-telemetry-otlp-rfc` 分支（PR #222/#231）上完成过一版，但从未进入 master：该提案将原始会话事件原样导出，法务评审未予通过。捕获侧设计（backend 契约、coordinator、handoff 游标、chunk 投影）本身合理且经过评审；导出侧的立场才是阻塞点。
+
+## Decision
+
+`packages/telemetry/` 以 SDK 立场复活这两个经过评审的包——harness 提供能力，部署方配置上报去向并对导出内容负责：
+
+- **`@deepseek-ai/dsh-session-telemetry`** —— seam 本体。`TelemetryBackend`（`emit`/`flush?`/`shutdown`）、服务注册形态的 `Telemetry`、以及拥有捕获侧的 `TelemetryCoordinator`：带游标回读的收养、逐 append 的 firehose（投影 → `structuredClone` → 脱敏 → `emit`，零 I/O）、固定的每 (turn, step) 首 chunk 投影、`agent/error` 转发、以及 dispose 时的 `shutdown` 记录。
+- **`telemetry/record` waterfall** —— 相对分支版本的增量，也是该 seam 的脱敏扩展点。每条记录抵达任何 backend 前必经此处；seam 自身不带任何规则——最内层 `next()` 原样透传，部署方以监听器挂载自己的规则（通过变换 `next()` 的返回值堆叠），抛异常的规则将该记录 fail-closed 扣下。脱敏只作用于导出副本；canonical log 永不改写。
+- **`@deepseek-ai/dsh-session-telemetry-otel`** —— 参考 backend：OTel JS SDK 日志管线（`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP exporter），经 `exporter`/`processor` passthrough 原样配置。`exporter.url` 必填且加载时校验；未挂载或未配置时，任何数据都不会离开进程。
+
+边界公理保持不变：harness 的职责止于 `emit()`。批处理、重试、排队与丢失策略属于 reporting SDK，经 passthrough 配置——投递是尽力而为（崩溃时至多一次），README 对此如实陈述。
+
+## Alternatives considered
+
+**实现 runtime-telemetry RFC 的 outbox（落盘 spool、每 sink 游标、at-least-once、persistence seam 的 `readCommitted` 方法）。** 推迟而非否决：SDK 立场使投递语义归属 reporting SDK，OTel SDK 自身的批处理管线是诚实的默认。outbox 是纯增量层（`emit()` 契约不动）；待某个部署提出遥测必须满足的崩溃丢失要求时再复活。
+
+**不设进程内脱敏点，交给接收端 collector processor。** 否决——接收端脱敏是先把秘密发出去再擦除。waterfall 在字节离开进程前提供一个可审计、可堆叠的擦除点；分支版本（PR #222 交付的形态）完全没有脱敏点，如今每条记录都必经其一。
+
+**在 waterfall 最内层 `next()` 内置一套保守规则集。** 否决：作为 SDK 我们无法预知某个部署里什么模式算秘密，内置列表只覆盖已知形状却会带来"脱敏已开启"的虚假信心，且误报会替从未要求过的消费者破坏导出 body。seam 拥有机制，部署方拥有策略——最内层 `next()` 原样透传，规则以监听器挂载。
+
+**映射到 OTel span（GenAI 语义约定）而非日志。** 本次复活否决：分支实现的日志映射已经过评审、形态可交付；span 模型对可 fork、可中断的会话有损，留给将来真正有 span 查询需求的消费者。
+
+**handoff 游标未存活时全量回放日志（重新导出构造函数种子）。** 首轮复活曾交付此方案，其后收窄：收养现在从会话的构造边界起回放（`Session.firstLiveSeq`，即构造函数种子长度，这一事实会话早已校验过却未曾暴露；`header.seedLength` 不能胜任：它是持久保存的 fork 谱系（lineage）值，而恢复会话的构造函数种子是其完整的已存储日志）。恢复会话的历史已由上一个进程以同一 id 发出，fork 继承的前缀也已在父会话的流中发出；再次导出任何一者，都会让每次恢复为其完整历史重复付费，并在没有原生摄取去重的 OTLP 后端上使查询时的计数翻倍。接收端基于 `session.parent_id` + `session.seed_length` 拼接 fork 谱系。此次收窄放弃的内容与至多一次立场一致：恢复不再回填上一个进程未能投递的记录（彼时遥测未挂载，或崩溃时仍在队列中）——这本是全量回放唯一的真实收益，代价却由常见情形承担。提出回填要求的部署需要的是上文已推迟的 outbox，而不是回放。该边界同样吞掉 `SessionPersistence.load()` 修复被崩溃打断的日志时写入的合成轮次关闭事件（它们落在 `firstLiveSeq` 之前，尽管在上一个进程中从未存在过）。这是有意为之，而非附带效果：远端轮次的真实尾部记录已随崩溃进程的队列一同消亡，导出合成关闭事件无法补全该轮次，只会让一个未完成的轮次看起来已经关闭。导出的流忠实于崩溃进程实际发出的内容；接收端会把恢复后的流中一个从未关闭的轮次读作「上一个进程死在了该轮次之内」（OTel README 陈述了这条规则），其后干净的 `shutdown` 标记也只证明恢复后进程自身的退出。若为让修复以实时事件的身份导出而将修复前边界贯穿 load/prepare 传递，将使三个包相互耦合，只为抹除这一信号。
+
+**将 seam 的轮次边界 `flush()` 提示转发到 OTel provider 的 `forceFlush()`。** 首轮复活曾交付此转发，其后移除：三轮评审在同一份包装层状态中各发现一条新的静默丢失路径——dispose 与进行中的 flush 之间的竞态（SDK 的并发 flush 防护会令 shutdown 的内部排空被跳过）、相互重叠的提示顶掉留存的 promise、以及 provider 固定的 30 秒 flush 超时在批处理器仍在排空时便 reject。这些路径存在的唯一原因，是该转发让这个后端成为进程内第二个执行 flush 的组件，面对的还是上游实验性（experimental）源码树中未见诸文档的 SDK 内部行为；不实现 `flush()` 时，批处理器就是唯一执行 flush 的组件，其 `scheduledDelayMillis`（已可由部署方经 `processor` passthrough 调优）决定导出节奏，`shutdown()` 的排空从构造上就是完整的。仅当某个部署提出 `scheduledDelayMillis` 无法满足的轮次边界延迟要求时才恢复此转发——且届时应调用留存的 `BatchLogRecordProcessor` 自身的 `forceFlush()`，绝不调用 provider 那个带超时包装的版本。
+
+## Consequences
+
+部署方在 `cordis.yml` 加一个带 OTLP endpoint 的条目即可把会话流接入任何 OTel 兼容体系；删除条目即退出，无残留状态。未挂载规则的部署导出的记录与捕获时完全一致——包括文件内容与命令输出中内嵌的任何凭据——因此跨信任边界的部署必须挂载 `telemetry/record` 监听器，两个 README 对此如实陈述。挂载规则后，导出的 body 可能与 canonical log 字节不同，接收端不得把遥测当作字节精确副本；日志仍是唯一事实源。崩溃持久性在上述 outbox 决定重启前明确不在范围内。