feat(telemetry): add feedback-gated OTEL modes

This commit is contained in:
Turtle
2026-08-05 12:43:35 +08:00
parent b8d51704f3
commit c836fcd416
41 changed files with 635 additions and 182 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md
2026-07-23-session-telemetry-otel-revival.md: a58598d8a956d47cb0cf6aa3e659f38314bc4b17
2026-07-23-session-telemetry-otel-revival.zh.md: cc09717e349d5ae2ab5157bf46de30b1823c775f
2026-07-23-session-telemetry-otel-revival.md: dcbff9757cbb730b66f456535fbd7ae471b6ffd1
2026-07-23-session-telemetry-otel-revival.zh.md: c3a098041795fa92bb4e0dd421ca09be94907cb8

View File

@@ -14,7 +14,7 @@ Every deployment that wants harness sessions in an observability stack must hand
- **`@deepseek-ai/dsh-session-telemetry`** — the seam. `TelemetryBackend` (`emit`/`flush?`/`shutdown`), the service-registered `Telemetry` form, and `TelemetryCoordinator` owning capture: adoption with cursor read-back, the per-append firehose (project → `structuredClone` → redact → `emit`, zero I/O), the fixed first-chunk-per-(turn, step) projection, the `agent/error` relay, and dispose-time `shutdown` records.
- **The `telemetry/record` waterfall** — the delta over the branch version and the seam's redaction extension point. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermost `next()` is a pass-through, deployments mount their rules as listeners (stacking by transforming `next()`'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten.
- **`@deepseek-ai/dsh-session-telemetry-otel`** — the reference backend: OTel JS SDK log pipeline (`LoggerProvider``BatchLogRecordProcessor` → OTLP/HTTP exporter), configured verbatim through `exporter`/`processor` passthroughs. `exporter.url` is required and validated at load; unmounted or unconfigured, nothing leaves the process.
- **`@deepseek-ai/dsh-session-telemetry-otel`** — the reference backend: OTel JS SDK log pipeline (`LoggerProvider``BatchLogRecordProcessor` → OTLP/HTTP exporter), configured verbatim through `exporter`/`processor` passthroughs. Its default `FULL` mode requires `exporter.url`; the later [feedback-gated telemetry decision](2026-08-05-feedback-gated-session-telemetry.md) adds `FEEDBACK_ONLY` and `DISABLED` delivery modes without moving the redaction or backend boundary.
The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly.
@@ -34,4 +34,4 @@ The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry
## Consequences
A deployment adds one `cordis.yml` entry with an OTLP endpoint and gets its session stream in any OTel-compatible stack; removing the entry is the opt-out, with no residual state. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount `telemetry/record` listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.
A deployment adds one `cordis.yml` entry with an OTLP endpoint and gets its session stream in any OTel-compatible stack. `FULL` preserves that behavior by default, `FEEDBACK_ONLY` withholds records until feedback releases a prefix, and `DISABLED` constructs no reporting pipeline; removing the entry remains a silent opt-out, while the disabled mode keeps the local feedback warning. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount `telemetry/record` listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.

View File

@@ -14,7 +14,7 @@ Status: implemented
- **`@deepseek-ai/dsh-session-telemetry`** —— seam 本体。`TelemetryBackend``emit`/`flush?`/`shutdown`)、服务注册形态的 `Telemetry`、以及拥有捕获侧的 `TelemetryCoordinator`:带游标回读的收养、逐 append 的 firehose投影 → `structuredClone` → 脱敏 → `emit`,零 I/O、固定的每 (turn, step) 首 chunk 投影、`agent/error` 转发、以及 dispose 时的 `shutdown` 记录。
- **`telemetry/record` waterfall** —— 相对分支版本的增量,也是该 seam 的脱敏扩展点。每条记录抵达任何 backend 前必经此处seam 自身不带任何规则——最内层 `next()` 原样透传,部署方以监听器挂载自己的规则(通过变换 `next()` 的返回值堆叠),抛异常的规则将该记录 fail-closed 扣下。脱敏只作用于导出副本canonical log 永不改写。
- **`@deepseek-ai/dsh-session-telemetry-otel`** —— 参考 backendOTel JS SDK 日志管线(`LoggerProvider``BatchLogRecordProcessor` → OTLP/HTTP exporter`exporter`/`processor` passthrough 原样配置。`exporter.url` 必填且加载时校验;未挂载或未配置时,任何数据都不会离开进程
- **`@deepseek-ai/dsh-session-telemetry-otel`** —— 参考 backendOTel JS SDK 日志管线(`LoggerProvider``BatchLogRecordProcessor` → OTLP/HTTP exporter`exporter`/`processor` passthrough 原样配置。其默认 `FULL` 模式要求 `exporter.url`;后续的[反馈门控遥测决策](2026-08-05-feedback-gated-session-telemetry.md)增加了 `FEEDBACK_ONLY``DISABLED` 投递模式,但未移动脱敏或后端边界
边界公理保持不变harness 的职责止于 `emit()`。批处理、重试、排队与丢失策略属于 reporting SDK经 passthrough 配置——投递是尽力而为崩溃时至多一次README 对此如实陈述。
@@ -34,4 +34,4 @@ Status: implemented
## Consequences
部署方在 `cordis.yml` 加一个带 OTLP endpoint 的条目即可把会话流接入任何 OTel 兼容体系;删除条目即退出,无残留状态。未挂载规则的部署导出的记录与捕获时完全一致——包括文件内容与命令输出中内嵌的任何凭据——因此跨信任边界的部署必须挂载 `telemetry/record` 监听器,两个 README 对此如实陈述。挂载规则后,导出的 body 可能与 canonical log 字节不同,接收端不得把遥测当作字节精确副本;日志仍是唯一事实源。崩溃持久性在上述 outbox 决定重启前明确不在范围内。
部署方在 `cordis.yml` 加一个带 OTLP endpoint 的条目即可把会话流接入任何 OTel 兼容体系`FULL` 默认保留该行为,`FEEDBACK_ONLY` 在反馈释放前暂存记录前缀,`DISABLED` 则不构造上报流水线;删除条目仍是静默退出方式,而禁用模式会保留本地反馈警告。未挂载规则的部署导出的记录与捕获时完全一致包括文件内容与命令输出中内嵌的任何凭据因此跨信任边界的部署必须挂载 `telemetry/record` 监听器,两个 README 对此如实陈述。挂载规则后,导出的 body 可能与 canonical log 字节不同,接收端不得把遥测当作字节精确副本;日志仍是源。崩溃持久性在上述 outbox 决定重启前明确不在范围内。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-28-feedback-command.md
2026-07-28-feedback-command.md: 1c093d0e37eb72dc66e3c5569bd642557dde56a1
2026-07-28-feedback-command.zh.md: 300946a71ac7485a4bc787dd70ae5357147627f3
2026-07-28-feedback-command.md: 963153ceb4332b74693ff5c1d248c616ff4e8de9
2026-07-28-feedback-command.zh.md: 4dd02dcfb8d0606436c22e269db8c0d6cf163cee

View File

@@ -18,7 +18,7 @@ The package declares the log-only `feedback/record { text }` session event and e
`dsh-commands` still writes its `command/run` / `command/done` lifecycle pair around `/feedback`, but this command sets `recordInput: false`. Its `command/run` therefore carries the command identity and source without `args`; the feedback text exists only in `feedback/record`, while `command/done` carries the acknowledgement outcome. All three records are log-only and non-surface. Their appends start persistence's ordinary eager drain; nothing forces a flush, so acknowledgement reports that the feedback is in the log rather than already on disk.
Capture is deliberately inert: nothing in this repository reads `feedback/record`.
Capture remains inert for the running agent and model. The optional OTel telemetry package later adds one infrastructure consumer: it observes `feedback/record` as a release trigger in `FEEDBACK_ONLY` mode and as the local-only warning trigger in `DISABLED` mode, without changing the feedback event or command path. See [Feedback-gated session telemetry](2026-08-05-feedback-gated-session-telemetry.md).
### Why feedback owns an event
@@ -34,7 +34,7 @@ Surrounding whitespace is discarded, but nothing else is parsed. `/feedback /pla
### A new group
`packages/feedback/` is a new group because no existing one owns this. `goal/` is objective state, `session-title/` is titles, `core/` is the product spine. The group holds one package; a consumer would join it rather than forcing this one to grow.
`packages/feedback/` is a new group because no existing one owns this. `goal/` is objective state, `session-title/` is titles, `core/` is the product spine. The group holds one producer package; cross-cutting consumers stay in their owning groups rather than forcing this one to grow.
## Alternatives considered
@@ -48,7 +48,7 @@ Surrounding whitespace is discarded, but nothing else is parsed. `/feedback /pla
**Register the command inside an existing package** such as `packages/ui/commands`. Avoids a new group and its README pair. Rejected: `ctx.commands` is the registry, not a home for arbitrary command implementations, and the requester asked for a standalone package.
**Parse structure out of the text** (category prefixes, severity markers). Rejected as speculative: no consumer exists to use the structure, and any control-word grammar makes the corresponding literal feedback unrecordable. Verbatim text is the widest surface a future consumer can narrow; a parsed one cannot be widened after the fact.
**Parse structure out of the text** (category prefixes, severity markers). Rejected as speculative: no consumer needs that structure, and any control-word grammar makes the corresponding literal feedback unrecordable. Verbatim text is the widest surface a future consumer can narrow; a parsed one cannot be widened after the fact.
**Add a model-facing tool instead of a slash command.** Rejected: feedback is a direct human observation. Routing it through the model spends a turn, lets the model paraphrase the user's words, and makes the record contingent on the model choosing to call the tool.
@@ -58,6 +58,6 @@ The TUI mounts the command unconditionally — no configuration, no dependency o
The package owns one independent append-only event with no cross-event or mutable-data relation for an invariant companion to check. The event follows the session log's existing replay, fork, persistence, and crash-tail behavior.
Deferred: no consumer; no structured fields; no amend or withdraw, since the log is append-only and this package adds no tombstone; and no explicit durability barrier, so an entry recorded immediately before a crash can be lost with any other unflushed tail.
Deferred: no product or model consumer; no structured fields; no amend or withdraw, since the log is append-only and this package adds no tombstone; and no explicit durability barrier, so an entry recorded immediately before a crash can be lost with any other unflushed tail. The optional telemetry consumer treats the event only as an export-policy trigger.
No snapshot accompanies this change. AGENTS.md asks for a keyless snapshot through a runnable example for product-user-visible behavior; this was skipped at the requester's explicit direction. The package tests plus a real Loader composition test over a `cordis.yml` are the whole of the evidence, alongside interactive verification in the assembled TUI.

View File

@@ -18,7 +18,7 @@ Status: implemented
`dsh-commands` 仍会围绕 `/feedback` 写入 `command/run` / `command/done` 生命周期配对,但该命令设置了 `recordInput: false`。因此,它的 `command/run` 携带命令标识与来源,但不携带 `args`;反馈文本只存在于 `feedback/record` 中,而 `command/done` 携带确认结果。三个记录都仅写入日志且非 surface。它们的追加会启动持久化的常规即时排空没有任何环节强制 flush因此确认文本报告的是反馈已进入日志而非已经落盘。
采集刻意不产生后续动作:本仓库中没有任何代码读取 `feedback/record`
采集对正在运行的 agent 与模型仍不产生后续动作。可选的 OTel 遥测包后续增加了一个基础设施消费方:它在 `FEEDBACK_ONLY` 模式下将 `feedback/record` 作为释放触发器,在 `DISABLED` 模式下将其作为本地警告触发器,且不改变反馈事件或命令路径。见[反馈门控的会话遥测](2026-08-05-feedback-gated-session-telemetry.md)
### 为何反馈拥有自己的事件
@@ -34,7 +34,7 @@ Status: implemented
### 一个新的分组
`packages/feedback/` 是新分组,因为现有分组都不拥有此职责:`goal/` 负责目标状态,`session-title/` 负责标题,`core/` 是产品主干。该分组目前只有一个包;未来的消费方应加入该分组,而不是迫使这个包不断膨胀。
`packages/feedback/` 是新分组,因为现有分组都不拥有此职责:`goal/` 负责目标状态,`session-title/` 负责标题,`core/` 是产品主干。该分组只包含一个生产方包;跨领域的消费方留在各自所属的分组,而不是迫使这个包不断膨胀。
## 考虑过的替代方案
@@ -48,7 +48,7 @@ Status: implemented
**在现有包中注册该命令**,例如 `packages/ui/commands`。可省去新分组及其双语 README。已否决`ctx.commands` 是注册表,而不是任意命令实现的归属地;且请求者明确要求独立的包。
**从文本中解析结构**(类别前缀、严重程度标记)。已否决,属于投机设计:目前没有消费方使用该结构,而任何控制词语法都会让对应的字面反馈无法记录。原样文本是未来消费方可以收窄的最宽接口;而已被解析的接口无法事后放宽。
**从文本中解析结构**(类别前缀、严重程度标记)。已否决,属于投机设计:没有消费方需要该结构,而任何控制词语法都会让对应的字面反馈无法记录。原样文本是未来消费方可以收窄的最宽接口;而已被解析的接口无法事后放宽。
**改为提供面向模型的工具。** 已否决:反馈是人类的直接观察。经由模型会消耗一个轮次、让模型改写用户的原话,并使记录取决于模型是否选择调用该工具。
@@ -58,6 +58,6 @@ TUI 无条件挂载该命令:没有配置,也不依赖 goal 栈。无头 CLI
本包拥有一个独立的仅追加事件不存在跨事件关系或可变数据关系可供不变式伴生插件检查。该事件遵循会话日志现有的回放、fork、持久化和崩溃尾部行为。
延期事项:没有消费方;没有结构化字段;不支持修改或撤回,因为日志仅追加且本包不新增 tombstone且没有显式持久化屏障因此紧临崩溃前记录的条目可能与其他未 flush 的尾部一同丢失。
延期事项:没有产品或模型消费方;没有结构化字段;不支持修改或撤回,因为日志仅追加且本包不新增 tombstone且没有显式持久化屏障因此紧临崩溃前记录的条目可能与其他未 flush 的尾部一同丢失。可选的遥测消费方只将该事件作为导出策略触发器。
本次变更不附带 snapshot。AGENTS.md 要求面向产品用户的可见行为变更通过可运行示例附带无密钥 snapshot此项按请求者的明确指示跳过。包测试连同一个基于真实 `cordis.yml` 的 Loader 组合测试即为全部证据,此外还有在组装后 TUI 中的交互验证。

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md
2026-08-05-feedback-gated-session-telemetry.md: 21a9028c603f3faaec39b2ddb8ef14644d6c84d4
2026-08-05-feedback-gated-session-telemetry.zh.md: ea94c743b962a93a5fc64bdc2e4ed103aadecc99

View File

@@ -0,0 +1,35 @@
# Agent Note: Feedback-gated session telemetry
Status: implemented
English | [中文](2026-08-05-feedback-gated-session-telemetry.zh.md)
## Problem
Session telemetry originally has one mounted behavior: every accepted record enters the reporting backend immediately. Deployments need two stricter policies without replacing the plugin: hold a session's telemetry unless its user records feedback, or disable reporting while still explaining what happens to feedback. The policy must preserve the existing full-export default and the telemetry seam's redaction-before-backend boundary.
## Decision
`@deepseek-ai/dsh-session-telemetry-otel` exposes three uppercase `mode` values:
- `FULL` is the default and preserves immediate delivery to the configured OTel pipeline.
- `FEEDBACK_ONLY` captures redacted copies in memory and releases the pending session prefix when `feedback/record` is appended. The released prefix includes the feedback event itself. Records appended after that event form another withheld prefix until another feedback event releases them.
- `DISABLED` constructs no exporter, processor, or logger provider. A `feedback/record` listener prints that nothing is shared and the feedback remains local.
The generic telemetry coordinator owns the delivery distinction as `immediate` or `held`. Both paths project, clone, and run `telemetry/record` listeners at capture time. Immediate delivery sends the accepted record to the backend and advances the session's handoff cursor. Held delivery retains the accepted record per session without moving that cursor. `release(session)` submits the retained records in order, contains each backend failure independently, advances the cursor only for submitted records, and removes the released prefix.
The OTel feedback listener is registered after the coordinator's session listener. Cordis therefore gives the coordinator the feedback append first, then the OTel listener releases a prefix that already contains that event. `exporter.url` is required in `FULL` and `FEEDBACK_ONLY`; `DISABLED` does not validate or use exporter configuration.
## Alternatives considered
**Open a session permanently after its first feedback.** Rejected because later work would be shared without another feedback act and the plugin would need additional open-session state. Releasing one pending prefix per feedback has the smaller state machine and the narrower sharing boundary.
**Buffer after `TelemetryCoordinator.emit()` in the OTel backend.** Rejected because the coordinator would advance its handoff cursor before a record became eligible for upload. A plugin rebuild would then lose the only retained copy and incorrectly treat the prefix as handed off.
**Replay the canonical session log when feedback arrives.** Rejected because replay would repeat projection and redaction, exclude telemetry operation records that are not session events, and require more lifecycle state to distinguish previously released prefixes.
**Use an unmounted plugin as the disabled state.** That remains the silent opt-out, but it cannot warn when feedback is recorded. The explicit disabled mode lets a deployment keep one configuration shape and communicate that the local feedback did not leave the process.
## Consequences
`FULL` remains source- and wire-compatible with the original default. `FEEDBACK_ONLY` retains deep-copied, already-redacted records in process memory until feedback or session collection; a crash before release uploads nothing from that prefix. A clean shutdown after the last feedback is part of the new withheld suffix, so feedback-only streams do not carry a reliable shutdown or crash signal. Each later feedback releases the suffix accumulated since the previous one. `DISABLED` can omit `exporter.url`, does no reporting work, and keeps feedback only in the canonical session log.

View File

@@ -0,0 +1,35 @@
# Agent Note反馈门控的会话遥测
Status: implemented
[English](2026-08-05-feedback-gated-session-telemetry.md) | 中文
## 问题
会话遥测原本只有一种已挂载行为:每条已接受记录都立即进入上报后端。部署方需要两种更严格的策略,且不替换插件:只有用户记录反馈时才释放该会话的遥测,或禁用上报并仍向用户说明反馈的去向。该策略必须保留现有的全量导出默认值,以及遥测 seam 在记录抵达后端之前脱敏的边界。
## 决策
`@deepseek-ai/dsh-session-telemetry-otel` 公开三个大写的 `mode` 值:
- `FULL` 是默认值,保留向已配置 OTel 流水线的即时投递。
- `FEEDBACK_ONLY` 在内存中捕获已脱敏副本,并在追加 `feedback/record` 时释放待处理的会话前缀。已释放前缀包含反馈事件本身。在该事件之后追加的记录会形成另一个暂存前缀,直到下一个反馈事件将其释放。
- `DISABLED` 不构造导出器、处理器或日志提供方。`feedback/record` 监听器会输出警告,说明什么都不会共享,且反馈仍留在本地。
通用遥测协调器以 `immediate``held` 的形式拥有这两种投递方式。两条路径都会在捕获时进行投影、深拷贝,并运行 `telemetry/record` 监听器。即时投递把已接受记录发送到后端,并推进会话的 handoff 游标。暂存投递按会话保留已接受记录,且不移动该游标。`release(session)` 按顺序提交保留的记录,独立隔离每个后端失败,仅为已提交的记录推进游标,并移除已释放前缀。
OTel 反馈监听器在协调器的会话监听器之后注册。因此Cordis 先将反馈追加交给协调器,再由 OTel 监听器释放已包含该事件的前缀。`exporter.url``FULL``FEEDBACK_ONLY` 中必填;`DISABLED` 不校验也不使用导出器配置。
## 考虑过的替代方案
**会话在首次反馈后永久开放。** 已否决,因为后续工作会在用户未再次提交反馈的情况下被共享,而且插件需要额外的会话开放状态。每次反馈只释放一个待处理前缀,状态机更小,共享边界也更窄。
**在 OTel 后端的 `TelemetryCoordinator.emit()` 之后缓冲。** 已否决,因为协调器会在记录具备上传资格前推进 handoff 游标。插件重建后,唯一保留的副本会丢失,而协调器会错误地将该前缀视为已交接。
**反馈到达时回放权威会话日志。** 已否决,因为回放会重复执行投影与脱敏,排除不属于会话事件的遥测运维记录,且需要更多生命周期状态才能区分已释放前缀。
**以不挂载插件表示禁用状态。** 这仍然是静默退出方式,但无法在记录反馈时输出警告。显式禁用模式让部署方可以保持同一种配置形态,并说明本地反馈未离开进程。
## 后果
`FULL` 与原有默认值保持源码及协议兼容。`FEEDBACK_ONLY` 会在进程内存中保留已深拷贝且已脱敏的记录,直到收到反馈或会话被回收;释放前发生崩溃时,该前缀不上传任何内容。上次反馈之后的干净关闭属于新的暂存后缀,因此仅反馈的流不携带可靠的关闭或崩溃信号。每个后续反馈都会释放从上一个反馈开始累积的后缀。`DISABLED` 可省略 `exporter.url`,不执行任何上报工作,并仅在权威会话日志中保留反馈。