refactor(telemetry): remove the OTel backend's flush forwarding

Three review rounds each found a new silent-loss path in the same
wrapper state (dispose racing an in-flight flush, overlapping hints
displacing the retained promise, the provider's fixed 30s flush timeout
rejecting while the processor still drains). Every path exists only
because forwarding the seam's turn-boundary hint to forceFlush() made
this backend the process's second flusher against undocumented SDK
internals from the upstream experimental tree.

The backend now implements no flush(): the batch processor is the only
flusher, its scheduledDelayMillis (already deployment-tunable through
the processor passthrough) governs export cadence, and shutdown()'s
drain is complete by construction. The two race-pin tests collapse into
one dispose-during-in-flight-batch drain pin; the seam's optional
flush() contract now tells implementers they own the concurrent-flush/
shutdown interaction. Removal rationale and the reinstatement trigger
(a stated turn-boundary latency requirement scheduledDelayMillis cannot
meet — and then via the processor's own forceFlush(), never the
provider's timeout-wrapped one) are recorded in the revival Agent Note,
both languages.
This commit is contained in:
kingwl
2026-07-26 17:55:54 +08:00
parent e8d8589219
commit 95315386b3
13 changed files with 56 additions and 86 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
README.md: 2e0c902e7b16726829319c852eab3f26a68847b3
README.zh.md: 4917ee1fcff91f07d29bbcb1b6957b80d9e73197
README.md: 28fffc5f43f960de1a44700aa70050d792b81d4f
README.zh.md: 821969fea09487d9d4450e981a0d791a66dad3a1

View File

@@ -17,7 +17,7 @@ The OpenTelemetry backend for [the telemetry seam](../session-telemetry/) — th
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
```
`exporter.url` is the one field this package validates itself — required, no default, must parse as `http(s)` — so a missing endpoint fails at plugin load. Everything else is the SDK's option shape, owned and documented by the SDK, and both blocks pass through whole: every `OTLPExporterNodeConfigBase` field (`headers`, `timeoutMillis`, `compression`, `keepAlive`, …) reaches the exporter, and batching, retry, queue bounds, and loss policy under sustained failure are the SDK's documented behavior, tuned through the `processor` passthrough. Removing this block from `cordis.yml` is the opt-out: no residual state, no `enabled` flag.
`exporter.url` is the one field this package validates itself — required, no default, must parse as `http(s)` — so a missing endpoint fails at plugin load. Everything else is the SDK's option shape, owned and documented by the SDK, and both blocks pass through whole: every `OTLPExporterNodeConfigBase` field (`headers`, `timeoutMillis`, `compression`, `keepAlive`, …) reaches the exporter, and batching, export cadence (`scheduledDelayMillis`), retry, queue bounds, and loss policy under sustained failure are the SDK's documented behavior, tuned through the `processor` passthrough. The backend deliberately implements no `flush()`: the batch processor is the only flusher in the process, which is what makes `shutdown()`'s drain complete. Removing this block from `cordis.yml` is the opt-out: no residual state, no `enabled` flag.
## What leaves the machine

View File

@@ -17,7 +17,7 @@
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
```
`exporter.url` 是本包唯一自行校验的字段:必填、无默认值、必须能解析为 `http(s)`,因此缺失端点会在插件加载时失败。其余全部是 SDK 自己的选项形态,由 SDK 拥有并在 SDK 文档中说明两个配置块都整体透传passthrough`OTLPExporterNodeConfigBase` 的每个字段(`headers``timeoutMillis``compression``keepAlive` 等)都会到达导出器;批处理、重试、队列上限,以及持续失败下的丢失策略,都是 SDK 的文档化行为,经 `processor` 透传调优。从 `cordis.yml` 中删除该配置块即为退出方式:无残留状态,也没有 `enabled` 开关。
`exporter.url` 是本包唯一自行校验的字段:必填、无默认值、必须能解析为 `http(s)`,因此缺失端点会在插件加载时失败。其余全部是 SDK 自己的选项形态,由 SDK 拥有并在 SDK 文档中说明两个配置块都整体透传passthrough`OTLPExporterNodeConfigBase` 的每个字段(`headers``timeoutMillis``compression``keepAlive` 等)都会到达导出器;批处理、导出节奏(`scheduledDelayMillis`)、重试、队列上限,以及持续失败下的丢失策略,都是 SDK 的文档化行为,经 `processor` 透传调优。该后端刻意不实现 `flush()`:批处理器是进程内唯一执行 flush 的组件,`shutdown()` 的排空正因如此才是完整的。`cordis.yml` 中删除该配置块即为退出方式:无残留状态,也没有 `enabled` 开关。
## 哪些数据会离开本机

View File

@@ -147,36 +147,26 @@ export class TelemetryOtel extends Telemetry {
})
}
/** Every not-yet-settled turn-boundary flush, retained so {@link shutdown} can order behind ALL of them. */
private inflightFlush: Promise<void> = Promise.resolve()
/** Forward the turn-boundary hint to the SDK's flush, fire-and-forget. */
override flush(): void {
// Best-effort hint: the SDK resolves forceFlush even when exports fail
// (failures go to its own diagnostics), and the coordinator stops calling
// this once the fiber is disposed — a rejection would be SDK drift. The
// settled promise is retained (not awaited): the SDK's concurrent-flush
// guard makes a flush that overlaps another return WITHOUT draining, so
// an overlapping hint resolves instantly and must JOIN the outstanding
// one, not displace it — shutdown orders behind the whole set.
/* v8 ignore next -- unreachable guard: forceFlush does not reject while the provider is alive */
const flush = this.provider.forceFlush().catch(() => {})
this.inflightFlush = Promise.all([this.inflightFlush, flush]).then(() => undefined)
}
// The seam's optional flush() hint is deliberately NOT implemented. The
// batch processor exports on its own cadence (`processor.scheduledDelayMillis`,
// the SDK's documented knob), and this backend is the SDK pipeline's only
// caller — forwarding the hint to `forceFlush()` was the sole source of
// concurrent flushes, whose undocumented interactions with shutdown's
// internal drain (concurrent-flush guard, provider-level flush timeout)
// silently dropped tail records. Removal history and the revival trigger:
// the revival Agent Note.
/**
* Delegate disposal to the SDK's shutdown contract: flush the queue and
* quiesce. Orders behind every outstanding turn-boundary flush first —
* shutdown's internal flush is a no-op while one is in flight (the SDK's
* concurrent-flush guard), which would silently drop everything enqueued
* after that flush snapshot, including the coordinator's dispose-time
* `shutdown` markers. Awaited (and error-contained) by the coordinator's
* disposer.
* Delegate disposal to the SDK's shutdown contract: drain the queue and
* quiesce. With no concurrent `forceFlush()` in the process (see above),
* shutdown's internal drain is complete — everything emitted before this
* call, including the coordinator's dispose-time `shutdown` markers, is
* exported before the exporter closes. Awaited (and error-contained) by
* the coordinator's disposer.
* @returns resolves when the SDK pipeline has quiesced.
*/
async shutdown(): Promise<void> {
await this.inflightFlush
await this.provider.shutdown()
shutdown(): Promise<void> {
return this.provider.shutdown()
}
}

View File

@@ -123,12 +123,14 @@ describe('TelemetryOtel wire', () => {
expect(ops[0]!.record.attributes).toContainEqual({ key: 'telemetry.op', value: { stringValue: 'shutdown' } })
})
it('delivers records enqueued while a turn-boundary flush is in flight (flush/shutdown race)', async () => {
// Hold the collector's response to the flush-triggered export open until
// after disposal has begun: the SDK's concurrent-flush guard makes the
// shutdown-internal flush return early while another flush is running, so
// without ordering in the backend the coordinator's dispose-time shutdown
// marker (enqueued after the flush snapshot) would be dropped silently.
it('drains records enqueued after a timer export began: dispose during an in-flight batch', async () => {
// The backend implements NO flush() — the batch processor exports on its
// own cadence, and shutdown's internal drain is complete exactly because
// nothing in the process calls forceFlush() concurrently (the SDK's
// concurrent-flush guard skips draining otherwise). Pin that: hold the
// collector's response to the timer-triggered export open across
// disposal, and the dispose-time shutdown marker (enqueued after that
// batch's snapshot) must still arrive.
const gate = Promise.withResolvers<boolean>()
const arrived = Promise.withResolvers<boolean>()
const { url, captures } = await mockCollector(async (index) => {
@@ -137,10 +139,14 @@ describe('TelemetryOtel wire', () => {
await gate.promise
}
})
const { ctx, fiber } = await boot(url)
const session = ctx.sessions.create(SessionId('race'), { meta: {} })
const ctx = new Context()
await ctx.plugin(SessionStore)
const fiber = await ctx.plugin(TelemetryOtel, {
exporter: { url },
processor: { scheduledDelayMillis: 10 },
})
const session = ctx.sessions.create(SessionId('drain'), { meta: {} })
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
ctx.telemetry.flush!()
await arrived.promise
const disposal = fiber.dispose()
@@ -149,40 +155,6 @@ describe('TelemetryOtel wire', () => {
gate.resolve(true)
await disposal
const records = allRecords(captures)
const ops = records.filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel/ops')
expect(ops).toHaveLength(1)
expect(ops[0]!.record.attributes).toContainEqual({ key: 'telemetry.op', value: { stringValue: 'shutdown' } })
})
it('orders shutdown behind the OLDEST in-flight flush when hints overlap', async () => {
// The SDK's concurrent-flush guard resolves an overlapping forceFlush()
// immediately; if the backend RETAINS only the latest flush promise, two
// back-to-back turn flushes leave shutdown awaiting the instantly-resolved
// second one while the first still exports — reopening the same silent
// drop the single-flush race test pins.
const gate = Promise.withResolvers<boolean>()
const arrived = Promise.withResolvers<boolean>()
const { url, captures } = await mockCollector(async (index) => {
if (index === 0) {
arrived.resolve(true)
await gate.promise
}
})
const { ctx, fiber } = await boot(url)
const session = ctx.sessions.create(SessionId('race2'), { meta: {} })
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
ctx.telemetry.flush!()
await arrived.promise
// Second hint while the first export is held open: resolves immediately
// under the SDK's guard and must not displace the outstanding one.
ctx.telemetry.flush!()
const disposal = fiber.dispose()
await new Promise(resolve => setTimeout(resolve, 50))
gate.resolve(true)
await disposal
const ops = allRecords(captures).filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel/ops')
expect(ops).toHaveLength(1)
expect(ops[0]!.record.attributes).toContainEqual({ key: 'telemetry.op', value: { stringValue: 'shutdown' } })
@@ -209,15 +181,14 @@ describe('TelemetryOtel wire', () => {
expect(types).toContain('turn/start')
})
it('maps the warn severity and forwards the flush hint to the SDK', async () => {
it('maps the warn severity and leaves the seam flush hint unimplemented', async () => {
const { url, captures } = await mockCollector()
const { ctx, fiber } = await boot(url)
const session = ctx.sessions.create(SessionId('warn'), { meta: {} })
session.append('prompt/blocked', { content: [], source: { kind: 'user' }, reason: 'vetoed' })
// The turn-boundary hint: safe, non-blocking, and enough to push the batch out.
expect(() => {
ctx.telemetry.flush!()
}).not.toThrow()
// No flush(): the coordinator's optional-call forwarding no-ops, and the
// batch processor owns export cadence end to end (see the backend note).
expect('flush' in ctx.telemetry && ctx.telemetry.flush !== undefined).toBe(false)
await fiber.dispose()
const blocked = allRecords(captures).find(r =>
r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'prompt/blocked'))

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
README.md: 73bc22053ad08725fecade43fb33fe944dfe3c60
README.zh.md: 8e63e14bd8e099980133c9d6d864135fba7f39af
README.md: df0384d0528f0b3a95ded414444e1b11ea7d52bf
README.zh.md: d86fede206a4b2363f1deda86ff32d868d1db71d

View File

@@ -6,7 +6,7 @@ The telemetry seam: the CAPTURE side of session-event reporting, behind a backen
## The backend contract
`TelemetryBackend` is three members: `emit(record)` (MUST be a non-blocking enqueue — it runs synchronously on the `session/event` hot path), optional `flush()` (a turn-boundary hint, fire-and-forget), and `shutdown()` (the lifecycle forward: flush-and-quiesce, awaited at dispose). `Telemetry` is its service-registered form under the `telemetry` context key — one implementation per context, duplicate load throws. A backend composes `TelemetryCoordinator` in its constructor.
`TelemetryBackend` is three members: `emit(record)` (MUST be a non-blocking enqueue — it runs synchronously on the `session/event` hot path), optional `flush()` (a turn-boundary hint, fire-and-forget; most backends leave it unimplemented and let their SDK's batching cadence govern export timing — an implementer owns the interaction between concurrent flushes and `shutdown()`'s drain), and `shutdown()` (the lifecycle forward: drain-and-quiesce, awaited at dispose). `Telemetry` is its service-registered form under the `telemetry` context key — one implementation per context, duplicate load throws. A backend composes `TelemetryCoordinator` in its constructor.
## Capture points

View File

@@ -6,7 +6,7 @@
## 后端契约
`TelemetryBackend` 只有三个成员:`emit(record)`(必须是非阻塞入队;它在 `session/event` 热路径上同步执行)、可选的 `flush()`(轮次边界提示,触发后不等待结果)、以及 `shutdown()`(生命周期转发点:flush 并完全停稳,在 dispose资源释放时被等待`Telemetry` 是它注册在 `telemetry` 上下文键下的服务形态:每个上下文只允许一个实现,重复加载会抛出异常。后端在其构造函数中组合 `TelemetryCoordinator`
`TelemetryBackend` 只有三个成员:`emit(record)`(必须是非阻塞入队;它在 `session/event` 热路径上同步执行)、可选的 `flush()`(轮次边界提示,触发后不等待结果;多数后端不实现它,而由其 SDK 的批处理节奏决定导出时机;并发 flush 与 `shutdown()` 的排空之间的交互由实现方自行负责)、以及 `shutdown()`(生命周期转发点:排空并完全停稳,在 dispose资源释放时被等待`Telemetry` 是它注册在 `telemetry` 上下文键下的服务形态:每个上下文只允许一个实现,重复加载会抛出异常。后端在其构造函数中组合 `TelemetryCoordinator`
## 捕获点

View File

@@ -103,7 +103,12 @@ export interface TelemetryBackend {
* Optional hint that a natural boundary (turn end) passed — a backend may
* forward it to its SDK's flush so records land at turn boundaries. Called
* fire-and-forget; implementations must not block and must not throw
* meaningfully (the coordinator contains exceptions).
* meaningfully (the coordinator contains exceptions). Most backends should
* leave this unimplemented and let their SDK's own batching cadence govern
* export timing: a backend that does implement it owns the interaction
* between its concurrent flushes and {@link shutdown}'s drain (the OTel
* backend removed its implementation for exactly that hazard — see the
* revival Agent Note).
*/
flush?(): void
/**