fix(telemetry): review fixes — flush/shutdown ordering, session retirement, whole-exporter passthrough
Three review findings, each pinned by a red test first: - The OTel backend retains the latest turn-boundary flush promise and shutdown() awaits it before provider.shutdown(): the SDK's concurrent-flush guard makes the shutdown-internal flush return early while one is in flight, silently dropping everything enqueued after the flush snapshot (including the coordinator's dispose-time shutdown markers). - The coordinator retires sessions from the adopted set on session/disposed (mirroring the persistence coordinator): a long-lived backend no longer retains closed sessions and their event logs, and final unload no longer stamps shutdown markers for dead sessions. - The exporter config passes through whole to OTLPLogExporter and its type/JSDoc now advertise the full OTLPExporterNodeConfigBase shape: rebuilding url/headers only silently dropped documented SDK options (timeoutMillis, compression, keepAlive, ...).
This commit is contained in:
@@ -8,7 +8,7 @@ The telemetry seam: the CAPTURE side of session-event reporting, behind a backen
|
||||
|
||||
## Capture points
|
||||
|
||||
The coordinator registers, all through the composing fiber's effects: `session/created` (adopt: record the header, read the log back through the projection — seed events from fork/resume never re-emit on the firehose), `session/event` (project, deep-copy, redact, hand off; zero I/O), `session/flush` (forward the optional `flush()` hint and return void — the loop's awaited parallel must never wait on telemetry), `agent/error` (the one live-bus relay; turn-enclosure structurally bars those errors from the log), a dispose effect (per adopted session emit its `shutdown` operational record, then await the backend's `shutdown()`; failures warn instead of throwing), and an adoption sweep of `ctx.sessions.list()` (a hot reload does not replay `session/created`).
|
||||
The coordinator registers, all through the composing fiber's effects: `session/created` (adopt: record the header, read the log back through the projection — seed events from fork/resume never re-emit on the firehose), `session/event` (project, deep-copy, redact, hand off; zero I/O), `session/flush` (forward the optional `flush()` hint and return void — the loop's awaited parallel must never wait on telemetry), `session/disposed` (retire: release the adopted entry so a long-lived backend neither retains closed sessions nor stamps dispose-time markers for them), `agent/error` (the one live-bus relay; turn-enclosure structurally bars those errors from the log), a dispose effect (per still-adopted session emit its `shutdown` operational record, then await the backend's `shutdown()`; failures warn instead of throwing), and an adoption sweep of `ctx.sessions.list()` (a hot reload does not replay `session/created`).
|
||||
|
||||
## The redact waterfall
|
||||
|
||||
|
||||
@@ -35,13 +35,20 @@ const handoffCursor = new WeakMap<Session, number>()
|
||||
* Registers the persistence-coordinator listener set plus the `agent/error`
|
||||
* relay, all through `ctx.effect()`/`ctx.on()` on the composing fiber, and
|
||||
* sweeps already-live sessions (a hot reload does not replay
|
||||
* `session/created`). Disposal emits each adopted session's `shutdown`
|
||||
* `session/created`). A `session/disposed` retires the session from the
|
||||
* adopted set — a long-lived backend must not retain closed sessions (and
|
||||
* their frozen event logs) or stamp dispose-time markers for sessions that
|
||||
* already ended. Disposal emits each still-adopted session's `shutdown`
|
||||
* operational record and then awaits the backend's `shutdown()`; a failure
|
||||
* there warns instead of throwing — best-effort reporting must not fail
|
||||
* application teardown.
|
||||
*/
|
||||
export class TelemetryCoordinator {
|
||||
/** Sessions adopted by THIS fiber, for dispose-time `shutdown` records and double-adoption protection. */
|
||||
/**
|
||||
* Sessions adopted by THIS fiber and still live, for dispose-time
|
||||
* `shutdown` records and double-adoption protection; `session/disposed`
|
||||
* retires entries.
|
||||
*/
|
||||
private readonly adopted = new Set<Session>()
|
||||
/** Per session, the `turn:step` keys whose first chunk already shipped; rebuilt from the log on re-adoption. */
|
||||
private readonly chunkSeen = new WeakMap<Session, Set<string>>()
|
||||
@@ -57,6 +64,11 @@ export class TelemetryCoordinator {
|
||||
ctx.on('session/created', (session) => {
|
||||
this.adopt(session)
|
||||
})
|
||||
// Retirement is observe-only: the projection/cursor WeakMaps die with the
|
||||
// Session object; only the strong adopted set needs the explicit release.
|
||||
ctx.on('session/disposed', (session) => {
|
||||
this.adopted.delete(session)
|
||||
})
|
||||
ctx.on('session/event', (session, event) => {
|
||||
this.contain(() => {
|
||||
this.capture(session, event)
|
||||
|
||||
@@ -108,9 +108,13 @@ export interface TelemetryBackend {
|
||||
flush?(): void
|
||||
/**
|
||||
* Forward the fiber's disposal to the SDK: flush whatever is queued and
|
||||
* reach quiescence, per the SDK's own shutdown contract. Awaited by the
|
||||
* coordinator's dispose; a rejection is logged as a warning and never
|
||||
* fails application teardown.
|
||||
* reach quiescence, per the SDK's own shutdown contract. Everything
|
||||
* emitted before this call must still be delivered — including records
|
||||
* enqueued while a {@link flush} hint is in flight, so a backend whose SDK
|
||||
* guards against concurrent flushes orders behind the outstanding one (the
|
||||
* coordinator emits its dispose-time `shutdown` markers immediately before
|
||||
* calling this). Awaited by the coordinator's dispose; a rejection is
|
||||
* logged as a warning and never fails application teardown.
|
||||
* @returns resolves when the backend's pipeline has quiesced.
|
||||
*/
|
||||
shutdown(): Promise<void>
|
||||
|
||||
@@ -262,6 +262,23 @@ describe('TelemetryCoordinator lifecycle and containment', () => {
|
||||
expect(ops.every(r => !('event.seq' in r.attributes) && !('event.type' in r.attributes))).toBe(true)
|
||||
})
|
||||
|
||||
it('retires a disposed session: no retention, no stale shutdown marker at unload', async () => {
|
||||
const { ctx, backend, fiber } = await setup()
|
||||
liveSession(ctx, 'survivor')
|
||||
// A session owned by its own fiber: disposing the fiber detaches it from
|
||||
// the store and emits `session/disposed` — the authoritative retirement
|
||||
// signal a long-lived telemetry backend must honor, or every closed
|
||||
// session (and its full event log) stays strongly held for the backend's
|
||||
// lifetime and final unload emits shutdown markers for dead sessions.
|
||||
const owner = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
inner.sessions.create(SessionId('ephemeral'), { meta: {} })
|
||||
}, { inject: ['sessions'] }))
|
||||
await owner.dispose()
|
||||
await fiber.dispose()
|
||||
const ops = backend.records.filter(r => r.channel === 'ops')
|
||||
expect(ops.map(r => r.attributes['session.id'])).toEqual(['survivor'])
|
||||
})
|
||||
|
||||
it('warns instead of throwing when backend shutdown fails', async () => {
|
||||
const backend = new FakeBackend()
|
||||
backend.shutdownError = new Error('exporter unreachable')
|
||||
|
||||
Reference in New Issue
Block a user