feat(telemetry): session-telemetry seam with mandatory redaction + OTel backend

Revive the reviewed session-telemetry packages from the closed
session-telemetry-otlp-rfc branch (PR #222/#231) on current master, renamed
to @deepseek-ai/dsh-session-telemetry{,-otel} (the SDK component-telemetry
package holds the dsh-telemetry name).

Delta over the branch version: every record now passes a telemetry/redact
waterfall between projection and emit() — the innermost next() applies a
non-configurable conservative credential-shape rule set, listeners stack
stricter rules, a throwing rule withholds the record fail-closed, and the
canonical log is never rewritten. This answers the export-side concern that
closed PR #222; the boundary axiom (our aspect ends at emit(); delivery is
the reporting SDK's) is unchanged, and the runtime-telemetry RFC's outbox /
readCommitted lane is recorded as deferred in the Agent Note.

Covered by seam/redact/OTel-wire unit tiers (100% per-file) and a keyless
Loader-composition e2e that boots the examples fixture against a mock OTLP
collector and pins redaction on the wire plus the untouched canonical log.
This commit is contained in:
kingwl
2026-07-23 03:06:57 +08:00
parent f7b36bd36d
commit cf2e184112
39 changed files with 2207 additions and 7 deletions

View File

@@ -0,0 +1,40 @@
# @deepseek-ai/dsh-session-telemetry
The telemetry seam: the CAPTURE side of session-event reporting, behind a backend contract any reporting SDK satisfies with zero bending. The boundary axiom that shapes everything here: **this package's aspect ends at `emit()`** — batching, retry, queueing, and loss policy belong to the backend's SDK and are neither specified nor wrapped. Rationale and rejected alternatives: [the revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md).
## The backend contract
`TelemetryBackend` is three members: `emit(record)` (MUST be a non-blocking enqueue — it runs synchronously on the `session/event` hot path), optional `flush()` (a turn-boundary hint, fire-and-forget), and `shutdown()` (the lifecycle forward: flush-and-quiesce, awaited at dispose). `Telemetry` is its service-registered form under the `telemetry` context key — one implementation per context, duplicate load throws. A backend composes `TelemetryCoordinator` in its constructor.
## Capture points
The coordinator registers, all through the composing fiber's effects: `session/created` (adopt: record the header, read the log back through the projection — seed events from fork/resume never re-emit on the firehose), `session/event` (project, deep-copy, redact, hand off; zero I/O), `session/flush` (forward the optional `flush()` hint and return void — the loop's awaited parallel must never wait on telemetry), `agent/error` (the one live-bus relay; turn-enclosure structurally bars those errors from the log), a dispose effect (per adopted session emit its `shutdown` operational record, then await the backend's `shutdown()`; failures warn instead of throwing), and an adoption sweep of `ctx.sessions.list()` (a hot reload does not replay `session/created`).
## The redact waterfall
Every record passes the `telemetry/redact` waterfall between projection and `emit()` — nothing reaches a backend unredacted. The innermost `next()` applies the built-in conservative rule set (`applyDefaultRedaction`: credential shapes — API keys, GitHub/Slack tokens, AWS/Google keys, JWTs, PEM blocks, URL userinfo — replaced with `[REDACTED]` in body strings and string attribute values). Listeners stack stricter rules by transforming `next()`'s return value; returning without `next()` replaces the default rule set, and a throwing listener withholds that one record fail-closed inside the coordinator's containment. The built-in pattern list is a security invariant, deliberately not configurable from cordis.yml. Redaction applies to the exported copy only; the canonical session log is never rewritten.
## The handoff cursor
A module-scope `WeakMap<Session, seq>` marks the highest seq HANDED OFF (not delivered) per session, advanced at emit time. It survives reloads that do not re-evaluate this module — config re-applies and backend source reloads, which is where iteration happens; that asymmetry is why the cursor lives in the seam. On re-adoption the coordinator re-hands only events past the cursor (events at or below it still rebuild the chunk-projection state); a missing cursor safely degrades to a full re-hand, absorbed by receiver-side dedupe on `(session.id, event.seq)`. This is a deliberate, narrow exception to the registrations-are-effects discipline: entries die with their sessions, the value is a monotonic watermark, and losing it is never an error.
## The fixed chunk projection
Only the first `assistant/chunk` of each `(turn, step)` ships; the rest are dropped at capture and never advance the cursor. That one chunk is the stream-started signal: `step/start` + first-chunk presence + `assistant/message` presence + the `turn/end` reason distinguish "the request never started" from "the stream died midway" without chunk volume, and time-to-first-token stays computable. Chunk elision makes `seq` gaps routine on the wire — a gap is never a loss signal. Every other event type, including ones merged by plugins this package never heard of, passes through whole.
## The logical record
`TelemetryRecord`: `channel` (`ledger` | `ops`), `time` (epoch ms), `severity` (pre-mapped: ERROR for `tool/result.isError`, `turn/end` error reasons, `compact/end` errors; WARN for `prompt/blocked`; INFO otherwise), identity-only `attributes` (`session.id`, `event.type`, `event.seq`, plus `session.cwd`/`session.parent_id` when the header has them), and the complete deep-copied `event.data` as `body` — post-redaction. Operational records carry `telemetry.op` (`agent-error` | `shutdown`) and `session.id`, and deliberately NO `event.seq`/`event.type` — signals to alert on, not entries to sum. Delivery downstream of the handoff is the backend SDK's; duplicates remain possible (cursor-less re-adoption, SDK retries), so receivers dedupe on `(session.id, event.seq)`.
## Model Experience
None, as the seam only observes the session stream and hands redacted copies to a reporting backend; it never contributes to a model request.
#### KV Cache effect
None; this package neither assembles nor sends a provider request.
## Known Limitations and Deferred Work
- **Best-effort delivery** — the cursor marks handed-off, not delivered; a session torn down inside a reload window cannot be re-adopted; whatever sits in a backend queue at crash time is lost. A durable outbox (spool, per-sink cursors, at-least-once) is deferred until a deployment states a crash-loss requirement — see [the revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md).
- **Redaction is shape-based** — the default rules catch known credential shapes, not every secret; a deployment with stricter needs stacks `telemetry/redact` listeners, and exported data is only as clean as the mounted rules.

View File

@@ -0,0 +1,41 @@
{
"name": "@deepseek-ai/dsh-session-telemetry",
"description": "Telemetry seam for the DeepSeek Harness: session-event capture, projection, redaction, and handoff to a reporting backend",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./invariant": {
"types": "./lib/types/invariant.d.ts",
"default": "./lib/invariant.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/invariant.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-agent": "^0.0.1",
"@deepseek-ai/dsh-invariants": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,244 @@
/**
* Capture coordinator: the seam's upstream half. Subscribes to the session
* firehose plus the one live-bus relay (`agent/error`), applies the fixed
* chunk projection, builds logical records, runs each through the
* `telemetry/redact` waterfall, and hands the redacted copy to the backend —
* synchronously, with every handler self-contained so a failing backend can
* never starve other subscribers (cordis `emit` is stop-on-throw) or touch
* the agent loop. Composed by a backend in its constructor.
*
* @module @deepseek-ai/dsh-session-telemetry/coordinator
*/
import type { Context } from 'cordis'
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
import type { Agent } from '@deepseek-ai/dsh-agent'
import type { TelemetryBackend, TelemetryRecord, TelemetrySeverity } from './index.ts'
import { applyDefaultRedaction } from './redact.ts'
/**
* The handoff cursor: per session, the highest `seq` handed to a backend.
* Deliberately MODULE-scope ambient state — a narrow, documented exception
* to the registrations-are-effects discipline: cordis has no HMR
* state-handover API, and keying by the `Session` object (which belongs to
* the session store and outlives any telemetry fiber) is the only in-process
* lifetime that lets a re-adopting fiber resume instead of re-handing
* history. Entries die with their sessions; a missing entry safely means
* "re-hand everything". Advanced only at emit time — the cursor marks
* handed-off, not delivered.
*/
const handoffCursor = new WeakMap<Session, number>()
/**
* Install the telemetry capture side onto a context for one backend.
*
* Registers the persistence-coordinator listener set plus the `agent/error`
* relay, all through `ctx.effect()`/`ctx.on()` on the composing fiber, and
* sweeps already-live sessions (a hot reload does not replay
* `session/created`). Disposal emits each adopted session's `shutdown`
* operational record and then awaits the backend's `shutdown()`; a failure
* there warns instead of throwing — best-effort reporting must not fail
* application teardown.
*/
export class TelemetryCoordinator {
/** Sessions adopted by THIS fiber, for dispose-time `shutdown` records and double-adoption protection. */
private readonly adopted = new Set<Session>()
/** Per session, the `turn:step` keys whose first chunk already shipped; rebuilt from the log on re-adoption. */
private readonly chunkSeen = new WeakMap<Session, Set<string>>()
/**
* @param ctx - the composing backend's context; listeners bind to its fiber.
* @param backend - the backend receiving records; owned elsewhere, never disposed here beyond `shutdown()` forwarding.
*/
constructor(
private readonly ctx: Context,
private readonly backend: TelemetryBackend,
) {
ctx.on('session/created', (session) => {
this.adopt(session)
})
ctx.on('session/event', (session, event) => {
this.contain(() => {
this.capture(session, event)
})
})
// Parallel listeners are awaited by the loop at turn end; returning void
// (not the SDK's flush promise) is the turn-latency contract.
ctx.on('session/flush', (session) => {
this.contain(() => {
this.hintFlush(session)
})
})
ctx.on('agent/error', (agent, turn, step, error) => {
this.contain(() => {
this.relayAgentError(agent, turn, step, error)
})
})
ctx.effect(() => async () => {
for (const session of this.adopted) {
this.contain(() => {
this.handOff(shutdownRecord(session))
})
}
try {
await this.backend.shutdown()
} catch (error) {
this.ctx.logger.warn(`telemetry: backend shutdown failed: ${String(error)}`)
}
}, 'telemetry capture')
for (const session of ctx.sessions.list()) {
this.adopt(session)
}
}
/**
* Adopt a session: replay its log THROUGH the projection from the handoff
* cursor (or from the start when no cursor survived), then rely on the
* firehose for everything after. Events at or below the cursor still feed
* the projection state (first-chunk tracking) without being re-handed, so
* a resumed fiber drops mid-step chunk continuations exactly like the
* fiber that saw the step begin.
* @param session - the live session to adopt; a second adoption is a no-op.
*/
private adopt(session: Session): void {
this.contain(() => {
if (this.adopted.has(session)) return
this.adopted.add(session)
const cursor = handoffCursor.get(session) ?? -1
for (const event of session.events) {
if (event.seq <= cursor) this.track(session, event)
else this.capture(session, event)
}
})
}
/** Feed the chunk projection without handing off — the ≤cursor half of re-adoption. */
private track(session: Session, event: SessionEvent): void {
if (event.type === 'assistant/chunk') {
this.seen(session).add(`${event.data.turn}:${event.data.step}`)
}
}
/** Project one event and hand it to the backend, advancing the cursor on handoff. */
private capture(session: Session, event: SessionEvent): void {
if (event.type === 'assistant/chunk') {
const key = `${event.data.turn}:${event.data.step}`
const seen = this.seen(session)
// Fixed chunk projection: only the first chunk of each (turn, step)
// ships — the stream-started signal; content is byte-complete in the
// step's assembled assistant/message. Dropped chunks do not advance
// the cursor, so re-adoption re-drops them deterministically.
if (seen.has(key)) return
seen.add(key)
}
this.handOff({
channel: 'ledger',
time: event.time,
severity: severityOf(event),
attributes: identityOf(session, event),
// The live event object is mutable and the backend serializes later;
// append-time validation guarantees this clone cannot throw.
body: structuredClone(event.data),
})
handoffCursor.set(session, event.seq)
}
/**
* Run the `telemetry/redact` waterfall over one record and hand the result
* to the backend. The innermost `next` applies the seam's conservative
* default rules, so an unconfigured deployment still never exports raw
* credential shapes; callers run inside {@link contain}, so a throwing
* rule withholds the record instead of reaching the loop (fail-closed).
*/
private handOff(record: TelemetryRecord): void {
this.backend.emit(this.ctx.waterfall('telemetry/redact', record, () => applyDefaultRedaction(record)))
}
/** Forward the turn-end boundary to the backend's optional flush hint. */
private hintFlush(session: Session): void {
if (this.adopted.has(session)) this.backend.flush?.()
}
/** Relay one `agent/error` bus emission as an `agent-error` operational record. */
private relayAgentError(agent: Agent, turn: number, step: number, error: Error): void {
this.handOff({
channel: 'ops',
time: Date.now(),
severity: 'error',
attributes: {
'telemetry.op': 'agent-error',
'session.id': String(agent.session.id),
'agent.id': agent.id,
'error.name': error.name,
turn,
step,
},
body: { name: error.name, message: error.message },
})
}
/** Lazily create the per-session first-chunk tracking set. */
private seen(session: Session): Set<string> {
let set = this.chunkSeen.get(session)
if (!set) this.chunkSeen.set(session, set = new Set())
return set
}
/**
* Run one capture-side step with its exception contained: cordis `emit`
* is stop-on-throw, so a throwing listener would starve every subscriber
* registered after this plugin — nothing from the backend may escape.
*/
private contain(step: () => void): void {
try {
step()
} catch (error) {
this.ctx.logger.warn(`telemetry: capture step failed: ${String(error)}`)
}
}
}
/** Build the per-session clean-exit marker emitted at dispose, before the backend's `shutdown()`. */
function shutdownRecord(session: Session): TelemetryRecord {
return {
channel: 'ops',
time: Date.now(),
severity: 'info',
attributes: { 'telemetry.op': 'shutdown', 'session.id': String(session.id) },
body: { op: 'shutdown' },
}
}
/** Map an event's own outcome flag to the pre-baked alerting severity. */
function severityOf(event: SessionEvent): TelemetrySeverity {
switch (event.type) {
case 'tool/result':
return event.data.isError ? 'error' : 'info'
case 'turn/end':
return event.data.reason.kind === 'error' ? 'error' : 'info'
case 'prompt/blocked':
return 'warn'
default: {
// Merge-extensible fall-through (no assertNever): types this seam does
// not depend on still get their RFC-pinned severity via a widened
// probe — `compact/end` is declared by dsh-compact, which the seam
// deliberately does not import.
const type: string = event.type
if (type === 'compact/end' && (event.data as { error?: unknown }).error !== undefined) return 'error'
return 'info'
}
}
}
/** Build the minimal identity attributes: envelope plus self-contained header facts. */
function identityOf(session: Session, event: SessionEvent): Record<string, string | number> {
const attributes: Record<string, string | number> = {
'session.id': String(session.id),
'event.type': event.type,
'event.seq': event.seq,
}
const { cwd, parentSession } = session.header
if (cwd !== undefined) attributes['session.cwd'] = cwd
if (parentSession !== undefined) attributes['session.parent_id'] = String(parentSession)
return attributes
}

View File

@@ -0,0 +1,145 @@
/**
* Telemetry seam for the DeepSeek Harness.
*
* The seam owns the CAPTURE side of session-event reporting — which records
* exist (the chunk projection), what they carry (the logical record), when
* they are handed over (adoption, the per-append firehose, lifecycle
* forwarding), and the HMR handoff cursor. Everything downstream of
* {@link Telemetry.emit} — batching, retry, queueing, loss policy — is the
* reporting SDK's territory and is deliberately not modelled here. The
* design and its trade-offs are pinned in
* .agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md.
*
* @module @deepseek-ai/dsh-session-telemetry
*/
import { Context, Service } from 'cordis'
declare module 'cordis' {
interface Context {
telemetry: Telemetry
}
interface Events {
/**
* Redact one outbound record before it reaches the backend. The innermost
* `next()` applies the seam's conservative default rule set
* (credential-shape scrubbing); listeners stack stricter rules by
* transforming its return value, and returning without `next()` replaces
* the default — the exported record is then only as clean as the
* replacing rule. Dispatched synchronously on the capture hot path inside
* the coordinator's containment: a throwing listener withholds that one
* record (fail-closed) and never reaches the agent loop. Redaction
* applies to the exported copy only; the canonical session log is never
* rewritten.
* @param record - the candidate record, already the coordinator's own deep
* copy; listeners return a (possibly new) record and must not mutate it.
* @mode waterfall
*/
'telemetry/redact'(record: TelemetryRecord, next: () => TelemetryRecord): TelemetryRecord
}
}
/**
* Severity of a telemetry record, pre-mapped at capture so a receiver can
* alert with zero configuration: `error` for events whose own outcome flag
* says so (`tool/result.isError`, `turn/end` error reasons, `compact/end`
* errors) and for `agent-error` operational records, `warn` for
* `prompt/blocked`, `info` for everything else.
*/
export type TelemetrySeverity = 'info' | 'warn' | 'error'
/**
* One logical record handed to a backend — the seam's whole outbound
* vocabulary. Ledger records mirror session-log events one-to-one;
* operational records (`channel: 'ops'`) carry the two signals with no log
* home (`agent-error`, `shutdown`) and deliberately omit `event.seq`-style
* identity so they can never be mistaken for ledger rows.
*/
export interface TelemetryRecord {
/** Ledger (session-log mirror) or ops (operational signal) channel; backends keep the two under separate instrumentation scopes. */
channel: 'ledger' | 'ops'
/** Unix epoch milliseconds — the source event's append time for ledger records, the emission time for ops records. */
time: number
/** Pre-mapped alerting severity; see {@link TelemetrySeverity}. */
severity: TelemetrySeverity
/**
* Identity attributes, deliberately minimal: ledger records carry
* `session.id`, `event.type`, `event.seq`, plus `session.cwd` /
* `session.parent_id` when the header has them; ops records carry
* `telemetry.op`, `session.id`, and (for `agent-error`) `agent.id`,
* `turn`, `step`, `error.name`. Anything recoverable from the body is
* intentionally NOT duplicated here.
*/
attributes: Record<string, string | number>
/**
* The complete payload: a deep copy of the session event's `data` for
* ledger records (JSON-serializable by `Session.append`'s own
* validation), or the op payload for ops records. Never mutated after
* handoff.
*/
body: unknown
}
/**
* The backend contract the coordinator hands records to — the minimum any
* reporting SDK satisfies with zero bending. {@link Telemetry} is its
* service-registered form; tests compose the coordinator with a bare
* implementation of this interface.
*/
export interface TelemetryBackend {
/**
* Hand one record to the backend's pipeline. MUST be a non-blocking
* enqueue — the coordinator calls this synchronously from the
* `session/event` hot path, so anything slower than a queue push would tax
* the agent loop. Errors thrown here are contained by the coordinator and
* logged; they never reach the loop.
* @param record - the logical record to report; owned by the backend after the call.
*/
emit(record: TelemetryRecord): void
/**
* Optional hint that a natural boundary (turn end) passed — a backend may
* forward it to its SDK's flush so records land at turn boundaries. Called
* fire-and-forget; implementations must not block and must not throw
* meaningfully (the coordinator contains exceptions).
*/
flush?(): void
/**
* Forward the fiber's disposal to the SDK: flush whatever is queued and
* reach quiescence, per the SDK's own shutdown contract. Awaited by the
* coordinator's dispose; a rejection is logged as a warning and never
* fails application teardown.
* @returns resolves when the backend's pipeline has quiesced.
*/
shutdown(): Promise<void>
}
/**
* The backend contract in its loadable form: one implementation per context —
* the cordis `Service` registration under the `telemetry` key throws on a
* duplicate, cordis' standard behavior. A backend composes a
* {@link TelemetryCoordinator} in its constructor to install the capture side.
*/
export abstract class Telemetry extends Service implements TelemetryBackend {
constructor(ctx: Context) {
super(ctx, 'telemetry')
}
/**
* See {@link TelemetryBackend.emit} — the seam declaration is the contract's one home.
* @param record - the logical record to report; owned by the backend after the call.
*/
abstract emit(record: TelemetryRecord): void
/** See {@link TelemetryBackend.flush}. */
flush?(): void
/**
* See {@link TelemetryBackend.shutdown}.
* @returns resolves when the backend's pipeline has quiesced.
*/
abstract shutdown(): Promise<void>
}
export { TelemetryCoordinator } from './coordinator.ts'
export { applyDefaultRedaction, REDACTION_PLACEHOLDER } from './redact.ts'

View File

@@ -0,0 +1,32 @@
/**
* Package-owned invariant companion for `@deepseek-ai/dsh-session-telemetry`.
* @module @deepseek-ai/dsh-session-telemetry/invariant
*/
/* jscpd:ignore-start */
import type { Context } from 'cordis'
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
const PACKAGE_NAME = '@deepseek-ai/dsh-session-telemetry'
/** Cordis companion plugin name. */
export const name = 'session-telemetry-invariant'
/** Service required before the companion can reserve package ownership. */
export const inject = ['invariants']
/**
* No runtime invariant: the seam's whole output is the backend handoff — a
* synchronous `emit()` call outside every authoritative event stream — and its
* capture side never appends session events, so no event/data relation exists
* for an independent companion to observe.
*/
const install: InvariantInstaller = () => {}
/**
* Register this package's invariant companion.
* @param ctx - Cordis context carrying the invariant service.
* @returns the installed registration's disposer after setup succeeds.
*/
export const apply = (ctx: Context): Promise<() => void> =>
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
/* jscpd:ignore-end */

View File

@@ -0,0 +1,77 @@
/**
* Conservative default redaction for outbound telemetry records.
*
* Session-event bodies carry file contents and command output that may embed
* credentials; nothing may cross the seam to a backend unredacted. This module
* is the innermost rule set of the `telemetry/redact` waterfall — always
* applied unless an outer listener deliberately replaces the whole chain. It
* scrubs credential-SHAPED substrings from every string in the record body,
* leaving structure (keys, nesting, surrounding prose) intact. The pattern
* list is a security invariant, deliberately not configurable; deployments
* add stricter rules by stacking `telemetry/redact` listeners.
*
* @module @deepseek-ai/dsh-session-telemetry/redact
*/
import type { TelemetryRecord } from './index.ts'
/** Replacement text substituted for each detected credential-shaped span. */
export const REDACTION_PLACEHOLDER = '[REDACTED]'
/**
* Well-known credential shapes. A match anywhere inside a body string is
* replaced; low-signal values (package names, versions, git SHAs, plain URLs)
* deliberately stay untouched — they are the observability signal.
*/
const SECRET_PATTERNS: readonly RegExp[] = [
/sk-(?:ant-)?[A-Za-z0-9_-]{10,}/g, // DeepSeek / OpenAI / Anthropic API keys
/gh[pousr]_[A-Za-z0-9]{16,}/g, // GitHub personal/oauth/server/refresh tokens
/github_pat_[A-Za-z0-9_]{20,}/g, // GitHub fine-grained PAT
/xox[baprs]-[A-Za-z0-9-]{10,}/g, // Slack tokens
/AKIA[0-9A-Z]{16}/g, // AWS access key id
/AIza[0-9A-Za-z_-]{35}/g, // Google API key
/eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g, // JWT
/-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----/g, // PEM blocks
/\b(?<scheme>[a-z][a-z0-9+.-]*):\/\/[^/\s:@]+:[^/\s:@]+@/g, // URL userinfo credentials
]
/** Replace every known credential shape inside one string. */
function scrub(text: string): string {
let out = text
for (const pattern of SECRET_PATTERNS) {
out = out.replace(pattern, REDACTION_PLACEHOLDER)
}
return out
}
/**
* Deep-scrub every string inside a lossless-JSON value, preserving structure.
* The record body is the coordinator's own `structuredClone` — mutation-free
* rebuilding keeps the exported copy independent of the canonical log either way.
*/
function scrubValue(value: unknown): unknown {
if (typeof value === 'string') return scrub(value)
if (Array.isArray(value)) return value.map(scrubValue)
if (value !== null && typeof value === 'object') {
const out: Record<string, unknown> = {}
for (const [key, entry] of Object.entries(value)) out[key] = scrubValue(entry)
return out
}
return value
}
/**
* Apply the conservative default rule set to one record — the innermost
* `next` of the `telemetry/redact` waterfall. Attribute VALUES are scrubbed
* alongside the body (identity attributes are seam-built and boring, but
* `session.cwd` is caller-supplied); attribute keys are seam-owned constants.
* @param record - the candidate record; not mutated.
* @returns a redacted copy safe to hand to a backend.
*/
export function applyDefaultRedaction(record: TelemetryRecord): TelemetryRecord {
const attributes: Record<string, string | number> = {}
for (const [key, value] of Object.entries(record.attributes)) {
attributes[key] = typeof value === 'string' ? scrub(value) : value
}
return { ...record, attributes, body: scrubValue(record.body) }
}

View File

@@ -0,0 +1,157 @@
/**
* Default redaction rules and the `telemetry/redact` waterfall contract:
* credential shapes scrubbed from bodies and attribute values, structure
* preserved, canonical log untouched, listener stacking/replacement, and the
* fail-closed containment of a throwing rule.
*/
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import {
applyDefaultRedaction,
REDACTION_PLACEHOLDER,
TelemetryCoordinator,
type TelemetryBackend,
type TelemetryRecord,
} from '../src/index.ts'
const SECRETS = {
deepseek: 'sk-abcdef1234567890abcdef',
anthropic: 'sk-ant-abcdef1234567890',
githubPat: 'ghp_ABCDEFGHIJKLMNOPqrstuv12345678',
finePat: 'github_pat_ABCDEFGHIJKLMNOPQRSTuvwx',
slack: 'xoxb-1234567890-abcdefghij',
aws: 'AKIAIOSFODNN7EXAMPLE',
google: 'AIzaSyA-1234567890abcdefghijklmnopqrstu',
jwt: 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpM',
pem: '-----BEGIN RSA PRIVATE KEY-----\nMIIEow\n-----END RSA PRIVATE KEY-----',
urlCreds: 'https://user:hunter2@internal.example.com/repo.git',
} as const
function record(body: unknown, attributes: Record<string, string | number> = {}): TelemetryRecord {
return { channel: 'ledger', time: 1, severity: 'info', attributes, body }
}
describe('applyDefaultRedaction', () => {
it('scrubs every known credential shape while preserving surrounding text', () => {
for (const secret of Object.values(SECRETS)) {
const out = applyDefaultRedaction(record(`before ${secret} after`))
expect(out.body, secret).not.toContain(secret.includes('\n') ? 'MIIEow' : secret)
expect(out.body).toContain('before ')
expect(out.body).toContain(' after')
expect(out.body).toContain(REDACTION_PLACEHOLDER)
}
})
it('scrubs URL userinfo credentials but leaves plain URLs alone', () => {
const out = applyDefaultRedaction(record(`${SECRETS.urlCreds} and https://example.com/path`))
expect(out.body).not.toContain('hunter2')
expect(out.body).toContain('https://example.com/path')
})
it('recurses through arrays and objects, preserving structure and non-strings', () => {
const out = applyDefaultRedaction(record({
list: [`key=${SECRETS.deepseek}`, 7, null, true],
nested: { text: SECRETS.githubPat, count: 3 },
}))
expect(out.body).toEqual({
list: [`key=${REDACTION_PLACEHOLDER}`, 7, null, true],
nested: { text: REDACTION_PLACEHOLDER, count: 3 },
})
})
it('leaves low-signal values untouched', () => {
const clean = {
pkg: '@deepseek-ai/dsh-session-telemetry@0.0.1',
sha: '342a4c3a9d3adf13cf4ad33b9f8d6e79170be5e2',
prose: 'ordinary sentence with kebab-case-identifier',
}
expect(applyDefaultRedaction(record(clean)).body).toEqual(clean)
})
it('scrubs string attribute values and keeps numeric ones', () => {
const out = applyDefaultRedaction(record(null, {
'session.cwd': `/home/${SECRETS.aws}/proj`,
'event.seq': 4,
}))
expect(out.attributes['session.cwd']).toBe(`/home/${REDACTION_PLACEHOLDER}/proj`)
expect(out.attributes['event.seq']).toBe(4)
})
it('never mutates its input', () => {
const input = record({ text: SECRETS.slack }, { 'session.cwd': SECRETS.aws })
applyDefaultRedaction(input)
expect((input.body as { text: string }).text).toBe(SECRETS.slack)
expect(input.attributes['session.cwd']).toBe(SECRETS.aws)
})
})
class CollectingBackend implements TelemetryBackend {
records: TelemetryRecord[] = []
emit(record: TelemetryRecord): void {
this.records.push(record)
}
async shutdown(): Promise<void> {}
}
async function setup() {
const backend = new CollectingBackend()
const ctx = new Context()
await ctx.plugin(SessionStore)
await ctx.plugin({
name: 'fake-telemetry',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, backend),
})
return { ctx, backend }
}
describe('telemetry/redact waterfall', () => {
it('applies the default rules when no listener is registered', async () => {
const { ctx, backend } = await setup()
const session = ctx.sessions.create(SessionId('w'))
session.append('user/message', { content: [{ type: 'text', text: `key ${SECRETS.deepseek}` }], source: { kind: 'user' } }, { surfaceOp: 'append' })
const body = backend.records[0]!.body as { content: { text: string }[] }
expect(body.content[0]!.text).toBe(`key ${REDACTION_PLACEHOLDER}`)
})
it('keeps the canonical log unredacted', async () => {
const { ctx } = await setup()
const session = ctx.sessions.create(SessionId('log'))
session.append('user/message', { content: [{ type: 'text', text: SECRETS.githubPat }], source: { kind: 'user' } }, { surfaceOp: 'append' })
const logged = session.events[0]!.data as { content: { text: string }[] }
expect(logged.content[0]!.text).toBe(SECRETS.githubPat)
})
it('lets a listener stack a stricter rule on top of the defaults', async () => {
const { ctx, backend } = await setup()
ctx.on('telemetry/redact', (_record, next) => {
const defaulted = next()
return { ...defaulted, body: { shapeOnly: true } }
})
const session = ctx.sessions.create(SessionId('stack'))
session.append('user/message', { content: [{ type: 'text', text: 'anything' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
expect(backend.records[0]!.body).toEqual({ shapeOnly: true })
})
it('a listener that skips next() replaces the default rules', async () => {
const { ctx, backend } = await setup()
ctx.on('telemetry/redact', record => record)
const session = ctx.sessions.create(SessionId('veto'))
session.append('user/message', { content: [{ type: 'text', text: SECRETS.slack }], source: { kind: 'user' } }, { surfaceOp: 'append' })
const body = backend.records[0]!.body as { content: { text: string }[] }
expect(body.content[0]!.text).toBe(SECRETS.slack)
})
it('a throwing rule withholds the record fail-closed without disturbing the log', async () => {
const { ctx, backend } = await setup()
ctx.on('telemetry/redact', () => {
throw new Error('rule exploded')
})
const session = ctx.sessions.create(SessionId('closed'))
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
expect(backend.records).toHaveLength(0)
expect(session.events).toHaveLength(1)
})
})

View File

@@ -0,0 +1,313 @@
/**
* Coordinator semantics against a bare fake backend — the RFC's named unit
* tier for the seam: adoption (fresh, seeded, re-adoption via the handoff
* cursor), the fixed chunk projection, deep-copy isolation, turn-latency and
* dispose-ordering pins, failure containment, and the `agent/error` relay.
*/
import { describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import SessionStore, { SessionId, type Session, type SessionEvent } from '@deepseek-ai/dsh-session'
import type { Agent } from '@deepseek-ai/dsh-agent'
import { TelemetryCoordinator, type TelemetryBackend, type TelemetryRecord } from '../src/index.ts'
declare module '@deepseek-ai/dsh-session' {
interface SessionEventMap {
/**
* Test-only merged event proving unknown types flow through unchanged.
* @mode emit
* @param payload - opaque test payload
*/
'telemetry-test/opaque': { payload: { nested: string[] } }
/**
* Test-only stand-in for dsh-compact's merge, exercising the widened severity probe.
* @mode emit
* @param error - failure text when the compaction failed
*/
'compact/end': { turn: number; error?: string }
}
}
class FakeBackend implements TelemetryBackend {
records: TelemetryRecord[] = []
calls: string[] = []
emitError: Error | undefined
shutdownError: Error | undefined
shutdownResolved = false
emit(record: TelemetryRecord): void {
if (this.emitError) throw this.emitError
this.records.push(record)
this.calls.push(`emit:${String(record.attributes['event.seq'] ?? record.attributes['telemetry.op'])}`)
}
flush = vi.fn()
async shutdown(): Promise<void> {
this.calls.push('shutdown')
await new Promise(resolve => setTimeout(resolve, 5))
if (this.shutdownError) throw this.shutdownError
this.shutdownResolved = true
}
ledger(): TelemetryRecord[] {
return this.records.filter(r => r.channel === 'ledger')
}
}
async function setup(backend: FakeBackend = new FakeBackend()) {
const ctx = new Context()
await ctx.plugin(SessionStore)
const fiber = await ctx.plugin({
name: 'fake-telemetry',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, backend),
})
return { ctx, backend, fiber }
}
function liveSession(ctx: Context, id = `s-${Math.random().toString(36).slice(2)}`): Session {
return ctx.sessions.create(SessionId(id), { meta: {} })
}
function appendTurn(session: Session): void {
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('user/message', { content: [{ type: 'text', text: 'hello' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
}
describe('TelemetryCoordinator capture', () => {
it('hands every appended event over with envelope identity and cloned body', async () => {
const { ctx, backend } = await setup()
const session = liveSession(ctx, 'cap')
appendTurn(session)
const start = backend.ledger()[0]!
const message = backend.ledger()[1]!
expect(start.attributes).toMatchObject({ 'session.id': 'cap', 'event.type': 'turn/start', 'event.seq': 0 })
expect(start.time).toBe(session.events[0]!.time)
expect(start.severity).toBe('info')
expect(message.attributes['event.seq']).toBe(1)
// Deep-copy isolation: mutating the handed-off body never reaches the log.
;(message.body as { content: { text: string }[] }).content[0]!.text = 'tampered'
const logged = session.events[1] as SessionEvent<'user/message'>
expect(logged.data.content[0]).toMatchObject({ text: 'hello' })
})
it('stamps header facts on every record when present', async () => {
const { ctx, backend } = await setup()
const parent = SessionId('parent')
const session = ctx.sessions.create(SessionId('child'), { meta: { cwd: '/tmp/proj', parentSession: parent } })
appendTurn(session)
for (const record of backend.ledger()) {
expect(record.attributes['session.cwd']).toBe('/tmp/proj')
expect(record.attributes['session.parent_id']).toBe('parent')
}
})
it('maps outcome flags to severity, including the widened merge-extensible probe', async () => {
const { ctx, backend } = await setup()
const session = liveSession(ctx)
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('tool/result', { turn: 1, step: 1, callId: 'c1' as never, content: [], isError: true }, { surfaceOp: 'append' })
session.append('tool/result', { turn: 1, step: 1, callId: 'c2' as never, content: [], isError: false }, { surfaceOp: 'append' })
session.append('prompt/blocked', { content: [], source: { kind: 'user' }, reason: 'vetoed' })
session.append('compact/end', { turn: 1, error: 'summarizer died' })
session.append('compact/end', { turn: 1 })
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, message: 'boom' } })
const severities = backend.ledger().map(r => [r.attributes['event.type'], r.severity])
expect(severities).toEqual([
['turn/start', 'info'],
['tool/result', 'error'],
['tool/result', 'info'],
['prompt/blocked', 'warn'],
['compact/end', 'error'],
['compact/end', 'info'],
['turn/end', 'error'],
])
})
it('passes unknown merged event types through unchanged', async () => {
const { ctx, backend } = await setup()
const session = liveSession(ctx)
session.append('telemetry-test/opaque', { payload: { nested: ['a', 'b'] } })
const record = backend.ledger()[0]!
expect(record.attributes['event.type']).toBe('telemetry-test/opaque')
expect(record.severity).toBe('info')
expect(record.body).toEqual({ payload: { nested: ['a', 'b'] } })
})
it('ships only the first chunk of each (turn, step), per session', async () => {
const { ctx, backend } = await setup()
const a = liveSession(ctx, 'a')
const b = liveSession(ctx, 'b')
const chunk = (s: Session, turn: number, step: number, text: string) =>
s.append('assistant/chunk', { turn, step, chunk: { type: 'text-delta', index: 0, text } })
chunk(a, 1, 1, 'a11-first')
chunk(a, 1, 1, 'a11-second')
chunk(a, 1, 2, 'a12-first')
chunk(b, 1, 1, 'b11-first')
chunk(b, 1, 1, 'b11-second')
const shipped = backend.ledger().map(r => [r.attributes['session.id'], (r.body as { chunk: { text: string } }).chunk.text])
expect(shipped).toEqual([
['a', 'a11-first'],
['a', 'a12-first'],
['b', 'b11-first'],
])
})
})
describe('TelemetryCoordinator adoption', () => {
it('reads seeded events back at adoption (fork/resume seeds never re-emit)', async () => {
const backend = new FakeBackend()
const ctx = new Context()
await ctx.plugin(SessionStore)
const parent = liveSession(ctx, 'seed-parent')
appendTurn(parent)
ctx.sessions.create(SessionId('seeded'), { seed: [...parent.events], meta: {} })
await ctx.plugin({
name: 'fake-telemetry',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, backend),
})
const seqs = backend.ledger().map(r => [r.attributes['session.id'], r.attributes['event.seq']])
expect(seqs).toEqual(expect.arrayContaining([
['seed-parent', 0], ['seed-parent', 1],
['seeded', 0], ['seeded', 1],
]))
})
it('adopts exactly once when created fires after the sweep', async () => {
const backend = new FakeBackend()
const ctx = new Context()
await ctx.plugin(SessionStore)
// The enter/announce window: prepare+enter puts the session in the store
// (visible to the constructor sweep) before `session/created` fires, so a
// coordinator loaded inside that window sees the session twice — sweep
// first, created second. The second adoption must be a no-op.
const session = ctx.sessions.prepare(SessionId('overlap'))
appendTurn(session)
ctx.sessions.enter(session)
await ctx.plugin({
name: 'fake-telemetry',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, backend),
})
expect(backend.ledger()).toHaveLength(2)
ctx.sessions.announce(session)
expect(backend.ledger()).toHaveLength(2)
})
it('resumes from the handoff cursor across a reload, re-dropping mid-step chunks', async () => {
const backend = new FakeBackend()
const { ctx, fiber } = await setup(backend)
const session = liveSession(ctx, 'hmr')
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('assistant/chunk', { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'first' } })
expect(backend.ledger()).toHaveLength(2)
await fiber.dispose()
// The reload window: appends while no telemetry listener is registered.
session.append('assistant/chunk', { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'mid-step continuation' } })
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
const second = new FakeBackend()
await ctx.plugin({
name: 'fake-telemetry-2',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, second),
})
// Only the window events past the cursor are re-handed, and the mid-step
// continuation is re-dropped because ≤cursor events rebuilt the projection.
expect(second.ledger().map(r => r.attributes['event.type'])).toEqual(['turn/end'])
})
it('re-hands the full log when no cursor survived (fresh session object)', async () => {
const backend = new FakeBackend()
const ctx = new Context()
await ctx.plugin(SessionStore)
const session = liveSession(ctx, 'fresh')
appendTurn(session)
await ctx.plugin({
name: 'fake-telemetry',
inject: ['sessions'],
apply: (inner: Context) => void new TelemetryCoordinator(inner, backend),
})
expect(backend.ledger().map(r => r.attributes['event.seq'])).toEqual([0, 1])
})
})
describe('TelemetryCoordinator lifecycle and containment', () => {
it('forwards session/flush as a hint without awaiting backend work', async () => {
const { ctx, backend } = await setup()
const session = liveSession(ctx)
let settled = false
backend.flush.mockImplementation(() => {
// The backend may kick off arbitrary async work; the loop's parallel must not wait for it.
void new Promise(resolve => setTimeout(resolve, 50)).then(() => { settled = true })
})
await ctx.parallel('session/flush', session)
expect(backend.flush).toHaveBeenCalledTimes(1)
expect(settled).toBe(false)
})
it('ignores flush hints for sessions it never adopted', async () => {
const { ctx, backend } = await setup()
const stranger = ctx.sessions.prepare(SessionId('stranger'), { meta: {} })
await ctx.parallel('session/flush', stranger)
expect(backend.flush).not.toHaveBeenCalled()
})
it('emits each adopted session’s shutdown record before awaiting backend shutdown', async () => {
const { ctx, backend, fiber } = await setup()
liveSession(ctx, 's1')
liveSession(ctx, 's2')
await fiber.dispose()
expect(backend.calls).toEqual(['emit:shutdown', 'emit:shutdown', 'shutdown'])
expect(backend.shutdownResolved).toBe(true)
const ops = backend.records.filter(r => r.channel === 'ops')
expect(ops.map(r => r.attributes['session.id']).sort()).toEqual(['s1', 's2'])
expect(ops.every(r => r.attributes['telemetry.op'] === 'shutdown' && r.severity === 'info')).toBe(true)
expect(ops.every(r => !('event.seq' in r.attributes) && !('event.type' in r.attributes))).toBe(true)
})
it('warns instead of throwing when backend shutdown fails', async () => {
const backend = new FakeBackend()
backend.shutdownError = new Error('exporter unreachable')
const { ctx, fiber } = await setup(backend)
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
liveSession(ctx)
await expect(fiber.dispose()).resolves.not.toThrow()
expect(warn.mock.calls.some(args => String(args[0]).includes('shutdown failed'))).toBe(true)
})
it('contains emit failures: the append succeeds and capture heals', async () => {
const { ctx, backend } = await setup()
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
const session = liveSession(ctx)
backend.emitError = new Error('backend broke')
expect(() => session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })).not.toThrow()
expect(warn).toHaveBeenCalled()
backend.emitError = undefined
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
expect(backend.ledger().map(r => r.attributes['event.type'])).toEqual(['turn/end'])
})
it('relays agent/error as an ops record with identity and structured name', async () => {
const { ctx, backend } = await setup()
const session = liveSession(ctx, 'erring')
// Only the members the relay reads; the full Agent surface is irrelevant here.
const agent = { id: 'agent-1', session } as Agent
ctx.emit('agent/error', agent, 3, 2, new TypeError('adapter exploded'))
const record = backend.records.find(r => r.channel === 'ops')!
expect(record.severity).toBe('error')
expect(record.attributes).toMatchObject({
'telemetry.op': 'agent-error',
'session.id': 'erring',
'agent.id': 'agent-1',
'error.name': 'TypeError',
turn: 3,
step: 2,
})
expect(record.body).toEqual({ name: 'TypeError', message: 'adapter exploded' })
})
})

View File

@@ -0,0 +1,27 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../../core/session"
},
{
"path": "../../core/agent"
},
{
"path": "../../support/invariants"
}
]
}