refactor(telemetry): ship the redact waterfall without built-in rules

The seam keeps the telemetry/redact scrubbing interface but ships no rules
of its own: the innermost next() passes records through unchanged, and
deployments mount their rules as waterfall listeners. As an SDK we cannot
know which patterns are secrets in a given deployment; a shipped list
invites false confidence while catching only known shapes, and false
positives would corrupt exported bodies. Mechanism stays with the seam,
policy moves to the deployment; both READMEs and the Agent Note state the
raw-export default plainly.

The loader-composition e2e now mounts a deployment-style rule fixture and
pins the same wire behavior: secret absent, placeholder present, canonical
log untouched.
This commit is contained in:
kingwl
2026-07-23 11:58:47 +08:00
parent cf2e184112
commit 70febffe1a
18 changed files with 155 additions and 236 deletions

View File

@@ -2,10 +2,11 @@
* Capture coordinator: the seam's upstream half. Subscribes to the session
* firehose plus the one live-bus relay (`agent/error`), applies the fixed
* chunk projection, builds logical records, runs each through the
* `telemetry/redact` waterfall, and hands the redacted copy to the backend —
* synchronously, with every handler self-contained so a failing backend can
* never starve other subscribers (cordis `emit` is stop-on-throw) or touch
* the agent loop. Composed by a backend in its constructor.
* `telemetry/redact` waterfall (deployment-mounted rules; pass-through when
* none), and hands the result to the backend — synchronously, with every
* handler self-contained so a failing backend can never starve other
* subscribers (cordis `emit` is stop-on-throw) or touch the agent loop.
* Composed by a backend in its constructor.
*
* @module @deepseek-ai/dsh-session-telemetry/coordinator
*/
@@ -14,7 +15,6 @@ import type { Context } from 'cordis'
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
import type { Agent } from '@deepseek-ai/dsh-agent'
import type { TelemetryBackend, TelemetryRecord, TelemetrySeverity } from './index.ts'
import { applyDefaultRedaction } from './redact.ts'
/**
* The handoff cursor: per session, the highest `seq` handed to a backend.
@@ -145,13 +145,13 @@ export class TelemetryCoordinator {
/**
* Run the `telemetry/redact` waterfall over one record and hand the result
* to the backend. The innermost `next` applies the seam's conservative
* default rules, so an unconfigured deployment still never exports raw
* credential shapes; callers run inside {@link contain}, so a throwing
* to the backend. The innermost `next` passes the record through unchanged
* — the seam ships no rules; exported data is as clean as the listeners a
* deployment mounts. Callers run inside {@link contain}, so a throwing
* rule withholds the record instead of reaching the loop (fail-closed).
*/
private handOff(record: TelemetryRecord): void {
this.backend.emit(this.ctx.waterfall('telemetry/redact', record, () => applyDefaultRedaction(record)))
this.backend.emit(this.ctx.waterfall('telemetry/redact', record, () => record))
}
/** Forward the turn-end boundary to the backend's optional flush hint. */

View File

@@ -22,16 +22,17 @@ declare module 'cordis' {
interface Events {
/**
* Redact one outbound record before it reaches the backend. The innermost
* `next()` applies the seam's conservative default rule set
* (credential-shape scrubbing); listeners stack stricter rules by
* transforming its return value, and returning without `next()` replaces
* the default — the exported record is then only as clean as the
* replacing rule. Dispatched synchronously on the capture hot path inside
* the coordinator's containment: a throwing listener withholds that one
* record (fail-closed) and never reaches the agent loop. Redaction
* applies to the exported copy only; the canonical session log is never
* rewritten.
* Redact one outbound record before it reaches the backend — the seam's
* scrubbing extension point. The seam ships NO rules of its own: the
* innermost `next()` passes the record through unchanged, and with no
* listener mounted records reach the backend as captured, so exported
* data is exactly as clean as the rules a deployment mounts. Listeners
* stack by transforming `next()`'s return value; returning without
* `next()` replaces everything beneath. Dispatched synchronously on the
* capture hot path inside the coordinator's containment: a throwing
* listener withholds that one record (fail-closed) and never reaches the
* agent loop. Redaction applies to the exported copy only; the canonical
* session log is never rewritten.
* @param record - the candidate record, already the coordinator's own deep
* copy; listeners return a (possibly new) record and must not mutate it.
* @mode waterfall
@@ -142,4 +143,3 @@ export abstract class Telemetry extends Service implements TelemetryBackend {
}
export { TelemetryCoordinator } from './coordinator.ts'
export { applyDefaultRedaction, REDACTION_PLACEHOLDER } from './redact.ts'

View File

@@ -1,77 +0,0 @@
/**
* Conservative default redaction for outbound telemetry records.
*
* Session-event bodies carry file contents and command output that may embed
* credentials; nothing may cross the seam to a backend unredacted. This module
* is the innermost rule set of the `telemetry/redact` waterfall — always
* applied unless an outer listener deliberately replaces the whole chain. It
* scrubs credential-SHAPED substrings from every string in the record body,
* leaving structure (keys, nesting, surrounding prose) intact. The pattern
* list is a security invariant, deliberately not configurable; deployments
* add stricter rules by stacking `telemetry/redact` listeners.
*
* @module @deepseek-ai/dsh-session-telemetry/redact
*/
import type { TelemetryRecord } from './index.ts'
/** Replacement text substituted for each detected credential-shaped span. */
export const REDACTION_PLACEHOLDER = '[REDACTED]'
/**
* Well-known credential shapes. A match anywhere inside a body string is
* replaced; low-signal values (package names, versions, git SHAs, plain URLs)
* deliberately stay untouched — they are the observability signal.
*/
const SECRET_PATTERNS: readonly RegExp[] = [
/sk-(?:ant-)?[A-Za-z0-9_-]{10,}/g, // DeepSeek / OpenAI / Anthropic API keys
/gh[pousr]_[A-Za-z0-9]{16,}/g, // GitHub personal/oauth/server/refresh tokens
/github_pat_[A-Za-z0-9_]{20,}/g, // GitHub fine-grained PAT
/xox[baprs]-[A-Za-z0-9-]{10,}/g, // Slack tokens
/AKIA[0-9A-Z]{16}/g, // AWS access key id
/AIza[0-9A-Za-z_-]{35}/g, // Google API key
/eyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g, // JWT
/-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----/g, // PEM blocks
/\b(?<scheme>[a-z][a-z0-9+.-]*):\/\/[^/\s:@]+:[^/\s:@]+@/g, // URL userinfo credentials
]
/** Replace every known credential shape inside one string. */
function scrub(text: string): string {
let out = text
for (const pattern of SECRET_PATTERNS) {
out = out.replace(pattern, REDACTION_PLACEHOLDER)
}
return out
}
/**
* Deep-scrub every string inside a lossless-JSON value, preserving structure.
* The record body is the coordinator's own `structuredClone` — mutation-free
* rebuilding keeps the exported copy independent of the canonical log either way.
*/
function scrubValue(value: unknown): unknown {
if (typeof value === 'string') return scrub(value)
if (Array.isArray(value)) return value.map(scrubValue)
if (value !== null && typeof value === 'object') {
const out: Record<string, unknown> = {}
for (const [key, entry] of Object.entries(value)) out[key] = scrubValue(entry)
return out
}
return value
}
/**
* Apply the conservative default rule set to one record — the innermost
* `next` of the `telemetry/redact` waterfall. Attribute VALUES are scrubbed
* alongside the body (identity attributes are seam-built and boring, but
* `session.cwd` is caller-supplied); attribute keys are seam-owned constants.
* @param record - the candidate record; not mutated.
* @returns a redacted copy safe to hand to a backend.
*/
export function applyDefaultRedaction(record: TelemetryRecord): TelemetryRecord {
const attributes: Record<string, string | number> = {}
for (const [key, value] of Object.entries(record.attributes)) {
attributes[key] = typeof value === 'string' ? scrub(value) : value
}
return { ...record, attributes, body: scrubValue(record.body) }
}