refactor(token-meter): simplify singleton service (round 1)
This commit is contained in:
@@ -8,7 +8,7 @@ This is the implementation tier of the compaction capability — see the [interf
|
||||
|
||||
This backend owns the compaction policy:
|
||||
|
||||
- **Measurement** — the effective conversation model's `ModelTokenMeter` prices the provisional request envelope and current surface at one consumed-log revision. The current prompt and prefix override their logged values; the pre-step boundary reuses logged tools and call config.
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the provisional request envelope and current surface at one consumed-log revision. The current prompt and prefix override their logged values; the pre-step boundary reuses logged tools and call config.
|
||||
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
|
||||
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
|
||||
- **Summarization** — a direct `llm/stream` call uses the configured model and cap without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
|
||||
@@ -16,16 +16,16 @@ This backend owns the compaction policy:
|
||||
- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step, outside an open step, so a tool-heavy turn remains compactable and the loop derives history once after mutation.
|
||||
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Recoverable failure records an error end and leaves the surface unchanged.
|
||||
|
||||
`summarize()` is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on the conversation model's meter. The hook returns the summary blocks together with the call envelope it used (`{ summary, model, maxTokens? }`), which is logged on `compact/summary`.
|
||||
`summarize()` is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on `ctx.tokenMeter`. The hook returns the summary blocks together with the call envelope it used (`{ summary, model, maxTokens? }`), which is logged on `compact/summary`.
|
||||
|
||||
## Config (`BasicCompactConfig`)
|
||||
|
||||
Every common setting is optional. Every model known to `ctx.tokenMeter` receives the default compact policy lazily; named overrides merge only the fields supplied and must name a configured meter profile.
|
||||
Every setting is optional. The pressure and retention policy applies to the token meter's single context window.
|
||||
|
||||
| Key | Required | Meaning |
|
||||
|---|---|---|
|
||||
| `models.<model>.thresholdRatio` | no (default `0.8`) | Compact at `floor(contextWindow × ratio)`. |
|
||||
| `models.<model>.retainTokens` | no (default `floor(contextWindow × 0.16)`) | Recent surface budget kept verbatim; must be below the threshold. |
|
||||
| `thresholdRatio` | no (default `0.8`) | Compact at `floor(contextWindow × ratio)`. |
|
||||
| `retainTokens` | no (default `floor(contextWindow × 0.16)`) | Recent surface budget kept verbatim; must be below the threshold. |
|
||||
| `summarizationModel` | no (default `''`) | Empty resolves the latest logged routed model, then `AgentOptions.model`. |
|
||||
| `maxTokens` | no (default `8192`) | Provider generation cap for the summarization call; may include reasoning tokens. |
|
||||
| `compactionRetries` | no (default `1`) | Extra attempts after the first when pressure remains above threshold. |
|
||||
@@ -116,7 +116,7 @@ Rules:
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Pre-step sees a provisional request envelope** — the current prompt and prefix are exact, but routing and tool changes made later in `agent/request` are not logged yet. A router-only agent with no provisional model skips that check.
|
||||
- **Meter accuracy follows the selected profile** — missing provider usage falls back to the token meter's configured character density and structural overhead.
|
||||
- **Meter accuracy follows the fixed heuristic** — missing reusable provider usage falls back to character count plus structural overhead rather than exact tokenization.
|
||||
- **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting.
|
||||
- **Summarization failure fails closed with full, over-budget history** — including truncation at the summarization `maxTokens`, which hidden reasoning tokens can consume; the auto path logs a warning and proceeds.
|
||||
- **The summarization call has no transcript-snapshot coverage** — `dsh-llm-replay` derives calls from `assistant/chunk` events, so this chunk-less direct `ctx.llm.stream()` call cannot replay (named deferred replay infrastructure in [the seam RFC](../../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
|
||||
@@ -7,10 +7,6 @@
|
||||
import type { Context } from 'cordis'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import type { Message } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
TOKEN_METER_MODEL_UNCONFIGURED,
|
||||
TokenMeterError,
|
||||
} from '@deepseek-ai/dsh-token-meter'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
interface AutomaticCompactor {
|
||||
@@ -49,10 +45,6 @@ export function registerAutomaticCompaction(
|
||||
)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
// A named routed model without a meter profile is configuration failure,
|
||||
// not an optional operational compaction miss.
|
||||
if (error instanceof TokenMeterError
|
||||
&& error.code === TOKEN_METER_MODEL_UNCONFIGURED) throw error
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
ctx.logger.warn(`compaction failed: ${message}; proceeding with full history`)
|
||||
}
|
||||
|
||||
@@ -1,63 +1,49 @@
|
||||
/**
|
||||
* Runtime defaulting and per-model policy validation for compact-basic.
|
||||
* Runtime defaulting and policy validation for compact-basic.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-compact-basic/config
|
||||
*/
|
||||
|
||||
import { deepFreeze } from '@deepseek-ai/dsh-llm'
|
||||
import type { ModelTokenMeter, TokenMeterService } from '@deepseek-ai/dsh-token-meter'
|
||||
import type {
|
||||
BasicCompactConfig,
|
||||
ModelCompactConfig,
|
||||
ResolvedConfig,
|
||||
ResolvedModelCompactConfig,
|
||||
} from './types.ts'
|
||||
import type { TokenMeterService } from '@deepseek-ai/dsh-token-meter'
|
||||
import type { BasicCompactConfig, ResolvedConfig } from './types.ts'
|
||||
|
||||
/** Default request-pressure fraction for every metered model. */
|
||||
/** Default request-pressure fraction of the token meter's context window. */
|
||||
const DEFAULT_THRESHOLD_RATIO = 0.8
|
||||
|
||||
/** Default verbatim-tail fraction of a model's context window. */
|
||||
/** Default verbatim-tail fraction of the token meter's context window. */
|
||||
const DEFAULT_RETAIN_RATIO = 0.16
|
||||
|
||||
/**
|
||||
* Resolve common defaults and validate every named model override.
|
||||
* Resolve defaults and validate the service-wide compaction policy.
|
||||
* @param config - raw compact-basic configuration.
|
||||
* @param tokenMeter - owning meter service used to reject unknown override names.
|
||||
* @returns a detached deeply immutable top-level configuration.
|
||||
* @param tokenMeter - token meter supplying the context capacity.
|
||||
* @returns a detached deeply immutable configuration.
|
||||
*/
|
||||
export function resolveConfig(
|
||||
config: BasicCompactConfig = {},
|
||||
tokenMeter: TokenMeterService,
|
||||
): ResolvedConfig {
|
||||
const configuredModels: unknown = config.models
|
||||
const models = configuredModels === undefined ? {} : configuredModels
|
||||
if (typeof models !== 'object' || models === null || Array.isArray(models)) {
|
||||
throw new Error('BasicCompactConfig: models must be an object')
|
||||
}
|
||||
|
||||
const detachedModels: Record<string, ModelCompactConfig> = {}
|
||||
for (const [model, override] of Object.entries(models as Record<string, unknown>)) {
|
||||
if (typeof override !== 'object' || override === null || Array.isArray(override)) {
|
||||
throw new Error(`BasicCompactConfig: models.${model} must be an object`)
|
||||
}
|
||||
const meter = tokenMeter.resolve(model)
|
||||
detachedModels[model] = { ...override as ModelCompactConfig }
|
||||
resolveModelConfig({
|
||||
models: detachedModels,
|
||||
summarizationModel: '',
|
||||
maxTokens: 8192,
|
||||
compactionRetries: 1,
|
||||
auto: true,
|
||||
}, meter)
|
||||
}
|
||||
|
||||
const thresholdRatio = config.thresholdRatio ?? DEFAULT_THRESHOLD_RATIO
|
||||
const retainTokens = config.retainTokens
|
||||
?? Math.floor(tokenMeter.contextWindow * DEFAULT_RETAIN_RATIO)
|
||||
const resolved: ResolvedConfig = {
|
||||
models: detachedModels,
|
||||
thresholdRatio,
|
||||
retainTokens,
|
||||
summarizationModel: config.summarizationModel ?? '',
|
||||
maxTokens: config.maxTokens ?? 8192,
|
||||
compactionRetries: config.compactionRetries ?? 1,
|
||||
auto: config.auto ?? true,
|
||||
}
|
||||
|
||||
assertRatio('thresholdRatio', resolved.thresholdRatio)
|
||||
assertNonNegativeInteger('retainTokens', resolved.retainTokens)
|
||||
const thresholdTokens = Math.floor(tokenMeter.contextWindow * resolved.thresholdRatio)
|
||||
if (resolved.retainTokens >= thresholdTokens) {
|
||||
throw new Error(
|
||||
`BasicCompactConfig: retainTokens (${resolved.retainTokens}) must be less than threshold tokens ${thresholdTokens}`,
|
||||
)
|
||||
}
|
||||
assertPositiveInteger('maxTokens', resolved.maxTokens)
|
||||
assertNonNegativeInteger('compactionRetries', resolved.compactionRetries)
|
||||
if (typeof resolved.summarizationModel !== 'string') {
|
||||
@@ -66,36 +52,7 @@ export function resolveConfig(
|
||||
if (typeof resolved.auto !== 'boolean') {
|
||||
throw new Error('BasicCompactConfig: auto must be a boolean')
|
||||
}
|
||||
return deepFreeze(structuredClone(resolved))
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one effective model's default policy plus optional field overrides.
|
||||
* @param config - validated compact-basic configuration.
|
||||
* @param meter - effective model's token-meter handle and context capacity.
|
||||
* @returns a detached immutable model policy.
|
||||
*/
|
||||
export function resolveModelConfig(
|
||||
config: ResolvedConfig,
|
||||
meter: ModelTokenMeter,
|
||||
): ResolvedModelCompactConfig {
|
||||
const override = config.models[meter.model]
|
||||
const thresholdRatio = override?.thresholdRatio ?? DEFAULT_THRESHOLD_RATIO
|
||||
const retainTokens = override?.retainTokens ?? Math.floor(meter.contextWindow * DEFAULT_RETAIN_RATIO)
|
||||
assertRatio(`models.${meter.model}.thresholdRatio`, thresholdRatio)
|
||||
assertNonNegativeInteger(`models.${meter.model}.retainTokens`, retainTokens)
|
||||
const thresholdTokens = Math.floor(meter.contextWindow * thresholdRatio)
|
||||
if (retainTokens >= thresholdTokens) {
|
||||
throw new Error(
|
||||
`BasicCompactConfig: models.${meter.model}.retainTokens (${retainTokens}) must be less than threshold tokens ${thresholdTokens}`,
|
||||
)
|
||||
}
|
||||
return deepFreeze({
|
||||
model: meter.model,
|
||||
contextWindow: meter.contextWindow,
|
||||
thresholdRatio,
|
||||
retainTokens,
|
||||
})
|
||||
return deepFreeze(resolved)
|
||||
}
|
||||
|
||||
function assertPositiveInteger(name: string, value: number): void {
|
||||
|
||||
@@ -11,24 +11,20 @@ import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import { canonicalHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { EpochHeader, Session } from '@deepseek-ai/dsh-session'
|
||||
import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { ModelTokenMeter } from '@deepseek-ai/dsh-token-meter'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { registerAutomaticCompaction } from './automatic.ts'
|
||||
import { resolveConfig, resolveModelConfig } from './config.ts'
|
||||
import { resolveConfig } from './config.ts'
|
||||
import { compactSurfaceRegion, selectCompactableRange } from './region.ts'
|
||||
import { summarizeWithLlm } from './summarizer.ts'
|
||||
import type {
|
||||
BasicCompactConfig,
|
||||
ResolvedConfig,
|
||||
ResolvedModelCompactConfig,
|
||||
} from './types.ts'
|
||||
|
||||
export { resolveConfig, resolveModelConfig } from './config.ts'
|
||||
export { resolveConfig } from './config.ts'
|
||||
export type {
|
||||
BasicCompactConfig,
|
||||
ModelCompactConfig,
|
||||
ResolvedConfig,
|
||||
ResolvedModelCompactConfig,
|
||||
} from './types.ts'
|
||||
|
||||
/** Resolve the latest actual routed model, then the agent's configured fallback. */
|
||||
@@ -61,28 +57,24 @@ function provisionalHeader(
|
||||
* retention, provenance, and summary-convergence pricing.
|
||||
*
|
||||
* `summarize()` is the sole subclass customization hook; the replay and durable
|
||||
* mutation strategy stays fixed so every pricing decision uses one effective
|
||||
* conversation-model meter.
|
||||
* mutation strategy stays fixed so every pricing decision uses the singleton
|
||||
* token meter.
|
||||
*/
|
||||
export class BasicCompactService extends CompactService {
|
||||
static inject = ['llm', 'tokenMeter']
|
||||
|
||||
static Config: z<BasicCompactConfig> = z.object({
|
||||
models: z.dict(z.object({
|
||||
thresholdRatio: z.number(),
|
||||
retainTokens: z.number().step(1),
|
||||
})),
|
||||
thresholdRatio: z.number().default(0.8),
|
||||
retainTokens: z.number().step(1),
|
||||
summarizationModel: z.string().default(''),
|
||||
maxTokens: z.number().step(1).min(1).default(8192),
|
||||
compactionRetries: z.number().step(1).min(0).default(1),
|
||||
auto: z.boolean().default(true),
|
||||
})
|
||||
|
||||
/** Resolved and validated common configuration plus named partial overrides. */
|
||||
/** Resolved and validated compaction configuration. */
|
||||
readonly config: ResolvedConfig
|
||||
|
||||
private readonly modelConfigs = new Map<string, ResolvedModelCompactConfig>()
|
||||
|
||||
constructor(ctx: Context, config: BasicCompactConfig = {}) {
|
||||
super(ctx)
|
||||
this.config = resolveConfig(config, ctx.tokenMeter)
|
||||
@@ -107,9 +99,8 @@ export class BasicCompactService extends CompactService {
|
||||
|
||||
/**
|
||||
* Check replayed pressure for the provisional pre-step envelope and compact
|
||||
* a tool-balanced head until it falls below the effective model threshold.
|
||||
* A genuinely model-less router-first step skips this provisional check;
|
||||
* naming an unconfigured model throws the token meter's typed error.
|
||||
* a tool-balanced head until it falls below the service-wide threshold.
|
||||
* A genuinely model-less router-first step skips this provisional check.
|
||||
* @param agent - agent whose session and provisional model are measured.
|
||||
* @param fullSystemPrompt - current assembled system prompt override.
|
||||
* @param sessionPrefix - current request-only prefix override.
|
||||
@@ -124,10 +115,9 @@ export class BasicCompactService extends CompactService {
|
||||
): Promise<CompactionResult | null> {
|
||||
const model = effectiveModel(agent)
|
||||
if (model === undefined || model.length === 0) return null
|
||||
const meter = this.ctx.tokenMeter.resolve(model)
|
||||
const policy = this._modelConfig(meter)
|
||||
const meter = this.ctx.tokenMeter
|
||||
const requestHeader = provisionalHeader(model, agent.session, fullSystemPrompt, sessionPrefix)
|
||||
const threshold = Math.floor(policy.contextWindow * policy.thresholdRatio)
|
||||
const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio)
|
||||
let measurement = meter.measure(agent.session, requestHeader)
|
||||
if (measurement.totalTokens < threshold) return null
|
||||
|
||||
@@ -139,7 +129,7 @@ export class BasicCompactService extends CompactService {
|
||||
`compaction: pressure revision ${measurement.logRevision} does not match surface revision ${surface.logRevision}`,
|
||||
)
|
||||
}
|
||||
const range = selectCompactableRange(agent.session, surface, policy.retainTokens)
|
||||
const range = selectCompactableRange(agent.session, surface, this.config.retainTokens)
|
||||
if (range === null) {
|
||||
/* v8 ignore else -- concrete replacement preserves a compactable checkpoint; subclass hooks cannot mutate it. */
|
||||
if (result === null) return null
|
||||
@@ -159,12 +149,12 @@ export class BasicCompactService extends CompactService {
|
||||
|
||||
/**
|
||||
* Compact one inclusive positional surface range using the effective
|
||||
* conversation model for all retention and shrink pricing. Reject an agent
|
||||
* that does not own the exact target before any resolution or mutation.
|
||||
* token meter for all retention and shrink pricing. Reject an agent that does
|
||||
* not own the exact target before any mutation.
|
||||
* @param session - session whose surface is mutated; must equal `agent.session`.
|
||||
* @param start - inclusive first surface-node seq.
|
||||
* @param end - inclusive last surface-node seq.
|
||||
* @param agent - owner of the target session, used by the summarizer and model resolver.
|
||||
* @param agent - owner of the target session, used by the summarizer.
|
||||
* @param signal - optional summarization cancellation signal.
|
||||
* @returns the successful durable compaction result.
|
||||
*/
|
||||
@@ -178,27 +168,11 @@ export class BasicCompactService extends CompactService {
|
||||
if (session !== agent.session) {
|
||||
throw new Error('compactRegion: agent.session must be the exact target session')
|
||||
}
|
||||
const model = effectiveModel(agent)
|
||||
if (model === undefined || model.length === 0) {
|
||||
throw new Error('compactRegion: no routed or configured conversation model is available for token pricing')
|
||||
}
|
||||
const meter = this.ctx.tokenMeter.resolve(model)
|
||||
this._modelConfig(meter)
|
||||
return compactSurfaceRegion({
|
||||
meter,
|
||||
meter: this.ctx.tokenMeter,
|
||||
summarize: (text, owner, abort) => this.summarize(text, owner, abort),
|
||||
}, session, start, end, agent, signal)
|
||||
}
|
||||
|
||||
/** Resolve and memoize one lazy default/override model policy. */
|
||||
private _modelConfig(meter: ModelTokenMeter): ResolvedModelCompactConfig {
|
||||
let modelConfig = this.modelConfigs.get(meter.model)
|
||||
if (modelConfig === undefined) {
|
||||
modelConfig = resolveModelConfig(this.config, meter)
|
||||
this.modelConfigs.set(meter.model, modelConfig)
|
||||
}
|
||||
return modelConfig
|
||||
}
|
||||
}
|
||||
|
||||
export default BasicCompactService
|
||||
|
||||
@@ -10,14 +10,14 @@ import {
|
||||
toolPairingBalancedBefore,
|
||||
} from '@deepseek-ai/dsh-compact'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import type { ModelTokenMeter, TokenSurfaceMeasurement } from '@deepseek-ai/dsh-token-meter'
|
||||
import type { TokenMeterService, TokenSurfaceMeasurement } from '@deepseek-ai/dsh-token-meter'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { frameSummary } from './summarizer.ts'
|
||||
import type { SummaryResult } from './summarizer.ts'
|
||||
|
||||
interface RegionDependencies {
|
||||
readonly meter: ModelTokenMeter
|
||||
readonly meter: TokenMeterService
|
||||
summarize(text: string, agent: Agent, signal?: AbortSignal): Promise<SummaryResult>
|
||||
}
|
||||
|
||||
|
||||
@@ -4,18 +4,12 @@
|
||||
* @module @deepseek-ai/dsh-compact-basic/types
|
||||
*/
|
||||
|
||||
/** Optional pressure and retention policy for one metered model. */
|
||||
export interface ModelCompactConfig {
|
||||
/** Compact at this fraction of the model's configured context window. Defaults to `0.8`. */
|
||||
/** Basic compaction configuration; every common field has a deployment default. */
|
||||
export interface BasicCompactConfig {
|
||||
/** Compact at this fraction of the token meter's context window. Defaults to `0.8`. */
|
||||
thresholdRatio?: number
|
||||
/** Recent surface tokens retained verbatim. Defaults to `floor(contextWindow * 0.16)`. */
|
||||
retainTokens?: number
|
||||
}
|
||||
|
||||
/** Basic compaction configuration; every common field has a deployment default. */
|
||||
export interface BasicCompactConfig {
|
||||
/** Field-wise pressure/retention overrides keyed by configured token-meter model name. */
|
||||
models?: Record<string, ModelCompactConfig>
|
||||
/** Summary model; `''` resolves the latest routed model, then `AgentOptions.model`. Defaults to `''`. */
|
||||
summarizationModel?: string
|
||||
/** Provider generation cap for summarization. Defaults to `8192`. */
|
||||
@@ -26,19 +20,12 @@ export interface BasicCompactConfig {
|
||||
auto?: boolean
|
||||
}
|
||||
|
||||
/** Validated top-level defaults plus detached per-model partial overrides. */
|
||||
/** Validated and detached compaction configuration. */
|
||||
export interface ResolvedConfig {
|
||||
readonly models: Readonly<Record<string, Readonly<ModelCompactConfig>>>
|
||||
readonly thresholdRatio: number
|
||||
readonly retainTokens: number
|
||||
readonly summarizationModel: string
|
||||
readonly maxTokens: number
|
||||
readonly compactionRetries: number
|
||||
readonly auto: boolean
|
||||
}
|
||||
|
||||
/** Fully resolved pressure/retention policy for one effective model. */
|
||||
export interface ResolvedModelCompactConfig {
|
||||
readonly model: string
|
||||
readonly contextWindow: number
|
||||
readonly thresholdRatio: number
|
||||
readonly retainTokens: number
|
||||
}
|
||||
|
||||
@@ -1,31 +1,21 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import BasicCompactService, {
|
||||
resolveConfig,
|
||||
resolveModelConfig,
|
||||
} from '@deepseek-ai/dsh-compact-basic'
|
||||
import BasicCompactService, { resolveConfig } from '@deepseek-ai/dsh-compact-basic'
|
||||
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
|
||||
import { selectCompactableRange } from '@deepseek-ai/dsh-compact-basic/src/region.ts'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import LlmService, { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import TokenMeterService, {
|
||||
TOKEN_METER_MODEL_UNCONFIGURED,
|
||||
TokenMeterError,
|
||||
} from '@deepseek-ai/dsh-token-meter'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
const SIGNAL = new AbortController().signal
|
||||
const MODEL = 'test-model'
|
||||
|
||||
function createContext(
|
||||
models: Record<string, { contextWindow?: number; charsPerToken?: number }> = {
|
||||
[MODEL]: { contextWindow: 100, charsPerToken: 1_000 },
|
||||
},
|
||||
): Context {
|
||||
function createContext(contextWindow = 1_000): Context {
|
||||
const ctx = new Context()
|
||||
void new TokenMeterService(ctx, { models })
|
||||
void new TokenMeterService(ctx, { contextWindow })
|
||||
return ctx
|
||||
}
|
||||
|
||||
@@ -34,7 +24,7 @@ function agent(session: Session, model?: string): Agent {
|
||||
}
|
||||
|
||||
/** Closed two-message turns followed by one open turn for durable compaction events. */
|
||||
function conversation(turns = 4, text = 'fixture'): Session {
|
||||
function conversation(turns = 4, text = 'fixture '.repeat(40).trim()): Session {
|
||||
const session = new Session(SessionId(`conversation-${turns}`))
|
||||
for (let turn = 1; turn <= turns; turn += 1) {
|
||||
session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
@@ -128,83 +118,64 @@ async function compactIfNeeded(
|
||||
}
|
||||
|
||||
describe('compact configuration and defaults', () => {
|
||||
it('uses low-friction common and per-profile defaults', () => {
|
||||
const ctx = createContext({
|
||||
[MODEL]: { contextWindow: 100, charsPerToken: 1_000 },
|
||||
large: { contextWindow: 1_000, charsPerToken: 4 },
|
||||
})
|
||||
it('uses low-friction service-wide defaults', () => {
|
||||
const ctx = createContext()
|
||||
const resolved = resolveConfig({}, ctx.tokenMeter)
|
||||
|
||||
expect(resolved).toEqual({
|
||||
models: {},
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 160,
|
||||
summarizationModel: '',
|
||||
maxTokens: 8192,
|
||||
compactionRetries: 1,
|
||||
auto: true,
|
||||
})
|
||||
expect(resolveModelConfig(resolved, ctx.tokenMeter.resolve(MODEL))).toEqual({
|
||||
model: MODEL,
|
||||
contextWindow: 100,
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 16,
|
||||
})
|
||||
expect(resolveModelConfig(resolved, ctx.tokenMeter.resolve('large')).retainTokens).toBe(160)
|
||||
expect(Object.isFrozen(resolved)).toBe(true)
|
||||
})
|
||||
|
||||
it('merges threshold and retention overrides field-wise', () => {
|
||||
it('resolves threshold and retention overrides independently', () => {
|
||||
const ctx = createContext()
|
||||
const thresholdOnly = resolveConfig({
|
||||
models: { [MODEL]: { thresholdRatio: 0.5 } },
|
||||
}, ctx.tokenMeter)
|
||||
expect(resolveModelConfig(thresholdOnly, ctx.tokenMeter.resolve(MODEL))).toMatchObject({
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 16,
|
||||
}, ctx.tokenMeter)
|
||||
expect(thresholdOnly).toMatchObject({
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 160,
|
||||
})
|
||||
|
||||
const retentionOnly = resolveConfig({
|
||||
models: { [MODEL]: { retainTokens: 7 } },
|
||||
retainTokens: 70,
|
||||
}, ctx.tokenMeter)
|
||||
expect(resolveModelConfig(retentionOnly, ctx.tokenMeter.resolve(MODEL))).toMatchObject({
|
||||
expect(retentionOnly).toMatchObject({
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 7,
|
||||
retainTokens: 70,
|
||||
})
|
||||
})
|
||||
|
||||
it('validates common values and model policy invariants', () => {
|
||||
it('validates common values and pressure-policy invariants', () => {
|
||||
const ctx = createContext()
|
||||
const bad = [
|
||||
[{ maxTokens: 0 }, /maxTokens/],
|
||||
[{ compactionRetries: -1 }, /compactionRetries/],
|
||||
[{ auto: 'yes' }, /auto must be a boolean/],
|
||||
[{ summarizationModel: 1 }, /summarizationModel must be a string/],
|
||||
[{ models: null }, /models must be an object/],
|
||||
[{ models: { [MODEL]: null } }, /must be an object/],
|
||||
[{ models: { [MODEL]: { thresholdRatio: 0 } } }, /number in \(0, 1\]/],
|
||||
[{ models: { [MODEL]: { thresholdRatio: 1.1 } } }, /number in \(0, 1\]/],
|
||||
[{ models: { [MODEL]: { retainTokens: -1 } } }, /non-negative integer/],
|
||||
[{ models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 50 } } }, /less than threshold/],
|
||||
[{ thresholdRatio: 0 }, /number in \(0, 1\]/],
|
||||
[{ thresholdRatio: 1.1 }, /number in \(0, 1\]/],
|
||||
[{ retainTokens: -1 }, /non-negative integer/],
|
||||
[{ thresholdRatio: 0.5, retainTokens: 500 }, /less than threshold/],
|
||||
] as Array<[unknown, RegExp]>
|
||||
|
||||
for (const [config, pattern] of bad) {
|
||||
expect(() => resolveConfig(config as BasicCompactConfig, ctx.tokenMeter)).toThrow(pattern)
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects an override for an unknown meter profile with the exact typed error', () => {
|
||||
const ctx = createContext()
|
||||
expect(() => resolveConfig({ models: { missing: { retainTokens: 1 } } }, ctx.tokenMeter))
|
||||
.toThrow(expect.objectContaining({
|
||||
code: TOKEN_METER_MODEL_UNCONFIGURED,
|
||||
model: 'missing',
|
||||
}))
|
||||
})
|
||||
})
|
||||
|
||||
describe('pressure measurement and retention', () => {
|
||||
const compactConfig: BasicCompactConfig = {
|
||||
auto: false,
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
}
|
||||
|
||||
it('skips the provisional check only when no routed or fallback model exists', async () => {
|
||||
@@ -214,10 +185,10 @@ describe('pressure measurement and retention', () => {
|
||||
expect(compact.calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('throws for a named unconfigured model instead of swallowing it', async () => {
|
||||
it('meters any routed model without profile resolution', async () => {
|
||||
const compact = service(compactConfig)
|
||||
await expect(compactIfNeeded(compact, conversation(), 'missing'))
|
||||
.rejects.toMatchObject({ code: TOKEN_METER_MODEL_UNCONFIGURED, model: 'missing' })
|
||||
await expect(compactIfNeeded(compact, conversation(), 'unlisted-model'))
|
||||
.resolves.not.toBeNull()
|
||||
})
|
||||
|
||||
it('does nothing below threshold and compacts a priced head above threshold', async () => {
|
||||
@@ -234,38 +205,39 @@ describe('pressure measurement and retention', () => {
|
||||
it('counts the current prompt and request prefix without putting either on the surface', async () => {
|
||||
const compact = service({
|
||||
auto: false,
|
||||
models: { [MODEL]: { thresholdRatio: 0.7, retainTokens: 9 } },
|
||||
thresholdRatio: 0.7,
|
||||
retainTokens: 50,
|
||||
})
|
||||
const session = conversation(2, 'x'.repeat(2_000))
|
||||
const session = conversation(2, 'x'.repeat(200))
|
||||
expect(await compactIfNeeded(compact, session)).toBeNull()
|
||||
|
||||
const prefix: Message[] = [{
|
||||
role: 'user',
|
||||
content: [{ type: 'text', text: 'p'.repeat(10_000) }],
|
||||
content: [{ type: 'text', text: 'p'.repeat(1_000) }],
|
||||
}]
|
||||
const result = await compactIfNeeded(compact, session, MODEL, 's'.repeat(5_000), prefix)
|
||||
const result = await compactIfNeeded(compact, session, MODEL, 's'.repeat(1_000), prefix)
|
||||
expect(result).not.toBeNull()
|
||||
expect(prefix).toHaveLength(1)
|
||||
expect(session.events.some(event => event.type === 'context/message')).toBe(false)
|
||||
})
|
||||
|
||||
it('uses the latest logged routed model instead of AgentOptions.model', async () => {
|
||||
const ctx = createContext({
|
||||
actual: { contextWindow: 100, charsPerToken: 1_000 },
|
||||
fallback: { contextWindow: 10_000, charsPerToken: 1_000 },
|
||||
})
|
||||
it('uses the latest logged routed model in the provisional request envelope', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = service({
|
||||
auto: false,
|
||||
models: { actual: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
}, ctx)
|
||||
const session = conversation(4)
|
||||
session.append('request/header', {
|
||||
header: { config: { model: 'actual' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
const measure = vi.spyOn(ctx.tokenMeter, 'measure')
|
||||
|
||||
const result = await compactIfNeeded(compact, session, 'fallback')
|
||||
expect(result).not.toBeNull()
|
||||
expect(measure.mock.calls[0]?.[1]?.config.model).toBe('actual')
|
||||
})
|
||||
|
||||
it('declines when envelope pressure is high but the surface has no compactable range', async () => {
|
||||
@@ -279,7 +251,7 @@ describe('pressure measurement and retention', () => {
|
||||
|
||||
it('detects scalar/surface revision disagreement', async () => {
|
||||
const ctx = createContext()
|
||||
const meter = ctx.tokenMeter.resolve(MODEL)
|
||||
const meter = ctx.tokenMeter
|
||||
const original = meter.measureSurface.bind(meter)
|
||||
vi.spyOn(meter, 'measureSurface').mockImplementation((session) => {
|
||||
const measurement = original(session)
|
||||
@@ -294,7 +266,8 @@ describe('pressure measurement and retention', () => {
|
||||
const compact = service({
|
||||
auto: false,
|
||||
compactionRetries: 0,
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.3,
|
||||
retainTokens: 180,
|
||||
})
|
||||
compact.summary = Array.from({ length: 7 }, (_, index) => ({
|
||||
type: 'text',
|
||||
@@ -308,8 +281,9 @@ describe('pressure measurement and retention', () => {
|
||||
it('rounds a retention cut head-ward to preserve tool-call/result pairing', async () => {
|
||||
const compact = service({
|
||||
auto: false,
|
||||
models: { [MODEL]: { thresholdRatio: 0.8, retainTokens: 8 } },
|
||||
})
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 80,
|
||||
}, createContext(4_000))
|
||||
const session = toolConversation()
|
||||
const result = await compactIfNeeded(compact, session)
|
||||
expect(result).not.toBeNull()
|
||||
@@ -327,7 +301,7 @@ describe('pressure measurement and retention', () => {
|
||||
it('rejects a priced surface that is not the current positional surface', () => {
|
||||
const ctx = createContext()
|
||||
const session = conversation(2)
|
||||
const priced = ctx.tokenMeter.resolve(MODEL).measureSurface(session)
|
||||
const priced = ctx.tokenMeter.measureSurface(session)
|
||||
expect(() => selectCompactableRange(session, {
|
||||
...priced,
|
||||
nodes: priced.nodes.slice(1),
|
||||
@@ -355,7 +329,7 @@ describe('pressure measurement and retention', () => {
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
|
||||
const priced = ctx.tokenMeter.resolve(MODEL).measureSurface(session)
|
||||
const priced = ctx.tokenMeter.measureSurface(session)
|
||||
expect(selectCompactableRange(session, priced, 1)).toBeNull()
|
||||
})
|
||||
})
|
||||
@@ -497,7 +471,7 @@ describe('compaction region transaction', () => {
|
||||
|
||||
it('rejects a meter snapshot that changed before summarization began', async () => {
|
||||
const ctx = createContext()
|
||||
const meter = ctx.tokenMeter.resolve(MODEL)
|
||||
const meter = ctx.tokenMeter
|
||||
const original = meter.measureSurface.bind(meter)
|
||||
vi.spyOn(meter, 'measureSurface').mockImplementationOnce((session) => {
|
||||
const measurement = original(session)
|
||||
@@ -569,7 +543,7 @@ describe('compaction region transaction', () => {
|
||||
|
||||
it('rejects a non-shrinking framed summary under the conversation meter', async () => {
|
||||
const compact = service()
|
||||
compact.summary = Array.from({ length: 20 }, (_, index) => ({
|
||||
compact.summary = Array.from({ length: 100 }, (_, index) => ({
|
||||
type: 'text',
|
||||
text: `verbose ${index}`,
|
||||
}))
|
||||
@@ -585,7 +559,7 @@ describe('compaction region transaction', () => {
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
|
||||
})
|
||||
|
||||
it('requires a conversation model for pricing', async () => {
|
||||
it('lets a model-independent custom summarizer compact without a conversation model', async () => {
|
||||
const compact = service()
|
||||
const session = conversation(1)
|
||||
const nodes = session.surface.nodes
|
||||
@@ -594,7 +568,7 @@ describe('compaction region transaction', () => {
|
||||
nodes[0]!.seq,
|
||||
nodes[1]!.seq,
|
||||
agent(session),
|
||||
)).rejects.toThrow(/no routed or configured conversation model/)
|
||||
)).resolves.toMatchObject({ shadowedSeqs: [nodes[0]!.seq, nodes[1]!.seq] })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -632,7 +606,7 @@ async function summarizerHarness(
|
||||
): Promise<{ ctx: Context; adapter: ScriptedAdapter; compact: BasicCompactService }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
void new TokenMeterService(ctx, { models: { [model]: { contextWindow: 100 } } })
|
||||
void new TokenMeterService(ctx, { contextWindow: 1_000 })
|
||||
const adapter = new ScriptedAdapter(blocks, finish)
|
||||
ctx.llm.registerAdapter([model], adapter)
|
||||
const compact = new BasicCompactService(ctx, config)
|
||||
@@ -724,7 +698,8 @@ describe('automatic listener and loader composition', () => {
|
||||
it('compacts above threshold and remains idle below it', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx, {
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
const pressured = conversation(4)
|
||||
await preStep(ctx, agent(pressured, MODEL))
|
||||
@@ -741,7 +716,8 @@ describe('automatic listener and loader composition', () => {
|
||||
const warnings: string[] = []
|
||||
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
|
||||
const compact = new TestCompactService(ctx, {
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
compact.error = 'temporary failure'
|
||||
const session = conversation(4)
|
||||
@@ -751,20 +727,12 @@ describe('automatic listener and loader composition', () => {
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
|
||||
})
|
||||
|
||||
it('propagates a named unknown-model configuration failure', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx)
|
||||
await expect(preStep(ctx, agent(conversation(4), 'missing'))).rejects.toMatchObject({
|
||||
code: TOKEN_METER_MODEL_UNCONFIGURED,
|
||||
model: 'missing',
|
||||
})
|
||||
})
|
||||
|
||||
it('auto:false installs no listener', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
const session = conversation(4)
|
||||
await preStep(ctx, agent(session, MODEL))
|
||||
@@ -777,7 +745,7 @@ describe('automatic listener and loader composition', () => {
|
||||
const meterFiber = await ctx.plugin(TokenMeterService)
|
||||
const compactFiber = await ctx.plugin(BasicCompactService, { auto: false })
|
||||
|
||||
expect(ctx.tokenMeter.resolve('deepseek-v4-flash').contextWindow).toBe(128_000)
|
||||
expect(ctx.tokenMeter.contextWindow).toBe(128_000)
|
||||
expect(ctx.get('compact')).toBeInstanceOf(BasicCompactService)
|
||||
await compactFiber.dispose()
|
||||
expect(ctx.get('compact')).toBeUndefined()
|
||||
@@ -788,11 +756,10 @@ describe('automatic listener and loader composition', () => {
|
||||
it('removes its automatic listener with the plugin fiber', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(TokenMeterService, {
|
||||
models: { [MODEL]: { contextWindow: 100, charsPerToken: 1_000 } },
|
||||
})
|
||||
await ctx.plugin(TokenMeterService, { contextWindow: 1_000 })
|
||||
const fiber = await ctx.plugin(TestCompactService, {
|
||||
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
await fiber.dispose()
|
||||
|
||||
@@ -801,17 +768,3 @@ describe('automatic listener and loader composition', () => {
|
||||
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('typed unknown-model boundary', () => {
|
||||
it('uses TokenMeterError identity rather than message matching', () => {
|
||||
const ctx = createContext()
|
||||
let thrown: unknown
|
||||
try {
|
||||
ctx.tokenMeter.resolve('missing')
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(TokenMeterError)
|
||||
expect(thrown).toMatchObject({ code: TOKEN_METER_MODEL_UNCONFIGURED })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -62,9 +62,7 @@ async function harness(toolSteps: number): Promise<{ ctx: Context; compact: Repr
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(TokenMeterService, {
|
||||
models: { mock: { contextWindow: 64, charsPerToken: 1_000 } },
|
||||
})
|
||||
await ctx.plugin(TokenMeterService, { contextWindow: 400 })
|
||||
ctx.llm.registerAdapter(['mock'], new StepwiseToolAdapter(toolSteps))
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'work',
|
||||
@@ -74,11 +72,12 @@ async function harness(toolSteps: number): Promise<{ ctx: Context; compact: Repr
|
||||
return [{ type: 'text', text: 'work result' }]
|
||||
},
|
||||
}))
|
||||
// Tiny window so a couple of tool steps cross the threshold and compaction
|
||||
// fires within the runaway turn.
|
||||
// Small window so several tool steps cross the threshold and compaction
|
||||
// fires within the runaway turn after enough history can shrink.
|
||||
const compact = new ReproCompactService(ctx, {
|
||||
auto: true,
|
||||
models: { mock: { thresholdRatio: 0.5, retainTokens: 20 } },
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 50,
|
||||
summarizationModel: '',
|
||||
maxTokens: 8192,
|
||||
compactionRetries: 1,
|
||||
|
||||
@@ -57,10 +57,7 @@ describe('real Loader composition', () => {
|
||||
.filter(entry => entry.fiber === undefined && !entry.disabled)
|
||||
.map(entry => entry.options.name)
|
||||
expect(unloaded).toEqual([])
|
||||
expect(context.tokenMeter.resolve('deepseek-v4-flash')).toMatchObject({
|
||||
contextWindow: 128_000,
|
||||
charsPerToken: 4,
|
||||
})
|
||||
expect(context.tokenMeter.contextWindow).toBe(128_000)
|
||||
expect(context.get('compact')).toBeInstanceOf(BasicCompactService)
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user