Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge origin/master into worktree/ci-under-minute

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-21 20:15:19 +08:00
parent f1202d0dd8 8672fa881a
commit e0775de689
66 changed files with 1697 additions and 622 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
packages/compact/compact-basic/README.md
View File

@@ -1,6 +1,6 @@
# @deepseek-ai/dsh-compact-basic
The **basic compaction backend**: a `BasicCompactService` implementing the `@deepseek-ai/dsh-compact` seam with reusable `ctx.tokenMeter` pressure, token-budget retention, and summarization as a direct one-shot `ctx.llm.stream()` call (interceptable at `llm/stream`).
The **basic compaction backend**: a `BasicCompactService` implementing the `@deepseek-ai/dsh-compact` seam with reusable `ctx.tokenMeter` pressure, token-budget retention, and summarization as a direct one-shot `ctx.llm.stream()` call that replays the conversation prefix to reuse the provider's KV cache (interceptable at `llm/stream`).
This is the implementation tier of the compaction capability — see the [interface package](../compact/README.md) for the seam and the [capability-seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-18-compaction-capability-seam.md) for the design.
@@ -9,32 +9,39 @@ This is the implementation tier of the compaction capability — see the [interf
This backend owns the compaction policy:
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering.
- **Routed policy** — proactive pressure resolves capacity from the adapter that owns the latest durable provider/model route, then scales the default policy plus an optional exact-target override into concrete token budgets. Model discovery remains advisory and is not consulted.
- **Model-free pruning** — after pressure or canonical overflow qualifies, the optional [`ctx.toolResultPrune`](../compact-tool-result-prune/README.md) service rewrites oversized tool results before range selection. Compact-basic remeasures through `ctx.tokenMeter`, skips summarization when pressure becomes safe, and otherwise summarizes the pruned surface. Below-pressure post-step checks never prune.
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes. The optional pruner can repair an oversized closed tool unit when its text-bearing result is the removable bulk; indivisible non-tool units and non-prunable tool remainders remain out of scope.
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
- **Summarization** — a direct `llm/stream` call uses the configured provider/model pair and cap, falling back to the latest logged request target and then the agent target, without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
- **Summarization** — a direct `llm/stream` call uses the configured provider/model pair and cap, falling back to the latest logged request target and then the agent target, without running the loop-only `agent/request` seam. The call replays the conversation's own system prompt, tools, and shadowed-region messages verbatim and appends the compaction instruction as the final user message, so it reuses the provider's warm prefix cache instead of invalidating it. Only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
- **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
- **Lifecycle** — `compactRegion()` mutates `agent.session` and records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes.
- **Overflow recovery** — below-threshold overflow bypasses normal retention, prunes, then attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized whenever `surface.replaceGeneration` advances, including when pruning lands before later summary work throws. No replacement, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
- **Overflow recovery** — provider-confirmed overflow needs no capacity metadata: it bypasses normal pressure and retention, prunes, then attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized whenever `surface.replaceGeneration` advances, including when pruning lands before later summary work throws. No replacement, an exhausted target-specific cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no summary replacement landed. A region failure records an error end; the surface remains unchanged unless pruning already landed. Operational post-step failures warn and continue, while overflow-recovery failure preserves the original provider error only when no earlier replacement advanced the surface. Cancellation remains authoritative after any progress.
The protected `summarize()` method is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on `ctx.tokenMeter`. The hook returns the summary blocks together with the call envelope it used (`{ summary, provider, model, maxTokens? }`), which is logged on `compact/summary`.
## Config (`BasicCompactConfig`)
Every setting is optional. The pressure and retention policy applies to the token meter's single context window. Unrecognized top-level keys are rejected.
Every setting is optional. Top-level policy fields are defaults for every routed model; `modelPolicies` applies partial overrides to exact provider/model pairs. At pressure time, compact-basic asks the owning LLM adapter for that route's context capacity and resolves absolute budgets. Unrecognized keys, duplicate targets, mutually exclusive retention forms, and a merged `retainRatio` that is not below `thresholdRatio` fail plugin load. An absolute `retainTokens` budget that is not below its scaled threshold fails on the first resolvable target because that comparison requires model capacity.
| Key | Required | Meaning |
|---|---|---|
| `thresholdRatio` | no (default `0.8`) | Compact at `floor(contextWindow × ratio)`. |
| `retainTokens` | no (default `floor(contextWindow × 0.16)`) | Recent surface budget kept verbatim; must be below the threshold. |
| `thresholdRatio` | no (default `0.8`) | Compact at `floor(routedContextWindow × ratio)`. |
| `retainRatio` | no (default `0.16`) | Recent surface budget kept verbatim as a fraction of the routed context window; mutually exclusive with `retainTokens`. |
| `retainTokens` | no | Absolute recent surface budget kept verbatim; mutually exclusive with `retainRatio` and must be below the resolved threshold. |
| `summarizationProvider` | no (default `''`) | Set together with `summarizationModel`; an empty pair resolves the latest logged request target, then the `AgentOptions` pair. |
| `summarizationModel` | no (default `''`) | Set together with `summarizationProvider`; an empty pair resolves the latest logged request target, then the `AgentOptions` pair. |
| `maxTokens` | no (default `8192`) | Provider generation cap for the summarization call; may include reasoning tokens. |
| `compactionRetries` | no (default `1`) | Extra attempts after the first when pressure remains above threshold. |
| `maxOverflowRetries` | no (default `1`) | Maximum retries after canonical context-window overflow; `0` disables recovery only. |
| `modelPolicies` | no (default `[]`) | Exact `{ provider, model, ...partialPolicy }` overrides; matching uses both fields and does not depend on `listModels()`. |
| `auto` | no (default `true`) | Register post-step pressure and overflow-recovery listeners. Set `false` for manual-only. |
Every `modelPolicies` entry accepts the policy fields above except `auto` and `modelPolicies` itself. If an entry supplies either retention field, it replaces the default policy's retention choice; otherwise retention is inherited. Summarization provider/model remain a pair inside each entry.
An adapter may return no capacity for a valid dynamic route, and resolved capacity may expose an invalid absolute retention budget. Manual pressure checks then throw a target-specific configuration error; the automatic listener warns once for that exact target and continues with full history. Unrelated operational failures remain independently visible. Canonical provider overflow still attempts recovery because the provider has already established that compaction is necessary.
## Usage
```ts
@@ -53,6 +60,20 @@ export function apply(ctx: Context): void {
Loading the plugin registers `ctx.compact`. Add [`dsh-compact-tool-result-prune`](../compact-tool-result-prune/README.md) as a sibling before this plugin to enable the optional model-free pass. With `auto: true` (the default) it compacts automatically under token pressure; a consumer (a future `/compact` tool) can also call `ctx.compact.compactIfNeeded(...)` or `ctx.compact.compactRegion(...)` directly.
For example, the same compact plugin can safely serve models with different capacities and one target-specific policy:
```yaml
- name: '@deepseek-ai/dsh-compact-basic'
config:
thresholdRatio: 0.8
retainRatio: 0.16
modelPolicies:
- provider: local
model: small-context
thresholdRatio: 0.7
retainTokens: 2048
```
## Model Experience
### Conversation history
@@ -75,30 +96,16 @@ Model-free pruning can avoid the auxiliary call entirely; otherwise it reduces t
Replacing rather than append-only. Each checkpoint invalidates reuse from the first replaced history token; the unchanged request prefix before that range remains reusable.
### Auxiliary summarizer user message
### Auxiliary summarizer request
#### What the model sees
The summarization model receives exactly `Summarize this conversation history:` followed by a blank line, the data-dependent [`renderTranscript()`](../compact/README.md) output, another blank line, and `Summary:`. The conversation model never sees this private request or its reasoning; only returned text is stored.
The summarization model receives the conversation replayed verbatim — the same system prompt, tool schemas, and messages the last routed request sent for the shadowed region — followed by one final user message: the compaction instruction below. The conversation model never sees this private request or its reasoning; only returned text is stored.
#### Token effect
This is a separate model call with data-dependent input and `maxTokens`-capped output. Convergence retries can pay this cost more than once.
#### KV Cache effect
Independent of the conversation request cache. An auxiliary call can reuse an exact transcript prefix, while a different selected range or rendering invalidates reuse from its first changed token.
### Auxiliary summarizer system prompt
#### What the model sees
The summarization model receives the checkpoint-writing instruction below.
##### Auxiliary summarizer system prompt
##### Compaction instruction (final user message)
```markdown
You are a compaction engine for an AI coding assistant. Condense the conversation transcript into a structured checkpoint that lets another model resume the work with no loss of essential context.
You are now acting as a compaction engine for this AI coding assistant. Condense the conversation ABOVE into a structured checkpoint that lets another model resume the work with no loss of essential context.
Output EXACTLY the Markdown structure below: keep every section, in order. Use terse bullets, not prose paragraphs. Write "(none)" for an empty section — never drop a section.
@@ -129,17 +136,18 @@ Output EXACTLY the Markdown structure below: keep every section, in order. Use t
Rules:
- Preserve exact file paths, commands, error strings, identifiers, and function signatures.
- Capture user feedback and explicit instructions faithfully, especially corrections.
- Do NOT mention this summarization process or that the context was compacted.
- If the transcript already contains a <compacted-summary> block, it is a PRIOR checkpoint. Do not copy it forward verbatim: preserve still-true facts, drop stale ones, and merge newer information into a single consolidated summary under the same structure.
- Do NOT mention this summarization request or that the context was compacted.
- Output only the checkpoint text: do not call any tool or take any other action.
- If the conversation already contains a <compacted-summary> block, it is a PRIOR checkpoint. Do not copy it forward verbatim: preserve still-true facts, drop stale ones, and merge newer information into a single consolidated summary under the same structure.
```
#### Token effect
Fixed auxiliary input cost plus the data-dependent transcript on every summarization attempt.
This is a separate model call: the replayed conversation prefix plus the fixed instruction as input, with `maxTokens`-capped output. Convergence retries can pay this cost more than once.
#### KV Cache effect
Prefix-stable for auxiliary calls while this instruction and the summarizer route are unchanged. Changing either starts a different prefix; transcript changes occur after the instruction.
The replayed system prompt, tools, and shadowed-region messages match the conversation's last routed request byte-for-byte, so the provider's warm prefix cache is reused up to the trailing instruction; only that instruction, and the summary output, is uncached. Routing the summarizer to a different provider/model, or compacting a non-head range, forgoes this reuse.
## Known Limitations and Deferred Work

321
packages/compact/compact-basic/src/config.ts
View File

@@ -1,111 +1,310 @@
/**
* Runtime defaulting and policy validation for compact-basic.
* Load-time validation and routed-model policy resolution for compact-basic.
*
* @module @deepseek-ai/dsh-compact-basic/config
*/
import { deepFreeze } from '@deepseek-ai/dsh-llm'
import type { TokenMeterService } from '@deepseek-ai/dsh-token-meter'
import type { BasicCompactConfig, ResolvedConfig } from './types.ts'
import type { LlmCallConfig } from '@deepseek-ai/dsh-llm'
import type {
BasicCompactConfig,
CompactPolicyConfig,
ModelCompactPolicyConfig,
ResolvedCompactSpec,
ResolvedConfig,
ResolvedRetention,
ResolvedTargetPolicy,
} from './types.ts'
/** Default request-pressure fraction of the token meter's context window. */
/** Default request-pressure fraction for every routed model. */
const DEFAULT_THRESHOLD_RATIO = 0.8
/** Default verbatim-tail fraction of the token meter's context window. */
/** Default verbatim-tail fraction for every routed model. */
const DEFAULT_RETAIN_RATIO = 0.16
/** Complete public configuration key set. */
const BASIC_COMPACT_CONFIG_KEYS: ReadonlySet<string> = new Set([
/** Fields shared by top-level defaults and exact-target overrides. */
const POLICY_CONFIG_KEYS = [
'thresholdRatio',
'retainRatio',
'retainTokens',
'summarizationProvider',
'summarizationModel',
'maxTokens',
'compactionRetries',
'maxOverflowRetries',
] as const
/** Complete public top-level configuration key set. */
const BASIC_COMPACT_CONFIG_KEYS: ReadonlySet<string> = new Set([
...POLICY_CONFIG_KEYS,
'modelPolicies',
'auto',
])
/** Reject stale or misspelled keys before defaults can hide them. */
function validateConfigKeys(config: BasicCompactConfig): void {
for (const key of Object.keys(config)) {
if (!BASIC_COMPACT_CONFIG_KEYS.has(key)) {
throw new Error(
`BasicCompactConfig: unknown key "${key}" `
+ '(allowed: thresholdRatio, retainTokens, summarizationProvider, summarizationModel, '
+ 'maxTokens, compactionRetries, maxOverflowRetries, auto)',
)
}
/** Complete exact-target override key set. */
const MODEL_POLICY_KEYS: ReadonlySet<string> = new Set([
'provider',
'model',
...POLICY_CONFIG_KEYS,
])
/** Target-specific pressure configuration failure eligible for warning suppression. */
export class TargetPressureConfigError extends Error {
/**
* @param targetKey - exact provider/model route used as the warning key.
* @param message - actionable configuration failure detail.
*/
constructor(readonly targetKey: string, message: string) {
super(message)
}
}
/**
* Resolve defaults and validate the service-wide compaction policy.
* @param config - raw compact-basic configuration.
* @param tokenMeter - token meter supplying the context capacity.
* @returns a detached deeply immutable configuration.
* Resolve and validate service defaults plus exact-target partial overrides.
* @param config - untrusted plugin configuration after Loader normalization.
* @returns detached immutable defaults and validated exact-target overrides.
*/
export function resolveConfig(
config: BasicCompactConfig = {},
tokenMeter: TokenMeterService,
): ResolvedConfig {
validateConfigKeys(config)
export function resolveConfig(config: BasicCompactConfig = {}): ResolvedConfig {
validateKeys(config, BASIC_COMPACT_CONFIG_KEYS, 'BasicCompactConfig')
validatePolicy(config, 'BasicCompactConfig')
if (config.auto !== undefined && typeof config.auto !== 'boolean') {
throw new Error('BasicCompactConfig: auto must be a boolean')
}
const thresholdRatio = config.thresholdRatio ?? DEFAULT_THRESHOLD_RATIO
const retainTokens = config.retainTokens
?? Math.floor(tokenMeter.contextWindow * DEFAULT_RETAIN_RATIO)
const resolved: ResolvedConfig = {
const retention = resolveRetention(config, { retainRatio: DEFAULT_RETAIN_RATIO })
validateRatioRetention(thresholdRatio, retention, 'BasicCompactConfig')
const modelPolicies = resolveModelPolicies(config.modelPolicies)
for (const [index, policy] of modelPolicies.entries()) {
validateRatioRetention(
policy.thresholdRatio ?? thresholdRatio,
resolveRetention(policy, retention),
`BasicCompactConfig: modelPolicies[${index}]`,
)
}
return deepFreeze({
thresholdRatio,
retainTokens,
...retention,
summarizationProvider: config.summarizationProvider ?? '',
summarizationModel: config.summarizationModel ?? '',
maxTokens: config.maxTokens ?? 8192,
compactionRetries: config.compactionRetries ?? 1,
maxOverflowRetries: config.maxOverflowRetries ?? 1,
modelPolicies,
auto: config.auto ?? true,
}
})
}
assertRatio('thresholdRatio', resolved.thresholdRatio)
assertNonNegativeInteger('retainTokens', resolved.retainTokens)
const thresholdTokens = Math.floor(tokenMeter.contextWindow * resolved.thresholdRatio)
if (resolved.retainTokens >= thresholdTokens) {
throw new Error(
`BasicCompactConfig: retainTokens (${resolved.retainTokens}) must be less than threshold tokens ${thresholdTokens}`,
/**
* Merge the exact provider/model override over the validated default policy.
* @param config - validated service defaults and override table.
* @param target - exact durable provider/model route to match.
* @returns detached immutable policy before model-capacity scaling.
*/
export function resolveTargetPolicy(
config: ResolvedConfig,
target: Pick<LlmCallConfig, 'provider' | 'model'>,
): ResolvedTargetPolicy {
const override = config.modelPolicies.find(policy => (
policy.provider === target.provider && policy.model === target.model
))
const inheritedRetention: ResolvedRetention = config.retainTokens === undefined
? { retainRatio: config.retainRatio }
: { retainTokens: config.retainTokens }
return deepFreeze({
target: { provider: target.provider, model: target.model },
thresholdRatio: override?.thresholdRatio ?? config.thresholdRatio,
...resolveRetention(override ?? {}, inheritedRetention),
summarizationProvider: override?.summarizationProvider ?? config.summarizationProvider,
summarizationModel: override?.summarizationModel ?? config.summarizationModel,
maxTokens: override?.maxTokens ?? config.maxTokens,
compactionRetries: override?.compactionRetries ?? config.compactionRetries,
maxOverflowRetries: override?.maxOverflowRetries ?? config.maxOverflowRetries,
})
}
/**
* Scale one routed policy into concrete token budgets for its model capacity.
* @param policy - merged policy for the exact routed target.
* @param contextWindow - positive adapter-owned capacity for that target.
* @returns detached immutable pressure and retention budgets.
*/
export function resolveCompactSpec(
policy: ResolvedTargetPolicy,
contextWindow: number,
): ResolvedCompactSpec {
const targetKey = `${policy.target.provider}/${policy.target.model}`
if (!Number.isInteger(contextWindow) || contextWindow <= 0) {
throw new TargetPressureConfigError(
targetKey,
`BasicCompactConfig: contextWindow (${contextWindow}) must be a positive integer`,
)
}
assertPositiveInteger('maxTokens', resolved.maxTokens)
assertNonNegativeInteger('compactionRetries', resolved.compactionRetries)
assertNonNegativeInteger('maxOverflowRetries', resolved.maxOverflowRetries)
if (typeof resolved.summarizationProvider !== 'string') {
throw new Error('BasicCompactConfig: summarizationProvider must be a string')
}
if (typeof resolved.summarizationModel !== 'string') {
throw new Error('BasicCompactConfig: summarizationModel must be a string')
}
if ((resolved.summarizationProvider.length === 0) !== (resolved.summarizationModel.length === 0)) {
throw new Error(
'BasicCompactConfig: summarizationProvider and summarizationModel must both be set or both be empty',
const thresholdTokens = Math.floor(contextWindow * policy.thresholdRatio)
const retainTokens = policy.retainTokens === undefined
? Math.floor(contextWindow * policy.retainRatio)
: policy.retainTokens
if (retainTokens >= thresholdTokens) {
throw new TargetPressureConfigError(
targetKey,
`BasicCompactConfig: ${policy.target.provider}/${policy.target.model} retainTokens `
+ `(${retainTokens}) must be less than threshold tokens ${thresholdTokens}`,
)
}
if (typeof resolved.auto !== 'boolean') {
throw new Error('BasicCompactConfig: auto must be a boolean')
}
return deepFreeze(resolved)
return deepFreeze({
target: { ...policy.target },
contextWindow,
thresholdRatio: policy.thresholdRatio,
thresholdTokens,
retainTokens,
summarizationProvider: policy.summarizationProvider,
summarizationModel: policy.summarizationModel,
maxTokens: policy.maxTokens,
compactionRetries: policy.compactionRetries,
maxOverflowRetries: policy.maxOverflowRetries,
})
}
function assertPositiveInteger(name: string, value: number): void {
if (!Number.isInteger(value) || value <= 0) {
throw new Error(`BasicCompactConfig: ${name} (${value}) must be a positive integer`)
/** Choose an explicit retention form or inherit the already-resolved fallback. */
function resolveRetention(
config: CompactPolicyConfig,
fallback: ResolvedRetention,
): ResolvedRetention {
if (config.retainTokens !== undefined) return { retainTokens: config.retainTokens }
if (config.retainRatio !== undefined) return { retainRatio: config.retainRatio }
return fallback
}
/** Reject a capacity-independent retention conflict at plugin load. */
function validateRatioRetention(
thresholdRatio: number,
retention: ResolvedRetention,
name: string,
): void {
if (retention.retainRatio !== undefined && retention.retainRatio >= thresholdRatio) {
throw new Error(
`${name}: retainRatio (${retention.retainRatio}) must be less than `
+ `the resolved thresholdRatio (${thresholdRatio})`,
)
}
}
function assertNonNegativeInteger(name: string, value: number): void {
if (!Number.isInteger(value) || value < 0) {
throw new Error(`BasicCompactConfig: ${name} (${value}) must be a non-negative integer`)
/** Validate, detach, and reject duplicate exact-target policies. */
function resolveModelPolicies(configured: unknown): ModelCompactPolicyConfig[] {
if (configured === undefined) return []
if (!Array.isArray(configured)) {
throw new Error('BasicCompactConfig: modelPolicies must be an array')
}
const seen = new Set<string>()
return configured.map((source: unknown, index) => {
const name = `BasicCompactConfig: modelPolicies[${index}]`
assertModelPolicy(source, name)
const key = `${source.provider}\u0000${source.model}`
if (seen.has(key)) {
throw new Error(
`BasicCompactConfig: duplicate model policy for ${source.provider}/${source.model}`,
)
}
seen.add(key)
return { ...source }
})
}
/** Validate one untrusted exact-target override and narrow its public type. */
function assertModelPolicy(
source: unknown,
name: string,
): asserts source is ModelCompactPolicyConfig {
if (!isUnknownRecord(source)) throw new Error(`${name} must be an object`)
validateKeys(source, MODEL_POLICY_KEYS, name)
assertNonEmptyString(`${name}.provider`, source.provider)
assertNonEmptyString(`${name}.model`, source.model)
validatePolicy(source, name)
}
/** Validate the fields common to defaults and exact-target partial overrides. */
function validatePolicy(
config: CompactPolicyConfig | Record<string, unknown>,
name: string,
): void {
const thresholdRatio = config.thresholdRatio
const retainRatio = config.retainRatio
const retainTokens = config.retainTokens
const maxTokens = config.maxTokens
const compactionRetries = config.compactionRetries
const maxOverflowRetries = config.maxOverflowRetries
if (thresholdRatio !== undefined) assertRatio(`${name}.thresholdRatio`, thresholdRatio)
if (retainRatio !== undefined) assertRatio(`${name}.retainRatio`, retainRatio)
if (retainTokens !== undefined) assertNonNegativeInteger(`${name}.retainTokens`, retainTokens)
if (retainRatio !== undefined && retainTokens !== undefined) {
throw new Error(`${name}: retainRatio and retainTokens are mutually exclusive`)
}
if (maxTokens !== undefined) assertPositiveInteger(`${name}.maxTokens`, maxTokens)
if (compactionRetries !== undefined) {
assertNonNegativeInteger(`${name}.compactionRetries`, compactionRetries)
}
if (maxOverflowRetries !== undefined) {
assertNonNegativeInteger(`${name}.maxOverflowRetries`, maxOverflowRetries)
}
validateSummarizationPair(config, name)
}
/** Require one scope to omit, clear, or replace the summarization target as a pair. */
function validateSummarizationPair(
config: CompactPolicyConfig | Record<string, unknown>,
name: string,
): void {
const provider = config.summarizationProvider
const model = config.summarizationModel
if (provider !== undefined && typeof provider !== 'string') {
throw new Error(`${name}.summarizationProvider must be a string`)
}
if (model !== undefined && typeof model !== 'string') {
throw new Error(`${name}.summarizationModel must be a string`)
}
if (provider === undefined && model === undefined) return
if (provider === undefined || model === undefined
|| (provider.length === 0) !== (model.length === 0)) {
throw new Error(
`${name}: summarizationProvider and summarizationModel must be set together `
+ 'as an empty or non-empty pair',
)
}
}
function assertRatio(name: string, value: number): void {
/** Reject stale or misspelled keys before defaults can hide them. */
function validateKeys(config: object, keys: ReadonlySet<string>, name: string): void {
for (const key of Object.keys(config)) {
if (!keys.has(key)) throw new Error(`${name}: unknown key "${key}"`)
}
}
function isUnknownRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value)
}
function assertNonEmptyString(name: string, value: unknown): asserts value is string {
if (typeof value !== 'string' || value.length === 0) {
throw new Error(`${name} must be a non-empty string`)
}
}
function assertPositiveInteger(name: string, value: unknown): asserts value is number {
if (typeof value !== 'number' || !Number.isInteger(value) || value <= 0) {
throw new Error(`${name} (${String(value)}) must be a positive integer`)
}
}
function assertNonNegativeInteger(name: string, value: unknown): asserts value is number {
if (typeof value !== 'number' || !Number.isInteger(value) || value < 0) {
throw new Error(`${name} (${String(value)}) must be a non-negative integer`)
}
}
function assertRatio(name: string, value: unknown): asserts value is number {
if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0 || value > 1) {
throw new Error(`BasicCompactConfig: ${name} (${value}) must be a number in (0, 1]`)
throw new Error(`${name} (${String(value)}) must be a number in (0, 1]`)
}
}

178
packages/compact/compact-basic/src/index.ts
View File

@@ -10,29 +10,79 @@ import { CompactService } from '@deepseek-ai/dsh-compact'
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
import type { Session } from '@deepseek-ai/dsh-session'
import { CONTEXT_WINDOW_EXCEEDED_CODE, assertNever } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, LlmCallConfig } from '@deepseek-ai/dsh-llm'
import type { Agent } from '@deepseek-ai/dsh-agent'
// Type-only: makes the optional sibling service available to `ctx.get()`.
import type {} from '@deepseek-ai/dsh-compact-tool-result-prune'
import { resolveConfig } from './config.ts'
import {
resolveCompactSpec,
resolveConfig,
resolveTargetPolicy,
TargetPressureConfigError,
} from './config.ts'
import { compactSurfaceRegion, selectCompactableRange } from './region.ts'
import { summarizeWithLlm } from './summarizer.ts'
import type { SummarizationInput } from './summarizer.ts'
import type {
BasicCompactConfig,
ModelCompactPolicyConfig,
ResolvedConfig,
} from './types.ts'
export type {
BasicCompactConfig,
CompactPolicyConfig,
ModelCompactPolicyConfig,
ResolvedCompactSpec,
ResolvedConfig,
ResolvedRetention,
ResolvedTargetPolicy,
} from './types.ts'
/** Resolve the exact model durably routed for the latest provider request. */
function routedModel(session: Session): string | undefined {
const model = session.requestHeader()?.config.model
return model === undefined || model.length === 0 ? undefined : model
/** Resolve the exact provider/model durably routed for the latest request. */
function routedTarget(
session: Session,
): Pick<LlmCallConfig, 'provider' | 'model'> | undefined {
const config = session.requestHeader()?.config
if (config === undefined || config.provider.length === 0 || config.model.length === 0) {
return undefined
}
return { provider: config.provider, model: config.model }
}
/** Resolve the conversation target used to select an optional policy override. */
function conversationTarget(
agent: Agent,
): Pick<LlmCallConfig, 'provider' | 'model'> | undefined {
const routed = routedTarget(agent.session)
if (routed !== undefined) return routed
if (agent.options.provider === undefined || agent.options.provider.length === 0
|| agent.options.model === undefined || agent.options.model.length === 0) return undefined
return { provider: agent.options.provider, model: agent.options.model }
}
const thresholdRatioSchema = z.number()
const retainRatioSchema = z.number()
const retainTokensSchema = z.number().step(1).min(0)
const summarizationProviderSchema = z.string()
const summarizationModelSchema = z.string()
const maxTokensSchema = z.number().step(1).min(1)
const compactionRetriesSchema = z.number().step(1).min(0)
const maxOverflowRetriesSchema = z.number().step(1).min(0)
const modelPolicy: z<ModelCompactPolicyConfig> = z.object({
provider: z.string().required(),
model: z.string().required(),
thresholdRatio: thresholdRatioSchema,
retainRatio: retainRatioSchema,
retainTokens: retainTokensSchema,
summarizationProvider: summarizationProviderSchema,
summarizationModel: summarizationModelSchema,
maxTokens: maxTokensSchema,
compactionRetries: compactionRetriesSchema,
maxOverflowRetries: maxOverflowRetriesSchema,
})
/**
* Dependency-light compaction backend using `ctx.tokenMeter` for pressure,
* retention, provenance, and summary-convergence pricing.
@@ -45,22 +95,26 @@ export class BasicCompactService extends CompactService {
static inject = ['llm', 'tokenMeter']
static Config: z<BasicCompactConfig> = z.object({
thresholdRatio: z.number().default(0.8),
retainTokens: z.number().step(1),
summarizationProvider: z.string().default(''),
summarizationModel: z.string().default(''),
maxTokens: z.number().step(1).min(1).default(8192),
compactionRetries: z.number().step(1).min(0).default(1),
maxOverflowRetries: z.number().step(1).min(0).default(1),
auto: z.boolean().default(true),
thresholdRatio: thresholdRatioSchema,
retainRatio: retainRatioSchema,
retainTokens: retainTokensSchema,
summarizationProvider: summarizationProviderSchema,
summarizationModel: summarizationModelSchema,
maxTokens: maxTokensSchema,
compactionRetries: compactionRetriesSchema,
maxOverflowRetries: maxOverflowRetriesSchema,
modelPolicies: z.array(modelPolicy),
auto: z.boolean(),
})
/** Resolved and validated compaction configuration. */
readonly config: ResolvedConfig
private readonly warnedPressureConfigTargets = new Set<string>()
constructor(ctx: Context, config: BasicCompactConfig = {}) {
super(ctx)
this.config = resolveConfig(config, ctx.tokenMeter)
this.config = resolveConfig(config)
if (this.config.auto) this._registerAutomaticCompaction()
}
@@ -90,16 +144,33 @@ export class BasicCompactService extends CompactService {
const result = await this.compactIfNeeded(agent, 'pressure', signal)
if (result !== null) logResult(result, 'post-step pressure')
} catch (error: unknown) {
if (error instanceof TargetPressureConfigError) {
if (this.warnedPressureConfigTargets.has(error.targetKey)) return
this.warnedPressureConfigTargets.add(error.targetKey)
}
const message = error instanceof Error ? error.message : String(error)
ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`)
}
})
ctx.on('agent/request-error', async (agent, _turn, _step, _error, failure, priorFailures, signal, next) => {
const priorOverflowFailures = priorFailures.filter(item => item.code === CONTEXT_WINDOW_EXCEEDED_CODE).length
if (failure.code !== CONTEXT_WINDOW_EXCEEDED_CODE
|| priorOverflowFailures >= this.config.maxOverflowRetries
|| signal.aborted) return next()
ctx.on('agent/request-error', async (
agent,
_turn,
_step,
_error,
failure,
priorFailures,
signal,
next,
) => {
const priorOverflowFailures = priorFailures.filter(
item => item.code === CONTEXT_WINDOW_EXCEEDED_CODE,
).length
if (failure.code !== CONTEXT_WINDOW_EXCEEDED_CODE || signal.aborted) return next()
const target = routedTarget(agent.session)
if (target === undefined) return next()
const policy = resolveTargetPolicy(this.config, target)
if (priorOverflowFailures >= policy.maxOverflowRetries) return next()
const generation = agent.session.surface.replaceGeneration
let result: CompactionResult | null
@@ -135,19 +206,25 @@ export class BasicCompactService extends CompactService {
}
/**
* Summarize a rendered region through a direct one-shot `ctx.llm.stream()`
* call. Override this sole hook for a template or remote summarizer.
* @param text - plain-text conversation region to condense.
* Summarize the replayed conversation region through a direct one-shot
* `ctx.llm.stream()` call whose prefix reuses the conversation's own system
* prompt, tools, and messages so the provider's KV cache is not invalidated.
* Override this sole hook for a template or remote summarizer.
* @param input - replayed conversation prefix (system, tools, and leading messages) to condense.
* @param agent - supplies routed-model history, fallback model, and session id.
* @param signal - optional cancellation forwarded to the adapter.
* @returns safe text summary blocks and exact auxiliary-call provenance.
*/
protected async summarize(
text: string,
input: SummarizationInput,
agent: Agent,
signal?: AbortSignal,
): Promise<{ summary: ContentBlock[]; provider: string; model: string; maxTokens?: number }> {
return summarizeWithLlm(this.ctx, this.config, text, agent, signal)
const target = conversationTarget(agent)
const config = target === undefined
? this.config
: resolveTargetPolicy(this.config, target)
return summarizeWithLlm(this.ctx, config, input, agent, signal)
}
/**
@@ -165,16 +242,15 @@ export class BasicCompactService extends CompactService {
trigger: CompactionTrigger,
signal: AbortSignal,
): Promise<CompactionResult | null> {
const model = routedModel(agent.session)
if (model === undefined) return null
const target = routedTarget(agent.session)
if (target === undefined) return null
const policy = resolveTargetPolicy(this.config, target)
const meter = this.ctx.tokenMeter
const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio)
let measurement = meter.measure(agent.session)
switch (trigger) {
case 'context-overflow':
break
case 'pressure':
if (measurement.totalTokens < threshold) return null
break
/* v8 ignore next -- closed-union exhaustiveness guard */
default:
@@ -182,25 +258,43 @@ export class BasicCompactService extends CompactService {
}
// Pruning is optional so compact-basic remains independently composable.
// Once either trigger qualifies, land the model-free pass before choosing
// a summary range, then remeasure through the singleton replay fold.
// Overflow always qualifies; pressure first resolves the routed model's
// capacity and checks its target-specific threshold.
const prune = this.ctx.get('toolResultPrune')
if (prune !== undefined) {
prune.pruneSession(agent.session)
measurement = meter.measure(agent.session)
}
if (trigger === 'context-overflow') {
if (prune !== undefined) {
prune.pruneSession(agent.session)
measurement = meter.measure(agent.session)
}
const range = selectCompactableRange(agent.session, measurement, 0)
if (range === null) return null
return this.compactRegion(range.start, range.end, agent, signal)
}
if (measurement.totalTokens < threshold) return null
const context = await this.ctx.llm.resolveModelContext(target.provider, target.model)
const targetKey = `${target.provider}/${target.model}`
if (context === undefined) {
throw new TargetPressureConfigError(
targetKey,
`compact-basic: no context capacity for ${targetKey}; `
+ 'configure contextWindow on that adapter model',
)
}
const spec = resolveCompactSpec(policy, context.contextWindow)
if (measurement.totalTokens < spec.thresholdTokens) return null
// Once pressure qualifies, land the model-free pass before choosing a
// summary range, then remeasure through the singleton replay fold.
if (prune !== undefined) {
prune.pruneSession(agent.session)
measurement = meter.measure(agent.session)
}
if (measurement.totalTokens < spec.thresholdTokens) return null
let result: CompactionResult | null = null
for (let attempt = 0; attempt <= this.config.compactionRetries; attempt += 1) {
const range = selectCompactableRange(agent.session, measurement, this.config.retainTokens)
for (let attempt = 0; attempt <= spec.compactionRetries; attempt += 1) {
const range = selectCompactableRange(agent.session, measurement, spec.retainTokens)
if (range === null) {
/* v8 ignore else -- concrete replacement preserves a compactable checkpoint; subclass hooks cannot mutate it. */
if (result === null) return null
@@ -209,12 +303,12 @@ export class BasicCompactService extends CompactService {
}
result = await this.compactRegion(range.start, range.end, agent, signal)
measurement = meter.measure(agent.session)
if (measurement.totalTokens < threshold) return result
if (measurement.totalTokens < spec.thresholdTokens) return result
}
throw new Error(
`compaction still above threshold after ${this.config.compactionRetries + 1} compaction attempts `
+ `(${measurement.totalTokens} estimated tokens >= threshold ${threshold})`,
`compaction still above threshold after ${spec.compactionRetries + 1} compaction attempts `
+ `(${measurement.totalTokens} estimated tokens >= threshold ${spec.thresholdTokens})`,
)
}
@@ -236,7 +330,7 @@ export class BasicCompactService extends CompactService {
const session = agent.session
return compactSurfaceRegion({
meter: this.ctx.tokenMeter,
summarize: (text, owner, abort) => this.summarize(text, owner, abort),
summarize: (input, owner, abort) => this.summarize(input, owner, abort),
}, session, start, end, agent, signal)
}
}

38
packages/compact/compact-basic/src/region.ts
View File

@@ -5,20 +5,20 @@
*/
import {
renderTranscript,
toolPairingBalancedAfter,
toolPairingBalancedBefore,
} from '@deepseek-ai/dsh-compact'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import type { Message } from '@deepseek-ai/dsh-llm'
import type { TokenMeasurement, TokenMeterService } from '@deepseek-ai/dsh-token-meter'
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
import type { Agent } from '@deepseek-ai/dsh-agent'
import { frameSummary } from './summarizer.ts'
import type { SummaryResult } from './summarizer.ts'
import type { SummarizationInput, SummaryResult } from './summarizer.ts'
interface RegionDependencies {
readonly meter: TokenMeterService
summarize(text: string, agent: Agent, signal?: AbortSignal): Promise<SummaryResult>
summarize(input: SummarizationInput, agent: Agent, signal?: AbortSignal): Promise<SummaryResult>
}
/**
@@ -122,8 +122,8 @@ export async function compactSurfaceRegion(
throw new Error('compaction: selected surface changed before summarization began')
}
const shadowedTokenCount = selected.reduce((total, node) => total + node.tokens, 0)
const text = renderTranscript(session.events, shadowedSeqs)
const { summary, provider, model, maxTokens } = await dependencies.summarize(text, agent, signal)
const summarizationInput = buildSummarizationInput(session, shadowedSeqs)
const { summary, provider, model, maxTokens } = await dependencies.summarize(summarizationInput, agent, signal)
const currentMeasurement = dependencies.meter.measure(session)
if (currentMeasurement.logRevision !== lockedMeasurement.logRevision) {
@@ -173,6 +173,34 @@ export async function compactSurfaceRegion(
}
}
/**
* Reconstruct the last routed request's cacheable prefix for the shadowed
* region: its system prompt and tool schemas, then the request-only message
* prefix followed by the region's own derived messages in surface order. The
* summarizer appends only the compaction instruction after this, so the call
* is a genuine prefix of the conversation and reuses the provider's KV cache.
* @param session - session supplying the request header and per-node projection.
* @param shadowedSeqs - the surface-node seqs, in order, being compacted.
* @returns the replayed conversation prefix to condense.
*/
function buildSummarizationInput(
session: Session,
shadowedSeqs: readonly number[],
): SummarizationInput {
const header = session.requestHeader()
const events = session.events
const regionMessages = shadowedSeqs
// shadowedSeqs are current surface seqs, so each is a valid log index.
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
.map(seq => session.deriveEventMessage(events[seq]!))
.filter((message): message is Message => message !== null)
return {
...header?.system === undefined ? {} : { system: header.system },
...header?.tools === undefined ? {} : { tools: header.tools },
messages: [...header?.messagePrefix ?? [], ...regionMessages],
}
}
/** Inspect the current turn boundary and latest compaction bracket once. */
function inspectTurnTail(
events: readonly SessionEvent[],

63
packages/compact/compact-basic/src/summarizer.ts
View File

@@ -6,17 +6,28 @@
import type { Context } from 'cordis'
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, FinishReason, GenerateOptions } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, FinishReason, GenerateOptions, Message, ToolSchema } from '@deepseek-ai/dsh-llm'
import type { Agent } from '@deepseek-ai/dsh-agent'
import type { ResolvedConfig } from './types.ts'
interface SummaryConfig {
readonly summarizationProvider: string
readonly summarizationModel: string
readonly maxTokens: number
}
/** Tags wrapping the structured summary inside the landed checkpoint node. */
const SUMMARY_OPEN_TAG = '<compacted-summary>'
const SUMMARY_CLOSE_TAG = '</compacted-summary>'
/** Fixed structure required from the auxiliary summarization call. */
const SUMMARIZE_SYSTEM_PROMPT = [
'You are a compaction engine for an AI coding assistant. Condense the conversation transcript into a structured checkpoint that lets another model resume the work with no loss of essential context.',
/**
* The summarization directive, delivered as the FINAL user message after the
* replayed conversation rather than as a distinct summarizer system prompt.
* Keeping the conversation's own system prompt, tools, and message prefix in
* front of it makes the auxiliary call a genuine prefix of the last routed
* request, so the provider's KV cache is reused instead of invalidated.
*/
const COMPACTION_INSTRUCTION = [
'You are now acting as a compaction engine for this AI coding assistant. Condense the conversation ABOVE into a structured checkpoint that lets another model resume the work with no loss of essential context.',
'',
'Output EXACTLY the Markdown structure below: keep every section, in order. Use terse bullets, not prose paragraphs. Write "(none)" for an empty section — never drop a section.',
'',
@@ -47,14 +58,30 @@ const SUMMARIZE_SYSTEM_PROMPT = [
'Rules:',
'- Preserve exact file paths, commands, error strings, identifiers, and function signatures.',
'- Capture user feedback and explicit instructions faithfully, especially corrections.',
'- Do NOT mention this summarization process or that the context was compacted.',
`- If the transcript already contains a ${SUMMARY_OPEN_TAG} block, it is a PRIOR checkpoint. Do not copy it forward verbatim: preserve still-true facts, drop stale ones, and merge newer information into a single consolidated summary under the same structure.`,
'- Do NOT mention this summarization request or that the context was compacted.',
'- Output only the checkpoint text: do not call any tool or take any other action.',
`- If the conversation already contains a ${SUMMARY_OPEN_TAG} block, it is a PRIOR checkpoint. Do not copy it forward verbatim: preserve still-true facts, drop stale ones, and merge newer information into a single consolidated summary under the same structure.`,
].join('\n')
/** Framing that makes the replacement user message established context. */
const CHECKPOINT_PREAMBLE =
'This is an automatically generated checkpoint condensing an earlier span of the conversation to free up context. Treat the captured context as established background and build on it without restating it. Continue the task directly from the messages that follow, without acknowledging this checkpoint.'
/**
* The replayed conversation surface the summarizer condenses. Reproducing the
* last routed request's system prompt, tools, and leading messages verbatim
* lets the auxiliary call reuse the provider's warm prefix cache; the trailing
* compaction instruction is then the only novel input.
*/
export interface SummarizationInput {
/** The conversation's own system prompt, reused for prefix-cache alignment; absent for a system-less request. */
readonly system?: string
/** The conversation's tool schemas, reused for prefix-cache alignment; absent when the request carried none. */
readonly tools?: readonly ToolSchema[]
/** The request prefix followed by the shadowed region, in surface order, that precedes the compaction instruction. */
readonly messages: readonly Message[]
}
/** Safe summary content plus the exact auxiliary call envelope recorded in provenance. */
export interface SummaryResult {
summary: ContentBlock[]
@@ -64,18 +91,20 @@ export interface SummaryResult {
}
/**
* Run the default direct `ctx.llm.stream()` summarization call.
* Run the default cache-reusing `ctx.llm.stream()` summarization call: replay
* the conversation prefix, then append the compaction instruction as the final
* user message so the provider's warm prefix cache is reused.
* @param ctx - context providing the LLM service.
* @param config - resolved backend configuration.
* @param text - rendered transcript region to summarize.
* @param input - replayed conversation prefix (system, tools, and leading messages) to condense.
* @param agent - supplies routed-model history, fallback model, and session id.
* @param signal - optional cancellation forwarded to the adapter.
* @returns safe text-only summary blocks and exact call provenance.
*/
export async function summarizeWithLlm(
ctx: Context,
config: ResolvedConfig,
text: string,
config: SummaryConfig,
input: SummarizationInput,
agent: Agent,
signal?: AbortSignal,
): Promise<SummaryResult> {
@@ -97,14 +126,16 @@ export async function summarizeWithLlm(
}
const assembler = new BlockAssembler()
const messages: Message[] = [
...input.messages,
{ role: 'user', content: [{ type: 'text', text: COMPACTION_INSTRUCTION }] },
]
const options: GenerateOptions = {
provider: target.provider,
model: target.model,
messages: [{
role: 'user',
content: [{ type: 'text', text: `Summarize this conversation history:\n\n${text}\n\nSummary:` }],
}],
system: SUMMARIZE_SYSTEM_PROMPT,
messages,
...input.system === undefined ? {} : { system: input.system },
...input.tools === undefined ? {} : { tools: [...input.tools] },
maxTokens: config.maxTokens,
sessionId: agent.session.id,
...signal === undefined ? {} : { signal },

57
packages/compact/compact-basic/src/types.ts
View File

@@ -4,15 +4,19 @@
* @module @deepseek-ai/dsh-compact-basic/types
*/
/** Basic compaction configuration; every common field has a deployment default. */
export interface BasicCompactConfig {
/** Compact at this fraction of the token meter's context window. Defaults to `0.8`. */
import type { LlmCallConfig } from '@deepseek-ai/dsh-llm'
/** Policy fields shared by the default policy and exact model overrides. */
export interface CompactPolicyConfig {
/** Compact at this fraction of the model's context window. Defaults to `0.8`. */
thresholdRatio?: number
/** Recent surface tokens retained verbatim. Defaults to `floor(contextWindow * 0.16)`. */
/** Recent context retained as a fraction of the model's window. Defaults to `0.16`. */
retainRatio?: number
/** Absolute recent-context budget; mutually exclusive with `retainRatio`. */
retainTokens?: number
/** Summary provider; `''` resolves the latest routed pair, then the agent pair. Defaults to `''`. */
/** Summary provider; set together with `summarizationModel`, or inherit the conversation target. */
summarizationProvider?: string
/** Summary model; `''` resolves the latest routed pair, then the agent pair. Defaults to `''`. */
/** Summary model; set together with `summarizationProvider`, or inherit the conversation target. */
summarizationModel?: string
/** Provider generation cap for summarization. Defaults to `8192`. */
maxTokens?: number
@@ -20,18 +24,53 @@ export interface BasicCompactConfig {
compactionRetries?: number
/** Maximum retries after canonical context overflow; `0` disables recovery. Defaults to `1`. */
maxOverflowRetries?: number
}
/** Exact provider/model override merged over the default compaction policy. */
export interface ModelCompactPolicyConfig extends CompactPolicyConfig {
/** Registered provider route to match. */
provider: string
/** Exact routed model id to match within `provider`. */
model: string
}
/** Basic compaction configuration with an optional exact-target policy table. */
export interface BasicCompactConfig extends CompactPolicyConfig {
/** Exact provider/model overrides; duplicate targets fail plugin load. */
modelPolicies?: ModelCompactPolicyConfig[]
/** Enable automatic post-step pressure and overflow-recovery listeners. Defaults to `true`. */
auto?: boolean
}
/** Validated and detached compaction configuration. */
export interface ResolvedConfig {
/** Exactly one validated retention form. */
export type ResolvedRetention =
| { readonly retainRatio: number; readonly retainTokens?: never }
| { readonly retainRatio?: never; readonly retainTokens: number }
/** Validated policy fields shared before and after exact-target matching. */
interface ResolvedPolicyFields {
readonly thresholdRatio: number
readonly retainTokens: number
readonly summarizationProvider: string
readonly summarizationModel: string
readonly maxTokens: number
readonly compactionRetries: number
readonly maxOverflowRetries: number
}
/** Validated immutable config whose target-specific defaults remain unresolved. */
export type ResolvedConfig = ResolvedPolicyFields & ResolvedRetention & {
readonly modelPolicies: readonly Readonly<ModelCompactPolicyConfig>[]
readonly auto: boolean
}
/** Fully merged policy for one routed conversation target, before capacity scaling. */
export type ResolvedTargetPolicy = ResolvedPolicyFields & ResolvedRetention & {
readonly target: Pick<LlmCallConfig, 'provider' | 'model'>
}
/** One routed model's concrete pressure and retention budget. */
export type ResolvedCompactSpec = Omit<ResolvedTargetPolicy, 'retainRatio' | 'retainTokens'> & {
readonly contextWindow: number
readonly thresholdTokens: number
readonly retainTokens: number
}

471
packages/compact/compact-basic/tests/compact-basic.spec.ts
View File

@@ -3,11 +3,23 @@ import { Context } from 'cordis'
import BasicCompactService from '@deepseek-ai/dsh-compact-basic'
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
import { selectCompactableRange } from '@deepseek-ai/dsh-compact-basic/src/region.ts'
import type { SummarizationInput } from '@deepseek-ai/dsh-compact-basic/src/summarizer.ts'
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
import { resolveConfig } from '@deepseek-ai/dsh-compact-basic/src/config.ts'
import {
resolveCompactSpec,
resolveConfig,
resolveTargetPolicy,
} from '@deepseek-ai/dsh-compact-basic/src/config.ts'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, LlmFailure, StreamChunk } from '@deepseek-ai/dsh-llm'
import type {
ContentBlock,
GenerateOptions,
LlmFailure,
LlmModelContext,
Message,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
import { agentEvents, type Agent } from '@deepseek-ai/dsh-agent'
@@ -16,9 +28,40 @@ import ToolResultPruneService from '@deepseek-ai/dsh-compact-tool-result-prune'
const SIGNAL = new AbortController().signal
const MODEL = 'test-model'
class ContextAdapter extends LlmAdapter {
constructor(private readonly contextWindow: number) {
super()
}
override resolveModelContext(): Promise<LlmModelContext> {
return Promise.resolve({ contextWindow: this.contextWindow })
}
override async * stream(): AsyncIterable<StreamChunk> {
yield { type: 'finish', reason: { kind: 'stop' } }
}
}
class RoutedContextAdapter extends LlmAdapter {
constructor(private readonly windows: Readonly<Record<string, number>>) {
super()
}
override resolveModelContext(provider: string): Promise<LlmModelContext | undefined> {
const contextWindow = this.windows[provider]
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
}
override async * stream(): AsyncIterable<StreamChunk> {
yield { type: 'finish', reason: { kind: 'stop' } }
}
}
function createContext(contextWindow = 1_000): Context {
const ctx = new Context()
void new TokenMeterService(ctx, { contextWindow })
void new LlmService(ctx)
void new TokenMeterService(ctx)
ctx.llm.registerAdapter([MODEL, 'actual', 'unlisted-provider'], new ContextAdapter(contextWindow))
return ctx
}
@@ -26,6 +69,21 @@ function agent(session: Session, model?: string): Agent {
return { session, options: model === undefined ? {} : { provider: model, model } } as Agent
}
/** Flatten every text fragment the summarizer received, recursing tool-result blocks. */
function summarizedText(input: SummarizationInput): string {
const collect = (blocks: readonly ContentBlock[]): string =>
blocks.map(block =>
block.type === 'text' ? block.text
: block.type === 'tool-result' ? collect(block.content)
: '').join('\n')
return input.messages.map(message => collect(message.content)).join('\n')
}
/** A minimal replayed prefix carrying one user message of the given text. */
function promptInput(text: string): SummarizationInput {
return { messages: [{ role: 'user', content: [{ type: 'text', text }] }] }
}
/** Closed two-message turns followed by one open turn for durable compaction events. */
function conversation(turns = 4, text = 'fixture '.repeat(40).trim()): Session {
const session = new Session(SessionId(`conversation-${turns}`))
@@ -141,14 +199,14 @@ class TestCompactService extends BasicCompactService {
summaryModel = 'summary-model'
error: unknown
mutateDuringSummary: (() => void) | undefined
calls: Array<{ text: string; signal: AbortSignal | undefined }> = []
calls: Array<{ input: SummarizationInput; signal: AbortSignal | undefined }> = []
override async summarize(
text: string,
input: SummarizationInput,
_agent: Agent,
signal?: AbortSignal,
): Promise<{ summary: ContentBlock[]; provider: string; model: string; maxTokens?: number }> {
this.calls.push({ text, signal })
this.calls.push({ input, signal })
this.mutateDuringSummary?.()
if (this.error !== undefined) throw this.error
return {
@@ -178,43 +236,131 @@ async function compactIfNeeded(
describe('compact configuration and defaults', () => {
it('uses low-friction service-wide defaults', () => {
const ctx = createContext()
const resolved = resolveConfig({}, ctx.tokenMeter)
const resolved = resolveConfig({})
expect(resolved).toEqual({
thresholdRatio: 0.8,
retainTokens: 160,
retainRatio: 0.16,
summarizationProvider: '',
summarizationModel: '',
maxTokens: 8192,
compactionRetries: 1,
maxOverflowRetries: 1,
modelPolicies: [],
auto: true,
})
expect(Object.isFrozen(resolved)).toBe(true)
})
it('resolves threshold and retention overrides independently', () => {
const ctx = createContext()
const thresholdOnly = resolveConfig({
thresholdRatio: 0.5,
}, ctx.tokenMeter)
})
expect(thresholdOnly).toMatchObject({
thresholdRatio: 0.5,
retainTokens: 160,
retainRatio: 0.16,
})
const retentionOnly = resolveConfig({
retainTokens: 70,
}, ctx.tokenMeter)
})
expect(retentionOnly).toMatchObject({
thresholdRatio: 0.8,
retainTokens: 70,
})
expect(retentionOnly).not.toHaveProperty('retainRatio')
})
it('merges exact provider/model policy overrides and scales ratios per model', () => {
const config = resolveConfig({
thresholdRatio: 0.8,
retainRatio: 0.1,
modelPolicies: [{
provider: 'small-provider',
model: 'shared-id',
thresholdRatio: 0.5,
retainTokens: 120,
}],
})
const small = resolveTargetPolicy(config, {
provider: 'small-provider',
model: 'shared-id',
})
const otherProvider = resolveTargetPolicy(config, {
provider: 'large-provider',
model: 'shared-id',
})
expect(resolveCompactSpec(small, 1_000)).toMatchObject({
thresholdTokens: 500,
retainTokens: 120,
})
expect(resolveCompactSpec(otherProvider, 2_000)).toMatchObject({
thresholdTokens: 1_600,
retainTokens: 200,
})
const ratioOverride = resolveTargetPolicy(resolveConfig({
retainTokens: 200,
modelPolicies: [{
provider: 'ratio-provider',
model: 'ratio-model',
thresholdRatio: 0.6,
retainRatio: 0.2,
summarizationProvider: 'summary-provider',
summarizationModel: 'summary-model',
maxTokens: 512,
compactionRetries: 2,
maxOverflowRetries: 3,
}],
}), { provider: 'ratio-provider', model: 'ratio-model' })
expect(resolveCompactSpec(ratioOverride, 2_000)).toMatchObject({
thresholdTokens: 1_200,
retainTokens: 400,
summarizationProvider: 'summary-provider',
summarizationModel: 'summary-model',
maxTokens: 512,
compactionRetries: 2,
maxOverflowRetries: 3,
})
})
it('inherits, clears, and replaces the summarization target as a pair', () => {
const config = resolveConfig({
summarizationProvider: 'default-provider',
summarizationModel: 'default-model',
modelPolicies: [
{ provider: 'inherit-provider', model: MODEL },
{
provider: 'clear-provider',
model: MODEL,
summarizationProvider: '',
summarizationModel: '',
},
{
provider: 'replace-provider',
model: MODEL,
summarizationProvider: 'replacement-provider',
summarizationModel: 'replacement-model',
},
],
})
expect(resolveTargetPolicy(config, { provider: 'inherit-provider', model: MODEL }))
.toMatchObject({
summarizationProvider: 'default-provider',
summarizationModel: 'default-model',
})
expect(resolveTargetPolicy(config, { provider: 'clear-provider', model: MODEL }))
.toMatchObject({ summarizationProvider: '', summarizationModel: '' })
expect(resolveTargetPolicy(config, { provider: 'replace-provider', model: MODEL }))
.toMatchObject({
summarizationProvider: 'replacement-provider',
summarizationModel: 'replacement-model',
})
})
it('validates common values and pressure-policy invariants', () => {
const ctx = createContext()
const bad = [
[{ maxTokens: 0 }, /maxTokens/],
[{ compactionRetries: -1 }, /compactionRetries/],
@@ -222,19 +368,60 @@ describe('compact configuration and defaults', () => {
[{ auto: 'yes' }, /auto must be a boolean/],
[{ summarizationProvider: 1 }, /summarizationProvider must be a string/],
[{ summarizationModel: 1 }, /summarizationModel must be a string/],
[{ summarizationProvider: MODEL }, /must both be set or both be empty/],
[{ summarizationModel: MODEL }, /must both be set or both be empty/],
[{ summarizationProvider: MODEL }, /must be set together/],
[{ summarizationModel: MODEL }, /must be set together/],
[{ summarizationProvider: '' }, /must be set together/],
[{ summarizationModel: '' }, /must be set together/],
[{ thresholdRatio: 0 }, /number in \(0, 1\]/],
[{ thresholdRatio: 1.1 }, /number in \(0, 1\]/],
[{ retainRatio: 0.9 }, /retainRatio \(0.9\) must be less than the resolved thresholdRatio \(0.8\)/],
[{ thresholdRatio: 0.1 }, /retainRatio \(0.16\) must be less than the resolved thresholdRatio \(0.1\)/],
[{ retainTokens: -1 }, /non-negative integer/],
[{ thresholdRatio: 0.5, retainTokens: 500 }, /less than threshold/],
[{ retainRatio: 0.2, retainTokens: 100 }, /mutually exclusive/],
[{ modelPolicies: {} }, /modelPolicies must be an array/],
[{ modelPolicies: [1] }, /modelPolicies\[0\] must be an object/],
[{ modelPolicies: [null] }, /modelPolicies\[0\] must be an object/],
[{ modelPolicies: [[]] }, /modelPolicies\[0\] must be an object/],
[{ modelPolicies: [{ provider: 1, model: MODEL }] }, /provider must be a non-empty string/],
[{ modelPolicies: [{ provider: '', model: MODEL }] }, /provider must be a non-empty string/],
[{ modelPolicies: [{ provider: MODEL, model: 1 }] }, /model must be a non-empty string/],
[{ modelPolicies: [{ provider: MODEL, model: '' }] }, /model must be a non-empty string/],
[{ modelPolicies: [{ provider: MODEL, model: MODEL, summarizationProvider: 1 }] }, /summarizationProvider must be a string/],
[{
summarizationProvider: 'default-provider',
summarizationModel: 'default-model',
modelPolicies: [{ provider: MODEL, model: MODEL, summarizationModel: '' }],
}, /modelPolicies\[0\].*must be set together/],
[{
summarizationProvider: 'default-provider',
summarizationModel: 'default-model',
modelPolicies: [{ provider: MODEL, model: MODEL, summarizationProvider: '' }],
}, /modelPolicies\[0\].*must be set together/],
[{ modelPolicies: [{ provider: MODEL, model: MODEL, retainRatio: 0.2, retainTokens: 100 }] }, /mutually exclusive/],
[
{ modelPolicies: [{ provider: MODEL, model: MODEL, thresholdRatio: 0.1 }] },
/modelPolicies\[0\]: retainRatio \(0.16\).*thresholdRatio \(0.1\)/,
],
[
{ modelPolicies: [{ provider: MODEL, model: MODEL, retainRatio: 0.9 }] },
/modelPolicies\[0\]: retainRatio \(0.9\).*thresholdRatio \(0.8\)/,
],
[{ modelPolicies: [{ provider: MODEL, model: MODEL }, { provider: MODEL, model: MODEL }] }, /duplicate model policy/],
[{ models: { [MODEL]: { retainTokens: 10 } } }, /BasicCompactConfig: unknown key "models"/],
[{ thresholdRato: 0.5 }, /BasicCompactConfig: unknown key "thresholdRato"/],
] as Array<[unknown, RegExp]>
for (const [config, pattern] of bad) {
expect(() => resolveConfig(config as BasicCompactConfig, ctx.tokenMeter)).toThrow(pattern)
expect(() => resolveConfig(config as BasicCompactConfig)).toThrow(pattern)
}
const invalidPressure = resolveTargetPolicy(resolveConfig({
thresholdRatio: 0.5,
retainTokens: 500,
}), { provider: MODEL, model: MODEL })
expect(() => resolveCompactSpec(invalidPressure, 1_000)).toThrow(/less than threshold/)
expect(() => resolveCompactSpec(invalidPressure, 1.5)).toThrow(/positive integer/)
expect(() => resolveCompactSpec(invalidPressure, 0)).toThrow(/positive integer/)
})
})
@@ -255,7 +442,7 @@ describe('pressure measurement and retention', () => {
expect(compact.calls).toHaveLength(0)
})
it('meters any routed model without profile resolution', async () => {
it('meters an unlisted model when its provider adapter supplies context metadata', async () => {
const compact = service(compactConfig)
const session = conversation()
session.append('request/header', {
@@ -266,6 +453,52 @@ describe('pressure measurement and retention', () => {
.resolves.not.toBeNull()
})
it('re-resolves capacity after a same-model-id provider switch in one session', async () => {
const ctx = new Context()
void new LlmService(ctx)
void new TokenMeterService(ctx)
ctx.llm.registerAdapter(['large', 'small'], new RoutedContextAdapter({
large: 10_000,
small: 1_000,
}))
const compact = service({
auto: false,
thresholdRatio: 0.5,
retainRatio: 0.1,
}, ctx)
const session = conversation(4)
session.append('request/header', {
header: { config: { provider: 'large', model: 'shared-id' } },
reason: 'resume',
})
await expect(compactIfNeeded(compact, session)).resolves.toBeNull()
session.append('request/header', {
header: { config: { provider: 'small', model: 'shared-id' } },
reason: 'change',
})
await expect(compactIfNeeded(compact, session)).resolves.not.toBeNull()
})
it('requires capacity only for proactive pressure, not provider-confirmed overflow', async () => {
const ctx = new Context()
void new LlmService(ctx)
void new TokenMeterService(ctx)
ctx.llm.registerAdapter(['unknown-context'], new ContextAdapter(1_000))
vi.spyOn(ctx.llm, 'resolveModelContext').mockResolvedValue(undefined)
const compact = service(compactConfig, ctx)
const session = conversation(4)
session.append('request/header', {
header: { config: { provider: 'unknown-context', model: 'model' } },
reason: 'resume',
})
await expect(compactIfNeeded(compact, session, 'pressure'))
.rejects.toThrow(/no context capacity for unknown-context\/model/)
await expect(compactIfNeeded(compact, session, 'context-overflow'))
.resolves.not.toBeNull()
})
it('declines forced overflow when the whole surface is one indivisible tool pair', async () => {
const compact = service(compactConfig)
const session = new Session(SessionId('single-tool-pair'))
@@ -504,8 +737,8 @@ describe('optional model-free tool-result pruning', () => {
expect(await compactIfNeeded(compact, session)).not.toBeNull()
expect(compact.calls).toHaveLength(1)
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
expect(compact.calls[0]!.text).not.toContain('result 1 '.repeat(300))
expect(summarizedText(compact.calls[0]!.input)).toContain('tool result middle pruned')
expect(summarizedText(compact.calls[0]!.input)).not.toContain('result 1 '.repeat(300))
})
it('retains the original compact-basic behavior without the optional plugin', async () => {
@@ -542,7 +775,7 @@ describe('compaction region transaction', () => {
expect(result.shadowedSeqs).toEqual(before.slice(0, 4))
expect(result.shadowedTokenCount).toBeGreaterThan(0)
expect(compact.calls[0]).toMatchObject({ signal: SIGNAL })
expect(compact.calls[0]?.text).toContain('fixture user 1')
expect(summarizedText(compact.calls[0]!.input)).toContain('fixture user 1')
const summary = session.events.findLast(event => event.type === 'compact/summary')
expect(summary?.data).toMatchObject({
shadowedSeqs: result.shadowedSeqs,
@@ -560,6 +793,25 @@ describe('compaction region transaction', () => {
expect(replay.deriveMessages()).toEqual(session.deriveMessages())
})
it('replays the latest routed header prefix so the summarizer reuses the cache', async () => {
const compact = service()
const session = conversation(3)
const tools = [{ name: 'do_thing', description: 'd', parameters: { type: 'object' } }]
const messagePrefix: Message[] = [{ role: 'user', content: [{ type: 'text', text: 'SESSION PREFIX' }] }]
session.append('request/header', {
header: { config: { provider: MODEL, model: MODEL }, system: 'CONVERSATION SYSTEM', tools, messagePrefix },
reason: 'resume',
})
const nodes = session.surface.nodes
await compact.compactRegion(nodes[0]!, nodes[1]!, agent(session, MODEL), SIGNAL)
const { input } = compact.calls[0]!
expect(input.system).toBe('CONVERSATION SYSTEM')
expect(input.tools).toEqual(tools)
expect(input.messages[0]).toEqual(messagePrefix[0])
expect(summarizedText(input)).toContain('fixture user 1')
})
it.each([
['start missing', 9_001, undefined, /start seq 9001 not found/],
['end missing', undefined, 9_002, /end seq 9002 not found/],
@@ -773,11 +1025,11 @@ class ScriptedAdapter extends LlmAdapter {
class ExposedCompactService extends BasicCompactService {
runSummarize(
text: string,
input: SummarizationInput,
owner: Agent,
signal?: AbortSignal,
): Promise<{ summary: ContentBlock[]; provider: string; model: string; maxTokens?: number }> {
return this.summarize(text, owner, signal)
return this.summarize(input, owner, signal)
}
}
@@ -789,7 +1041,7 @@ async function summarizerHarness(
): Promise<{ ctx: Context; adapter: ScriptedAdapter; compact: ExposedCompactService }> {
const ctx = new Context()
await ctx.plugin(LlmService)
void new TokenMeterService(ctx, { contextWindow: 1_000 })
void new TokenMeterService(ctx)
const adapter = new ScriptedAdapter(blocks, finish)
ctx.llm.registerAdapter([model], adapter)
const compact = new ExposedCompactService(ctx, config)
@@ -809,7 +1061,7 @@ describe('default one-shot summarizer', () => {
maxTokens: 321,
})
const session = conversation(1)
const output = await compact.runSummarize('transcript', agent(session, 'fallback'), SIGNAL)
const output = await compact.runSummarize(promptInput('transcript'), agent(session, 'fallback'), SIGNAL)
expect(output).toEqual({
summary: [{ type: 'text', text: 'public summary' }],
@@ -824,7 +1076,68 @@ describe('default one-shot summarizer', () => {
signal: SIGNAL,
sessionId: session.id,
})
expect(adapter.lastOptions?.system).toContain('## Primary Request and Intent')
const instruction = adapter.lastOptions?.messages.at(-1)?.content[0]
expect(instruction?.type === 'text' ? instruction.text : '').toContain('## Primary Request and Intent')
})
it('replays the conversation prefix and appends the instruction as the final message', async () => {
const { adapter, compact } = await summarizerHarness([{ type: 'text', text: 'summary' }])
const tools = [{ name: 'do_thing', description: 'd', parameters: { type: 'object' } }]
const prefix: Message = { role: 'user', content: [{ type: 'text', text: 'earlier turn' }] }
await compact.runSummarize({
system: 'REPLAYED SYSTEM',
tools,
messages: [prefix],
}, agent(conversation(1), MODEL))
expect(adapter.lastOptions?.system).toBe('REPLAYED SYSTEM')
expect(adapter.lastOptions?.tools).toEqual(tools)
const messages = adapter.lastOptions?.messages ?? []
expect(messages[0]).toEqual(prefix)
const last = messages.at(-1)?.content[0]
const lastText = last?.type === 'text' ? last.text : ''
expect(lastText).toContain('Condense the conversation ABOVE')
expect(lastText).toContain('## Primary Request and Intent')
})
it('applies the routed model policy without changing the replayed prefix', async () => {
const { ctx, compact } = await summarizerHarness(
[{ type: 'text', text: 'unused default summary' }],
undefined,
MODEL,
{
auto: false,
maxTokens: 111,
modelPolicies: [{
provider: MODEL,
model: MODEL,
summarizationProvider: 'policy-summary',
summarizationModel: 'policy-summary',
maxTokens: 222,
}],
},
)
const policyAdapter = new ScriptedAdapter([{ type: 'text', text: 'policy summary' }])
ctx.llm.registerAdapter(['policy-summary'], policyAdapter)
const prefix: Message = { role: 'user', content: [{ type: 'text', text: 'warm prefix' }] }
const output = await compact.runSummarize({
system: 'WARM SYSTEM',
messages: [prefix],
}, agent(conversation(1), 'fallback'))
expect(output).toMatchObject({
provider: 'policy-summary',
model: 'policy-summary',
maxTokens: 222,
})
expect(policyAdapter.lastOptions).toMatchObject({
provider: 'policy-summary',
model: 'policy-summary',
maxTokens: 222,
system: 'WARM SYSTEM',
})
expect(policyAdapter.lastOptions?.messages[0]).toEqual(prefix)
})
it('resolves the latest routed provider/model before the AgentOptions pair', async () => {
@@ -834,7 +1147,7 @@ describe('default one-shot summarizer', () => {
header: { config: { provider: 'routed', model: 'routed' } },
reason: 'initial',
})
const output = await compact.runSummarize('history', agent(session, 'fallback'))
const output = await compact.runSummarize(promptInput('history'), agent(session, 'fallback'))
expect(output.provider).toBe('routed')
expect(output.model).toBe('routed')
expect(adapter.lastOptions?.provider).toBe('routed')
@@ -868,7 +1181,32 @@ describe('default one-shot summarizer', () => {
await ctx.plugin(LlmService)
void new TokenMeterService(ctx)
const compact = new ExposedCompactService(ctx, { auto: false })
await expect(compact.runSummarize('history', agent(new Session(SessionId('model-less')))))
await expect(compact.runSummarize(promptInput('history'), agent(new Session(SessionId('model-less')))))
.rejects.toThrow(/no provider\/model available for summarization/)
})
it('uses a complete AgentOptions target when no durable route exists', async () => {
const { adapter, compact } = await summarizerHarness([{ type: 'text', text: 'summary' }])
const session = new Session(SessionId('headerless-summary'))
await expect(compact.runSummarize(promptInput('history'), agent(session, MODEL))).resolves.toMatchObject({
provider: MODEL,
model: MODEL,
})
expect(adapter.lastOptions).toMatchObject({ provider: MODEL, model: MODEL })
})
it.each([
{ provider: '', model: MODEL },
{ provider: MODEL },
{ provider: MODEL, model: '' },
])('rejects incomplete AgentOptions target %#', async (options) => {
const { compact } = await summarizerHarness([{ type: 'text', text: 'unused' }])
const owner = {
session: new Session(SessionId(`incomplete-${String(options.model)}`)),
options,
} as Agent
await expect(compact.runSummarize(promptInput('history'), owner))
.rejects.toThrow(/no provider\/model available for summarization/)
})
@@ -883,7 +1221,7 @@ describe('default one-shot summarizer', () => {
const { compact } = await summarizerHarness([], finish)
let thrown: unknown
try {
await compact.runSummarize('history', agent(conversation(1), MODEL))
await compact.runSummarize(promptInput('history'), agent(conversation(1), MODEL))
} catch (error: unknown) {
thrown = error
}
@@ -895,7 +1233,7 @@ describe('default one-shot summarizer', () => {
it('rejects empty or reasoning-only successful output', async () => {
const { compact } = await summarizerHarness([{ type: 'reasoning', text: 'private' }])
await expect(compact.runSummarize('history', agent(conversation(1), MODEL)))
await expect(compact.runSummarize(promptInput('history'), agent(conversation(1), MODEL)))
.rejects.toThrow(/no text summary content/)
})
})
@@ -972,6 +1310,43 @@ describe('automatic listener and loader composition', () => {
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
})
it('warns once per routed target when proactive pressure has no context metadata', async () => {
const ctx = createContext()
const warnings: string[] = []
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
vi.spyOn(ctx.llm, 'resolveModelContext').mockResolvedValue(undefined)
void new TestCompactService(ctx, {
thresholdRatio: 0.5,
retainTokens: 180,
})
const session = conversation(4)
await postStep(ctx, agent(session, MODEL))
await postStep(ctx, agent(session, MODEL))
expect(warnings).toEqual([
expect.stringContaining(`no context capacity for ${MODEL}/${MODEL}`),
])
})
it('warns once per routed target when absolute retention exceeds its resolved threshold', async () => {
const ctx = createContext()
const warnings: string[] = []
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
void new TestCompactService(ctx, {
thresholdRatio: 0.5,
retainTokens: 500,
})
const session = conversation(4)
await postStep(ctx, agent(session, MODEL))
await postStep(ctx, agent(session, MODEL))
expect(warnings).toEqual([
expect.stringContaining('retainTokens (500) must be less than threshold tokens 500'),
])
})
it('force-compacts below normal pressure for canonical overflow and retries only after replacement', async () => {
const ctx = createContext(10_000)
void new TestCompactService(ctx, {
@@ -1026,7 +1401,7 @@ describe('automatic listener and loader composition', () => {
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
expect(session.events.some(event => event.type === 'compact/summary')).toBe(true)
expect(compact.calls).toHaveLength(1)
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
expect(summarizedText(compact.calls[0]!.input)).toContain('tool result middle pruned')
})
it('retries from a durable prune when later overflow summarization throws', async () => {
@@ -1187,6 +1562,18 @@ describe('automatic listener and loader composition', () => {
.toEqual({ action: 'retry' })
})
it('delegates canonical overflow when no durable routed target exists', async () => {
const ctx = createContext()
void new TestCompactService(ctx)
const session = new Session(SessionId('headerless-overflow'))
session.append('turn/start', {
turn: 1,
trigger: { kind: 'message', source: { kind: 'user' } },
})
await expect(recover(ctx, agent(session, MODEL), overflow())).resolves.toEqual({ action: 'fail' })
})
it('honors retry caps, non-context failures, and cancellation', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx, { maxOverflowRetries: 1 })
@@ -1202,6 +1589,23 @@ describe('automatic listener and loader composition', () => {
expect(compactSpy).not.toHaveBeenCalled()
})
it('applies the routed model override to the overflow retry cap', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx, {
maxOverflowRetries: 2,
modelPolicies: [{
provider: MODEL,
model: MODEL,
maxOverflowRetries: 1,
}],
})
const compactSpy = vi.spyOn(compact, 'compactIfNeeded')
expect(await recover(ctx, agent(conversation(3), MODEL), overflow(), 1))
.toEqual({ action: 'fail' })
expect(compactSpy).not.toHaveBeenCalled()
})
it('does not retry when cancellation lands during an awaited compaction', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx)
@@ -1249,7 +1653,6 @@ describe('automatic listener and loader composition', () => {
const meterFiber = await ctx.plugin(TokenMeterService)
const compactFiber = await ctx.plugin(BasicCompactService, { auto: false })
expect(ctx.tokenMeter.contextWindow).toBe(128_000)
expect(ctx.get('compact')).toBeInstanceOf(BasicCompactService)
await compactFiber.dispose()
expect(ctx.get('compact')).toBeUndefined()
@@ -1260,7 +1663,7 @@ describe('automatic listener and loader composition', () => {
it('removes its automatic listener with the plugin fiber', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(TokenMeterService, { contextWindow: 1_000 })
await ctx.plugin(TokenMeterService)
const fiber = await ctx.plugin(TestCompactService, {
thresholdRatio: 0.5,
retainTokens: 180,

22
packages/compact/compact-basic/tests/compact-loop-repro.spec.ts
View File

@@ -41,6 +41,10 @@ class StepwiseToolAdapter extends LlmAdapter {
super()
}
override resolveModelContext(): Promise<{ contextWindow: number }> {
return Promise.resolve({ contextWindow: 400 })
}
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
const n = this.calls
this.calls += 1
@@ -72,8 +76,17 @@ class OverflowRecoveryAdapter extends LlmAdapter {
super()
}
override resolveModelContext(): Promise<{ contextWindow: number }> {
return Promise.resolve({ contextWindow: 128 })
}
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
if (options.system?.includes('You are a compaction engine')) {
// The cache-reusing summarizer replays the conversation prefix and marks
// its call only by the compaction instruction in the trailing user message.
const trailing = options.messages.at(-1)?.content
.map(block => (block.type === 'text' ? block.text : ''))
.join('') ?? ''
if (trailing.includes('acting as a compaction engine')) {
this.summaryRequests.push(options)
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'RECOVERY CHECKPOINT' } }
@@ -119,7 +132,7 @@ async function harness(toolSteps: number): Promise<{ ctx: Context; compact: Repr
await mountAgentLoopTestDependencies(ctx)
await mountInvariants(ctx)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(TokenMeterService, { contextWindow: 400 })
await ctx.plugin(TokenMeterService)
ctx.llm.registerAdapter(['mock'], new StepwiseToolAdapter(toolSteps))
ctx.tools.register(defineTool({
name: 'work',
@@ -135,7 +148,6 @@ async function harness(toolSteps: number): Promise<{ ctx: Context; compact: Repr
auto: true,
thresholdRatio: 0.5,
retainTokens: 50,
summarizationModel: '',
maxTokens: 8192,
compactionRetries: 1,
})
@@ -267,7 +279,7 @@ describe('context-overflow recovery across the real loop and compact-basic', ()
await mountAgentLoopTestDependencies(ctx)
await mountInvariants(ctx)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(TokenMeterService, { contextWindow: 128 })
await ctx.plugin(TokenMeterService)
ctx.llm.registerAdapter(['mock'], adapter)
ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, provider: 'mock', model: 'mock' }))
await ctx.plugin(BasicCompactService, {
@@ -335,7 +347,7 @@ describe('context-overflow recovery across the real loop and compact-basic', ()
jitterRatio: 0,
})
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(TokenMeterService, { contextWindow: 128 })
await ctx.plugin(TokenMeterService)
ctx.llm.registerAdapter(['mock'], adapter)
await ctx.plugin(BasicCompactService, {
thresholdRatio: 1,

40
packages/compact/compact-basic/tests/loader-composition.spec.ts
View File

@@ -56,8 +56,6 @@ describe('real Loader composition', () => {
const loaded = await loadYaml([
"- name: '@deepseek-ai/dsh-llm'",
"- name: '@deepseek-ai/dsh-token-meter'",
' config:',
' contextWindow: 4096',
"- name: '@deepseek-ai/dsh-compact-tool-result-prune'",
' config:',
' thresholdChars: 100',
@@ -66,7 +64,7 @@ describe('real Loader composition', () => {
"- name: '@deepseek-ai/dsh-compact-basic'",
' config:',
' thresholdRatio: 0.5',
' retainTokens: 512',
' retainRatio: 0.125',
' auto: false',
])
@@ -74,12 +72,11 @@ describe('real Loader composition', () => {
.filter(entry => entry.fiber === undefined && !entry.disabled)
.map(entry => entry.options.name)
expect(unloaded).toEqual([])
expect(loaded.tokenMeter.contextWindow).toBe(4096)
expect(loaded.get('toolResultPrune')).toBeInstanceOf(ToolResultPruneService)
expect(loaded.get('compact')).toBeInstanceOf(BasicCompactService)
expect((loaded.compact as BasicCompactService).config).toMatchObject({
thresholdRatio: 0.5,
retainTokens: 512,
retainRatio: 0.125,
auto: false,
})
})
@@ -87,8 +84,8 @@ describe('real Loader composition', () => {
it('rejects stale token-meter config after Schemastery normalization', async () => {
context = new Context()
await expect(context.plugin(TokenMeterService, {
models: { legacy: { contextWindow: 4096 } },
} as never)).rejects.toThrow(/TokenMeterConfig: unknown key "models"/)
contextWindow: 4096,
} as never)).rejects.toThrow(/TokenMeterConfig: unknown key "contextWindow"/)
})
it('rejects stale compact-basic config after Schemastery normalization', async () => {
@@ -99,4 +96,33 @@ describe('real Loader composition', () => {
models: { legacy: { thresholdRatio: 0.5 } },
} as never)).rejects.toThrow(/BasicCompactConfig: unknown key "models"/)
})
it('rejects a capacity-independent merged ratio conflict during plugin load', async () => {
context = new Context()
await context.plugin(LlmService)
await context.plugin(TokenMeterService)
await expect(context.plugin(BasicCompactService, {
retainRatio: 0.2,
modelPolicies: [{
provider: 'test-provider',
model: 'test-model',
thresholdRatio: 0.1,
}],
})).rejects.toThrow(/modelPolicies\[0\]: retainRatio \(0.2\).*thresholdRatio \(0.1\)/)
})
it('rejects an incomplete model-policy summarization pair during plugin load', async () => {
context = new Context()
await context.plugin(LlmService)
await context.plugin(TokenMeterService)
await expect(context.plugin(BasicCompactService, {
summarizationProvider: 'default-provider',
summarizationModel: 'default-model',
modelPolicies: [{
provider: 'test-provider',
model: 'test-model',
summarizationModel: '',
}],
})).rejects.toThrow(/modelPolicies\[0\].*must be set together/)
})
})

18
packages/compact/compact/README.md
View File

@@ -6,7 +6,7 @@ This package is the interface tier of the compaction capability, split so each c
| Package | Role |
|---|---|
| `@deepseek-ai/dsh-compact` (this) | the interface: abstract service + `compact/*` events + `CompactionResult` + tool-pairing boundary helpers + the shared transcript renderer (`renderTranscript`/`renderContentBlocks`) |
| `@deepseek-ai/dsh-compact` (this) | the interface: abstract service + `compact/*` events + `CompactionResult` + tool-pairing boundary helpers |
| `@deepseek-ai/dsh-compact-basic` | a backend: `ctx.tokenMeter` pressure + token-budget retention + `llm.stream()` summarization |
| `@deepseek-ai/dsh-tool-compact` (deferred) | the model-facing `/compact` tool over `ctx.compact` |
@@ -63,7 +63,7 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and
#### What the model sees
A successful implementation replaces an older surface range with one user-role summary checkpoint; the raw events stay logged but stop appearing in derived model messages. The seam itself performs no rewrite.
A successful implementation replaces an older surface range with one user-role summary checkpoint — a `user/message` carrying `surfaceOp: { op: 'replace', start, end }`; the raw events stay logged but stop appearing in derived model messages. The seam itself performs no rewrite.
#### Token effect
@@ -73,20 +73,6 @@ Zero direct tokens from this interface. A backend trades many retained history t
A successful backend replacement invalidates reuse from the first shadowed history token; the seam itself does not alter a request.
### Transcript supplied to a compaction consumer
#### What the model sees
`renderTranscript()` joins entries with one blank line and renders them exactly as `User: <content>`, `Assistant: <content>`, `Tool result (call <callId>): <content>`, `Tool error (call <callId>): <content>`, `[Context: <content>]`, or `[Steering: <content>]`. Non-text blocks render exactly as `[reasoning: <text>]`, `[tool-call: <name>(<arguments>)]`, `[tool-result: <content>]`, `[tool-result]`, or `[<block-type>]`.
#### Token effect
Data-dependent input tokens are paid only by the auxiliary model or consumer that requests this transcript; the conversation model does not receive a duplicate transcript.
#### KV Cache effect
No conversation-cache invalidation. A consumer's auxiliary request can reuse only the exact prefix produced by this rendering; changed or compacted entries invalidate reuse from their first difference.
## Known Limitations and Deferred Work
- **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener.

1
packages/compact/compact/src/index.ts
View File

@@ -12,7 +12,6 @@ import type { Session } from '@deepseek-ai/dsh-session'
import type { CompactionResult } from './types.ts'
export type { CompactionResult } from './types.ts'
export { renderContentBlocks, renderTranscript } from './render.ts'
export { toolPairingBalancedAfter, toolPairingBalancedBefore } from './tool-pairing.ts'
/** Why automatic policy is asking a backend to consider compaction. */

95
packages/compact/compact/src/render.ts
View File

@@ -1,95 +0,0 @@
/**
* Pure shared transcript projection for summarization and recall, so both
* render the same log span byte-for-byte under replay.
* @module @deepseek-ai/dsh-compact/render
*/
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
/**
* Render text directly, reasoning as a tagged span, and every other block as a
* type-tagged placeholder. Tool results recurse into nested content; empty
* blocks contribute nothing and rendered blocks join with newlines.
*
* @param blocks - the content blocks to render.
* @returns the newline-joined plain-text rendering; empty string when nothing renders.
*/
export function renderContentBlocks(blocks: readonly ContentBlock[]): string {
const parts: string[] = []
for (const block of blocks) {
switch (block.type) {
case 'text':
if (block.text) parts.push(block.text)
break
case 'reasoning':
if (block.text) parts.push(`[reasoning: ${block.text}]`)
break
case 'tool-call':
parts.push(`[tool-call: ${block.name}(${block.arguments})]`)
break
case 'tool-result': {
const inner = renderContentBlocks(block.content)
parts.push(inner ? `[tool-result: ${inner}]` : '[tool-result]')
break
}
// ContentBlockMap is merge-extensible — render an unknown block as a
// bare type-tagged placeholder so a plugin-added block type is still
// signalled to the reader rather than dropped.
default:
parts.push(`[${(block as ContentBlock).type}]`)
}
}
return parts.join('\n')
}
/**
* Render message-producing events as a role-labeled transcript. `seqs` are
* walked in caller-supplied surface order, which may differ from numeric log
* order after replacement; non-surface and unknown merged events are skipped.
*
* @param events - the session log the seqs index into (`session.events`).
* @param seqs - the surface-node seqs to render, in surface order.
* @returns the transcript, entries joined by blank lines; empty string when nothing renders.
*/
export function renderTranscript(events: readonly SessionEvent[], seqs: readonly number[]): string {
const lines: string[] = []
for (const seq of seqs) {
const event = events[seq]
if (!event) continue
switch (event.type) {
case 'user/message': {
const text = renderContentBlocks(event.data.content)
if (text) lines.push(`User: ${text}`)
break
}
case 'assistant/message': {
const text = renderContentBlocks(event.data.content)
if (text) lines.push(`Assistant: ${text}`)
break
}
case 'tool/result': {
const text = renderContentBlocks(event.data.content)
const label = event.data.isError ? 'Tool error' : 'Tool result'
if (text) lines.push(`${label} (call ${event.data.callId}): ${text}`)
break
}
case 'context/message': {
const text = renderContentBlocks(event.data.content)
if (text) lines.push(`[Context: ${text}]`)
break
}
case 'steering/message': {
const text = renderContentBlocks(event.data.content)
if (text) lines.push(`[Steering: ${text}]`)
break
}
default:
break
}
}
return lines.join('\n\n')
}

138
packages/compact/compact/tests/render.spec.ts
View File

@@ -1,138 +0,0 @@
import { describe, expect, it } from 'vitest'
import { renderContentBlocks, renderTranscript } from '@deepseek-ai/dsh-compact'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { CallId } from '@deepseek-ai/dsh-llm'
function session(): Session {
return new Session(SessionId('render-spec'))
}
describe('renderContentBlocks', () => {
it('renders text blocks verbatim and skips empty ones', () => {
expect(renderContentBlocks([
{ type: 'text', text: 'hello' },
{ type: 'text', text: '' },
{ type: 'text', text: 'world' },
])).toBe('hello\nworld')
})
it('wraps reasoning, skipping empty reasoning', () => {
expect(renderContentBlocks([
{ type: 'reasoning', text: 'think' },
{ type: 'reasoning', text: '' },
])).toBe('[reasoning: think]')
})
it('renders tool-call as a name(args) placeholder', () => {
expect(renderContentBlocks([
{ type: 'tool-call', id: CallId('c1'), name: 'read', arguments: '{"filePath":"a"}' },
])).toBe('[tool-call: read({"filePath":"a"})]')
})
it('renders tool-result with nested content, and bare when empty', () => {
expect(renderContentBlocks([
{ type: 'tool-result', toolCallId: CallId('c1'), content: [{ type: 'text', text: 'ok' }] },
{ type: 'tool-result', toolCallId: CallId('c2'), content: [] },
])).toBe('[tool-result: ok]\n[tool-result]')
})
it('renders an unknown (merge-extended) block type as a bare type tag', () => {
const unknown = { type: 'image', data: 'zzz' } as unknown as ContentBlock
expect(renderContentBlocks([unknown])).toBe('[image]')
})
it('returns the empty string for no blocks', () => {
expect(renderContentBlocks([])).toBe('')
})
})
describe('renderTranscript', () => {
it('renders each surface event type with its label, in the seq order given', () => {
const s = session()
const user = s.append('user/message', {
content: [{ type: 'text', text: 'fix the bug' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
const assistant = s.append('assistant/message', { provenance: { provider: 'mock', model: 'mock' },
turn: 0, step: 0,
content: [{ type: 'text', text: 'looking' }],
}, { surfaceOp: 'append' })
const result = s.append('tool/result', {
turn: 0, step: 0, callId: CallId('c1'),
content: [{ type: 'text', text: 'exit 0' }],
isError: false,
}, { surfaceOp: 'append' })
const context = s.append('context/message', {
content: [{ type: 'text', text: 'file changed' }],
source: { kind: 'plugin', plugin: 'fs' },
}, { surfaceOp: 'append' })
const steering = s.append('steering/message', {
turn: 0,
content: [{ type: 'text', text: 'stop that' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
expect(renderTranscript(s.events, [user.seq, assistant.seq, result.seq, context.seq, steering.seq])).toBe([
'User: fix the bug',
'Assistant: looking',
'Tool result (call c1): exit 0',
'[Context: file changed]',
'[Steering: stop that]',
].join('\n\n'))
})
it('labels an error tool result "Tool error"', () => {
const s = session()
const result = s.append('tool/result', {
turn: 0, step: 0, callId: CallId('c9'),
content: [{ type: 'text', text: 'boom' }],
isError: true,
}, { surfaceOp: 'append' })
expect(renderTranscript(s.events, [result.seq])).toBe('Tool error (call c9): boom')
})
it('renders NON-log-order seqs in the order given (surface order after a replace)', () => {
const s = session()
const first = s.append('user/message', {
content: [{ type: 'text', text: 'first' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
const second = s.append('user/message', {
content: [{ type: 'text', text: 'second' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
expect(renderTranscript(s.events, [second.seq, first.seq])).toBe('User: second\n\nUser: first')
})
it('skips events that render to nothing, non-message events, and seqs with no event', () => {
const s = session()
const empty = s.append('user/message', {
content: [{ type: 'text', text: '' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
const emptyAssistant = s.append('assistant/message', { provenance: { provider: 'mock', model: 'mock' },
turn: 0, step: 0,
content: [{ type: 'text', text: '' }],
}, { surfaceOp: 'append' })
const emptyResult = s.append('tool/result', {
turn: 0, step: 0, callId: CallId('c3'),
content: [{ type: 'text', text: '' }],
isError: false,
}, { surfaceOp: 'append' })
const emptyContext = s.append('context/message', {
content: [{ type: 'text', text: '' }],
source: { kind: 'plugin', plugin: 'fs' },
}, { surfaceOp: 'append' })
const emptySteering = s.append('steering/message', {
turn: 0,
content: [{ type: 'text', text: '' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
// A log-only (non-surface) event type: contributes nothing to a transcript.
const lock = s.append('compact/start', { turn: 0 })
expect(renderTranscript(s.events, [
empty.seq, emptyAssistant.seq, emptyResult.seq, emptyContext.seq, emptySteering.seq, lock.seq, 9999,
])).toBe('')
})
})

8
packages/cordis/tool-cordis/src/api-catalog.ts
View File

@@ -340,6 +340,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
signature: 'async listModels(provider: string): Promise<LlmModelInfo[]>',
jsDoc: '/**\n * Discover models advertised by one registered provider. Catalog membership\n * is advisory and never changes routing or request validation.\n * @param provider - registered provider route to inspect.\n * @returns detached model metadata in adapter-preferred order.\n */',
},
{
signature: 'async resolveModelContext( provider: string, model: string, ): Promise<LlmModelContext | undefined>',
jsDoc: '/**\n * Resolve context capacity from the adapter that owns one exact route.\n * This query is independent of the advisory model catalog: an unlisted model\n * may return metadata, while `undefined` never rejects later routing.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @returns detached context metadata, or `undefined` when the adapter has none.\n */',
},
{
signature: 'stream(options: GenerateOptions): AsyncIterable<StreamChunk>',
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection, dispatch, and iteration failures retain their original\n * Error identity and are tagged in a call-local scope for narrow agent-loop\n * request recovery; middleware and nested-call failures remain untagged for\n * the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
@@ -1366,6 +1370,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'LlmFailure',
declaration: 'export interface LlmFailure {\n readonly message: string;\n readonly code: string;\n readonly status?: number;\n readonly providerRetryAfterMs?: number;\n readonly requestId?: ProviderRequestId;\n}',
},
{
name: 'LlmModelContext',
declaration: 'export interface LlmModelContext {\n contextWindow: number;\n}',
},
{
name: 'LlmModelInfo',
declaration: 'export interface LlmModelInfo {\n provider: string;\n id: string;\n name: string;\n description?: string;\n}',

2
packages/llm/README.md
View File

@@ -10,4 +10,4 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) |
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
The interface lives at `llm/llm/`; adapters, retry policy, and reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership.
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the interface or consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.

6
packages/llm/llm-deepseek/README.md
View File

@@ -20,11 +20,15 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
models: # optional; defaults to V4 Flash and V4 Pro
- id: deepseek-v4-flash
name: DeepSeek V4 Flash
contextWindow: 128000
- id: private-reasoner
description: Company-hosted reasoning model
contextWindow: 64000
```
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns it only for an exact configured id; omission or an unlisted pass-through model returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` wire field is not sent and the server applies its own default for the model. The only accepted values are `high` and `max` (DeepSeek's official effort levels). It is meaningful only with thinking enabled (the provider default).

18
packages/llm/llm-deepseek/src/adapter.ts
View File

@@ -6,7 +6,13 @@
*/
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type {
GenerateOptions,
LlmModelContext,
LlmModelInfo,
LlmProviderInfo,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
import { serializeRequest } from './serialize.ts'
import type { RequestDefaults } from './serialize.ts'
@@ -22,6 +28,8 @@ export interface DeepSeekCatalogModel {
name?: string
/** Optional selector detail for deployments with similar model variants. */
description?: string
/** Known combined request/response context capacity; omitted when deployment metadata is unavailable. */
contextWindow?: number
}
/** Constructor options for {@link DeepSeekAdapter}; the plugin's `apply` resolves them from Config + environment. */
@@ -111,6 +119,14 @@ export class DeepSeekAdapter extends LlmAdapter {
})))
}
override resolveModelContext(
_provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const contextWindow = this.options.models?.find(entry => entry.id === model)?.contextWindow
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
}
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
const consumer = new AbortController()
const upstream = options.signal === undefined

12
packages/llm/llm-deepseek/src/index.ts
View File

@@ -21,8 +21,8 @@ export const name = 'llm-deepseek'
export const inject = ['llm']
const DEFAULT_MODELS: DeepSeekCatalogModel[] = [
{ id: 'deepseek-v4-flash' },
{ id: 'deepseek-v4-pro' },
{ id: 'deepseek-v4-flash', contextWindow: 128_000 },
{ id: 'deepseek-v4-pro', contextWindow: 128_000 },
]
/**
@@ -50,6 +50,7 @@ const catalogModel: z<DeepSeekCatalogModel> = z.object({
id: z.string().required(),
name: z.string(),
description: z.string(),
contextWindow: z.number().step(1).min(1),
})
export const Config: z<Config> = z.object({
@@ -72,12 +73,19 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee
if (model.name !== undefined && model.name.length === 0) {
throw new Error(`llm-deepseek: catalog model "${model.id}" has an empty name`)
}
if (model.contextWindow !== undefined
&& (!Number.isInteger(model.contextWindow) || model.contextWindow <= 0)) {
throw new Error(
`llm-deepseek: catalog model "${model.id}" contextWindow must be a positive integer`,
)
}
if (seen.has(model.id)) throw new Error(`llm-deepseek: duplicate catalog model "${model.id}"`)
seen.add(model.id)
return {
id: model.id,
...model.name === undefined ? {} : { name: model.name },
...model.description === undefined ? {} : { description: model.description },
...model.contextWindow === undefined ? {} : { contextWindow: model.contextWindow },
}
})
}

30
packages/llm/llm-deepseek/tests/adapter.spec.ts
View File

@@ -517,6 +517,8 @@ describe('plugin registration and config', () => {
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'deepseek-v4-flash' },
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'deepseek-v4-pro' },
])
await expect(ctx.llm.resolveModelContext('deepseek', 'deepseek-v4-flash'))
.resolves.toEqual({ contextWindow: 128_000 })
})
it('uses the default model catalog when apply is called directly', async () => {
@@ -536,14 +538,23 @@ describe('plugin registration and config', () => {
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
models: [
{ id: 'private-fast' },
{ id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget' },
{ id: 'private-fast', contextWindow: 32_000 },
{
id: 'private-reasoner',
name: 'Private Reasoner',
description: 'Higher reasoning budget',
contextWindow: 64_000,
},
],
})
await expect(ctx.llm.listModels('deepseek')).resolves.toEqual([
{ provider: 'deepseek', id: 'private-fast', name: 'private-fast' },
{ provider: 'deepseek', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget' },
])
await expect(ctx.llm.resolveModelContext('deepseek', 'private-fast'))
.resolves.toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'arbitrary-unlisted'))
.resolves.toBeUndefined()
})
it('allows an explicit empty model catalog', async () => {
@@ -560,6 +571,8 @@ describe('plugin registration and config', () => {
it.each([
[[{ id: '' }], /ids must be non-empty/],
[[{ id: 'm', name: '' }], /empty name/],
[[{ id: 'm', contextWindow: 0 }], /contextWindow/],
[[{ id: 'm', contextWindow: 1.5 }], /contextWindow/],
[[{ id: 'm' }, { id: 'm' }], /duplicate catalog model/],
] as const)('rejects invalid advisory model config', async (models, message) => {
const ctx = new Context()
@@ -572,6 +585,19 @@ describe('plugin registration and config', () => {
expect(ctx.llm.listProviders()).toEqual([])
})
it('rejects invalid context capacity when apply is called directly', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
expect(() => {
LlmDeepSeek.apply(ctx, {
apiKey: 'k',
baseURL: 'http://127.0.0.1:1',
models: [{ id: 'invalid-context', contextWindow: 0 }],
})
}).toThrow(/contextWindow must be a positive integer/)
expect(ctx.llm.listProviders()).toEqual([])
})
it('falls back to DEEPSEEK_API_KEY and DEEPSEEK_BASE_URL env vars', async () => {
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://127.0.0.1:1')

2
packages/llm/llm-pi-ai/README.md
View File

@@ -28,7 +28,7 @@ Configure credentials and deployment-specific transport settings per provider. O
Each provider name must exist in pi-ai's installed catalog and may appear only once in this plugin instance. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry.
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelContext(provider, model)` performs the same exact descriptor lookup and returns its context window, keeping capacity metadata on the route-owning adapter rather than a consuming plugin.
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.

18
packages/llm/llm-pi-ai/src/adapter.ts
View File

@@ -15,7 +15,7 @@ import type {
SimpleStreamOptions,
} from '@earendil-works/pi-ai'
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelContext, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
import { resolveProfiles } from './config.ts'
import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts'
@@ -89,6 +89,22 @@ export class PiAiAdapter extends LlmAdapter {
})))
}
override resolveModelContext(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const profile = this.profiles.get(provider)
if (profile === undefined) {
return Promise.reject(new LlmError(
`pi-ai adapter does not own provider "${provider}"`,
'NO_ADAPTER',
))
}
return Promise.resolve().then(() => ({
contextWindow: resolveModel(profile, model).contextWindow,
}))
}
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
if (options.stop !== undefined) {
throw new LlmError('llm-pi-ai does not support GenerateOptions.stop', 'UNSUPPORTED_OPTION')

7
packages/llm/llm-pi-ai/tests/adapter.spec.ts
View File

@@ -322,6 +322,9 @@ describe('provider profile lifecycle', () => {
provider: 'openai', id: 'gpt-4.1', name: 'GPT-4.1',
})
expect(models.every(model => model.provider === 'openai')).toBe(true)
const context = await ctx.llm.resolveModelContext('openai', 'gpt-4.1')
expect(context).toBeDefined()
expect(typeof context?.contextWindow).toBe('number')
})
it('accepts absent credentials for pi-ai ambient authentication', async () => {
@@ -373,6 +376,10 @@ describe('provider profile lifecycle', () => {
it('constructs the adapter directly and rejects routes it does not own', async () => {
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai' }] })
await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(adapter.resolveModelContext('anthropic', 'claude-sonnet-4'))
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(adapter.resolveModelContext('openai', 'not-a-catalog-model'))
.rejects.toMatchObject({ code: 'UNKNOWN_MODEL' })
await expect((async () => {
for await (const _chunk of adapter.stream({ provider: 'anthropic', model: 'claude-sonnet-4', messages: [] })) { /* drain */ }
})()).rejects.toMatchObject({ code: 'NO_ADAPTER' })

5
packages/llm/llm/README.md
View File

@@ -11,12 +11,15 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
### Events
| Event | Mode | Purpose |
@@ -25,7 +28,7 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
### Extension points
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models.
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)

47
packages/llm/llm/src/index.ts
View File

@@ -7,7 +7,15 @@
*/
import { Context, Service } from 'cordis'
import type { GenerateOptions, LlmFailure, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
import type {
GenerateOptions,
LlmFailure,
LlmModelContext,
LlmModelInfo,
LlmProviderInfo,
Message,
StreamChunk,
} from './types.ts'
import type { ProviderRequestId } from './brand.ts'
import { deepFreeze } from './call-config.ts'
import { HarnessError } from './error.ts'
@@ -122,6 +130,20 @@ export abstract class LlmAdapter {
return Promise.resolve([])
}
/**
* Resolve context capacity for one model accepted by this adapter. Absence
* means the adapter does not know the capacity, not that routing is invalid.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns provider-owned context metadata, or `undefined` when unavailable.
*/
resolveModelContext(
_provider: string,
_model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(undefined)
}
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
@@ -217,6 +239,29 @@ export class LlmService extends Service {
})
}
/**
* Resolve context capacity from the adapter that owns one exact route.
* This query is independent of the advisory model catalog: an unlisted model
* may return metadata, while `undefined` never rejects later routing.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @returns detached context metadata, or `undefined` when the adapter has none.
*/
async resolveModelContext(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
if (context === undefined) return undefined
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
throw new LlmError(
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_CONTEXT',
)
}
return { contextWindow: context.contextWindow }
}
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
const registration = this.adapters.get(provider)
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')

6
packages/llm/llm/src/types.ts
View File

@@ -155,6 +155,12 @@ export interface LlmModelInfo {
description?: string
}
/** Provider-owned context capacity for one exact provider/model route. */
export interface LlmModelContext {
/** Maximum combined request and response context in tokens. */
contextWindow: number
}
/**
* Raw streaming protocol emitted by adapters.
* Block indexes correlate interleaved deltas, and `block-end` carries the

44
packages/llm/llm/tests/service.spec.ts
View File

@@ -13,7 +13,7 @@ import LlmService, {
ProviderRequestId,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
class ScriptedAdapter extends LlmAdapter {
constructor(private script: StreamChunk[]) {
@@ -48,6 +48,7 @@ class CatalogAdapter extends ScriptedAdapter {
constructor(
private readonly provider: LlmProviderInfo,
private readonly models: readonly LlmModelInfo[],
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
) {
super(SCRIPT)
}
@@ -59,6 +60,13 @@ class CatalogAdapter extends ScriptedAdapter {
override listModels(_provider: string): Promise<readonly LlmModelInfo[]> {
return Promise.resolve(this.models)
}
override resolveModelContext(
_provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(this.contexts[model])
}
}
const SCRIPT: StreamChunk[] = [
@@ -645,8 +653,42 @@ describe('LlmService', () => {
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
})
it('resolves detached model context independently of advisory catalog membership', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const source = { contextWindow: 32_000 }
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{ unlisted: source },
))
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
expect(resolved).toEqual({ contextWindow: 32_000 })
source.contextWindow = 64_000
expect(resolved).toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
})
it.each([0, -1, 1.5, Number.NaN])(
'rejects invalid adapter model context %s',
async (contextWindow) => {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{ model: { contextWindow } },
))
await expect(ctx.llm.resolveModelContext('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
},
)
it.each([
[{ id: 1, name: 'Name' }, 'non-string id'],
[{ id: 'other', name: 'Name' }, 'mismatched id'],

14
packages/llm/token-meter/README.md
View File

@@ -4,11 +4,7 @@ Replay-aware token measurement through the singleton `ctx.tokenMeter` service. I
## Configuration
| Key | Default | Contract |
|---|---:|---|
| `contextWindow` | `128000` | Positive integer service-wide context capacity. |
The estimator intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. `contextWindow` is the only deployment setting. Direct construction validates it; Loader mounts first apply the package's Schemastery shape validation. Unrecognized top-level keys are rejected.
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelContext()`.
## Measurement contract
@@ -30,13 +26,7 @@ Usage accounting sums disjoint input, cache-read, cache-write, and output bucket
- name: '@deepseek-ai/dsh-compact-basic'
```
Both plugins have usable defaults. A deployment with a different capacity configures the meter once:
```yaml
- name: '@deepseek-ai/dsh-token-meter'
config:
contextWindow: 32768
```
Both plugins have usable defaults. The meter remains independent of model routing and optional compaction. A deployment configures capacity on its LLM adapter and compaction policy on `dsh-compact-basic`.
## Model Experience

37
packages/llm/token-meter/src/index.ts
View File

@@ -19,12 +19,6 @@ import type {
export type * from './types.ts'
/** Default service-wide provider context capacity. */
const DEFAULT_CONTEXT_WINDOW = 128_000
/** Complete public configuration key set. */
const TOKEN_METER_CONFIG_KEYS: ReadonlySet<string> = new Set(['contextWindow'])
/** Fixed text-density estimate used until exact tokenization is needed. */
const CHARS_PER_TOKEN = 4
@@ -74,28 +68,10 @@ function optionalHeaderEquals(
/** Reject stale or misspelled keys before defaults can hide them. */
function validateConfigKeys(config: TokenMeterConfig): void {
for (const key of Object.keys(config)) {
if (!TOKEN_METER_CONFIG_KEYS.has(key)) {
throw new Error(
`TokenMeterConfig: unknown key "${key}" (allowed: contextWindow)`,
)
}
throw new Error(`TokenMeterConfig: unknown key "${key}" (no settings are supported)`)
}
}
/** Resolve and validate the one service-wide context capacity. */
function resolveContextWindow(config: TokenMeterConfig): number {
validateConfigKeys(config)
const contextWindow = config.contextWindow === undefined
? DEFAULT_CONTEXT_WINDOW
: config.contextWindow
if (!Number.isInteger(contextWindow) || contextWindow <= 0) {
throw new Error(
`TokenMeterConfig: contextWindow (${contextWindow}) must be a positive integer`,
)
}
return contextWindow
}
declare module 'cordis' {
interface Context {
tokenMeter: TokenMeterService
@@ -104,18 +80,15 @@ declare module 'cordis' {
/** Replay owner for one service-wide estimator and isolated per-session folds. */
export class TokenMeterService extends Service {
static Config: z<TokenMeterConfig> = z.object({
contextWindow: z.number().step(1).min(1).default(DEFAULT_CONTEXT_WINDOW),
})
/** Provider context-window capacity used by pressure consumers. */
readonly contextWindow: number
// Schemastery preserves untrusted loader keys on an empty object schema;
// the public type excludes settings while validateConfigKeys rejects them.
static Config: z<TokenMeterConfig> = z.object({}) as unknown as z<TokenMeterConfig>
private readonly states = new WeakMap<Session, ReplayState>()
constructor(ctx: Context, config: TokenMeterConfig = {}) {
super(ctx, 'tokenMeter')
this.contextWindow = resolveContextWindow(config)
validateConfigKeys(config)
// Readers catch up independently, while eager observation bounds ordinary
// read latency without creating state for sessions no consumer has read.

7
packages/llm/token-meter/src/types.ts
View File

@@ -6,11 +6,8 @@
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
/** Token-meter plugin configuration. */
export interface TokenMeterConfig {
/** Service-wide context-window capacity in tokens. Defaults to `128000`. */
contextWindow?: number
}
/** Token-meter plugin configuration; the fixed estimator has no settings. */
export type TokenMeterConfig = Record<string, never>
/** The baseline from which a signed surface delta produces current pressure. */
export type TokenMeasurementBaseline =

37
packages/llm/token-meter/tests/token-meter.spec.ts
View File

@@ -1,4 +1,4 @@
import { describe, expect, it } from 'vitest'
import { describe, expect, expectTypeOf, it } from 'vitest'
import { Context } from 'cordis'
import { CallId } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, Message, TokenUsage } from '@deepseek-ai/dsh-llm'
@@ -87,29 +87,18 @@ function expectSurfaceTotal(measurement: TokenMeasurement): void {
}
describe('TokenMeterService configuration and registration', () => {
it('provides one zero-config context window', () => {
const service = meter()
expect(service.contextWindow).toBe(128_000)
it('exposes an empty public configuration type', () => {
expectTypeOf<{}>().toExtend<TokenMeterConfig>()
expectTypeOf<{ contextWindow: number }>().not.toExtend<TokenMeterConfig>()
})
it('accepts one service-wide context-window override', () => {
expect(meter({ contextWindow: 32_000 }).contextWindow).toBe(32_000)
})
it.each(['models', 'contextWidow'])('rejects unknown top-level config key %s', (key) => {
expect(() => meter({ [key]: {} }))
.toThrow(`TokenMeterConfig: unknown key "${key}"`)
})
it.each([
{ contextWindow: 0 },
{ contextWindow: -1 },
{ contextWindow: 1.5 },
{ contextWindow: Number.NaN },
{ contextWindow: null },
] as unknown as TokenMeterConfig[])('rejects invalid context capacity %#', (config) => {
expect(() => meter(config)).toThrow(/contextWindow .* positive integer/)
})
it.each(['models', 'contextWindow', 'contextWidow'])(
'rejects stale or unknown top-level config key %s',
(key) => {
expect(() => meter({ [key]: {} } as unknown as TokenMeterConfig))
.toThrow(`TokenMeterConfig: unknown key "${key}"`)
},
)
it('registers and unregisters ctx.tokenMeter with its plugin fiber', async () => {
const ctx = new Context()
@@ -123,7 +112,7 @@ describe('TokenMeterService configuration and registration', () => {
describe('TokenMeterService pricing', () => {
it('prices every built-in content shape and merge-extended blocks with one fixed heuristic', () => {
const service = meter({ contextWindow: 100 })
const service = meter()
const blocks: ContentBlock[] = [
{ type: 'text', text: 'abcd' },
{ type: 'reasoning', text: 'ab' },
@@ -331,7 +320,7 @@ describe('replay anchors and surface folds', () => {
})
it('keeps only the latest successful request anchor across model switches', () => {
const service = meter({ contextWindow: 1_000 })
const service = meter()
const session = new Session(SessionId('switch'))
const alphaHeader = header('alpha', { system: 'same envelope' })
appendSuccessfulCall(session, alphaHeader, { usage: USAGE, providerText: 'alpha' })

3
packages/support/llm-replay/README.md
View File

@@ -23,7 +23,7 @@ Replay keys every call by its calling session id (`GenerateOptions.sessionId`, s
| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the primary (parent) `session.jsonl` fixture. Required (config or env). |
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional path to a `ReplayEntry[]` sidecar that replaces the PRIMARY session's derived script. |
| `childFiles` | string[] | `$DSH_SNAPSHOT_CHILD_FILES` (path-delimited) | Recorded subagent child-session logs for a nested scenario; empty for a single-session scenario. |
| `providers` | `ReplayProviderConfig[]` | — | Optional replay-only provider and model catalog. Configured routes dispatch through the replay adapter and never perform provider I/O. |
| `providers` | `ReplayProviderConfig[]` | — | Optional replay-only provider and model catalog. Each model may publish `contextWindow`; configured routes dispatch through the replay adapter and never perform provider I/O. |
```yaml
- id: llm-replay
@@ -34,6 +34,7 @@ Replay keys every call by its calling session id (`GenerateOptions.sessionId`, s
name: DeepSeek
models:
- id: deepseek-v4-flash
contextWindow: 128000
- id: deepseek-v4-pro
# file/overrideFile/childFiles default to $DSH_SNAPSHOT_FILE /
# $DSH_SNAPSHOT_OVERRIDE / $DSH_SNAPSHOT_CHILD_FILES, set by the snapshot

12
packages/support/llm-replay/src/index.ts
View File

@@ -10,7 +10,7 @@ import { existsSync, readFileSync } from 'node:fs'
import { delimiter as pathDelimiter } from 'node:path'
import type { Context } from 'cordis'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelContext, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { LlmAdapter, LlmError, assertNever } from '@deepseek-ai/dsh-llm'
/**
@@ -31,6 +31,8 @@ export interface ReplayModelConfig {
name?: string
/** Optional selector description. */
description?: string
/** Optional positive integer context capacity published by the replay adapter. */
contextWindow?: number
}
/** One provider route exposed by the replay adapter. */
@@ -260,6 +262,14 @@ class ReplayAdapter extends LlmAdapter {
})))
}
override resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined> {
const configured = this.providers.get(provider)
/* v8 ignore next -- LlmService only asks about routes registered from this same map. */
if (configured === undefined) return Promise.resolve(undefined)
const contextWindow = configured.models?.find(candidate => candidate.id === model)?.contextWindow
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
}
override stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
return this.replay(options)
}

6
packages/support/llm-replay/tests/llm-replay.spec.ts
View File

@@ -228,7 +228,7 @@ describe('installLlmReplay (through the real LlmService)', () => {
id: 'deepseek',
name: 'DeepSeek',
models: [
{ id: 'flash' },
{ id: 'flash', contextWindow: 128_000 },
{ id: 'pro', name: 'Pro', description: 'Larger model' },
],
},
@@ -245,6 +245,10 @@ describe('installLlmReplay (through the real LlmService)', () => {
{ provider: 'deepseek', id: 'pro', name: 'Pro', description: 'Larger model' },
])
await expect(ctx.llm.listModels('empty')).resolves.toEqual([])
await expect(ctx.llm.resolveModelContext('deepseek', 'flash')).resolves.toEqual({ contextWindow: 128_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'pro')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('deepseek', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('empty', 'unlisted')).resolves.toBeUndefined()
expect(await drain(ctx.llm.stream({ provider: 'deepseek', model: 'pro', messages: [] }))).toEqual(TEXT_CHUNKS)
dispose()

2
packages/ui/tui/README.md
View File

@@ -8,7 +8,7 @@ Interactive terminals on macOS, Linux, and Windows are supported. Windows uses p
This package owns interactive terminal presentation and input only. It injects `agents`, [`commands`](../commands/README.md), `llm`, `systemPrompt`, `tokenMeter`, `tools`, and `userInteraction`, then drives an agent created or resumed by app or developer code. Agent lifecycle, persistence, and the model-facing [`ask_user_question`](../tool-ask-user/README.md) tool remain separate composition entries.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view shows token-meter context occupancy, tool-card mode, and the current model with reasoning state; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view compares token-meter pressure with `ctx.llm.resolveModelContext()` for the current route, displays `context unknown` when the adapter has no capacity metadata, and also shows tool-card mode and the current model with reasoning state; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
Before model output, session events, tool presenters, questions, configuration, or diagnostics reach pi-tui's ANSI-aware renderers or the terminal title, the TUI renders C0 and C1 controls other than line feeds as visible `\xNN` text. Those sources cannot add terminal control sequences; the TUI and pi-tui retain ownership of terminal rendering and styling.

37
packages/ui/tui/src/index.ts
View File

@@ -737,7 +737,7 @@ class FooterComponent implements Component {
private readonly showReasoning: () => boolean,
private readonly tokens: () => { input: number; output: number },
private readonly currentModel: () => string | undefined,
private readonly contextPercent: () => number,
private readonly contextPercent: () => number | undefined,
private readonly runningSeconds: () => number,
) {}
@@ -755,7 +755,8 @@ class FooterComponent implements Component {
const counters = `↑${formatTokens(input)} ↓${formatTokens(output)}`
const model = displayText(this.currentModel() ?? 'model unset')
const modelState = `${model}(reasoning:${this.showReasoning() ? 'on' : 'off'})`
const context = `${this.contextPercent()}% context`
const contextPercent = this.contextPercent()
const context = contextPercent === undefined ? 'context unknown' : `${contextPercent}% context`
const fullRight = `${context} tools:${this.toolsExpanded() ? 'expanded' : 'compact'} ${modelState}`
const compactRight = `${context} ${modelState}`
if (visibleWidth(counters) + visibleWidth(compactRight) + 1 > width) {
@@ -1058,6 +1059,11 @@ export function createTuiChat(
let activeQuestion: PendingQuestion | undefined
let modelOverlay: OverlayHandle | undefined
const target: AgentLlmTargetRef = { current: initialTarget(agent), assembled: undefined }
let contextWindow: number | undefined
let contextResolution: Promise<
| { readonly kind: 'resolved'; readonly contextWindow: number | undefined }
| { readonly kind: 'error'; readonly error: unknown }
> | undefined
let modelCommands = Promise.resolve()
const now = (): number => runtime.now?.() ?? Date.now()
@@ -1070,7 +1076,9 @@ export function createTuiChat(
() => showReasoning,
() => tokens,
() => target.current?.model,
() => Math.min(100, Math.round(ctx.tokenMeter.measure(agent.session).totalTokens / ctx.tokenMeter.contextWindow * 100)),
() => contextWindow === undefined
? undefined
: Math.min(100, Math.round(ctx.tokenMeter.measure(agent.session).totalTokens / contextWindow * 100)),
() => runningStartedAt === undefined ? 0 : Math.max(0, Math.floor((now() - runningStartedAt) / 1_000)),
)
ui.addChild(header)
@@ -1096,12 +1104,34 @@ export function createTuiChat(
const disposeTargetListeners = installAgentLlmTarget(agent.ctx, target)
const resolveContextWindow = (selected: AgentLlmTarget | undefined): void => {
contextWindow = undefined
const resolution = selected === undefined
? Promise.resolve({ kind: 'resolved', contextWindow: undefined } as const)
: ctx.llm.resolveModelContext(selected.provider, selected.model).then(
context => ({ kind: 'resolved', contextWindow: context?.contextWindow } as const),
(error: unknown) => ({ kind: 'error', error } as const),
)
contextResolution = resolution
void resolution.then((result) => {
if (contextResolution !== resolution) return
if (result.kind === 'error') {
appendNotice(`Could not resolve model context: ${errorChain(result.error)}`, 'error')
return
}
contextWindow = result.contextWindow
requestRender()
})
}
resolveContextWindow(target.current)
const selectModel = (selected: ModelChoice): void => {
if (target.current?.provider === selected.provider && target.current.model === selected.model) {
appendNotice(`Model is already ${targetLabel(selected)}.`)
return
}
target.current = { provider: selected.provider, model: selected.model }
resolveContextWindow(target.current)
appendNotice(`Model selected: ${targetLabel(selected)}. New steps will use it.`)
}
@@ -1432,6 +1462,7 @@ export function createTuiChat(
const shutdown = (exitProcess: boolean): Promise<void> => {
shuttingDown ??= (async () => {
disposed = true
contextResolution = undefined
clearStatus()
modelOverlay?.hide()
modelOverlay = undefined

8
packages/ui/tui/tests/harness.ts
View File

@@ -1,7 +1,7 @@
import { Context } from 'cordis'
import type { Terminal } from '@earendil-works/pi-tui'
import AgentRegistry, { type Agent, type AgentOptions, type AgentStatus } from '@deepseek-ai/dsh-agent'
import type { ContentBlock, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
import CommandService from '@deepseek-ai/dsh-commands'
import SessionStore, { SessionId, type Session } from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
@@ -31,6 +31,7 @@ export interface TuiHarnessOptions {
providers: LlmProviderInfo[]
models: LlmModelInfo[]
listModels?: (provider: string) => Promise<LlmModelInfo[]>
resolveModelContext?: (provider: string, model: string) => Promise<LlmModelContext | undefined>
}
}
@@ -75,9 +76,12 @@ export async function createTuiTestHarness<TerminalType extends Terminal, Exit e
return catalog.listModels?.(provider)
?? Promise.resolve(catalog.models.filter(model => model.provider === provider).map(model => ({ ...model })))
},
resolveModelContext(provider: string, model: string) {
return catalog.resolveModelContext?.(provider, model)
?? Promise.resolve({ contextWindow: options.contextWindow ?? 128_000 })
},
} as never)
ctx.provide('tokenMeter', {
contextWindow: options.contextWindow ?? 128_000,
measure() {
return { totalTokens: options.contextTokens ?? 0 }
},

30
packages/ui/tui/tests/tui.spec.ts
View File

@@ -115,7 +115,6 @@ async function dispose(setupResult: Awaited<ReturnType<typeof setup>>): Promise<
function provideTokenMeter(ctx: Context): void {
ctx.provide('tokenMeter', {
contextWindow: 128_000,
measure() {
return { totalTokens: 0 }
},
@@ -540,8 +539,10 @@ describe('pi-tui chat lifecycle and transcript', () => {
})
it('opens a keyboard selector and switches the session model without sending slash text to the agent', async () => {
const initialContext = Promise.withResolvers<{ contextWindow: number }>()
const result = await setup({
agentOptions: { provider: 'alpha', model: 'a1' },
contextTokens: 50,
catalog: {
providers: [{ id: 'alpha', name: 'Alpha' }, { id: 'beta', name: 'Beta' }],
models: [
@@ -550,6 +551,9 @@ describe('pi-tui chat lifecycle and transcript', () => {
{ provider: 'beta', id: 'b1', name: 'Beta One' },
{ provider: 'beta', id: 'shared', name: 'Beta Shared' },
],
resolveModelContext: (provider, model) => provider === 'alpha' && model === 'a1'
? initialContext.promise
: Promise.resolve({ contextWindow: 200 }),
},
})
@@ -577,6 +581,9 @@ describe('pi-tui chat lifecycle and transcript', () => {
expect(result.terminal.output).toContain('Model selected: beta/b1')
expect(result.agent.sent).toEqual([])
expect(result.agent.steered).toEqual([])
initialContext.resolve({ contextWindow: 100 })
await tick()
expect(result.terminal.output).not.toContain('50% context tools:compact b1(reasoning:on)')
result.terminal.send('/model')
result.terminal.send('\r')
@@ -587,7 +594,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
result.agent.status = 'idle'
result.ctx.emit('agent/status', result.agent, 'idle')
await tick()
expect(result.terminal.output).toContain('tools:compact b1(reasoning:on)')
expect(result.terminal.output).toContain('25% context tools:compact b1(reasoning:on)')
const assembly = await result.ctx.systemPrompt.assemble(assembleContextFor(result.agent))
expect(assembly.variables).toMatchObject({ provider: 'beta', model: 'b1' })
@@ -623,6 +630,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'alpha', name: 'Alpha' }],
models: [{ provider: 'alpha', id: 'a1', name: 'Alpha One' }],
resolveModelContext: () => Promise.resolve(undefined),
},
})
unset.terminal.send('/model')
@@ -631,6 +639,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
unset.terminal.send('\r')
await tick()
expect(unset.terminal.output).toContain('Model selected: alpha/a1')
expect(unset.terminal.output).toContain('context unknown tools:compact a1(reasoning:on)')
await dispose(unset)
const empty = await setup({ agentOptions: {}, catalog: { providers: [], models: [] } })
@@ -652,12 +661,14 @@ describe('pi-tui chat lifecycle and transcript', () => {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
listModels: () => Promise.reject(new Error('catalog offline')),
resolveModelContext: () => Promise.reject(new Error('capacity offline')),
},
})
failed.terminal.send('/model')
failed.terminal.send('\r')
await tick()
expect(failed.terminal.output).toContain('Could not read the model catalog: catalog offline')
expect(failed.terminal.output).toContain('Could not resolve model context: capacity offline')
await dispose(failed)
})
@@ -693,6 +704,21 @@ describe('pi-tui chat lifecycle and transcript', () => {
await tick()
expect(rejectedResult.terminal.output).not.toContain('late catalog failure')
await rejectedResult.ctx.fiber.dispose()
const context = Promise.withResolvers<{ contextWindow: number }>()
const contextResult = await setup({
contextTokens: 99,
catalog: {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
resolveModelContext: () => context.promise,
},
})
await contextResult.controller.dispose()
context.resolve({ contextWindow: 100 })
await tick()
expect(contextResult.terminal.output).not.toContain('99% context')
await contextResult.ctx.fiber.dispose()
})
it('discovers and executes plugin commands, then removes TUI-local commands on disposal', async () => {