Merge remote-tracking branch 'origin/master' into codex/enforce-tool-cancellation
# Conflicts: # packages/core/agent-loop/tests/tool-calls.spec.ts
This commit is contained in:
@@ -8,6 +8,14 @@ These package-specific rules supplement the repo-wide [conventions](../AGENTS.md
|
||||
- **Typed same-process service and plugin calls are contracts, not serialization boundaries.** Prefer readonly borrowed values; materialize or defensively validate only at parser/config, queued, model/tool JSON, durable/file, worker, process, or wire boundaries.
|
||||
- **Initiator-owned private chains derive, then capture.** Under `ctx.agents.withInitiator()`, recover the Agent at each orchestration entry, derive `agent.session`, and let operation-local helpers close over it. Keep `Agent` and `Session` explicit at lifecycle, session-log, service, authority, worker/process, persistence, and wire interfaces; do not widen a leaf helper from `Session` to `Context` merely to hide a parameter ([rationale](../docs/rfc/implemented/architecture/2026-07-15-agent-initiator-scope.md)).
|
||||
- **Represent one asynchronous operation with one lifecycle controller or transaction.** Separate readiness, cancellation, disposal, reservation, or sentinel state requires an independent owner or settlement boundary; otherwise fold it while preserving rollback, callback containment, and quiescence.
|
||||
- **Shape capability interfaces around all current consumers.** Keep tool-schema, Loader, UI, transport, and backend-specific behavior in the consumer or adapter; do not let one consumer dictate the interface ([capability-seam rationale](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)).
|
||||
- **Require a current owner and need.** Tie each abstraction, state machine, option, defensive copy, and compatibility path to a current contract or production consumer, and keep behavior in its owning plugin or service.
|
||||
- **Require evidence for public choices.** Configurability does not justify an unsupported default, public operation set, format, or imported external concept. Use current-consumer evidence or relevant prior art; otherwise require an explicit value or defer the choice.
|
||||
- **Write model-facing contracts from the model's perspective.** Prompts, tool schemas, results, and diagnostics contain only task-relevant concepts, not UI, transport, or implementation vocabulary. Pin stable model-visible text verbatim and dynamic behavior through snapshots or end-to-end coverage.
|
||||
- **Enforce at the operation boundary that owns the decision.** Schema omission, prompt filtering, facades, wrappers, and listener order are not enforcement when direct or alternate callers can bypass them; test denial through the executor.
|
||||
- **Publish state only at its commit point.** Emit each notification and update derived state only after the success boundary that makes it true; derive caches, prompts, UI echoes, replay, and query views from one authoritative source.
|
||||
- **Apply bounds to the complete result.** Enforce byte, token, item, and time limits where the complete emitted or retained value, including wrappers and metadata, is known; test tiny and exact limits, oversized single chunks, and multibyte byte limits.
|
||||
- **Registry contributions prove disposal.** Add the HMR-safety test required by the [testing policy](../docs/testing.md): dispose the contributing fiber and observe removal.
|
||||
|
||||
Naming notes:
|
||||
|
||||
|
||||
@@ -8,13 +8,14 @@ This is the implementation tier of the compaction capability — see the [interf
|
||||
|
||||
This backend owns the compaction policy:
|
||||
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the provisional request envelope and current surface at one consumed-log revision. The current prompt and prefix override their logged values; the pre-step boundary reuses logged tools and call config.
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering.
|
||||
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
|
||||
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
|
||||
- **Summarization** — a direct `llm/stream` call uses the configured provider/model pair and cap, falling back to the latest logged request target and then the agent target, without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
|
||||
- **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
|
||||
- **Lifecycle** — `compactRegion()` mutates `agent.session` and records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step, outside an open step, so a tool-heavy turn remains compactable and the loop derives history once after mutation.
|
||||
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Recoverable failure records an error end and leaves the surface unchanged.
|
||||
- **Lifecycle** — `compactRegion()` mutates `agent.session` and records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes.
|
||||
- **Overflow recovery** — below-threshold overflow bypasses normal retention and attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized only when `surface.replaceGeneration` advances; no range, no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
|
||||
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. A region failure records an error end and leaves the surface unchanged. Operational post-step failures warn and continue; overflow-recovery failure preserves the original provider error.
|
||||
|
||||
The protected `summarize()` method is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on `ctx.tokenMeter`. The hook returns the summary blocks together with the call envelope it used (`{ summary, provider, model, maxTokens? }`), which is logged on `compact/summary`.
|
||||
|
||||
@@ -30,7 +31,8 @@ Every setting is optional. The pressure and retention policy applies to the toke
|
||||
| `summarizationModel` | no (default `''`) | Set together with `summarizationProvider`; an empty pair resolves the latest logged request target, then the `AgentOptions` pair. |
|
||||
| `maxTokens` | no (default `8192`) | Provider generation cap for the summarization call; may include reasoning tokens. |
|
||||
| `compactionRetries` | no (default `1`) | Extra attempts after the first when pressure remains above threshold. |
|
||||
| `auto` | no (default `true`) | Register the `agent/pre-step` automatic listener. Set `false` for manual-only. |
|
||||
| `maxOverflowRetries` | no (default `1`) | Maximum retries after canonical context-window overflow; `0` disables recovery only. |
|
||||
| `auto` | no (default `true`) | Register post-step pressure and overflow-recovery listeners. Set `false` for manual-only. |
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -40,7 +42,7 @@ import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
|
||||
export const name = 'compact-basic'
|
||||
export const inject = ['llm']
|
||||
export const inject = ['llm', 'tokenMeter']
|
||||
|
||||
export function apply(ctx: Context): void {
|
||||
ctx.plugin(TokenMeterService)
|
||||
@@ -54,7 +56,7 @@ Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it c
|
||||
|
||||
### Conversation history
|
||||
|
||||
**What the model sees**: Before a step whose estimated envelope and history exceed the threshold, the conversation model receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. This one checkpoint replaces the selected older range and is followed by the retained recent units.
|
||||
**What the model sees**: After a successful step crosses the threshold, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from that replacement. This one checkpoint replaces the selected older range and is followed by the retained recent units.
|
||||
|
||||
**Token effect**: The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget.
|
||||
|
||||
@@ -116,8 +118,9 @@ Rules:
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Pre-step sees a provisional request envelope** — the current prompt and prefix are exact, but routing and tool changes made later in `agent/request` are not logged yet. A router-only agent with no provisional provider/model pair skips that check.
|
||||
- **Meter accuracy follows the fixed heuristic** — missing reusable provider usage falls back to character count plus structural overhead rather than exact tokenization.
|
||||
- **Overflow classification is adapter-maintained** — provider wording can change; both DeepSeek adapters normalize currently recognized context-limit failures to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- **Single-unit and envelope-only overflow remain outside surface compaction** — recovery cannot split one indivisible message/tool unit or shrink system/tools/prefix.
|
||||
- **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting.
|
||||
- **Summarization failure fails closed with full, over-budget history** — including truncation at the summarization `maxTokens`, which hidden reasoning tokens can consume; the auto path logs a warning and proceeds.
|
||||
- **The summarization call has no transcript-snapshot coverage** — `dsh-llm-replay` derives calls from `assistant/chunk` events, so this chunk-less direct `ctx.llm.stream()` call cannot replay (named deferred replay infrastructure in [the seam RFC](../../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
/**
|
||||
* Automatic pre-step pressure listener for compact-basic.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-compact-basic/automatic
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import type { Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
interface AutomaticCompactor {
|
||||
compactIfNeeded(
|
||||
agent: Agent,
|
||||
fullSystemPrompt: string,
|
||||
sessionPrefix: readonly Message[],
|
||||
signal: AbortSignal,
|
||||
): Promise<CompactionResult | null>
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the implementation-owned automatic compaction listener.
|
||||
* @param ctx - context owning the listener effect and logger.
|
||||
* @param service - compactor whose public methods remain dynamically dispatched.
|
||||
*/
|
||||
export function registerAutomaticCompaction(
|
||||
ctx: Context,
|
||||
service: AutomaticCompactor,
|
||||
): void {
|
||||
ctx.on('agent/pre-step', async (
|
||||
agent: Agent,
|
||||
_turn: number,
|
||||
_step: number,
|
||||
fullSystemPrompt: string,
|
||||
sessionPrefix: readonly Message[],
|
||||
signal: AbortSignal,
|
||||
) => {
|
||||
try {
|
||||
const result = await service.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)
|
||||
if (result !== null) {
|
||||
ctx.logger.info(
|
||||
`compaction: shadowed ${result.shadowedSeqs.length} surface nodes `
|
||||
+ `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, `
|
||||
+ `~${result.shadowedTokenCount} tokens)`,
|
||||
)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
ctx.logger.warn(`compaction failed: ${message}; proceeding with full history`)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -22,6 +22,7 @@ const BASIC_COMPACT_CONFIG_KEYS: ReadonlySet<string> = new Set([
|
||||
'summarizationModel',
|
||||
'maxTokens',
|
||||
'compactionRetries',
|
||||
'maxOverflowRetries',
|
||||
'auto',
|
||||
])
|
||||
|
||||
@@ -31,7 +32,8 @@ function validateConfigKeys(config: BasicCompactConfig): void {
|
||||
if (!BASIC_COMPACT_CONFIG_KEYS.has(key)) {
|
||||
throw new Error(
|
||||
`BasicCompactConfig: unknown key "${key}" `
|
||||
+ '(allowed: thresholdRatio, retainTokens, summarizationProvider, summarizationModel, maxTokens, compactionRetries, auto)',
|
||||
+ '(allowed: thresholdRatio, retainTokens, summarizationProvider, summarizationModel, '
|
||||
+ 'maxTokens, compactionRetries, maxOverflowRetries, auto)',
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -58,6 +60,7 @@ export function resolveConfig(
|
||||
summarizationModel: config.summarizationModel ?? '',
|
||||
maxTokens: config.maxTokens ?? 8192,
|
||||
compactionRetries: config.compactionRetries ?? 1,
|
||||
maxOverflowRetries: config.maxOverflowRetries ?? 1,
|
||||
auto: config.auto ?? true,
|
||||
}
|
||||
|
||||
@@ -71,6 +74,7 @@ export function resolveConfig(
|
||||
}
|
||||
assertPositiveInteger('maxTokens', resolved.maxTokens)
|
||||
assertNonNegativeInteger('compactionRetries', resolved.compactionRetries)
|
||||
assertNonNegativeInteger('maxOverflowRetries', resolved.maxOverflowRetries)
|
||||
if (typeof resolved.summarizationProvider !== 'string') {
|
||||
throw new Error('BasicCompactConfig: summarizationProvider must be a string')
|
||||
}
|
||||
|
||||
@@ -7,12 +7,11 @@
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { CompactService } from '@deepseek-ai/dsh-compact'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import { canonicalHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { EpochHeader, Session } from '@deepseek-ai/dsh-session'
|
||||
import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { registerAutomaticCompaction } from './automatic.ts'
|
||||
import { resolveConfig } from './config.ts'
|
||||
import { compactSurfaceRegion, selectCompactableRange } from './region.ts'
|
||||
import { summarizeWithLlm } from './summarizer.ts'
|
||||
@@ -26,35 +25,10 @@ export type {
|
||||
ResolvedConfig,
|
||||
} from './types.ts'
|
||||
|
||||
/** Resolve the latest actual routed provider/model, then the complete agent fallback pair. */
|
||||
function effectiveTarget(agent: Agent): { provider: string; model: string } | undefined {
|
||||
const latest = agent.session.requestHeader()?.config
|
||||
if (latest !== undefined) return { provider: latest.provider, model: latest.model }
|
||||
const { provider, model } = agent.options
|
||||
if (provider === undefined || provider.length === 0 || model === undefined || model.length === 0) {
|
||||
return undefined
|
||||
}
|
||||
return { provider, model }
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the provisional pre-step request envelope. Prompt and prefix are exact;
|
||||
* tools and non-model call config come from the latest logged request because
|
||||
* later request middleware has not run yet.
|
||||
*/
|
||||
function provisionalHeader(
|
||||
target: { provider: string; model: string },
|
||||
session: Session,
|
||||
fullSystemPrompt: string,
|
||||
sessionPrefix: readonly Message[],
|
||||
): EpochHeader {
|
||||
const latest = session.requestHeader()
|
||||
return canonicalHeader({
|
||||
config: latest === undefined ? target : { ...latest.config, ...target },
|
||||
...fullSystemPrompt.length === 0 ? {} : { system: fullSystemPrompt },
|
||||
...latest?.tools === undefined ? {} : { tools: latest.tools },
|
||||
...sessionPrefix.length === 0 ? {} : { messagePrefix: [...sessionPrefix] },
|
||||
})
|
||||
/** Resolve the exact model durably routed for the latest provider request. */
|
||||
function routedModel(session: Session): string | undefined {
|
||||
const model = session.requestHeader()?.config.model
|
||||
return model === undefined || model.length === 0 ? undefined : model
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -75,6 +49,7 @@ export class BasicCompactService extends CompactService {
|
||||
summarizationModel: z.string().default(''),
|
||||
maxTokens: z.number().step(1).min(1).default(8192),
|
||||
compactionRetries: z.number().step(1).min(0).default(1),
|
||||
maxOverflowRetries: z.number().step(1).min(0).default(1),
|
||||
auto: z.boolean().default(true),
|
||||
})
|
||||
|
||||
@@ -84,7 +59,63 @@ export class BasicCompactService extends CompactService {
|
||||
constructor(ctx: Context, config: BasicCompactConfig = {}) {
|
||||
super(ctx)
|
||||
this.config = resolveConfig(config, ctx.tokenMeter)
|
||||
if (this.config.auto) registerAutomaticCompaction(ctx, this)
|
||||
if (this.config.auto) this._registerAutomaticCompaction()
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the automatic post-step pressure and context-overflow recovery
|
||||
* listeners. `compactIfNeeded` stays dynamically dispatched so subclass
|
||||
* overrides are honored at event time.
|
||||
*/
|
||||
private _registerAutomaticCompaction(): void {
|
||||
const { ctx } = this
|
||||
const logResult = (result: CompactionResult, trigger: string): void => {
|
||||
ctx.logger.info(
|
||||
`compaction (${trigger}): shadowed ${result.shadowedSeqs.length} surface nodes `
|
||||
+ `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, `
|
||||
+ `~${result.shadowedTokenCount} tokens)`,
|
||||
)
|
||||
}
|
||||
|
||||
ctx.on('agent/post-step', async (
|
||||
agent: Agent,
|
||||
_turn: number,
|
||||
_step: number,
|
||||
signal: AbortSignal,
|
||||
) => {
|
||||
if (signal.aborted) return
|
||||
try {
|
||||
const result = await this.compactIfNeeded(agent, 'pressure', signal)
|
||||
if (result !== null) logResult(result, 'post-step pressure')
|
||||
} catch (error: unknown) {
|
||||
const message = error instanceof Error ? error.message : String(error)
|
||||
ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`)
|
||||
}
|
||||
})
|
||||
|
||||
ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => {
|
||||
if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE
|
||||
|| retryAttempt >= this.config.maxOverflowRetries
|
||||
|| signal.aborted) return next()
|
||||
|
||||
let generation: number
|
||||
let result: CompactionResult | null
|
||||
try {
|
||||
generation = agent.session.surface.replaceGeneration
|
||||
result = await this.compactIfNeeded(agent, 'context-overflow', signal)
|
||||
} catch (recoveryError: unknown) {
|
||||
const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError)
|
||||
ctx.logger.warn(
|
||||
`context-overflow compaction failed: ${message}; preserving the original request error`,
|
||||
)
|
||||
return next()
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited.
|
||||
if (signal.aborted || result === null
|
||||
|| agent.session.surface.replaceGeneration <= generation) return next()
|
||||
logResult(result, 'context overflow recovery')
|
||||
return { action: 'retry' }
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -104,27 +135,39 @@ export class BasicCompactService extends CompactService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check replayed pressure for the provisional pre-step envelope and compact
|
||||
* a tool-balanced head until it falls below the service-wide threshold.
|
||||
* A genuinely model-less router-first step skips this provisional check.
|
||||
* @param agent - agent whose session and provisional provider/model are measured.
|
||||
* @param fullSystemPrompt - current assembled system prompt override.
|
||||
* @param sessionPrefix - current request-only prefix override.
|
||||
* @param signal - live step cancellation signal forwarded to summarization.
|
||||
* Compact for replayed post-step pressure or one provider-confirmed context
|
||||
* overflow. Both triggers price the latest durable routed request envelope;
|
||||
* overflow bypasses the normal threshold and retained-tail policy so it can
|
||||
* force one useful balanced reduction.
|
||||
* @param agent - agent whose latest durable routed request is measured.
|
||||
* @param trigger - normal post-step pressure or context-overflow recovery.
|
||||
* @param signal - live turn cancellation signal forwarded to summarization.
|
||||
* @returns the latest compaction result, or `null` when no check/work applies.
|
||||
*/
|
||||
override async compactIfNeeded(
|
||||
agent: Agent,
|
||||
fullSystemPrompt: string,
|
||||
sessionPrefix: readonly Message[],
|
||||
trigger: CompactionTrigger,
|
||||
signal: AbortSignal,
|
||||
): Promise<CompactionResult | null> {
|
||||
const target = effectiveTarget(agent)
|
||||
if (target === undefined) return null
|
||||
const model = routedModel(agent.session)
|
||||
if (model === undefined) return null
|
||||
const meter = this.ctx.tokenMeter
|
||||
const requestHeader = provisionalHeader(target, agent.session, fullSystemPrompt, sessionPrefix)
|
||||
switch (trigger) {
|
||||
case 'context-overflow': {
|
||||
const measurement = meter.measure(agent.session)
|
||||
const range = selectCompactableRange(agent.session, measurement, 0)
|
||||
if (range === null) return null
|
||||
return this.compactRegion(range.start, range.end, agent, signal)
|
||||
}
|
||||
case 'pressure':
|
||||
break
|
||||
/* v8 ignore next -- closed-union exhaustiveness guard */
|
||||
default:
|
||||
assertNever(trigger, 'compaction trigger')
|
||||
}
|
||||
|
||||
const threshold = Math.floor(meter.contextWindow * this.config.thresholdRatio)
|
||||
let measurement = meter.measure(agent.session, requestHeader)
|
||||
let measurement = meter.measure(agent.session)
|
||||
if (measurement.totalTokens < threshold) return null
|
||||
|
||||
let result: CompactionResult | null = null
|
||||
@@ -137,7 +180,7 @@ export class BasicCompactService extends CompactService {
|
||||
break
|
||||
}
|
||||
result = await this.compactRegion(range.start, range.end, agent, signal)
|
||||
measurement = meter.measure(agent.session, requestHeader)
|
||||
measurement = meter.measure(agent.session)
|
||||
if (measurement.totalTokens < threshold) return result
|
||||
}
|
||||
|
||||
|
||||
@@ -119,8 +119,8 @@ export async function summarizeWithLlm(
|
||||
}
|
||||
return {
|
||||
summary,
|
||||
provider: target.provider,
|
||||
model: target.model,
|
||||
provider: options.provider,
|
||||
model: options.model,
|
||||
maxTokens: config.maxTokens,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18,7 +18,9 @@ export interface BasicCompactConfig {
|
||||
maxTokens?: number
|
||||
/** Extra attempts after the first compaction when pressure remains above threshold. Defaults to `1`. */
|
||||
compactionRetries?: number
|
||||
/** Enable the automatic `agent/pre-step` pressure listener. Defaults to `true`. */
|
||||
/** Maximum retries after canonical context overflow; `0` disables recovery. Defaults to `1`. */
|
||||
maxOverflowRetries?: number
|
||||
/** Enable automatic post-step pressure and overflow-recovery listeners. Defaults to `true`. */
|
||||
auto?: boolean
|
||||
}
|
||||
|
||||
@@ -30,5 +32,6 @@ export interface ResolvedConfig {
|
||||
readonly summarizationModel: string
|
||||
readonly maxTokens: number
|
||||
readonly compactionRetries: number
|
||||
readonly maxOverflowRetries: number
|
||||
readonly auto: boolean
|
||||
}
|
||||
|
||||
@@ -3,10 +3,11 @@ import { Context } from 'cordis'
|
||||
import BasicCompactService from '@deepseek-ai/dsh-compact-basic'
|
||||
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
|
||||
import { selectCompactableRange } from '@deepseek-ai/dsh-compact-basic/src/region.ts'
|
||||
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
|
||||
import { resolveConfig } from '@deepseek-ai/dsh-compact-basic/src/config.ts'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import LlmService, { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
@@ -34,6 +35,12 @@ function conversation(turns = 4, text = 'fixture '.repeat(40).trim()): Session {
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
if (turn === 1) {
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL } },
|
||||
reason: 'initial',
|
||||
})
|
||||
}
|
||||
session.append('assistant/message', {
|
||||
provenance: { provider: MODEL, model: MODEL },
|
||||
turn,
|
||||
@@ -60,6 +67,12 @@ function toolConversation(): Session {
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
if (turn === 1) {
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL } },
|
||||
reason: 'initial',
|
||||
})
|
||||
}
|
||||
session.append('assistant/message', {
|
||||
provenance: { provider: MODEL, model: MODEL },
|
||||
turn,
|
||||
@@ -119,11 +132,10 @@ function service(
|
||||
async function compactIfNeeded(
|
||||
compact: BasicCompactService,
|
||||
session: Session,
|
||||
trigger: 'pressure' | 'context-overflow' = 'pressure',
|
||||
model: string | undefined = MODEL,
|
||||
system = '',
|
||||
prefix: readonly Message[] = [],
|
||||
): Promise<CompactionResult | null> {
|
||||
return compact.compactIfNeeded(agent(session, model), system, prefix, SIGNAL)
|
||||
return compact.compactIfNeeded(agent(session, model), trigger, SIGNAL)
|
||||
}
|
||||
|
||||
describe('compact configuration and defaults', () => {
|
||||
@@ -138,6 +150,7 @@ describe('compact configuration and defaults', () => {
|
||||
summarizationModel: '',
|
||||
maxTokens: 8192,
|
||||
compactionRetries: 1,
|
||||
maxOverflowRetries: 1,
|
||||
auto: true,
|
||||
})
|
||||
expect(Object.isFrozen(resolved)).toBe(true)
|
||||
@@ -167,6 +180,7 @@ describe('compact configuration and defaults', () => {
|
||||
const bad = [
|
||||
[{ maxTokens: 0 }, /maxTokens/],
|
||||
[{ compactionRetries: -1 }, /compactionRetries/],
|
||||
[{ maxOverflowRetries: -1 }, /maxOverflowRetries/],
|
||||
[{ auto: 'yes' }, /auto must be a boolean/],
|
||||
[{ summarizationProvider: 1 }, /summarizationProvider must be a string/],
|
||||
[{ summarizationModel: 1 }, /summarizationModel must be a string/],
|
||||
@@ -193,19 +207,58 @@ describe('pressure measurement and retention', () => {
|
||||
retainTokens: 180,
|
||||
}
|
||||
|
||||
it('skips the provisional check only when no routed or fallback model exists', async () => {
|
||||
it('skips when no durable routed model exists instead of using AgentOptions fallback', async () => {
|
||||
const compact = service(compactConfig)
|
||||
const session = conversation()
|
||||
expect(await compact.compactIfNeeded(agent(session), '', [], SIGNAL)).toBeNull()
|
||||
const session = new Session(SessionId('headerless'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(compact.compactIfNeeded(agent(session, MODEL), 'pressure', SIGNAL))
|
||||
.resolves.toBeNull()
|
||||
expect(compact.calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('meters any routed model without profile resolution', async () => {
|
||||
const compact = service(compactConfig)
|
||||
await expect(compactIfNeeded(compact, conversation(), 'unlisted-model'))
|
||||
const session = conversation()
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'unlisted-provider', model: 'unlisted-model' } },
|
||||
reason: 'resume',
|
||||
})
|
||||
await expect(compactIfNeeded(compact, session))
|
||||
.resolves.not.toBeNull()
|
||||
})
|
||||
|
||||
it('declines forced overflow when the whole surface is one indivisible tool pair', async () => {
|
||||
const compact = service(compactConfig)
|
||||
const session = new Session(SessionId('single-tool-pair'))
|
||||
const callId = CallId('single-call')
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('assistant/message', {
|
||||
provenance: { provider: MODEL, model: MODEL },
|
||||
turn: 1,
|
||||
step: 1,
|
||||
content: [{ type: 'tool-call', id: callId, name: 'read', arguments: '{}' }],
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('tool/call', { turn: 1, step: 1, callId, name: 'read', arguments: '{}' })
|
||||
session.append('tool/result', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
callId,
|
||||
content: [{ type: 'text', text: 'result' }],
|
||||
isError: false,
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const generation = session.surface.replaceGeneration
|
||||
|
||||
await expect(compactIfNeeded(compact, session, 'context-overflow')).resolves.toBeNull()
|
||||
expect(session.surface.replaceGeneration).toBe(generation)
|
||||
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
|
||||
it('does nothing below threshold and compacts a priced head above threshold', async () => {
|
||||
const compact = service(compactConfig)
|
||||
expect(await compactIfNeeded(compact, conversation(2))).toBeNull()
|
||||
@@ -217,26 +270,31 @@ describe('pressure measurement and retention', () => {
|
||||
expect(session.surface.nodes.length).toBeLessThan(8)
|
||||
})
|
||||
|
||||
it('counts the current prompt and request prefix without putting either on the surface', async () => {
|
||||
it('counts the durable routed request envelope without putting its prefix on the surface', async () => {
|
||||
const compact = service({
|
||||
auto: false,
|
||||
thresholdRatio: 0.7,
|
||||
thresholdRatio: 0.9,
|
||||
retainTokens: 50,
|
||||
})
|
||||
const session = conversation(2, 'x'.repeat(200))
|
||||
const session = conversation(2, 'x'.repeat(600))
|
||||
expect(await compactIfNeeded(compact, session)).toBeNull()
|
||||
|
||||
const prefix: Message[] = [{
|
||||
role: 'user',
|
||||
content: [{ type: 'text', text: 'p'.repeat(1_000) }],
|
||||
}]
|
||||
const result = await compactIfNeeded(compact, session, MODEL, 's'.repeat(1_000), prefix)
|
||||
const prefix = [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'p'.repeat(600) }] }]
|
||||
session.append('request/header', {
|
||||
header: {
|
||||
config: { provider: MODEL, model: MODEL },
|
||||
system: 's'.repeat(600),
|
||||
messagePrefix: prefix,
|
||||
},
|
||||
reason: 'resume',
|
||||
})
|
||||
const result = await compactIfNeeded(compact, session)
|
||||
expect(result).not.toBeNull()
|
||||
expect(prefix).toHaveLength(1)
|
||||
expect(session.events.some(event => event.type === 'context/message')).toBe(false)
|
||||
})
|
||||
|
||||
it('uses the latest logged routed model in the provisional request envelope', async () => {
|
||||
it('uses the latest logged request envelope without an AgentOptions override', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = service({
|
||||
auto: false,
|
||||
@@ -250,19 +308,28 @@ describe('pressure measurement and retention', () => {
|
||||
})
|
||||
const measure = vi.spyOn(ctx.tokenMeter, 'measure')
|
||||
|
||||
const result = await compactIfNeeded(compact, session, 'fallback')
|
||||
const result = await compactIfNeeded(compact, session, 'pressure', 'fallback')
|
||||
expect(result).not.toBeNull()
|
||||
expect(measure.mock.calls[0]?.[1]?.config.provider).toBe('actual')
|
||||
expect(measure.mock.calls[0]?.[1]?.config.model).toBe('actual')
|
||||
expect(session.requestHeader()?.config.model).toBe('actual')
|
||||
expect(measure.mock.calls[0]).toEqual([session])
|
||||
})
|
||||
|
||||
it('declines when envelope pressure is high but the surface has no compactable range', async () => {
|
||||
const compact = service(compactConfig)
|
||||
const empty = new Session(SessionId('empty'))
|
||||
expect(await compactIfNeeded(compact, empty, MODEL, 'x'.repeat(100_000))).toBeNull()
|
||||
empty.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
empty.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL }, system: 'x'.repeat(100_000) },
|
||||
reason: 'initial',
|
||||
})
|
||||
expect(await compactIfNeeded(compact, empty)).toBeNull()
|
||||
|
||||
const retained = conversation(1)
|
||||
expect(await compactIfNeeded(compact, retained, MODEL, 'x'.repeat(100_000))).toBeNull()
|
||||
retained.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL }, system: 'x'.repeat(100_000) },
|
||||
reason: 'resume',
|
||||
})
|
||||
expect(await compactIfNeeded(compact, retained)).toBeNull()
|
||||
})
|
||||
|
||||
it('uses one unified measurement for each pressure-and-retention decision', async () => {
|
||||
@@ -544,7 +611,20 @@ describe('compaction region transaction', () => {
|
||||
|
||||
it('lets a model-independent custom summarizer compact without a conversation model', async () => {
|
||||
const compact = service()
|
||||
const session = conversation(1)
|
||||
const session = new Session(SessionId('model-less-region'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', {
|
||||
content: [{ type: 'text', text: 'history '.repeat(100) }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('assistant/message', {
|
||||
provenance: { provider: 'historical', model: 'historical' },
|
||||
turn: 1,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: 'answer '.repeat(100) }],
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const nodes = session.surface.nodes
|
||||
await expect(compact.compactRegion(
|
||||
nodes[0]!,
|
||||
@@ -650,6 +730,28 @@ describe('default one-shot summarizer', () => {
|
||||
expect(adapter.lastOptions?.model).toBe('routed')
|
||||
})
|
||||
|
||||
it('records the model actually dispatched after one-shot stream routing', async () => {
|
||||
const { ctx, compact } = await summarizerHarness([{ type: 'text', text: 'unused' }])
|
||||
const routedAdapter = new ScriptedAdapter([{ type: 'text', text: 'routed summary' }])
|
||||
ctx.llm.registerAdapter(['routed-summary-provider'], routedAdapter)
|
||||
ctx.on('llm/stream', (options, next) => {
|
||||
options.provider = 'routed-summary-provider'
|
||||
options.model = 'routed-summary-model'
|
||||
return next()
|
||||
})
|
||||
|
||||
const session = conversation(3, 'large history '.repeat(500))
|
||||
const nodes = session.surface.nodes
|
||||
await compact.compactRegion(nodes[0]!, nodes[3]!, agent(session, MODEL), SIGNAL)
|
||||
expect(session.events.findLast(event => event.type === 'compact/summary')?.data).toMatchObject({
|
||||
summary: [{ type: 'text', text: 'routed summary' }],
|
||||
provider: 'routed-summary-provider',
|
||||
model: 'routed-summary-model',
|
||||
})
|
||||
expect(routedAdapter.lastOptions?.provider).toBe('routed-summary-provider')
|
||||
expect(routedAdapter.lastOptions?.model).toBe('routed-summary-model')
|
||||
})
|
||||
|
||||
it('fails clearly when no complete summarization target can be resolved', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -688,26 +790,57 @@ describe('default one-shot summarizer', () => {
|
||||
})
|
||||
|
||||
describe('automatic listener and loader composition', () => {
|
||||
function preStep(ctx: Context, owner: Agent): Promise<unknown> {
|
||||
return ctx.serial('agent/pre-step', owner, 1, 1, '', [], SIGNAL)
|
||||
function postStep(ctx: Context, owner: Agent, signal = SIGNAL): Promise<unknown> {
|
||||
return ctx.serial('agent/post-step', owner, 1, 1, signal)
|
||||
}
|
||||
|
||||
it('compacts above threshold and remains idle below it', async () => {
|
||||
function recover(
|
||||
ctx: Context,
|
||||
owner: Agent,
|
||||
error: Error & { code?: string },
|
||||
retryAttempt = 0,
|
||||
signal = SIGNAL,
|
||||
next: () => Promise<{ action: 'fail' | 'retry' }> = () => Promise.resolve({ action: 'fail' }),
|
||||
): Promise<{ action: 'fail' | 'retry' }> {
|
||||
return ctx.waterfall('agent/request-error', owner, 1, 1, error, retryAttempt, signal, next)
|
||||
}
|
||||
|
||||
function overflow(message = 'provider overflow'): Error & { code: string } {
|
||||
return Object.assign(new Error(message), { code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
}
|
||||
|
||||
it('compacts post-step above threshold using the durable routed model and remains idle below it', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
const pressured = conversation(4)
|
||||
await preStep(ctx, agent(pressured, MODEL))
|
||||
await postStep(ctx, agent(pressured, 'unconfigured-agent-fallback'))
|
||||
expect(pressured.events.some(event => event.type === 'compact/summary')).toBe(true)
|
||||
|
||||
const small = conversation(1)
|
||||
await preStep(ctx, agent(small, MODEL))
|
||||
await postStep(ctx, agent(small, MODEL))
|
||||
expect(small.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
expect(compact.calls).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('skips post-step pressure when the step signal is already aborted', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
const pressured = conversation(4)
|
||||
const compactIfNeeded = vi.spyOn(compact, 'compactIfNeeded')
|
||||
|
||||
await expect(postStep(ctx, agent(pressured, MODEL), AbortSignal.abort('step aborted')))
|
||||
.resolves.toBeUndefined()
|
||||
|
||||
expect(compactIfNeeded).not.toHaveBeenCalled()
|
||||
expect(pressured.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
})
|
||||
|
||||
it('warns and continues after operational failures, including non-Errors', async () => {
|
||||
const ctx = createContext()
|
||||
const warnings: string[] = []
|
||||
@@ -719,12 +852,187 @@ describe('automatic listener and loader composition', () => {
|
||||
compact.error = 'temporary failure'
|
||||
const session = conversation(4)
|
||||
|
||||
await expect(preStep(ctx, agent(session, MODEL))).resolves.toBeUndefined()
|
||||
await expect(postStep(ctx, agent(session, MODEL))).resolves.toBeUndefined()
|
||||
expect(warnings).toContainEqual(expect.stringContaining('temporary failure'))
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
|
||||
})
|
||||
|
||||
it('auto:false installs no listener', async () => {
|
||||
it('force-compacts below normal pressure for canonical overflow and retries only after replacement', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
void new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 900,
|
||||
})
|
||||
const session = conversation(3)
|
||||
const beforeGeneration = session.surface.replaceGeneration
|
||||
const retainedSeq = session.surface.nodes.at(-1)!
|
||||
const threshold = 10_000
|
||||
expect(ctx.tokenMeter.measure(session).totalTokens).toBeLessThan(threshold)
|
||||
const decision = await recover(ctx, agent(session, 'unconfigured-agent-fallback'), overflow())
|
||||
|
||||
expect(decision).toEqual({ action: 'retry' })
|
||||
expect(session.surface.replaceGeneration).toBe(beforeGeneration + 1)
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(true)
|
||||
expect(session.surface.nodes).toContain(retainedSeq)
|
||||
})
|
||||
|
||||
it('preserves the newest whole tool-call/result pair during forced overflow compaction', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 90,
|
||||
})
|
||||
const session = toolConversation()
|
||||
const newestAssistant = session.surface.nodes.at(-2)!
|
||||
const newestResult = session.surface.nodes.at(-1)!
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
|
||||
const currentAssistant = session.surface.nodes.find(node => node === newestAssistant)
|
||||
const currentResult = session.surface.nodes.find(node => node === newestResult)
|
||||
expect(currentAssistant).toBeDefined()
|
||||
expect(currentResult).toBeDefined()
|
||||
expect(toolPairingBalancedBefore(session, currentAssistant!)).toBe(true)
|
||||
expect(toolPairingBalancedAfter(session, currentResult!)).toBe(true)
|
||||
})
|
||||
|
||||
it('does not retry when a backend reports success without replacing the surface', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx)
|
||||
const session = conversation(2)
|
||||
const fakeResult: CompactionResult = {
|
||||
startSeq: 1,
|
||||
summarySeq: 2,
|
||||
endSeq: 3,
|
||||
summary: [{ type: 'text', text: 'fake' }],
|
||||
shadowedRange: { start: 1, end: 2 },
|
||||
shadowedSeqs: [1, 2],
|
||||
shadowedTokenCount: 10,
|
||||
}
|
||||
vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(fakeResult)
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
|
||||
expect(session.surface.replaceGeneration).toBe(0)
|
||||
})
|
||||
|
||||
it('delegates downstream exactly once when no replacement is available', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx)
|
||||
vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(null)
|
||||
const downstream = new Error('downstream recovery failed')
|
||||
let calls = 0
|
||||
|
||||
await expect(recover(
|
||||
ctx,
|
||||
agent(conversation(2), MODEL),
|
||||
overflow(),
|
||||
0,
|
||||
SIGNAL,
|
||||
() => {
|
||||
calls += 1
|
||||
return Promise.reject(downstream)
|
||||
},
|
||||
)).rejects.toBe(downstream)
|
||||
expect(calls).toBe(1)
|
||||
})
|
||||
|
||||
it('preserves the original provider error when recovery throws', async () => {
|
||||
const ctx = createContext()
|
||||
const warnings: string[] = []
|
||||
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
|
||||
const compact = new TestCompactService(ctx)
|
||||
compact.error = new Error('summary unavailable')
|
||||
const original = overflow('original provider overflow')
|
||||
|
||||
expect(await recover(ctx, agent(conversation(3), MODEL), original)).toEqual({ action: 'fail' })
|
||||
expect(original).toMatchObject({
|
||||
message: 'original provider overflow',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
})
|
||||
expect(warnings).toContainEqual(expect.stringContaining('preserving the original request error'))
|
||||
})
|
||||
|
||||
it('delegates once when overflow recovery throws a non-Error value', async () => {
|
||||
const ctx = createContext()
|
||||
const warnings: string[] = []
|
||||
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
|
||||
const compact = new TestCompactService(ctx)
|
||||
compact.error = 'non-error recovery failure'
|
||||
const session = conversation(3)
|
||||
const generation = session.surface.replaceGeneration
|
||||
const original = overflow('original provider failure')
|
||||
let delegations = 0
|
||||
|
||||
const decision = await recover(ctx, agent(session, MODEL), original, 0, SIGNAL, () => {
|
||||
delegations += 1
|
||||
return Promise.resolve({ action: 'fail' })
|
||||
})
|
||||
|
||||
expect(decision).toEqual({ action: 'fail' })
|
||||
expect(delegations).toBe(1)
|
||||
expect(session.surface.replaceGeneration).toBe(generation)
|
||||
expect(original).toMatchObject({
|
||||
message: 'original provider failure',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
})
|
||||
expect(warnings).toContainEqual(expect.stringContaining('non-error recovery failure'))
|
||||
})
|
||||
|
||||
it('recovers an overflow for an unlisted routed model', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx)
|
||||
const session = conversation(2)
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'unknown-routed-provider', model: 'unknown-routed-model' } },
|
||||
reason: 'resume',
|
||||
})
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow('unlisted-model overflow')))
|
||||
.toEqual({ action: 'retry' })
|
||||
})
|
||||
|
||||
it('honors retry caps, non-context failures, and cancellation', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx, { maxOverflowRetries: 1 })
|
||||
const compactSpy = vi.spyOn(compact, 'compactIfNeeded')
|
||||
const owner = agent(conversation(3), MODEL)
|
||||
expect(await recover(ctx, owner, Object.assign(new Error('rate limit'), { code: 'RATE_LIMIT' })))
|
||||
.toEqual({ action: 'fail' })
|
||||
expect(await recover(ctx, owner, overflow(), 1)).toEqual({ action: 'fail' })
|
||||
|
||||
const controller = new AbortController()
|
||||
controller.abort('cancelled')
|
||||
expect(await recover(ctx, owner, overflow(), 0, controller.signal)).toEqual({ action: 'fail' })
|
||||
expect(compactSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not retry when cancellation lands during an awaited compaction', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx)
|
||||
const controller = new AbortController()
|
||||
compact.mutateDuringSummary = () => { controller.abort('cancelled during summary') }
|
||||
const session = conversation(3)
|
||||
const generation = session.surface.replaceGeneration
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow(), 0, controller.signal))
|
||||
.toEqual({ action: 'fail' })
|
||||
expect(session.surface.replaceGeneration).toBe(generation + 1)
|
||||
})
|
||||
|
||||
it('maxOverflowRetries:0 disables recovery without disabling post-step pressure', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx, {
|
||||
maxOverflowRetries: 0,
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 180,
|
||||
})
|
||||
const session = conversation(4)
|
||||
await postStep(ctx, agent(session, MODEL))
|
||||
const summaries = session.events.filter(event => event.type === 'compact/summary').length
|
||||
expect(summaries).toBe(1)
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
|
||||
expect(session.events.filter(event => event.type === 'compact/summary')).toHaveLength(summaries)
|
||||
})
|
||||
|
||||
it('auto:false installs neither automatic listener', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
@@ -732,8 +1040,9 @@ describe('automatic listener and loader composition', () => {
|
||||
retainTokens: 180,
|
||||
})
|
||||
const session = conversation(4)
|
||||
await preStep(ctx, agent(session, MODEL))
|
||||
await postStep(ctx, agent(session, MODEL))
|
||||
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
|
||||
})
|
||||
|
||||
it('loads and disposes the real zero-config service stack', async () => {
|
||||
@@ -761,7 +1070,8 @@ describe('automatic listener and loader composition', () => {
|
||||
await fiber.dispose()
|
||||
|
||||
const session = conversation(4)
|
||||
await preStep(ctx, agent(session, MODEL))
|
||||
await postStep(ctx, agent(session, MODEL))
|
||||
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
@@ -55,6 +56,45 @@ class StepwiseToolAdapter extends LlmAdapter {
|
||||
}
|
||||
}
|
||||
|
||||
/** First conversation request overflows, then the rebuilt retry succeeds. */
|
||||
class OverflowRecoveryAdapter extends LlmAdapter {
|
||||
readonly conversationRequests: GenerateOptions[] = []
|
||||
readonly summaryRequests: GenerateOptions[] = []
|
||||
|
||||
constructor(private readonly delivery: 'thrown' | 'in-band') {
|
||||
super()
|
||||
}
|
||||
|
||||
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (options.system?.includes('You are a compaction engine')) {
|
||||
this.summaryRequests.push(options)
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'RECOVERY CHECKPOINT' } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
return
|
||||
}
|
||||
|
||||
this.conversationRequests.push(options)
|
||||
if (this.conversationRequests.length === 1) {
|
||||
if (this.delivery === 'thrown') {
|
||||
throw new LlmError('request too large for model context', CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
}
|
||||
yield {
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
message: 'request too large for model context',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
},
|
||||
}
|
||||
return
|
||||
}
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
}
|
||||
}
|
||||
|
||||
async function harness(toolSteps: number): Promise<{ ctx: Context; compact: ReproCompactService }> {
|
||||
const ctx = new Context()
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
@@ -95,6 +135,54 @@ function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
}
|
||||
|
||||
describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => {
|
||||
it('uses the model actually routed by agent/request for post-step pressure', async () => {
|
||||
const { ctx } = await harness(8)
|
||||
ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, provider: 'mock', model: 'mock' }))
|
||||
try {
|
||||
const agent = ctx.agentLoop.create(SessionId('routed-pressure'), {
|
||||
provider: 'unconfigured-agent-fallback',
|
||||
model: 'unconfigured-agent-fallback',
|
||||
})
|
||||
agent.send([{ type: 'text', text: 'do a routed multi-step task' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(agent.session.requestHeader()?.config.model).toBe('mock')
|
||||
expect(agent.session.events.some(event => event.type === 'compact/summary')).toBe(true)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'completed' } },
|
||||
})
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('runs automatic pressure after the current tool result and before step/end', async () => {
|
||||
const { ctx } = await harness(8)
|
||||
try {
|
||||
const agent = ctx.agentLoop.create(SessionId('post-step-order'), { provider: 'mock', model: 'mock' })
|
||||
agent.send([{ type: 'text', text: 'do tool work' }])
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const events = [...agent.session.events]
|
||||
const compactStart = events.find(event => event.type === 'compact/start')
|
||||
expect(compactStart).toBeDefined()
|
||||
const precedingResult = events.findLast(event =>
|
||||
event.type === 'tool/result' && event.seq < compactStart!.seq,
|
||||
)
|
||||
if (precedingResult?.type !== 'tool/result') throw new Error('expected a durable tool result before compaction')
|
||||
const stepEnd = events.find(event =>
|
||||
event.type === 'step/end'
|
||||
&& event.data.step === precedingResult.data.step
|
||||
&& event.seq > compactStart!.seq,
|
||||
)
|
||||
expect(precedingResult.seq).toBeLessThan(compactStart!.seq)
|
||||
expect(compactStart!.seq).toBeLessThan(stepEnd!.seq)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('the head checkpoint the loop lands is a balanced cut on both sides', async () => {
|
||||
const { ctx } = await harness(8)
|
||||
try {
|
||||
@@ -127,3 +215,88 @@ describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', ()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('context-overflow recovery across the real loop and compact-basic', () => {
|
||||
it.each(['thrown', 'in-band'] as const)(
|
||||
'force-compacts a %s overflow between failed and retry steps',
|
||||
async (delivery) => {
|
||||
const ctx = new Context()
|
||||
const adapter = new OverflowRecoveryAdapter(delivery)
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await ctx.plugin(Invariants)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(TokenMeterService, { contextWindow: 128 })
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, provider: 'mock', model: 'mock' }))
|
||||
await ctx.plugin(BasicCompactService, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 100,
|
||||
maxTokens: 64,
|
||||
compactionRetries: 0,
|
||||
maxOverflowRetries: 1,
|
||||
})
|
||||
|
||||
try {
|
||||
const agent = ctx.agentLoop.create(SessionId(`overflow-${delivery}`), {
|
||||
provider: 'unconfigured-agent-fallback',
|
||||
model: 'unconfigured-agent-fallback',
|
||||
})
|
||||
for (let turn = 1; turn <= 2; turn += 1) {
|
||||
const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY'
|
||||
agent.session.append('turn/start', {
|
||||
turn,
|
||||
trigger: { kind: 'message', source: { kind: 'user' } },
|
||||
})
|
||||
agent.session.append('user/message', {
|
||||
content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/start', { turn, step: 1 })
|
||||
agent.session.append('assistant/message', {
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
turn,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }],
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/end', { turn, step: 1 })
|
||||
agent.session.append('turn/end', { turn, reason: { kind: 'completed' } })
|
||||
}
|
||||
|
||||
agent.send([{ type: 'text', text: 'continue from history' }])
|
||||
await agent.whenIdle()
|
||||
|
||||
expect(adapter.conversationRequests).toHaveLength(2)
|
||||
expect(adapter.summaryRequests).toHaveLength(1)
|
||||
expect(JSON.stringify(adapter.conversationRequests[0]!.messages)).toContain('OLD HISTORY SENTINEL')
|
||||
const retry = JSON.stringify(adapter.conversationRequests[1]!.messages)
|
||||
expect(retry).toContain('RECOVERY CHECKPOINT')
|
||||
expect(retry).not.toContain('OLD HISTORY SENTINEL')
|
||||
|
||||
const events = [...agent.session.events]
|
||||
const failedEnd = events.find(event =>
|
||||
event.type === 'step/end' && event.data.turn === 3 && event.data.step === 1,
|
||||
)!
|
||||
const retryStart = events.find(event =>
|
||||
event.type === 'step/start' && event.data.turn === 3 && event.data.step === 2,
|
||||
)!
|
||||
const compaction = events.filter(event =>
|
||||
event.type === 'compact/start'
|
||||
|| event.type === 'compact/summary'
|
||||
|| event.type === 'compact/end',
|
||||
)
|
||||
expect(compaction.map(event => event.type)).toEqual([
|
||||
'compact/start',
|
||||
'compact/summary',
|
||||
'compact/end',
|
||||
])
|
||||
expect(compaction.every(event => event.seq > failedEnd.seq && event.seq < retryStart.seq)).toBe(true)
|
||||
expect(events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'completed' } },
|
||||
})
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
},
|
||||
)
|
||||
})
|
||||
|
||||
@@ -18,7 +18,7 @@ Both methods are **abstract** — the backend owns trigger policy, retention, ev
|
||||
|
||||
| Member | Semantics |
|
||||
|---|---|
|
||||
| `compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` | Estimate the surface-derived history size; if over the backend's threshold, compact an older range via `compactRegion`, keeping recent context intact. Returns the `CompactionResult`, or `null` if nothing needed compacting. All parameters required — the loop's `agent/pre-step` checkpoint supplies the agent, assembled `fullSystemPrompt`, composed `sessionPrefix` (request-only messages every request carries but the derived history omits — the pressure estimate must count them), and turn `signal`. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. |
|
||||
| `compactIfNeeded(agent, trigger, signal)` | Consider automatic compaction for `trigger: 'pressure' \| 'context-overflow'`. A pressure trigger may apply the backend's threshold and retained-tail policy; a confirmed overflow may force a useful balanced reduction. Returns the `CompactionResult`, or `null` when no safe range exists. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. |
|
||||
| `compactRegion(start, end, agent, signal?)` | Forcibly summarize surface nodes `[start, end]` (inclusive seqs) from `agent.session` into a single replacement node. **Throws** if a compaction is already in progress, if `start`/`end` aren't surface nodes, or if `start` is positioned after `end` on the surface. The range is a SURFACE-POSITION span, not a numeric seq interval — after a prior replace lands a fresh high-seq summary node at the shadowed range's position, surface order no longer tracks seq order. |
|
||||
|
||||
`CompactionResult` keeps the raw summary and bookkeeping-event seqs available to callers alongside the shadowed range and token accounting; its drift-checked shape lives in the [compaction data-structure reference](../../../docs/core-data-structures/compaction.md#compactionresult).
|
||||
@@ -74,6 +74,5 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener.
|
||||
- **Single-unit overflow is out of contract** — one retained unit (a closed step or a large pasted `user/message`) alone exceeding the budget cannot be compacted; the call may go out over-budget.
|
||||
- **A session prefix that alone approaches the window is a configuration error no backend fixes** — compaction shrinks derived history, never the prefix.
|
||||
- **Request context injected by downstream `agent/request` listeners sits outside pressure accounting** — `compactIfNeeded` counts prefix, derived history, and system prompt only.
|
||||
- **Single-unit overflow is out of contract** — one indivisible unit (a closed tool pair or a large pasted `user/message`) alone exceeding the budget cannot be compacted.
|
||||
- **An envelope that alone approaches the window is not surface-compaction work** — compaction shrinks derived history, never the system prompt, tools, or session prefix.
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import type { CompactionResult } from './types.ts'
|
||||
|
||||
@@ -16,10 +15,13 @@ export type { CompactionResult } from './types.ts'
|
||||
export { renderContentBlocks, renderTranscript } from './render.ts'
|
||||
export { toolPairingBalancedAfter, toolPairingBalancedBefore } from './tool-pairing.ts'
|
||||
|
||||
/** Why automatic policy is asking a backend to consider compaction. */
|
||||
export type CompactionTrigger = 'pressure' | 'context-overflow'
|
||||
|
||||
/** Minimal agent context compaction needs without depending on the agent package. */
|
||||
export interface CompactAgentContext {
|
||||
session: Session
|
||||
options: { model?: string }
|
||||
options: { provider?: string; model?: string }
|
||||
}
|
||||
|
||||
declare module 'cordis' {
|
||||
@@ -41,24 +43,20 @@ export abstract class CompactService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Check token pressure and compact if the conversation is too large.
|
||||
* Estimate the next request, including its session prefix, derived history,
|
||||
* and system prompt. Above threshold, compact a head-anchored range ending at
|
||||
* a balanced tool boundary and reconsolidate any prior automatic checkpoint.
|
||||
* Return `null` when no compaction is needed or an open tail leaves no safe
|
||||
* cutoff. A single oversized retained unit or prefix cannot be repaired here.
|
||||
* Consider automatic compaction for one explicit trigger. Pressure policy
|
||||
* uses the latest durable routed request, while context-overflow policy may
|
||||
* force a useful balanced reduction even below the normal threshold. Return
|
||||
* `null` when no safe range can be compacted. A single oversized retained
|
||||
* unit or request envelope cannot be repaired through surface compaction.
|
||||
*
|
||||
* @param agent - agent context owning the session surface and model options.
|
||||
* @param fullSystemPrompt - assembled system prompt, counted toward the estimate.
|
||||
* @param sessionPrefix - the instance's composed session prefix, counted toward the
|
||||
* estimate.
|
||||
* @param agent - agent context owning the session surface and routing options.
|
||||
* @param trigger - normal pressure or provider-confirmed context overflow.
|
||||
* @param signal - cancellation signal; model-backed implementations must forward it.
|
||||
* @returns the compaction result, or `null` if no compaction was needed.
|
||||
*/
|
||||
abstract compactIfNeeded(
|
||||
agent: CompactAgentContext,
|
||||
fullSystemPrompt: string,
|
||||
sessionPrefix: readonly Message[],
|
||||
trigger: CompactionTrigger,
|
||||
signal: AbortSignal,
|
||||
): Promise<CompactionResult | null>
|
||||
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { CompactService } from '@deepseek-ai/dsh-compact'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import type { Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { CompactAgentContext } from '@deepseek-ai/dsh-compact'
|
||||
|
||||
@@ -18,8 +17,7 @@ class StubCompactService extends CompactService {
|
||||
|
||||
override async compactIfNeeded(
|
||||
_agent: CompactAgentContext,
|
||||
_fullSystemPrompt: string,
|
||||
_sessionPrefix: readonly Message[],
|
||||
_trigger: CompactionTrigger,
|
||||
signal: AbortSignal,
|
||||
): Promise<CompactionResult | null> {
|
||||
this.lastSignal = signal
|
||||
@@ -82,7 +80,7 @@ describe('CompactService seam', () => {
|
||||
const ctx = new Context()
|
||||
const svc = new StubCompactService(ctx)
|
||||
const session = new Session(SessionId('s'))
|
||||
expect(await svc.compactIfNeeded(stubAgent(session), '', [], new AbortController().signal)).toBeNull()
|
||||
expect(await svc.compactIfNeeded(stubAgent(session), 'pressure', new AbortController().signal)).toBeNull()
|
||||
})
|
||||
|
||||
it('compact/* events merge into SessionEventMap and are log-only', async () => {
|
||||
@@ -115,7 +113,7 @@ describe('CompactService seam', () => {
|
||||
await svc.compactRegion(0, 0, stubAgent(session, 'm'), controller.signal)
|
||||
expect(svc.lastSignal).toBe(controller.signal)
|
||||
|
||||
await svc.compactIfNeeded(stubAgent(session), '', [], controller.signal)
|
||||
await svc.compactIfNeeded(stubAgent(session), 'context-overflow', controller.signal)
|
||||
expect(svc.lastSignal).toBe(controller.signal)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import type { Message } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Cordis plugin name used by loader diagnostics. */
|
||||
export const name = 'time-context'
|
||||
@@ -162,8 +161,6 @@ export function apply(ctx: Context, config: Config): void {
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
_fullSystemPrompt: string,
|
||||
_sessionPrefix: readonly Message[],
|
||||
signal: AbortSignal,
|
||||
) => {
|
||||
if (signal.aborted) return
|
||||
|
||||
@@ -83,7 +83,7 @@ async function fire(
|
||||
step: number,
|
||||
signal: AbortSignal = SIGNAL,
|
||||
): Promise<void> {
|
||||
await ctx.serial('agent/pre-step', agent, turn, step, '', [], signal)
|
||||
await ctx.serial('agent/pre-step', agent, turn, step, signal)
|
||||
}
|
||||
|
||||
function textResponse(text: string): StreamChunk[] {
|
||||
|
||||
@@ -203,8 +203,8 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
summary: 'Abstract compaction service.',
|
||||
methods: [
|
||||
{
|
||||
signature: 'abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise<CompactionResult | null>',
|
||||
jsDoc: '/**\n * Check token pressure and compact if the conversation is too large.\n * Estimate the next request, including its session prefix, derived history,\n * and system prompt. Above threshold, compact a head-anchored range ending at\n * a balanced tool boundary and reconsolidate any prior automatic checkpoint.\n * Return `null` when no compaction is needed or an open tail leaves no safe\n * cutoff. A single oversized retained unit or prefix cannot be repaired here.\n *\n * @param agent - agent context owning the session surface and model options.\n * @param fullSystemPrompt - assembled system prompt, counted toward the estimate.\n * @param sessionPrefix - the instance\'s composed session prefix, counted toward the\n * estimate.\n * @param signal - cancellation signal; model-backed implementations must forward it.\n * @returns the compaction result, or `null` if no compaction was needed.\n */',
|
||||
signature: 'abstract compactIfNeeded( agent: CompactAgentContext, trigger: CompactionTrigger, signal: AbortSignal, ): Promise<CompactionResult | null>',
|
||||
jsDoc: '/**\n * Consider automatic compaction for one explicit trigger. Pressure policy\n * uses the latest durable routed request, while context-overflow policy may\n * force a useful balanced reduction even below the normal threshold. Return\n * `null` when no safe range can be compacted. A single oversized retained\n * unit or request envelope cannot be repaired through surface compaction.\n *\n * @param agent - agent context owning the session surface and routing options.\n * @param trigger - normal pressure or provider-confirmed context overflow.\n * @param signal - cancellation signal; model-backed implementations must forward it.\n * @returns the compaction result, or `null` if no compaction was needed.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'abstract compactRegion( start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise<CompactionResult>',
|
||||
@@ -268,7 +268,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
},
|
||||
{
|
||||
signature: 'stream(options: GenerateOptions): AsyncIterable<StreamChunk>',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Dispatches\n * through the `llm/stream` waterfall.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection, dispatch, and iteration failures retain their original\n * Error identity and are tagged in a call-local scope for narrow agent-loop\n * request recovery; middleware and nested-call failures remain untagged for\n * the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -634,12 +634,19 @@ export const EVENT_API: readonly EventApiEntry[] = [
|
||||
jsDoc: '/**\n * A step or turn errored. The loop reports a failure here (plus the logger)\n * even when the error has no in-turn position for a session `error` event.\n * @param agent - the agent whose turn errored.\n * @param turn - the turn in which the failure surfaced.\n * @param step - the step at which the failure surfaced.\n * @param error - the failure, verbatim.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode emit\n */',
|
||||
summary: 'A step or turn errored.',
|
||||
},
|
||||
{
|
||||
name: 'agent/post-step',
|
||||
mode: 'serial',
|
||||
signature: '\'agent/post-step\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void',
|
||||
jsDoc: '/**\n * Awaited serial checkpoint after the response, real or synthetic tool\n * results, injected context, and steering are durable but before `step/end`.\n * A cancelled tool batch reaches this checkpoint with an aborted signal.\n * @param agent - the agent whose step is settling.\n * @param turn - the open turn number.\n * @param step - the open step number.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode serial\n */',
|
||||
summary: 'Awaited serial checkpoint after the response, real or synthetic tool results, injected context, and steering are durable but before `step/end`.',
|
||||
},
|
||||
{
|
||||
name: 'agent/pre-step',
|
||||
mode: 'serial',
|
||||
signature: '\'agent/pre-step\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise<void> | void',
|
||||
jsDoc: '/**\n * Awaited serial checkpoint for session-surface mutation after prompt\n * assembly and before `step/start`; appends land outside the pending step.\n * The loop derives history once afterward, so compaction records and\n * replacements are included without rewriting an assembled request. The\n * prompt and prefix are the exact pressure inputs for that request, and\n * `signal` cancels listener work.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent opening the step.\n * @param turn - the open turn number.\n * @param step - the pending step number.\n * @param fullSystemPrompt - the assembled prompt.\n * @param sessionPrefix - the frozen request prefix.\n * @param signal - the turn abort signal.\n * @mode serial\n */',
|
||||
summary: 'Awaited serial checkpoint for session-surface mutation after prompt assembly and before `step/start`; appends land outside the pending step.',
|
||||
signature: '\'agent/pre-step\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void',
|
||||
jsDoc: '/**\n * Awaited serial checkpoint before `step/start`; appends land outside the\n * pending step and are included when the loop derives request history.\n * `signal` cancels listener work.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent opening the step.\n * @param turn - the open turn number.\n * @param step - the pending step number.\n * @param signal - the turn abort signal.\n * @mode serial\n */',
|
||||
summary: 'Awaited serial checkpoint before `step/start`; appends land outside the pending step and are included when the loop derives request history.',
|
||||
},
|
||||
{
|
||||
name: 'agent/prompt-submit',
|
||||
@@ -662,11 +669,18 @@ export const EVENT_API: readonly EventApiEntry[] = [
|
||||
jsDoc: '/**\n * Replace the frozen call configuration. Model-visible content must use\n * logged channels; this seam cannot mutate messages. Injection here joins\n * the next request because the current step boundary is already fixed.\n * @param agent - the agent making the model call.\n * @param turn - the open turn number.\n * @param step - the step whose request this is.\n * @param config - the config the loop would use (frozen); return a replacement to switch.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */',
|
||||
summary: 'Replace the frozen call configuration.',
|
||||
},
|
||||
{
|
||||
name: 'agent/request-error',
|
||||
mode: 'waterfall',
|
||||
signature: '\'agent/request-error\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise<RequestErrorDecision>): Promise<RequestErrorDecision>',
|
||||
jsDoc: '/**\n * Recover a model-request failure after its failed step has closed. `retry`\n * opens a new numbered step; `fail` preserves the original request error.\n * Call `next()` to delegate to the next recovery listener or the default.\n * @param agent - the agent whose request failed.\n * @param turn - the open turn number.\n * @param step - the failed step number.\n * @param error - the original model-request failure.\n * @param retryAttempt - zero-based number of prior recovery retries.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */',
|
||||
summary: 'Recover a model-request failure after its failed step has closed.',
|
||||
},
|
||||
{
|
||||
name: 'agent/session-prefix',
|
||||
mode: 'waterfall',
|
||||
signature: '\'agent/session-prefix\'(this: Scoped<Agent>, agent: Agent, prefix: Message[], signal: AbortSignal, next: () => Promise<Message[]>): Promise<Message[]>',
|
||||
jsDoc: '/**\n * Compose request-only messages placed before derived history. The frozen\n * result is computed once per loop instance, logged on its anchoring request\n * header, and reused so the provider prefix remains stable. Interrupted\n * composition is discarded. Composition precedes the first `agent/pre-step`\n * and request boundary, so listener appends join the current request and\n * pressure accounting sees the composed prefix. Changing context belongs in\n * history; contributors should prepend to `await next()` to preserve registration order.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent whose session prefix is being composed.\n * @param prefix - the frozen seed; return an extended replacement.\n * @param signal - aborts composition when the step is torn down.\n * @mode waterfall\n */',
|
||||
jsDoc: '/**\n * Compose request-only messages placed before derived history. The frozen\n * result is computed once per loop instance, logged on its anchoring request\n * header, and reused so the provider prefix remains stable. Interrupted\n * composition is discarded. Composition precedes the first `agent/pre-step`\n * and request boundary, so listener appends join the current request.\n * Changing context belongs in history; contributors should prepend to\n * `await next()` to preserve registration order.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @param agent - the agent whose session prefix is being composed.\n * @param prefix - the frozen seed; return an extended replacement.\n * @param signal - aborts composition when the step is torn down.\n * @mode waterfall\n */',
|
||||
summary: 'Compose request-only messages placed before derived history.',
|
||||
},
|
||||
{
|
||||
@@ -1028,12 +1042,16 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'CompactAgentContext',
|
||||
declaration: 'export interface CompactAgentContext {\n session: Session;\n options: {\n model?: string;\n };\n}',
|
||||
declaration: 'export interface CompactAgentContext {\n session: Session;\n options: {\n provider?: string;\n model?: string;\n };\n}',
|
||||
},
|
||||
{
|
||||
name: 'CompactionResult',
|
||||
declaration: 'export interface CompactionResult {\n startSeq: number;\n summarySeq: number;\n endSeq: number;\n summary: ContentBlock[];\n shadowedRange: {\n start: number;\n end: number;\n };\n shadowedSeqs: number[];\n shadowedTokenCount: number;\n}',
|
||||
},
|
||||
{
|
||||
name: 'CompactionTrigger',
|
||||
declaration: 'export type CompactionTrigger = \'pressure\' | \'context-overflow\';',
|
||||
},
|
||||
{
|
||||
name: 'ConfinedArgv',
|
||||
declaration: 'export interface ConfinedArgv {\n argv: string[];\n enforcement: SandboxEnforcement;\n denialSignatures: readonly string[];\n runnerFailureSignatures: readonly string[];\n}',
|
||||
|
||||
@@ -54,7 +54,7 @@ The driver owns one agent for its lifetime and runs inside `ctx.agents.withIniti
|
||||
|
||||
Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history.
|
||||
|
||||
Plugin failure ends the current turn, not the loop. Cancellation clears pending work and aborts the current step without leaking to the next prompt. Terminal continuation stops remain authoritative through turn close and durability flush.
|
||||
Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and `agent/post-step` remain ordinary turn failures. Recovery observes a closed failed step, and a retry rebuilds the request from the durable log in a new numbered step. Cancellation clears pending work and aborts the current step without leaking to the next prompt; undispatched model tool calls receive synthetic `tool/call` and aborted result pairs. Terminal continuation stops remain authoritative through turn close and durability flush.
|
||||
|
||||
Within a step, exclusive calls form barriers; parallel-safe calls use a bounded rolling pool and are reclassified before start. Only dispatch/body overlaps. Policy, durable results, and result context remain model-ordered. Abort stops new calls, drains started results, then drains accepted batch context before the turn closes through the normal abort path.
|
||||
|
||||
@@ -62,7 +62,7 @@ Within a step, exclusive calls form barriers; parallel-safe calls use a bounded
|
||||
|
||||
Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy:
|
||||
- Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute` → `tools/execute` → `tools/post-execute` → `tools/result` pipeline; exact signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md)
|
||||
- Compaction: `agent/pre-step`
|
||||
- Compaction: pressure on `agent/post-step`; canonical context overflow on `agent/request-error`
|
||||
- Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation
|
||||
- Sub-agents: implemented outside the loop as `ctx.subagents` providers; in-process providers use `ctx.agents.create()` and owned `AgentHandle` teardown, while generic [`ctx.tasks`](../../tasks/tasks/) plus [`dsh-tool-subagent`](../../subagent/tool-subagent/) own background collection.
|
||||
- Persistence: `session/event` + `session/flush`
|
||||
@@ -82,6 +82,12 @@ Everything that goes beyond "call the model, run the tools, repeat" belongs to p
|
||||
|
||||
**Token effect**: Input grows with every surface message until a compaction replacement shadows older nodes; a multi-step tool turn resends the accumulated prefix and history each step.
|
||||
|
||||
### Undispatched calls after cancellation
|
||||
|
||||
**What the model sees**: If a later request replays an aborted step, each tool call that cancellation prevented from dispatching has the error result text `Error: tool call skipped because the step was aborted before execution`.
|
||||
|
||||
**Token effect**: One fixed error result per skipped call remains in history until compaction shadows it.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Classification is unary** — calls whose safety depends on comparing siblings or resources must remain exclusive ([rationale](../../../docs/rfc/implemented/feature/2026-07-10-parallel-tool-call-execution.md)).
|
||||
|
||||
@@ -8,9 +8,9 @@
|
||||
import type { Context } from 'cordis'
|
||||
import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { isDeepStrictEqual } from 'node:util'
|
||||
import { BlockAssembler, HarnessError, deepFreeze } from '@deepseek-ai/dsh-llm'
|
||||
import { BlockAssembler, HarnessError, assertNever, deepFreeze, isLlmAdapterFailure } from '@deepseek-ai/dsh-llm'
|
||||
import { agentEvents, assembleContextFor } from '@deepseek-ai/dsh-agent'
|
||||
import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision } from '@deepseek-ai/dsh-agent'
|
||||
import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent'
|
||||
import { canonicalHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, TurnEndReason, TurnTrigger } from '@deepseek-ai/dsh-session'
|
||||
import { createTransmissionLog, recordRequestHeader } from './request-log.ts'
|
||||
@@ -21,24 +21,29 @@ import type {} from '@deepseek-ai/dsh-tools'
|
||||
import { executeToolCalls } from './tool-calls.ts'
|
||||
import type { Inbox } from './inbox.ts'
|
||||
|
||||
/** An Error with an optional machine-readable code (e.g., from LlmError or a throwing plugin). */
|
||||
type CodedError = Error & { code?: string }
|
||||
|
||||
/** Normalize thrown values while preserving an existing error code. */
|
||||
function toError(error: unknown): CodedError {
|
||||
function toError(error: unknown): RequestError {
|
||||
return error instanceof Error ? error : new HarnessError(String(error), 'UNKNOWN', { cause: error })
|
||||
}
|
||||
|
||||
/** Distinguishes final model-request failures from failures in later step processing. */
|
||||
class TerminalModelRequestFailure extends Error {
|
||||
constructor(readonly requestError: RequestError) {
|
||||
super(requestError.message, { cause: requestError })
|
||||
this.name = 'TerminalModelRequestFailure'
|
||||
}
|
||||
}
|
||||
|
||||
/** Convert terminal failure finishes into step errors; unknown extensible finishes remain successful. */
|
||||
function finishError(finish: FinishReason): CodedError | undefined {
|
||||
function finishError(finish: FinishReason): RequestError | undefined {
|
||||
switch (finish.kind) {
|
||||
case 'error': {
|
||||
const error: CodedError = new Error(finish.message)
|
||||
const error: RequestError = new Error(finish.message)
|
||||
if (finish.code !== undefined) error.code = finish.code
|
||||
return error
|
||||
}
|
||||
case 'aborted': {
|
||||
const error: CodedError = new Error('model stream aborted')
|
||||
const error: RequestError = new Error('model stream aborted')
|
||||
error.code = 'ABORTED'
|
||||
return error
|
||||
}
|
||||
@@ -52,7 +57,7 @@ function finishError(finish: FinishReason): CodedError | undefined {
|
||||
* Build the `{ message, code? }` part of an error payload, omitting the
|
||||
* `code` key entirely when absent (exactOptionalPropertyTypes-correct).
|
||||
*/
|
||||
function errorData(err: CodedError): { message: string; code?: string } {
|
||||
function errorData(err: RequestError): { message: string; code?: string } {
|
||||
return { message: err.message, ...typeof err.code === 'string' ? { code: err.code } : {} }
|
||||
}
|
||||
|
||||
@@ -186,6 +191,7 @@ async function runTurn(
|
||||
|
||||
let reason: TurnEndReason = { kind: 'completed' }
|
||||
let step = 0
|
||||
let requestRetryAttempt = 0
|
||||
let stepOpen = false
|
||||
let errorReported = false
|
||||
let terminalStopped = false
|
||||
@@ -198,7 +204,7 @@ async function runTurn(
|
||||
}
|
||||
|
||||
// Record the durable turn failure once and contain the live error notification.
|
||||
const failTurn = (err: CodedError): void => {
|
||||
const failTurn = (err: RequestError): void => {
|
||||
if (errorReported) return
|
||||
errorReported = true
|
||||
reason = { kind: 'error', step, ...errorData(err) }
|
||||
@@ -284,7 +290,7 @@ async function runTurn(
|
||||
const abort = new AbortController()
|
||||
handle.setAbort(abort)
|
||||
|
||||
// Assemble once before pre-step so pressure checks and the request share the same prompt.
|
||||
// Assemble once before pre-step so listener work and the request share one prompt value.
|
||||
const assembly = await ctx.systemPrompt.assemble(assembleContextFor(agent))
|
||||
const fullSystemPrompt = renderPrompt(assembly)
|
||||
|
||||
@@ -295,9 +301,9 @@ async function runTurn(
|
||||
break
|
||||
}
|
||||
|
||||
// Compose the request-only prefix once per loop instance before pressure
|
||||
// checks. It precedes all derived history and is recorded only in the
|
||||
// request header, not as session history.
|
||||
// Compose the request-only prefix once per loop instance before the first
|
||||
// request boundary. It precedes all derived history and is recorded only
|
||||
// in the request header, not as session history.
|
||||
if (transmission.sessionPrefix === undefined) {
|
||||
const emptyPrefix: Message[] = deepFreeze([])
|
||||
const composed = await events.waterfall(
|
||||
@@ -314,8 +320,8 @@ async function runTurn(
|
||||
transmission.sessionPrefix = deepFreeze(structuredClone(composed))
|
||||
}
|
||||
|
||||
// Await surface mutations outside the step; pressure checks receive the pending prefix.
|
||||
await events.serial('agent/pre-step', turn, step, fullSystemPrompt, transmission.sessionPrefix, abort.signal)
|
||||
// Await surface mutations outside the step before snapshotting history.
|
||||
await events.serial('agent/pre-step', turn, step, abort.signal)
|
||||
|
||||
// Interruption landing during the pre-step seam: do not open an empty step.
|
||||
if (handle.isCancelled() || handle.isDisposed()) {
|
||||
@@ -345,14 +351,69 @@ async function runTurn(
|
||||
break
|
||||
}
|
||||
|
||||
let stepOutcome: { hadToolCalls: boolean; finish: FinishReason } | { error: Error }
|
||||
let stepOutcome:
|
||||
| { hadToolCalls: boolean; finish: FinishReason }
|
||||
| { requestError: RequestError }
|
||||
| { error: RequestError }
|
||||
try {
|
||||
stepOutcome = await runStep(
|
||||
ctx, events, handle, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal)
|
||||
} catch (error: unknown) {
|
||||
stepOutcome = { error: toError(error) }
|
||||
} finally {
|
||||
if (error instanceof TerminalModelRequestFailure) {
|
||||
stepOutcome = { requestError: error.requestError }
|
||||
} else {
|
||||
stepOutcome = { error: toError(error) }
|
||||
}
|
||||
}
|
||||
|
||||
if ('requestError' in stepOutcome) {
|
||||
// Recovery observes a balanced failed step and the original provider
|
||||
// error while the failed step's signal remains the active owner.
|
||||
closeStep()
|
||||
if (handle.isDisposed() || abort.signal.aborted) {
|
||||
handle.setAbort(undefined)
|
||||
reason = handle.isDisposed()
|
||||
? { kind: 'disposed' }
|
||||
: { kind: 'aborted', reason: String(abort.signal.reason) }
|
||||
break
|
||||
}
|
||||
|
||||
const defaultDecision: RequestErrorDecision = { action: 'fail' }
|
||||
let recoveryDecision: RequestErrorDecision = defaultDecision
|
||||
try {
|
||||
recoveryDecision = await events.waterfall(
|
||||
'agent/request-error', turn, step, stepOutcome.requestError,
|
||||
requestRetryAttempt, abort.signal,
|
||||
() => Promise.resolve(defaultDecision),
|
||||
)
|
||||
} catch (recoveryError: unknown) {
|
||||
ctx.logger.warn(
|
||||
`agent "${agent.id}": request recovery failed at turn ${turn}, step ${step}: ${toError(recoveryError).message}`,
|
||||
)
|
||||
}
|
||||
handle.setAbort(undefined)
|
||||
|
||||
// Cancellation and disposal always win over either a recovery decision
|
||||
// or a recovery-listener failure.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
|
||||
if (handle.isDisposed() || abort.signal.aborted) {
|
||||
reason = handle.isDisposed()
|
||||
? { kind: 'disposed' }
|
||||
: { kind: 'aborted', reason: String(abort.signal.reason) }
|
||||
break
|
||||
}
|
||||
switch (recoveryDecision.action) {
|
||||
case 'retry':
|
||||
requestRetryAttempt += 1
|
||||
continue
|
||||
case 'fail':
|
||||
failTurn(stepOutcome.requestError)
|
||||
break
|
||||
/* v8 ignore next -- closed-union exhaustiveness guard */
|
||||
default:
|
||||
assertNever(recoveryDecision, 'agent request-error decision')
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
if ('error' in stepOutcome) {
|
||||
@@ -360,7 +421,9 @@ async function runTurn(
|
||||
// runLoop re-enqueues it as a queued message, so an abort-then-steer
|
||||
// starts a fresh turn instead of being silently consumed.
|
||||
closeStep()
|
||||
handle.setAbort(undefined)
|
||||
const { error } = stepOutcome
|
||||
/* v8 ignore next -- narrow race: disposal while non-request step work throws. */
|
||||
if (handle.isDisposed()) {
|
||||
reason = { kind: 'disposed' }
|
||||
} else if (abort.signal.aborted) {
|
||||
@@ -372,6 +435,8 @@ async function runTurn(
|
||||
break
|
||||
}
|
||||
|
||||
requestRetryAttempt = 0
|
||||
|
||||
// Preserve max-token completion unless a later disposal, abort, or error wins.
|
||||
const stepReason = stepFinishReason(stepOutcome.finish)
|
||||
if (stepReason) reason = stepReason
|
||||
@@ -379,7 +444,38 @@ async function runTurn(
|
||||
// Steering that arrived during streaming/tool execution.
|
||||
const steered = drainSteering()
|
||||
|
||||
try {
|
||||
await events.serial('agent/post-step', turn, step, abort.signal)
|
||||
} catch (error: unknown) {
|
||||
stepOutcome = { error: toError(error) }
|
||||
}
|
||||
|
||||
if ('error' in stepOutcome) {
|
||||
closeStep()
|
||||
handle.setAbort(undefined)
|
||||
/* v8 ignore next -- narrow race: disposal while a post-step listener throws. */
|
||||
if (handle.isDisposed()) {
|
||||
reason = { kind: 'disposed' }
|
||||
} else if (abort.signal.aborted) {
|
||||
/* v8 ignore next -- signal.reason always set by cancellation or disposal. */
|
||||
reason = { kind: 'aborted', reason: String(abort.signal.reason ?? 'aborted') }
|
||||
} else {
|
||||
failTurn(stepOutcome.error)
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
if (handle.isDisposed() || abort.signal.aborted) {
|
||||
reason = handle.isDisposed()
|
||||
? { kind: 'disposed' }
|
||||
: { kind: 'aborted', reason: String(abort.signal.reason) }
|
||||
closeStep()
|
||||
handle.setAbort(undefined)
|
||||
break
|
||||
}
|
||||
|
||||
closeStep()
|
||||
handle.setAbort(undefined)
|
||||
|
||||
const defaultDecision: ContinuationDecision = { action: stepOutcome.hadToolCalls || steered ? 'continue' : 'stop' }
|
||||
let decision: ContinuationDecision
|
||||
@@ -529,17 +625,23 @@ async function runStep(
|
||||
// --- Model call (streaming-first; raw chunks are the replay record) ---
|
||||
const assembler = new BlockAssembler()
|
||||
const chunkSeqs: number[] = []
|
||||
for await (const chunk of ctx.llm.stream(request)) {
|
||||
/* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */
|
||||
if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted'))
|
||||
const chunkEvent = session.append('assistant/chunk', { turn, step, chunk })
|
||||
chunkSeqs.push(chunkEvent.seq)
|
||||
assembler.push(chunk)
|
||||
const stream = ctx.llm.stream(request)
|
||||
try {
|
||||
for await (const chunk of stream) {
|
||||
/* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */
|
||||
if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted'))
|
||||
const chunkEvent = session.append('assistant/chunk', { turn, step, chunk })
|
||||
chunkSeqs.push(chunkEvent.seq)
|
||||
assembler.push(chunk)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
if (isLlmAdapterFailure(stream, error)) throw new TerminalModelRequestFailure(error)
|
||||
throw error
|
||||
}
|
||||
|
||||
// Normalize failure finish chunks into the same path as thrown stream errors.
|
||||
const stepError = finishError(assembler.finish)
|
||||
if (stepError) throw stepError
|
||||
if (stepError) throw new TerminalModelRequestFailure(stepError)
|
||||
|
||||
const recordAssistantMessage = (
|
||||
assembledContent: ContentBlock[],
|
||||
|
||||
@@ -4,14 +4,15 @@
|
||||
* Dispatch may overlap, while policy, results, and result context remain
|
||||
* model-ordered. Abort stops replenishment and drains started calls.
|
||||
*
|
||||
* Each started call records `tool/call`; `tool/result` commits in model order,
|
||||
* preserving derived history when audit events interleave with earlier results.
|
||||
* Each advertised call records a balanced `tool/call`/`tool/result` pair. Calls
|
||||
* skipped after abort receive synthetic error results so replay stays valid.
|
||||
* @module dsh-agent-loop/tool-calls
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { assertNever, type ToolCallBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { HookContext } from '@deepseek-ai/dsh-agent'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import { TOOL_REGISTRY_SCHEDULER, type ToolExecutionInput, type ToolExecutionMode, type ToolExecutionResult, type ToolRunContext } from '@deepseek-ai/dsh-tools'
|
||||
|
||||
/** One tool call after argument parsing, ready to schedule. */
|
||||
@@ -27,10 +28,17 @@ interface Slot {
|
||||
needsPost: boolean
|
||||
}
|
||||
|
||||
/** One scheduler group outcome, including a drained cancellation. */
|
||||
interface GroupOutcome {
|
||||
consumed: number
|
||||
aborted: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Schedule one assistant step's tool calls by their live concurrency mode.
|
||||
* Started calls receive ordered results. Abort drains them and rethrows after
|
||||
* accepting their context into the batch FIFO owned by the caller.
|
||||
* Started calls receive ordered results. Abort drains them, records synthetic
|
||||
* results for unstarted calls, and returns with the signal still aborted after
|
||||
* accepting started-call context into the batch FIFO owned by the caller.
|
||||
* The committed step's AgentLoop driver boundary supplies the initiating Agent
|
||||
* that becomes each explicit {@link ToolExecutionInput.agent}.
|
||||
*
|
||||
@@ -52,6 +60,7 @@ export async function executeToolCalls(
|
||||
acceptContext: (context: HookContext) => void,
|
||||
): Promise<void> {
|
||||
const agent = ctx.agents.requireInitiator()
|
||||
const { session } = agent
|
||||
|
||||
// Inputs are distinct because tools/execute wrappers may replace `exec.signal`.
|
||||
const planned: PlannedCall[] = toolCalls.map(block => ({
|
||||
@@ -72,7 +81,14 @@ export async function executeToolCalls(
|
||||
const first = planned[next]!
|
||||
const mode = ctx.tools.executionMode(first.exec).kind
|
||||
const group = mode === 'parallel' ? planned.slice(next) : [first]
|
||||
next += await runGroup(ctx, turn, step, group, mode, signal, maxParallel, acceptContext)
|
||||
const outcome = await runGroup(
|
||||
ctx, turn, step, group, mode, signal, maxParallel, acceptContext,
|
||||
)
|
||||
next += outcome.consumed
|
||||
if (outcome.aborted) {
|
||||
for (const call of planned.slice(next)) appendSkippedToolCall(session, turn, step, call.block)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -90,7 +106,8 @@ function parseArguments(raw: string): unknown {
|
||||
* before start; an exclusive reclassification waits for the current pool to
|
||||
* drain and remains for the caller's next barrier. Results and contexts commit
|
||||
* in model order. Abort stops starts, drains and commits started calls, accepts
|
||||
* their contexts into the owning batch, and throws.
|
||||
* their contexts into the owning batch, records results for skipped calls, and
|
||||
* returns an aborted outcome.
|
||||
*/
|
||||
async function runGroup(
|
||||
ctx: Context,
|
||||
@@ -101,33 +118,8 @@ async function runGroup(
|
||||
signal: AbortSignal,
|
||||
maxParallel: number,
|
||||
acceptContext: (context: HookContext) => void,
|
||||
): Promise<number> {
|
||||
): Promise<GroupOutcome> {
|
||||
const { session } = ctx.agents.requireInitiator()
|
||||
const appendToolCall = (block: ToolCallBlock): number => {
|
||||
return session.append('tool/call', {
|
||||
turn,
|
||||
step,
|
||||
callId: block.id,
|
||||
name: block.name,
|
||||
arguments: block.arguments,
|
||||
}).seq
|
||||
}
|
||||
const appendToolResult = (block: ToolCallBlock, result: ToolExecutionResult, callSeq: number): void => {
|
||||
session.append('tool/result', {
|
||||
turn,
|
||||
step,
|
||||
// Correlation stays with the loop's authoritative model-transcript call id;
|
||||
// registry results deliberately do not duplicate it.
|
||||
callId: block.id,
|
||||
content: result.content,
|
||||
isError: result.isError,
|
||||
...result.error ? { error: result.error } : {},
|
||||
// Persist presentation payloads so UI bridges reproduce result cards on replay.
|
||||
...result.meta !== undefined ? { meta: result.meta } : {},
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [callSeq] })
|
||||
}
|
||||
/* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */
|
||||
if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted'))
|
||||
const slots: (Slot | undefined)[] = group.map(() => undefined)
|
||||
// Started slots retain their tool/call seq for result provenance.
|
||||
const callSeqs: number[] = group.map(() => -1)
|
||||
@@ -146,7 +138,7 @@ async function runGroup(
|
||||
? await ctx.tools[TOOL_REGISTRY_SCHEDULER].finalize(slot.exec, slot.result)
|
||||
: ctx.tools[TOOL_REGISTRY_SCHEDULER].finish(slot.exec, slot.result)
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion -- bounded index
|
||||
appendToolResult(call!.block, result, callSeqs[committed]!)
|
||||
appendToolResult(session, turn, step, call!.block, result, callSeqs[committed]!)
|
||||
for (const context of result.additionalContexts ?? []) acceptContext(context)
|
||||
committed++
|
||||
}
|
||||
@@ -157,7 +149,7 @@ async function runGroup(
|
||||
const startCall = async (index: number): Promise<void> => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion -- bounded index
|
||||
const call = group[index]!
|
||||
callSeqs[index] = appendToolCall(call.block)
|
||||
callSeqs[index] = appendToolCall(session, turn, step, call.block)
|
||||
started++
|
||||
const prepared = await ctx.tools[TOOL_REGISTRY_SCHEDULER].prepare(call.exec)
|
||||
switch (prepared.kind) {
|
||||
@@ -205,17 +197,57 @@ async function runGroup(
|
||||
inFlight.delete(settledIndex)
|
||||
await commitReady()
|
||||
// Abort may arrive while a tool or ordered commit awaits.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
|
||||
|
||||
if (signal.aborted) aborted = true
|
||||
await fillPool()
|
||||
}
|
||||
|
||||
if (aborted) {
|
||||
// Started calls and accepted context settle before the turn records the abort.
|
||||
/* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */
|
||||
throw new Error(String(signal.reason ?? 'aborted'))
|
||||
// Started calls and accepted context settle first; every remaining model
|
||||
// call then receives an ordered synthetic result before the turn aborts.
|
||||
for (const call of group.slice(started)) appendSkippedToolCall(session, turn, step, call.block)
|
||||
return { consumed: group.length, aborted: true }
|
||||
}
|
||||
/* v8 ignore next -- unreachable: a non-aborted group commits every started call */
|
||||
if (committed !== started) throw new Error('tool-call scheduler: uncommitted settled calls')
|
||||
return started
|
||||
return { consumed: started, aborted: false }
|
||||
}
|
||||
|
||||
/** Append the durable call/result pair for a model call skipped after cancellation. */
|
||||
function appendSkippedToolCall(session: Session, turn: number, step: number, block: ToolCallBlock): void {
|
||||
const callSeq = appendToolCall(session, turn, step, block)
|
||||
appendToolResult(session, turn, step, block, {
|
||||
content: [{ type: 'text', text: 'Error: tool call skipped because the step was aborted before execution' }],
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: 'ABORTED' },
|
||||
}, callSeq)
|
||||
}
|
||||
|
||||
/** Append a started call and return its provenance sequence. */
|
||||
function appendToolCall(session: Session, turn: number, step: number, block: ToolCallBlock): number {
|
||||
const event = session.append('tool/call', { turn, step, callId: block.id, name: block.name, arguments: block.arguments })
|
||||
return event.seq
|
||||
}
|
||||
|
||||
/** Append a model-ordered result linked to its call event. */
|
||||
function appendToolResult(
|
||||
session: Session,
|
||||
turn: number,
|
||||
step: number,
|
||||
block: ToolCallBlock,
|
||||
result: ToolExecutionResult,
|
||||
callSeq: number,
|
||||
): void {
|
||||
session.append('tool/result', {
|
||||
turn, step,
|
||||
// Correlation stays with the loop's authoritative model-transcript call id;
|
||||
// registry results deliberately do not duplicate it.
|
||||
callId: block.id,
|
||||
content: result.content,
|
||||
isError: result.isError,
|
||||
...result.error ? { error: result.error } : {},
|
||||
// The tool's private presentation payload (e.g. a result-time diff),
|
||||
// persisted so a UI bridge reproduces the card on replay.
|
||||
...result.meta !== undefined ? { meta: result.meta } : {},
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [callSeq] })
|
||||
}
|
||||
|
||||
@@ -12,11 +12,10 @@ import { Context } from 'cordis'
|
||||
import LlmService, { type Message } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId, TurnEndReason } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { MockAdapter, textResponse } from './mock-adapter.ts'
|
||||
import { MockAdapter, textResponse, toolCallResponse } from './mock-adapter.ts'
|
||||
|
||||
function driverDone(agent: Agent): Promise<void> {
|
||||
return (agent as Agent & { done: Promise<void> }).done
|
||||
@@ -144,6 +143,59 @@ describe('Agent.cancel()', () => {
|
||||
expect(reasons).toEqual([{ kind: 'aborted', reason: 'cancelled' }])
|
||||
})
|
||||
|
||||
it('cancel from an assistant/message observer skips execution but balances replay', async () => {
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'danger', {}),
|
||||
textResponse('recovered after cancellation'),
|
||||
])
|
||||
const ctx = await harness(adapter)
|
||||
let executions = 0
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'danger',
|
||||
description: 'must not run after cancellation',
|
||||
parameters: {},
|
||||
async execute() {
|
||||
executions += 1
|
||||
return [{ type: 'text', text: 'ran' }]
|
||||
},
|
||||
}))
|
||||
const agent = ctx.agentLoop.create(SessionId('cancel-after-assistant-message'), { provider: 'mock', model: 'mock' })
|
||||
const dispose = ctx.on('session/event', (session, event) => {
|
||||
if (session === agent.session && event.type === 'assistant/message') {
|
||||
agent.cancel('cancelled after assistant message')
|
||||
}
|
||||
})
|
||||
|
||||
const reasons: TurnEndReason[] = []
|
||||
ctx.on('session/event', (_session, event) => { if (event.type === 'turn/end') reasons.push(event.data.reason) })
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
dispose()
|
||||
|
||||
expect(executions).toBe(0)
|
||||
expect(reasons).toEqual([{ kind: 'aborted', reason: 'cancelled after assistant message' }])
|
||||
const call = agent.session.events.find(event => event.type === 'tool/call')
|
||||
const result = agent.session.events.find(event => event.type === 'tool/result')
|
||||
expect(call?.type === 'tool/call' ? call.data.callId : undefined).toBe('c1')
|
||||
expect(result?.type === 'tool/result' ? result.data : undefined).toMatchObject({
|
||||
callId: 'c1',
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: 'ABORTED' },
|
||||
})
|
||||
|
||||
send(agent, 'continue safely')
|
||||
await waitForIdle(ctx, agent)
|
||||
const replayedResult = adapter.requests[1]!.messages
|
||||
.flatMap(message => message.content)
|
||||
.find(block => block.type === 'tool-result')
|
||||
expect(replayedResult).toMatchObject({ toolCallId: 'c1', isError: true })
|
||||
expect(reasons).toEqual([
|
||||
{ kind: 'aborted', reason: 'cancelled after assistant message' },
|
||||
{ kind: 'completed' },
|
||||
])
|
||||
})
|
||||
|
||||
it('a prompt sent AFTER a cancelled turn settles runs normally (marker reset)', async () => {
|
||||
const adapter = new MockAdapter(['hang', textResponse('second reply')])
|
||||
const ctx = await harness(adapter)
|
||||
|
||||
@@ -204,7 +204,7 @@ describe('successful provider completion survives agent/step-result failure', ()
|
||||
})
|
||||
|
||||
describe('abort during tool execution ends the turn', () => {
|
||||
it('aborting the in-flight step inside a tool prevents both remaining tools and the next model step', async () => {
|
||||
it('balances an aborted tool batch through context, steering, and post-step before closing', async () => {
|
||||
const adapter = new MockAdapter([
|
||||
// model asks for two tool calls in one step
|
||||
[
|
||||
@@ -223,16 +223,24 @@ describe('abort during tool execution ends the turn', () => {
|
||||
name: 'aborter',
|
||||
description: '',
|
||||
parameters: {},
|
||||
async execute() {
|
||||
async execute(_args, exec) {
|
||||
executed.push('aborter')
|
||||
// Fire the in-flight step's AbortController directly (the loop registers
|
||||
// it on the agent). This is the bare step-abort path — distinct from
|
||||
// cancel(), which would also clear the inbox; here the subject is the
|
||||
// loop's response to its running step being aborted mid-tool.
|
||||
exec.agent?.steer(
|
||||
[{ type: 'text', text: 'steering before abort' }],
|
||||
{ source: { kind: 'plugin', plugin: 'abort-test' } },
|
||||
)
|
||||
// Exercise bare step abort without `cancel()` clearing queued work.
|
||||
;(agent as unknown as { currentAbort?: AbortController }).currentAbort?.abort('user interrupt')
|
||||
return [{ type: 'text', text: 'done' }]
|
||||
},
|
||||
}))
|
||||
ctx.on('tools/post-execute', async exec => ({
|
||||
kind: 'accept',
|
||||
additionalContexts: [{
|
||||
content: [{ type: 'text', text: `context for ${exec.callId}` }],
|
||||
source: { kind: 'plugin', plugin: 'abort-test' },
|
||||
}],
|
||||
}))
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'second',
|
||||
description: '',
|
||||
@@ -244,14 +252,69 @@ describe('abort during tool execution ends the turn', () => {
|
||||
}))
|
||||
|
||||
const reasons: TurnEndReason[] = []
|
||||
ctx.on('session/event', (_s, event) => { if (event.type === 'turn/end') reasons.push(event.data.reason) })
|
||||
const order: string[] = []
|
||||
ctx.on('session/event', (session, event) => {
|
||||
if (session !== agent.session) return
|
||||
switch (event.type) {
|
||||
case 'assistant/message': order.push('assistant/message'); break
|
||||
case 'tool/call': order.push(`tool/call:${event.data.callId}`); break
|
||||
case 'tool/result': {
|
||||
const outcome = event.data.error?.code === 'ABORTED' ? 'aborted' : 'completed'
|
||||
order.push(`tool/result:${event.data.callId}:${outcome}`)
|
||||
break
|
||||
}
|
||||
case 'context/message': order.push('context/message'); break
|
||||
case 'steering/message': order.push('steering/message'); break
|
||||
case 'step/end': order.push('step/end'); break
|
||||
case 'turn/end': {
|
||||
reasons.push(event.data.reason)
|
||||
order.push(`turn/end:${event.data.reason.kind}`)
|
||||
break
|
||||
}
|
||||
}
|
||||
})
|
||||
let postSteps = 0
|
||||
ctx.on('agent/post-step', (subject, turn, step, signal) => {
|
||||
if (subject !== agent) return
|
||||
postSteps += 1
|
||||
expect({ turn, step, aborted: signal.aborted }).toEqual({ turn: 1, step: 1, aborted: true })
|
||||
order.push('agent/post-step')
|
||||
})
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(executed).toEqual(['aborter']) // second tool never ran
|
||||
expect(adapter.requests).toHaveLength(1) // no follow-up model call
|
||||
expect(executed).toEqual(['aborter'])
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(postSteps).toBe(1)
|
||||
expect(order).toEqual([
|
||||
'assistant/message',
|
||||
'tool/call:c1',
|
||||
'tool/result:c1:aborted',
|
||||
'tool/call:c2',
|
||||
'tool/result:c2:aborted',
|
||||
'context/message',
|
||||
'steering/message',
|
||||
'agent/post-step',
|
||||
'step/end',
|
||||
'turn/end:aborted',
|
||||
])
|
||||
expect(reasons).toEqual([{ kind: 'aborted', reason: 'user interrupt' }])
|
||||
const calls = agent.session.events.filter(event => event.type === 'tool/call')
|
||||
const results = agent.session.events.filter(event => event.type === 'tool/result')
|
||||
expect(calls.map(event => event.data.callId)).toEqual([CallId('c1'), CallId('c2')])
|
||||
expect(results).toHaveLength(2)
|
||||
expect(results[0]!.data).toMatchObject({
|
||||
callId: CallId('c1'),
|
||||
content: [{ type: 'text', text: 'Error: tool call aborted' }],
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: 'ABORTED' },
|
||||
})
|
||||
expect(results[1]!.data).toMatchObject({
|
||||
callId: CallId('c2'),
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: 'ABORTED' },
|
||||
})
|
||||
})
|
||||
|
||||
it('records context accepted before a tool-step abort in the same turn', async () => {
|
||||
|
||||
@@ -115,14 +115,11 @@ describe('agent/prompt-submit', () => {
|
||||
expect(ctxMsg?.type === 'context/message' && ctxMsg.data.source).toEqual({ kind: 'plugin', plugin: 'test' })
|
||||
expect(ctxMsg?.type === 'context/message' && ctxMsg.data.envelope).toBe('raw')
|
||||
expect(ctxMsg?.type === 'context/message' && ctxMsg.data.meta).toEqual(meta)
|
||||
// both the prompt and the injected context reach the model
|
||||
const sent = JSON.stringify(adapter.requests[0]!.messages)
|
||||
expect(sent).toContain('extra ctx')
|
||||
})
|
||||
|
||||
it('a prompt-submit rewrite + additionalContexts is VISIBLE to the agent/pre-step seam (merged ordering)', async () => {
|
||||
// Prompt rewrites and injected context land before `agent/pre-step`, so a
|
||||
// compaction listener measures the current surface before the single derive.
|
||||
it('runs pre-step after prompt rewrites and injected context become durable', async () => {
|
||||
const adapter = new MockAdapter([textResponse('ok')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('a1'), { provider: 'mock', model: 'mock' })
|
||||
@@ -134,8 +131,6 @@ describe('agent/prompt-submit', () => {
|
||||
additionalContexts: [{ content: [{ type: 'text', text: 'injected ctx' }], source: { kind: 'plugin', plugin: 'test' } }],
|
||||
}))
|
||||
|
||||
// The pre-step seam (where compaction lives) derives the surface it would act
|
||||
// on. Capture what it sees on the first step.
|
||||
let preStepDerived: string | undefined
|
||||
ctx.on('agent/pre-step', (subject, _turn, step) => {
|
||||
if (subject === agent && step === 1) preStepDerived = JSON.stringify(subject.session.deriveMessages())
|
||||
@@ -144,8 +139,6 @@ describe('agent/prompt-submit', () => {
|
||||
send(agent, 'ORIGINAL prompt')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
// The pre-step seam ran and saw BOTH the rewrite (not the original) and the
|
||||
// injected context — i.e. the prompt-submit effects landed before it.
|
||||
expect(preStepDerived).toBeDefined()
|
||||
expect(preStepDerived).toContain('REWRITTEN prompt')
|
||||
expect(preStepDerived).toContain('injected ctx')
|
||||
@@ -379,7 +372,7 @@ describe('agent/session-prefix', () => {
|
||||
expect(agent.session.deriveMessages()[0]).toEqual({ role: 'user', content: [{ type: 'text', text: 'go' }] })
|
||||
})
|
||||
|
||||
it('composes before the first pre-step and hands the prefix to the seam (pressure gates see the real value)', async () => {
|
||||
it('composes before the first pre-step and records the prefix on the request header', async () => {
|
||||
const adapter = new MockAdapter([textResponse('ok')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('a1'), { provider: 'mock', model: 'mock' })
|
||||
@@ -390,20 +383,15 @@ describe('agent/session-prefix', () => {
|
||||
order.push('compose')
|
||||
return [reminder, ...await next()]
|
||||
})
|
||||
const seen: (readonly Message[])[] = []
|
||||
ctx.on('agent/pre-step', (_agent, _turn, _step, _system, sessionPrefix) => {
|
||||
ctx.on('agent/pre-step', () => {
|
||||
order.push('pre-step')
|
||||
seen.push(sessionPrefix)
|
||||
})
|
||||
|
||||
send(agent, 'hi')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
// Composition precedes the pre-step seam, and the seam receives THIS
|
||||
// instance's composed prefix — a token-pressure gate (compaction) counts
|
||||
// what the request will actually carry, never a stale logged prefix.
|
||||
expect(order).toEqual(['compose', 'pre-step'])
|
||||
expect(seen[0]).toEqual([reminder])
|
||||
expect(agent.session.requestHeader()?.messagePrefix).toEqual([reminder])
|
||||
})
|
||||
|
||||
it('the canonical prepend pattern composes contributions in registration order', async () => {
|
||||
|
||||
@@ -577,10 +577,6 @@ describe('agent loop', () => {
|
||||
})
|
||||
|
||||
it('agent/pre-step fires once per step before the step is opened', async () => {
|
||||
// Two steps (a tool call, then a final text turn) → two model calls → two
|
||||
// pre-step fires, each carrying the assembled full system prompt, BEFORE
|
||||
// the step is opened and its request is derived (the request the adapter
|
||||
// sees reflects any surface state at fire time).
|
||||
const adapter = new MockAdapter([
|
||||
toolCallResponse('c1', 'echo', {}, 'calling echo'),
|
||||
textResponse('done'),
|
||||
@@ -592,21 +588,19 @@ describe('agent loop', () => {
|
||||
}))
|
||||
const agent = ctx.agentLoop.create(SessionId('a1'), { provider: 'mock', model: 'mock' })
|
||||
|
||||
const fires: { turn: number; step: number; fullSystemPrompt: string }[] = []
|
||||
ctx.on('agent/pre-step', (subject, turn, step, fullSystemPrompt) => {
|
||||
if (subject === agent) fires.push({ turn, step, fullSystemPrompt })
|
||||
const fires: { turn: number; step: number; signal: AbortSignal }[] = []
|
||||
ctx.on('agent/pre-step', (subject, turn, step, signal) => {
|
||||
if (subject === agent) fires.push({ turn, step, signal })
|
||||
})
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
// One fire per step, in order, each with the assembled system prompt
|
||||
// (here just the loop's own harness-identity section — no persona set).
|
||||
const HARNESS = 'You are an AI agent powered by the DeepSeek Harness SDK.'
|
||||
expect(fires).toEqual([
|
||||
{ turn: 1, step: 1, fullSystemPrompt: HARNESS },
|
||||
{ turn: 1, step: 2, fullSystemPrompt: HARNESS },
|
||||
expect(fires.map(({ turn, step }) => ({ turn, step }))).toEqual([
|
||||
{ turn: 1, step: 1 },
|
||||
{ turn: 1, step: 2 },
|
||||
])
|
||||
expect(fires.every(({ signal }) => signal instanceof AbortSignal)).toBe(true)
|
||||
})
|
||||
|
||||
it('agent/pre-step fires BEFORE the step it precedes opens (events land outside the step)', async () => {
|
||||
|
||||
516
packages/core/agent-loop/tests/request-recovery.spec.ts
Normal file
516
packages/core/agent-loop/tests/request-recovery.spec.ts
Normal file
@@ -0,0 +1,516 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, {
|
||||
CallId,
|
||||
CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { PostToolDecision } from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { maxTokensResponse, textResponse, toolCallResponse } from './mock-adapter.ts'
|
||||
|
||||
class FailureScriptAdapter extends LlmAdapter {
|
||||
requests: GenerateOptions[] = []
|
||||
|
||||
constructor(private readonly entries: (Error | StreamChunk[])[]) {
|
||||
super()
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests.push(options)
|
||||
const entry = this.entries.shift()
|
||||
if (entry === undefined) throw new Error('failure script exhausted')
|
||||
if (entry instanceof Error) throw entry
|
||||
yield* entry
|
||||
}
|
||||
}
|
||||
|
||||
class IteratorConstructionFailureAdapter extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
throw new LlmError('iterator construction failed', 'ITERATOR_CONSTRUCTION')
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class SynchronousDispatchFailureAdapter extends LlmAdapter {
|
||||
constructor(private readonly error: Error) {
|
||||
super()
|
||||
}
|
||||
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
throw this.error
|
||||
}
|
||||
}
|
||||
|
||||
class IteratorResultGetterFailureAdapter extends LlmAdapter {
|
||||
constructor(
|
||||
private readonly field: 'done' | 'value',
|
||||
private readonly error: Error,
|
||||
) {
|
||||
super()
|
||||
}
|
||||
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const result = this.field === 'done' ? {} : { done: false }
|
||||
Object.defineProperty(result, this.field, { get: () => { throw this.error } })
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return { next: () => Promise.resolve(result as unknown as IteratorResult<StreamChunk>) }
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const streamListenerFailureCases: readonly [string, (ctx: Context) => void][] = [
|
||||
['synchronous listener throw', (ctx) => {
|
||||
ctx.on('llm/stream', () => { throw new Error('synchronous stream listener failed') })
|
||||
}],
|
||||
['invalid listener iterable', (ctx) => {
|
||||
ctx.on('llm/stream', () => ({}) as AsyncIterable<StreamChunk>)
|
||||
}],
|
||||
['listener wrapper iteration failure', (ctx) => {
|
||||
ctx.on('llm/stream', (_options, next) => (async function * () {
|
||||
for await (const chunk of next()) {
|
||||
yield chunk
|
||||
throw new Error('stream listener wrapper failed')
|
||||
}
|
||||
})())
|
||||
}],
|
||||
]
|
||||
|
||||
async function harness(adapter?: LlmAdapter): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
if (adapter) ctx.llm.registerAdapter(['mock'], adapter)
|
||||
return ctx
|
||||
}
|
||||
|
||||
function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const dispose = ctx.on('agent/status', (subject, status) => {
|
||||
if (subject === agent && status === 'idle') {
|
||||
dispose()
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
function send(agent: Agent): void {
|
||||
agent.send([{ type: 'text', text: 'go' }])
|
||||
}
|
||||
|
||||
function contextError(message = 'context too large'): LlmError {
|
||||
return new LlmError(message, CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
}
|
||||
|
||||
describe('agent post-step and request-error lifecycle', () => {
|
||||
it('fires post-step after results, buffered context, and steering but before step/end', async () => {
|
||||
const twoCalls: StreamChunk[] = [
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: CallId('call-1'), name: 'work', arguments: '{}' } },
|
||||
{ type: 'block-start', index: 1, blockType: 'tool-call' },
|
||||
{ type: 'block-end', index: 1, block: { type: 'tool-call', id: CallId('call-2'), name: 'work', arguments: '{}' } },
|
||||
{ type: 'usage', usage: { inputTokens: 10, outputTokens: 5 } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
]
|
||||
const adapter = new FailureScriptAdapter([twoCalls, textResponse('done')])
|
||||
const ctx = await harness(adapter)
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'work',
|
||||
description: 'do work',
|
||||
parameters: {},
|
||||
async execute(_args, exec) {
|
||||
if (exec.callId === CallId('call-2')) {
|
||||
exec.agent?.steer([{ type: 'text', text: 'steered' }], { source: { kind: 'plugin', plugin: 'test' } })
|
||||
}
|
||||
return [{ type: 'text', text: 'worked' }]
|
||||
},
|
||||
}))
|
||||
ctx.on('tools/post-execute', async (exec, _result): Promise<PostToolDecision> => ({
|
||||
kind: 'accept',
|
||||
additionalContexts: [{
|
||||
content: [{ type: 'text', text: `context for ${exec.callId}` }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
}],
|
||||
}))
|
||||
const agent = ctx.agentLoop.create(SessionId('post-step-order'), { provider: 'mock', model: 'mock' })
|
||||
const order: string[] = []
|
||||
ctx.on('session/event', (_session, event) => {
|
||||
if (
|
||||
event.type === 'assistant/message' || event.type === 'tool/call'
|
||||
|| event.type === 'tool/result' || event.type === 'context/message'
|
||||
|| event.type === 'steering/message' || event.type === 'step/end'
|
||||
) {
|
||||
if (!('step' in event.data) || event.data.step === 1) order.push(event.type)
|
||||
}
|
||||
})
|
||||
ctx.on('agent/post-step', (subject, turn, step, signal) => {
|
||||
if (subject !== agent || step !== 1) return
|
||||
expect({ turn, step, aborted: signal.aborted }).toEqual({ turn: 1, step: 1, aborted: false })
|
||||
subject.inject([{ type: 'text', text: 'listener mutation' }], { source: { kind: 'plugin', plugin: 'post-step' } })
|
||||
order.push('agent/post-step')
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(order).toEqual([
|
||||
'assistant/message',
|
||||
'tool/call',
|
||||
'tool/result',
|
||||
'tool/call',
|
||||
'tool/result',
|
||||
'context/message',
|
||||
'context/message',
|
||||
'steering/message',
|
||||
'context/message',
|
||||
'agent/post-step',
|
||||
'step/end',
|
||||
])
|
||||
})
|
||||
|
||||
it('fires post-step for max-tokens and lets cancellation override that success', async () => {
|
||||
const adapter = new FailureScriptAdapter([maxTokensResponse('partial')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('cancel-post-step-max-tokens'), { provider: 'mock', model: 'mock' })
|
||||
let entered!: () => void
|
||||
const postStepEntered = new Promise<void>((resolve) => { entered = resolve })
|
||||
ctx.on('agent/post-step', async (_agent, turn, step, signal) => {
|
||||
expect({ turn, step }).toEqual({ turn: 1, step: 1 })
|
||||
entered()
|
||||
await new Promise<void>((resolve) => {
|
||||
signal.addEventListener('abort', () => { resolve() }, { once: true })
|
||||
})
|
||||
})
|
||||
|
||||
send(agent)
|
||||
const idle = waitForIdle(ctx, agent)
|
||||
await postStepEntered
|
||||
agent.cancel('cancelled during max-tokens post-step')
|
||||
await idle
|
||||
|
||||
expect(agent.session.events.find(event => event.type === 'assistant/message')).toMatchObject({
|
||||
data: { usage: { inputTokens: 10, outputTokens: 7 } },
|
||||
})
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'aborted', reason: 'cancelled during max-tokens post-step' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('closes the successful step as disposed when disposal lands during post-step', async () => {
|
||||
const adapter = new FailureScriptAdapter([
|
||||
toolCallResponse('dispose-call', 'work', {}),
|
||||
textResponse('must not continue'),
|
||||
])
|
||||
const ctx = await harness(adapter)
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'work',
|
||||
description: 'do work',
|
||||
parameters: {},
|
||||
async execute() { return [{ type: 'text', text: 'worked' }] },
|
||||
}))
|
||||
const agent = ctx.agentLoop.create(SessionId('dispose-post-step'), { provider: 'mock', model: 'mock' })
|
||||
let entered!: () => void
|
||||
const postStepEntered = new Promise<void>((resolve) => { entered = resolve })
|
||||
ctx.on('agent/post-step', async (_agent, turn, step, signal) => {
|
||||
expect({ turn, step }).toEqual({ turn: 1, step: 1 })
|
||||
entered()
|
||||
await new Promise<void>((resolve) => {
|
||||
signal.addEventListener('abort', () => { resolve() }, { once: true })
|
||||
})
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await postStepEntered
|
||||
await ctx.fiber.dispose()
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
const boundaries = agent.session.events.filter(event =>
|
||||
event.type === 'step/start' || event.type === 'step/end',
|
||||
)
|
||||
expect(boundaries.map(event => event.type)).toEqual(['step/start', 'step/end'])
|
||||
expect(boundaries.map(event => event.data)).toEqual([
|
||||
{ turn: 1, step: 1 },
|
||||
{ turn: 1, step: 1 },
|
||||
])
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'disposed' } },
|
||||
})
|
||||
})
|
||||
|
||||
it.each([
|
||||
['thrown', contextError()],
|
||||
['in-band', [{ type: 'finish', reason: { kind: 'error', message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE } }] satisfies StreamChunk[]],
|
||||
] as const)('recovers a %s request failure in a new reconstructable step', async (_style, failure) => {
|
||||
const adapter = new FailureScriptAdapter([failure, textResponse('recovered')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId(`recover-${_style}`), { provider: 'mock', model: 'mock' })
|
||||
const attempts: number[] = []
|
||||
ctx.on('agent/request-error', async (subject, turn, step, error, attempt) => {
|
||||
expect(subject).toBe(agent)
|
||||
expect({ turn, step, code: error.code }).toEqual({ turn: 1, step: 1, code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
attempts.push(attempt)
|
||||
subject.session.append('context/message', {
|
||||
content: [{ type: 'text', text: 'RECOVERY SURFACE MUTATION' }],
|
||||
source: { kind: 'plugin', plugin: 'test-recovery' },
|
||||
}, { surfaceOp: 'append' })
|
||||
return { action: 'retry' }
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(attempts).toEqual([0])
|
||||
expect(adapter.requests).toHaveLength(2)
|
||||
expect(JSON.stringify(adapter.requests[1]!.messages)).toContain('RECOVERY SURFACE MUTATION')
|
||||
const starts = agent.session.events.filter(event => event.type === 'step/start')
|
||||
const ends = agent.session.events.filter(event => event.type === 'step/end')
|
||||
expect(starts.map(event => event.data.step)).toEqual([1, 2])
|
||||
expect(ends.map(event => event.data.step)).toEqual([1, 2])
|
||||
const recovery = agent.session.events.find(event => event.type === 'context/message')!
|
||||
expect(ends[0]!.seq).toBeLessThan(recovery.seq)
|
||||
expect(recovery.seq).toBeLessThan(starts[1]!.seq)
|
||||
})
|
||||
|
||||
it.each(streamListenerFailureCases)('does not offer %s to request recovery', async (_name, install) => {
|
||||
const ctx = await harness(new FailureScriptAdapter([textResponse('unused')]))
|
||||
const agent = ctx.agentLoop.create(SessionId(`stream-plugin-${_name.replaceAll(' ', '-')}`), { provider: 'mock', model: 'mock' })
|
||||
let recoveries = 0
|
||||
install(ctx)
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => {
|
||||
recoveries += 1
|
||||
return next()
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(recoveries).toBe(0)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'error' } } })
|
||||
})
|
||||
|
||||
it('does not offer a nested model-call failure as the outer request failure', async () => {
|
||||
const outer = new FailureScriptAdapter([textResponse('outer adapter must not run')])
|
||||
const nested = new FailureScriptAdapter([contextError('nested overflow')])
|
||||
const ctx = await harness(outer)
|
||||
ctx.llm.registerAdapter(['nested'], nested)
|
||||
ctx.on('llm/stream', (options, next) => {
|
||||
if (options.provider !== 'mock') return next()
|
||||
return (async function* () {
|
||||
yield* ctx.llm.stream({
|
||||
provider: 'nested',
|
||||
model: 'nested',
|
||||
messages: [],
|
||||
...options.signal === undefined ? {} : { signal: options.signal },
|
||||
})
|
||||
yield* next()
|
||||
})()
|
||||
})
|
||||
const agent = ctx.agentLoop.create(SessionId('nested-stream-not-recoverable'), { provider: 'mock', model: 'mock' })
|
||||
let recoveries = 0
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => {
|
||||
recoveries += 1
|
||||
return next()
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(nested.requests).toHaveLength(1)
|
||||
expect(outer.requests).toHaveLength(0)
|
||||
expect(recoveries).toBe(0)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'error', message: 'nested overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } },
|
||||
})
|
||||
})
|
||||
|
||||
it.each(['prompt-submit', 'prompt-assembly', 'pre-step', 'request'] as const)(
|
||||
'does not offer %s middleware failures to request recovery',
|
||||
async (boundary) => {
|
||||
const adapter = new FailureScriptAdapter([textResponse('unused')])
|
||||
const ctx = await harness(adapter)
|
||||
if (boundary === 'prompt-submit') {
|
||||
ctx.on('agent/prompt-submit', () => { throw new Error('prompt submit failed') })
|
||||
} else if (boundary === 'prompt-assembly') {
|
||||
ctx.on('system-prompt/assemble', () => { throw new Error('prompt assembly failed') })
|
||||
} else if (boundary === 'pre-step') {
|
||||
ctx.on('agent/pre-step', () => { throw new Error('pre-step failed') })
|
||||
} else {
|
||||
ctx.on('agent/request', () => { throw new Error('request middleware failed') })
|
||||
}
|
||||
const agent = ctx.agentLoop.create(SessionId(`${boundary}-not-recoverable`), { provider: 'mock', model: 'mock' })
|
||||
let recoveries = 0
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => {
|
||||
recoveries += 1
|
||||
return next()
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(recoveries).toBe(0)
|
||||
expect(adapter.requests).toHaveLength(0)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'error' } } })
|
||||
},
|
||||
)
|
||||
|
||||
it('does not offer result, tool, or post-step plugin failures to request recovery', async () => {
|
||||
for (const failure of ['result', 'tool', 'post-step'] as const) {
|
||||
const adapter = new FailureScriptAdapter([
|
||||
failure === 'tool' ? toolCallResponse(`call-${failure}`, 'work', {}) : textResponse('done'),
|
||||
...(failure === 'tool' ? [textResponse('done')] : []),
|
||||
])
|
||||
const ctx = await harness(adapter)
|
||||
if (failure === 'result') ctx.on('agent/step-result', () => { throw new Error('result failed') })
|
||||
if (failure === 'post-step') ctx.on('agent/post-step', () => { throw new Error('post-step failed') })
|
||||
if (failure === 'tool') {
|
||||
vi.spyOn(ctx.tools, 'execute').mockRejectedValue(new Error('tool service failed'))
|
||||
}
|
||||
const agent = ctx.agentLoop.create(SessionId(`${failure}-not-recoverable`), { provider: 'mock', model: 'mock' })
|
||||
let recoveries = 0
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => {
|
||||
recoveries += 1
|
||||
return next()
|
||||
})
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
expect(recoveries, failure).toBe(0)
|
||||
}
|
||||
})
|
||||
|
||||
it.each([
|
||||
['synchronous dispatch', (error: Error) => new SynchronousDispatchFailureAdapter(error)],
|
||||
['done getter', (error: Error) => new IteratorResultGetterFailureAdapter('done', error)],
|
||||
['value getter', (error: Error) => new IteratorResultGetterFailureAdapter('value', error)],
|
||||
] as const)('preserves original Error identity for adapter %s', async (_name, makeAdapter) => {
|
||||
const original = contextError(`${_name} overflow`)
|
||||
const ctx = await harness(makeAdapter(original))
|
||||
const agent = ctx.agentLoop.create(SessionId(`identity-${_name.replaceAll(' ', '-')}`), { provider: 'mock', model: 'mock' })
|
||||
let seen: Error | undefined
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => {
|
||||
seen = error
|
||||
return next()
|
||||
})
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(seen).toBe(original)
|
||||
})
|
||||
|
||||
it('classifies iterator construction and explicit NO_ADAPTER as model-request failures', async () => {
|
||||
for (const scenario of ['iterator', 'no-adapter'] as const) {
|
||||
const ctx = scenario === 'iterator' ? await harness(new IteratorConstructionFailureAdapter()) : await harness()
|
||||
const agent = ctx.agentLoop.create(SessionId(`request-boundary-${scenario}`), { provider: 'mock', model: 'mock' })
|
||||
let seen = ''
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => {
|
||||
seen = error.code ?? ''
|
||||
return next()
|
||||
})
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
expect(seen).toBe(scenario === 'iterator' ? 'ITERATOR_CONSTRUCTION' : 'NO_ADAPTER')
|
||||
}
|
||||
})
|
||||
|
||||
it('tracks consecutive retry attempts and resets after a successful request', async () => {
|
||||
const capped = new FailureScriptAdapter([contextError('first overflow'), contextError('second overflow')])
|
||||
const cappedCtx = await harness(capped)
|
||||
const cappedAgent = cappedCtx.agentLoop.create(SessionId('retry-cap'), { provider: 'mock', model: 'mock' })
|
||||
const cappedAttempts: number[] = []
|
||||
cappedCtx.on('agent/request-error', async (_agent, _turn, _step, _error, attempt, _signal, next) => {
|
||||
cappedAttempts.push(attempt)
|
||||
return attempt < 1 ? { action: 'retry' } : next()
|
||||
})
|
||||
send(cappedAgent)
|
||||
await waitForIdle(cappedCtx, cappedAgent)
|
||||
expect(cappedAttempts).toEqual([0, 1])
|
||||
|
||||
const reset = new FailureScriptAdapter([
|
||||
contextError('first overflow'),
|
||||
toolCallResponse('retry-reset-call', 'work', {}),
|
||||
contextError('later overflow'),
|
||||
])
|
||||
const resetCtx = await harness(reset)
|
||||
resetCtx.tools.register(defineTool({
|
||||
name: 'work',
|
||||
description: 'continue',
|
||||
parameters: {},
|
||||
async execute() { return [{ type: 'text', text: 'worked' }] },
|
||||
}))
|
||||
const resetAgent = resetCtx.agentLoop.create(SessionId('retry-reset'), { provider: 'mock', model: 'mock' })
|
||||
const resetAttempts: { step: number; attempt: number }[] = []
|
||||
resetCtx.on('agent/request-error', async (_agent, _turn, step, _error, attempt, _signal, next) => {
|
||||
resetAttempts.push({ step, attempt })
|
||||
return resetAttempts.length === 1 ? { action: 'retry' } : next()
|
||||
})
|
||||
send(resetAgent)
|
||||
await waitForIdle(resetCtx, resetAgent)
|
||||
expect(resetAttempts).toEqual([{ step: 1, attempt: 0 }, { step: 3, attempt: 0 }])
|
||||
})
|
||||
|
||||
it('preserves the original provider error when recovery throws', async () => {
|
||||
const adapter = new FailureScriptAdapter([contextError('original overflow')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('recovery-throws'), { provider: 'mock', model: 'mock' })
|
||||
ctx.on('agent/request-error', () => { throw new Error('recovery exploded') })
|
||||
|
||||
send(agent)
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'error', message: 'original overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } },
|
||||
})
|
||||
})
|
||||
|
||||
it.each(['cancel', 'dispose'] as const)('keeps %s live through request recovery', async (action) => {
|
||||
const adapter = new FailureScriptAdapter([contextError()])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId(`${action}-recovery`), { provider: 'mock', model: 'mock' })
|
||||
let entered!: () => void
|
||||
const recoveryEntered = new Promise<void>((resolve) => { entered = resolve })
|
||||
ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, signal) => {
|
||||
entered()
|
||||
await new Promise<void>((resolve) => {
|
||||
signal.addEventListener('abort', () => { resolve() }, { once: true })
|
||||
})
|
||||
return { action: 'retry' }
|
||||
})
|
||||
|
||||
send(agent)
|
||||
const idle = waitForIdle(ctx, agent)
|
||||
await recoveryEntered
|
||||
if (action === 'cancel') {
|
||||
agent.cancel('cancelled during recovery')
|
||||
await idle
|
||||
} else {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
|
||||
expect(adapter.requests).toHaveLength(1)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: action === 'cancel' ? { kind: 'aborted', reason: 'cancelled during recovery' } : { kind: 'disposed' } },
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -469,8 +469,16 @@ describe('tool-call scheduler: abort handling', () => {
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(gated.started).toEqual([])
|
||||
expect(events(agent).filter(e => e.type === 'tool/call')).toEqual([])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result')).toEqual([])
|
||||
expect(events(agent).filter(e => e.type === 'tool/call').map(e => e.data.callId))
|
||||
.toEqual([CallId('c1'), CallId('c2')])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result').map(e => ({
|
||||
callId: e.data.callId,
|
||||
isError: e.data.isError,
|
||||
error: e.data.error,
|
||||
}))).toEqual([
|
||||
{ callId: CallId('c1'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } },
|
||||
{ callId: CallId('c2'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('skips dispatch and stops starting siblings when abort fires during ordered pre-execute', async () => {
|
||||
@@ -494,10 +502,15 @@ describe('tool-call scheduler: abort handling', () => {
|
||||
|
||||
expect(gated.started).toEqual([])
|
||||
expect(events(agent).filter(e => e.type === 'tool/call').map(e => e.data.callId))
|
||||
.toEqual([CallId('c1')])
|
||||
const results = events(agent).filter(e => e.type === 'tool/result')
|
||||
expect(results.map(e => e.data.callId)).toEqual([CallId('c1')])
|
||||
expect(results[0]?.data.error).toEqual({ name: 'AbortError', code: 'ABORTED' })
|
||||
.toEqual([CallId('c1'), CallId('c2')])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result').map(e => ({
|
||||
callId: e.data.callId,
|
||||
isError: e.data.isError,
|
||||
error: e.data.error,
|
||||
}))).toEqual([
|
||||
{ callId: CallId('c1'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } },
|
||||
{ callId: CallId('c2'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('stops replenishing after abort, commits started results, and drains accepted additional contexts', async () => {
|
||||
@@ -523,12 +536,17 @@ describe('tool-call scheduler: abort handling', () => {
|
||||
|
||||
expect(gated.started).toEqual(['1', '2'])
|
||||
expect(events(agent).filter(e => e.type === 'tool/call').map(e => e.data.callId))
|
||||
.toEqual([CallId('c1'), CallId('c2')])
|
||||
.toEqual([CallId('c1'), CallId('c2'), CallId('c3'), CallId('c4')])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result').map(e => e.data.callId))
|
||||
.toEqual([CallId('c1'), CallId('c2')])
|
||||
.toEqual([CallId('c1'), CallId('c2'), CallId('c3'), CallId('c4')])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result').slice(-2).map(e => e.data))
|
||||
.toEqual([
|
||||
expect.objectContaining({ callId: CallId('c3'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } }),
|
||||
expect.objectContaining({ callId: CallId('c4'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } }),
|
||||
])
|
||||
const settled = events(agent).filter(e => e.type === 'tool/result' || e.type === 'context/message')
|
||||
expect(settled.map(e => e.type))
|
||||
.toEqual(['tool/result', 'tool/result', 'context/message', 'context/message'])
|
||||
.toEqual(['tool/result', 'tool/result', 'tool/result', 'tool/result', 'context/message', 'context/message'])
|
||||
expect(settled.filter(e => e.type === 'context/message')
|
||||
.map(e => (e.data.content[0] as { text: string }).text))
|
||||
.toEqual(['ctx-c1', 'ctx-c2'])
|
||||
@@ -564,6 +582,8 @@ describe('tool-call scheduler: abort handling', () => {
|
||||
|
||||
expect(exclusive).toEqual([])
|
||||
expect(events(agent).filter(e => e.type === 'tool/call').map(e => e.data.callId))
|
||||
.toEqual([CallId('c1'), CallId('c2')])
|
||||
.toEqual([CallId('c1'), CallId('c2'), CallId('c3')])
|
||||
expect(events(agent).filter(e => e.type === 'tool/result').at(-1)?.data)
|
||||
.toMatchObject({ callId: CallId('c3'), isError: true, error: { name: 'AbortError', code: 'ABORTED' } })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -44,7 +44,7 @@ Agent *creation* is provided by the plugin implementing `AgentFactory` (`dsh-age
|
||||
|
||||
The lifecycle edges have two important local caveats. `agent/created` runs after scoped setup and after both session and agent registry entries exist. Setup is trusted composition-only code; the immediately following non-vetoing `agent/session-start` notification is the first supported startup injection point. `agent/disposed` always means the exact agent has left the registry. AgentLoop emits it after its driver is quiescent, while ordered teardown may still be detaching the session and unwinding the scope; custom agents registered directly own any stronger driver-ordering contract themselves.
|
||||
|
||||
Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` is a serial surface-mutation checkpoint, while `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale is in the [agent-scope runtime-design RFC](../../../docs/rfc/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way).
|
||||
Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` and `agent/post-step` are serial checkpoints around a step's durable work, while `agent/request-error` is the failed-model-request recovery waterfall: a retry opens a new numbered step after the failed step closes. `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale for scoped dispatch and terminal settlement is in the [agent-scope runtime-design RFC](../../../docs/rfc/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way).
|
||||
|
||||
`PromptDecision.additionalContexts` is an array so every injected context keeps its own source, envelope, and metadata. A `ContinuationDecision` reason is narrower: it becomes a `steering/message`, not a `context/message`, and therefore carries only content and source.
|
||||
|
||||
@@ -90,4 +90,3 @@ The handle every plugin programs against:
|
||||
- **No public step-only abort** — `cancel()` clears ALL pending work (queued + steering + in-flight); an abort that preserves queued prompts returns only with a named consumer ([stop-surface RFC](../../../docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md)).
|
||||
- **`HookContext` carries exactly one `MessageSource`** — contributions from several plugins merged onto one tool call collapse under one source; mixed provenance is unrepresentable.
|
||||
- **`SessionStartSource` reserves `'clear'`/`'compact'` with no emitter yet** — only `'startup'`/`'resume'` occur until the driving subsystems land (`TODO(compaction)`).
|
||||
- **`agent/pre-step`'s `fullSystemPrompt`/`sessionPrefix` parameters are a flagged smell** — compaction is their only consumer; a lazy prompt provider or a compaction-specific pressure seam is the marked revisit.
|
||||
|
||||
@@ -70,6 +70,12 @@ export type ContinuationDecision =
|
||||
| { action: 'stop' }
|
||||
| { action: 'continue'; reason?: { content: ContentBlock[]; source: MessageSource } }
|
||||
|
||||
/** Failed-request recovery decision; `retry` opens another numbered step while listeners delegate by calling `next()`. */
|
||||
export type RequestErrorDecision = { action: 'fail' } | { action: 'retry' }
|
||||
|
||||
/** Model-request failure with an optional machine-routable provider code. */
|
||||
export type RequestError = Error & { code?: string }
|
||||
|
||||
/**
|
||||
* The terminal subset of {@link ContinuationDecision}. A listener on
|
||||
* `agent/turn-stop` returns this to make the already-composed continuation
|
||||
@@ -185,23 +191,17 @@ declare module 'cordis' {
|
||||
|
||||
// ---- step/request extension seams (serial + waterfall) ----
|
||||
/**
|
||||
* Awaited serial checkpoint for session-surface mutation after prompt
|
||||
* assembly and before `step/start`; appends land outside the pending step.
|
||||
* The loop derives history once afterward, so compaction records and
|
||||
* replacements are included without rewriting an assembled request. The
|
||||
* prompt and prefix are the exact pressure inputs for that request, and
|
||||
* Awaited serial checkpoint before `step/start`; appends land outside the
|
||||
* pending step and are included when the loop derives request history.
|
||||
* `signal` cancels listener work.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @param agent - the agent opening the step.
|
||||
* @param turn - the open turn number.
|
||||
* @param step - the pending step number.
|
||||
* @param fullSystemPrompt - the assembled prompt.
|
||||
* @param sessionPrefix - the frozen request prefix.
|
||||
* @param signal - the turn abort signal.
|
||||
* @mode serial
|
||||
*/
|
||||
// TODO: Move prompt-pressure inputs behind a compaction-specific seam if no second consumer appears.
|
||||
'agent/pre-step'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise<void> | void
|
||||
'agent/pre-step'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void
|
||||
/**
|
||||
* Allow, rewrite, or block one drained prompt before it becomes a user
|
||||
* message. Call `next()` for the unchanged default.
|
||||
@@ -229,9 +229,9 @@ declare module 'cordis' {
|
||||
* result is computed once per loop instance, logged on its anchoring request
|
||||
* header, and reused so the provider prefix remains stable. Interrupted
|
||||
* composition is discarded. Composition precedes the first `agent/pre-step`
|
||||
* and request boundary, so listener appends join the current request and
|
||||
* pressure accounting sees the composed prefix. Changing context belongs in
|
||||
* history; contributors should prepend to `await next()` to preserve registration order.
|
||||
* and request boundary, so listener appends join the current request.
|
||||
* Changing context belongs in history; contributors should prepend to
|
||||
* `await next()` to preserve registration order.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @param agent - the agent whose session prefix is being composed.
|
||||
* @param prefix - the frozen seed; return an extended replacement.
|
||||
@@ -250,6 +250,32 @@ declare module 'cordis' {
|
||||
* @mode waterfall
|
||||
*/
|
||||
'agent/step-result'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, message: Message, next: () => Promise<Message>): Promise<Message>
|
||||
/**
|
||||
* Awaited serial checkpoint after the response, real or synthetic tool
|
||||
* results, injected context, and steering are durable but before `step/end`.
|
||||
* A cancelled tool batch reaches this checkpoint with an aborted signal.
|
||||
* @param agent - the agent whose step is settling.
|
||||
* @param turn - the open turn number.
|
||||
* @param step - the open step number.
|
||||
* @param signal - the turn abort signal.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @mode serial
|
||||
*/
|
||||
'agent/post-step'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void
|
||||
/**
|
||||
* Recover a model-request failure after its failed step has closed. `retry`
|
||||
* opens a new numbered step; `fail` preserves the original request error.
|
||||
* Call `next()` to delegate to the next recovery listener or the default.
|
||||
* @param agent - the agent whose request failed.
|
||||
* @param turn - the open turn number.
|
||||
* @param step - the failed step number.
|
||||
* @param error - the original model-request failure.
|
||||
* @param retryAttempt - zero-based number of prior recovery retries.
|
||||
* @param signal - the turn abort signal.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @mode waterfall
|
||||
*/
|
||||
'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise<RequestErrorDecision>): Promise<RequestErrorDecision>
|
||||
/**
|
||||
* Override whether the turn continues. The default continues after tool
|
||||
* calls or steering and stops otherwise; a continue reason becomes steering.
|
||||
|
||||
@@ -42,7 +42,7 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
|
||||
## Errors
|
||||
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), `HTTP_<status>` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: <REASON>}` chunks.
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: <REASON>}` chunks.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { serializeRequest } from './serialize.ts'
|
||||
import type { RequestDefaults } from './serialize.ts'
|
||||
@@ -38,12 +38,17 @@ export interface DeepSeekAdapterOptions {
|
||||
/**
|
||||
* Map an HTTP status to a stable LlmError code.
|
||||
* @param status - status of a non-2xx provider response.
|
||||
* @returns `AUTH` (401/403), `RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), or `HTTP_<status>` for anything else.
|
||||
* @param error - parsed provider error body, when available.
|
||||
* @returns the normalized harness error code.
|
||||
*/
|
||||
export function httpErrorCode(status: number): string {
|
||||
export function httpErrorCode(status: number, error?: WireError['error']): string {
|
||||
if (status === 401 || status === 403) return 'AUTH'
|
||||
if (status === 429) return 'RATE_LIMIT'
|
||||
if (status === 400) return 'INVALID_REQUEST'
|
||||
if (status === 400) {
|
||||
const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ')
|
||||
if (isContextWindowExceededError(detail)) return CONTEXT_WINDOW_EXCEEDED_CODE
|
||||
return 'INVALID_REQUEST'
|
||||
}
|
||||
if (status >= 500) return 'SERVER'
|
||||
return `HTTP_${status}`
|
||||
}
|
||||
@@ -92,16 +97,17 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const code = httpErrorCode(response.status)
|
||||
let message = `DeepSeek API error (HTTP ${response.status})`
|
||||
let providerError: WireError['error']
|
||||
try {
|
||||
const parsed = await response.json() as WireError
|
||||
if (parsed.error?.message) message = parsed.error.message
|
||||
providerError = parsed.error
|
||||
if (providerError?.message) message = providerError.message
|
||||
} catch {
|
||||
// Only swallow error-body parsing: the stable code and status-line message
|
||||
// are already captured, so malformed gateway JSON must not mask the failure.
|
||||
// Only swallow error-body parsing: the HTTP status still identifies the
|
||||
// failure, so malformed gateway JSON must not mask it.
|
||||
}
|
||||
throw new LlmError(message, code)
|
||||
throw new LlmError(message, httpErrorCode(response.status, providerError))
|
||||
}
|
||||
if (!response.body) {
|
||||
throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE')
|
||||
|
||||
@@ -2,7 +2,7 @@ import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { DeepSeekAdapter } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { httpErrorCode } from '../src/adapter.ts'
|
||||
@@ -170,6 +170,32 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
).resolves.toBe(code)
|
||||
})
|
||||
|
||||
it('classifies a thrown HTTP context-window rejection with the canonical code', async () => {
|
||||
const server = await mockServer([{
|
||||
kind: 'http-error',
|
||||
status: 400,
|
||||
body: JSON.stringify({
|
||||
error: {
|
||||
message: 'This model maximum context length is 128000 tokens; your input exceeds that limit.',
|
||||
type: 'invalid_request_error',
|
||||
code: 'context_length_exceeded',
|
||||
},
|
||||
}),
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
const code = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
.catch((error: unknown) => (error as LlmError).code)
|
||||
expect(code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
})
|
||||
|
||||
it('classifies only context-capacity HTTP 400 details as context overflow', () => {
|
||||
expect(httpErrorCode(400, { message: 'request too large for model context' }))
|
||||
.toBe(CONTEXT_WINDOW_EXCEEDED_CODE)
|
||||
expect(httpErrorCode(400, { message: 'invalid input: temperature exceeds maximum allowed value' }))
|
||||
.toBe('INVALID_REQUEST')
|
||||
expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413')
|
||||
})
|
||||
|
||||
it('keeps the status-line message for JSON error bodies without a message', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
@@ -43,7 +43,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
|
||||
## Vocabulary differences
|
||||
|
||||
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks.
|
||||
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks. Provider-specific error text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
|
||||
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
|
||||
|
||||
|
||||
@@ -115,7 +115,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
// Harness-owned and therefore win collisions.
|
||||
headers: requestHeaders(profile.headers),
|
||||
})
|
||||
yield* toStreamChunks(events)
|
||||
yield* toStreamChunks(events, model.contextWindow)
|
||||
} finally {
|
||||
options.signal?.removeEventListener('abort', onCallerAbort)
|
||||
controller.abort('consumer stopped streaming')
|
||||
|
||||
@@ -8,8 +8,9 @@
|
||||
* @module dsh-llm-pi-ai/stream
|
||||
*/
|
||||
|
||||
import { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { isContextOverflow } from '@earendil-works/pi-ai'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai'
|
||||
import { toPiReplayState } from './replay.ts'
|
||||
|
||||
@@ -38,9 +39,24 @@ function classifyPiAiError(message: string): string {
|
||||
/**
|
||||
* Map a terminal pi-ai event to the harness finish reason.
|
||||
* @param message - the assistant message carried by the `done` or `error` event.
|
||||
* @returns the harness reason; `error` yields `{kind: 'error'}` with a code classified from the error text.
|
||||
* @param contextWindow - resolved catalog capacity for usage-based overflow detection.
|
||||
* @returns the mapped harness reason. Recognized error text, `stop` usage above
|
||||
* `contextWindow`, and zero-output `length` usage that fills the window map
|
||||
* to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
*/
|
||||
export function mapStopReason(message: AssistantMessage): FinishReason {
|
||||
export function mapStopReason(message: AssistantMessage, contextWindow?: number): FinishReason {
|
||||
const piAiOverflow = isContextOverflow(message, contextWindow)
|
||||
const harnessOverflow = message.stopReason === 'error'
|
||||
&& message.errorMessage !== undefined
|
||||
&& isContextWindowExceededError(message.errorMessage)
|
||||
if (piAiOverflow || harnessOverflow) {
|
||||
return {
|
||||
kind: 'error',
|
||||
message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
}
|
||||
}
|
||||
|
||||
switch (message.stopReason) {
|
||||
case 'stop': return { kind: 'stop' }
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
@@ -58,10 +74,14 @@ export function mapStopReason(message: AssistantMessage): FinishReason {
|
||||
* mid-stream — failures arrive as `error` events, which become error/aborted
|
||||
* `finish` chunks (the harness protocol's other error-delivery style).
|
||||
* @param events - one assistant turn's pi-ai event stream.
|
||||
* @param contextWindow - resolved catalog capacity for usage-based overflow detection.
|
||||
* @returns the harness chunks, ending with `usage` then `finish`; throws
|
||||
* `LlmError` (`STREAM_CLOSED`) if the source ends without a terminal event.
|
||||
*/
|
||||
export async function* toStreamChunks(events: AsyncIterable<AssistantMessageEvent>): AsyncGenerator<StreamChunk> {
|
||||
export async function* toStreamChunks(
|
||||
events: AsyncIterable<AssistantMessageEvent>,
|
||||
contextWindow?: number,
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
// pi-ai contentIndex ↔ our block index map 1:1 (both count blocks from 0
|
||||
// in stream order), but we track ids per index for tool calls.
|
||||
const toolIds = new Map<number, { id: string; name: string }>()
|
||||
@@ -124,13 +144,17 @@ export async function* toStreamChunks(events: AsyncIterable<AssistantMessageEven
|
||||
break
|
||||
case 'done':
|
||||
yield { type: 'usage', usage: mapUsage(event.message.usage) }
|
||||
yield { type: 'finish', reason: mapStopReason(event.message), replayState: toPiReplayState(event.message) }
|
||||
yield {
|
||||
type: 'finish',
|
||||
reason: mapStopReason(event.message, contextWindow),
|
||||
replayState: toPiReplayState(event.message),
|
||||
}
|
||||
return
|
||||
case 'error':
|
||||
// In-stream error delivery (pi-ai's style) → error finish chunk
|
||||
// (the harness's other sanctioned error path besides throwing).
|
||||
yield { type: 'usage', usage: mapUsage(event.error.usage) }
|
||||
yield { type: 'finish', reason: mapStopReason(event.error) }
|
||||
yield { type: 'finish', reason: mapStopReason(event.error, contextWindow) }
|
||||
return
|
||||
// no default: AssistantMessageEvent is pi-ai's closed union; a new
|
||||
// event type should fail compilation here via tsc's exhaustiveness
|
||||
|
||||
@@ -2,9 +2,10 @@ import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { getModels } from '@earendil-works/pi-ai'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
|
||||
@@ -178,6 +179,29 @@ describe('PiAiAdapter provider routing', () => {
|
||||
const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.finish).toMatchObject({ kind: 'error', code })
|
||||
})
|
||||
|
||||
it('uses the resolved catalog context window for usage-based overflow detection', async () => {
|
||||
const model = getModels('deepseek').find(candidate => candidate.id === 'deepseek-v4-flash')
|
||||
if (model === undefined) throw new Error('deepseek-v4-flash missing from pi-ai test catalog')
|
||||
const events = [
|
||||
'{"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}',
|
||||
JSON.stringify({
|
||||
choices: [{ delta: {}, index: 0, finish_reason: 'stop' }],
|
||||
usage: { prompt_tokens: model.contextWindow + 1, completion_tokens: 0 },
|
||||
}),
|
||||
'[DONE]',
|
||||
]
|
||||
const server = await mockServer([{ events }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await assemble(ctx, { model: model.id, messages: [] })
|
||||
|
||||
expect(result.finish).toEqual({
|
||||
kind: 'error',
|
||||
message: `pi-ai detected context overflow for model "${model.id}"`,
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('provider profile lifecycle', () => {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai'
|
||||
import { toPiContext } from '../src/context.ts'
|
||||
@@ -523,6 +523,46 @@ describe('mapStopReason / mapUsage', () => {
|
||||
.toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' })))
|
||||
.toMatchObject({ kind: 'error', code: 'SERVER' })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: input exceeds the model context window limit',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: request too large for model context',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value',
|
||||
}))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' })
|
||||
})
|
||||
|
||||
it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => {
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'prompt is too long: 213462 tokens > 200000 maximum',
|
||||
}))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'ThrottlingException: Too many tokens, rate limit reached',
|
||||
}))).toMatchObject({ kind: 'error', code: 'RATE_LIMIT' })
|
||||
})
|
||||
|
||||
it('uses the resolved context window for silent and length-stop overflows', () => {
|
||||
const silent = assistant({ stopReason: 'stop', usage: usage(101, 0) })
|
||||
expect(mapStopReason(silent)).toEqual({ kind: 'stop' })
|
||||
expect(mapStopReason(silent, 100)).toEqual({
|
||||
kind: 'error',
|
||||
message: 'pi-ai detected context overflow for model "deepseek-v4-flash"',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
})
|
||||
|
||||
const truncated = assistant({ stopReason: 'length', usage: usage(80, 0, 19) })
|
||||
expect(mapStopReason(truncated)).toEqual({ kind: 'max-tokens' })
|
||||
expect(mapStopReason(truncated, 100)).toMatchObject({
|
||||
kind: 'error',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
})
|
||||
})
|
||||
|
||||
it('maps cache fields only when nonzero', () => {
|
||||
|
||||
@@ -13,6 +13,8 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
### Events
|
||||
@@ -46,6 +48,7 @@ Every product adapter sends application identity on provider HTTP requests. `att
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract.
|
||||
- `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail.
|
||||
|
||||
### Real adapters
|
||||
|
||||
@@ -57,7 +60,7 @@ None, as this adapter registry forwards an already assembled request without add
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No retry/caching/rate-limit layer ships** — `llm/stream` is the intended wrap seam and has no production listener, so provider 429/5xx failures surface immediately.
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([RFC](../../../docs/rfc/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
|
||||
67
packages/llm/llm/src/adapter-failure.ts
Normal file
67
packages/llm/llm/src/adapter-failure.ts
Normal file
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* Private provider-failure tagging shared by `LlmService` and its consumers.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm/adapter-failure
|
||||
*/
|
||||
|
||||
import { HarnessError } from './error.ts'
|
||||
import type { StreamChunk } from './types.ts'
|
||||
|
||||
/** Errors proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakSet<Error>
|
||||
|
||||
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
|
||||
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
|
||||
|
||||
/**
|
||||
* Bind one call's adapter-failure scope to a unique returned stream handle.
|
||||
* @param stream - the waterfall-selected stream for this call.
|
||||
* @param failures - errors tagged by this call's final adapter boundary.
|
||||
* @returns a unique stream handle that delegates iteration to `stream`.
|
||||
* @internal
|
||||
*/
|
||||
export function bindAdapterFailureScope(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
failures: AdapterFailureScope,
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const call = {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return stream[Symbol.asyncIterator]()
|
||||
},
|
||||
}
|
||||
adapterFailureScopes.set(call, failures)
|
||||
return call
|
||||
}
|
||||
|
||||
/**
|
||||
* Preserve an adapter's Error identity while tagging its provider origin.
|
||||
* @param failures - the call-local final-adapter failure scope.
|
||||
* @param value - arbitrary value thrown by adapter dispatch or iteration.
|
||||
* @returns the original Error, or a coded Error wrapping a non-Error throw.
|
||||
* @internal
|
||||
*/
|
||||
export function markLlmAdapterFailure(
|
||||
failures: AdapterFailureScope,
|
||||
value: unknown,
|
||||
): Error & { code?: string } {
|
||||
const error = value instanceof Error
|
||||
? value as Error & { code?: string }
|
||||
: new HarnessError(String(value), 'UNKNOWN', { cause: value })
|
||||
failures.add(error)
|
||||
return error
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a failure came from final adapter dispatch, iterator construction,
|
||||
* or iteration for the call represented by the exact returned stream handle.
|
||||
* @param stream - the exact stream returned by the model call being classified.
|
||||
* @param value - arbitrary failure caught by a model-call consumer.
|
||||
* @returns true only for errors tagged at that call's final adapter boundary.
|
||||
*/
|
||||
export function isLlmAdapterFailure(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
value: unknown,
|
||||
): value is Error & { code?: string } {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error && failures !== undefined && failures.has(value)
|
||||
}
|
||||
@@ -21,6 +21,47 @@ export class HarnessError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
/** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */
|
||||
export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED'
|
||||
|
||||
/** Structured codes and plain phrases that explicitly name a context bound being exceeded. */
|
||||
const STRUCTURED_CONTEXT_OVERFLOW = new RegExp(
|
||||
String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]`
|
||||
+ String.raw`(?:exceed(?:ed|s)?|overflow(?:ed)?|limit[\s_-]exceeded)(?:$|[^a-z0-9])`,
|
||||
'i',
|
||||
)
|
||||
|
||||
/** Request-size wording that ties "too large" directly to model context capacity. */
|
||||
const TOO_LARGE_FOR_CONTEXT = new RegExp(
|
||||
String.raw`\b(?:request|prompt|input|messages?)\s+(?:is\s+|are\s+)?`
|
||||
+ String.raw`too\s+(?:large|long)\s+for\s+(?:(?:this|the)\s+)?`
|
||||
+ String.raw`(?:model(?:'s)?\s+)?context(?:\s+window)?\b`,
|
||||
'i',
|
||||
)
|
||||
|
||||
/** "Exceeds" wording is safe only when its object is explicitly the model context. */
|
||||
const EXCEEDS_MODEL_CONTEXT = new RegExp(
|
||||
String.raw`\b(?:input|prompt|request|messages?)\b.{0,40}`
|
||||
+ String.raw`\b(?:exceed(?:s|ed)?|overflows?|is\s+larger\s+than)\b.{0,40}`
|
||||
+ String.raw`\b(?:the\s+)?(?:model(?:'s)?\s+)?context(?:\s+(?:length|window))?\b`,
|
||||
'i',
|
||||
)
|
||||
|
||||
/**
|
||||
* Recognize the context-overflow wording used by OpenAI-compatible providers
|
||||
* and library adapters. Adapters pass all available provider code, type, and
|
||||
* message text so both thrown and in-band delivery styles share one classifier.
|
||||
* @param detail - provider error code/type/message text joined into one string.
|
||||
* @returns true when the detail identifies a request exceeding the model context window.
|
||||
*/
|
||||
export function isContextWindowExceededError(detail: string): boolean {
|
||||
return STRUCTURED_CONTEXT_OVERFLOW.test(detail)
|
||||
|| /\b(?:maximum|max)(?:\s+(?:allowed|supported))?\s+context\s+(?:length|window)\b/i.test(detail)
|
||||
|| TOO_LARGE_FOR_CONTEXT.test(detail)
|
||||
|| /\b(?:input|prompt|request)\s+(?:is\s+)?too\s+(?:long|large)\s+for\s+(?:this|the)\s+model\b/i.test(detail)
|
||||
|| EXCEEDS_MODEL_CONTEXT.test(detail)
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams).
|
||||
* @param value - the caught value (`unknown` in catch clauses).
|
||||
|
||||
@@ -8,8 +8,10 @@
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { deepFreeze } from './call-config.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
|
||||
import type { AdapterFailureScope } from './adapter-failure.ts'
|
||||
|
||||
export * from './attribution.ts'
|
||||
export * from './brand.ts'
|
||||
@@ -19,6 +21,7 @@ export * from './types.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export { isLlmAdapterFailure } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -196,20 +199,72 @@ export class LlmService extends Service {
|
||||
return Object.isFrozen(options) ? deepFreeze(filtered) : filtered
|
||||
}
|
||||
|
||||
/**
|
||||
* Final adapter boundary. It tags only failures from adapter selection,
|
||||
* synchronous dispatch, iterator construction, or iteration while preserving
|
||||
* the original Error object. Middleware outside this generator remains
|
||||
* distinguishable as plugin work. An iteration failure skips adapter cleanup
|
||||
* so it cannot suppress the primary provider error. A downstream close awaits
|
||||
* adapter cleanup, whose failures remain ordinary untagged work.
|
||||
*/
|
||||
private async * adapterStream(
|
||||
options: GenerateOptions,
|
||||
failures: AdapterFailureScope,
|
||||
): AsyncGenerator<StreamChunk> {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const adapter = this.registration(options.provider).adapter
|
||||
const stream = adapter.stream(this.forAdapter(options, adapter))
|
||||
iterator = stream[Symbol.asyncIterator]()
|
||||
} catch (error: unknown) {
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
}
|
||||
|
||||
let completed = false
|
||||
let iterationFailed = false
|
||||
try {
|
||||
while (true) {
|
||||
let value: StreamChunk
|
||||
try {
|
||||
const item = await iterator.next()
|
||||
if (item.done) {
|
||||
completed = true
|
||||
return
|
||||
}
|
||||
value = item.value
|
||||
} catch (error: unknown) {
|
||||
iterationFailed = true
|
||||
throw markLlmAdapterFailure(failures, error)
|
||||
}
|
||||
// End the adapter-owned try before yielding: consumer/middleware
|
||||
// failures resumed into this generator must remain untagged.
|
||||
yield value
|
||||
}
|
||||
} finally {
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- the iteration catch sets its latch before entering finally.
|
||||
if (!completed && !iterationFailed) {
|
||||
const close = iterator.return?.bind(iterator)
|
||||
if (close) await close()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as raw chunks (token-level deltas). Throws
|
||||
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
|
||||
* `options.provider`. Replay state is retained only when the same adapter
|
||||
* instance owns its historical provider and the target provider. Dispatches
|
||||
* through the `llm/stream` waterfall.
|
||||
* instance owns its historical provider and the target provider. Final
|
||||
* adapter selection, dispatch, and iteration failures retain their original
|
||||
* Error identity and are tagged in a call-local scope for narrow agent-loop
|
||||
* request recovery; middleware and nested-call failures remain untagged for
|
||||
* the outer call.
|
||||
* @param options - the full request; `options.provider` selects the adapter.
|
||||
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
|
||||
*/
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return this.ctx.waterfall(this, 'llm/stream', options, () => {
|
||||
const adapter = this.registration(options.provider).adapter
|
||||
return adapter.stream(this.forAdapter(options, adapter))
|
||||
})
|
||||
const failures: AdapterFailureScope = new WeakSet<Error>()
|
||||
const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures))
|
||||
return bindAdapterFailureScope(stream, failures)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { GenerateOptions, LlmAdapter, LlmError, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, {
|
||||
GenerateOptions,
|
||||
HarnessError,
|
||||
isContextWindowExceededError,
|
||||
isLlmAdapterFailure,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
@@ -22,6 +30,16 @@ class RecordingAdapter extends ScriptedAdapter {
|
||||
}
|
||||
}
|
||||
|
||||
class ThrowingAdapter extends LlmAdapter {
|
||||
constructor(private readonly failure: Error) {
|
||||
super()
|
||||
}
|
||||
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
throw this.failure
|
||||
}
|
||||
}
|
||||
|
||||
class CatalogAdapter extends ScriptedAdapter {
|
||||
constructor(
|
||||
private readonly provider: LlmProviderInfo,
|
||||
@@ -46,6 +64,22 @@ const SCRIPT: StreamChunk[] = [
|
||||
]
|
||||
|
||||
describe('LlmService', () => {
|
||||
it('recognizes structured and model-capacity context-window overflow details', () => {
|
||||
expect(isContextWindowExceededError('context_length_exceeded maximum context length')).toBe(true)
|
||||
expect(isContextWindowExceededError('context-window-overflowed')).toBe(true)
|
||||
expect(isContextWindowExceededError('This model maximum context length is 128000 tokens')).toBe(true)
|
||||
expect(isContextWindowExceededError('input is too long for this model')).toBe(true)
|
||||
expect(isContextWindowExceededError('request too large for model context')).toBe(true)
|
||||
expect(isContextWindowExceededError('input exceeds the model context window limit')).toBe(true)
|
||||
})
|
||||
|
||||
it('does not mistake unrelated input validation for context-window overflow', () => {
|
||||
expect(isContextWindowExceededError('invalid request: malformed tool arguments')).toBe(false)
|
||||
expect(isContextWindowExceededError('invalid input: temperature exceeds maximum allowed value')).toBe(false)
|
||||
expect(isContextWindowExceededError('input exceeds maximum allowed value')).toBe(false)
|
||||
expect(isContextWindowExceededError('context window size must be positive')).toBe(false)
|
||||
})
|
||||
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -59,9 +93,308 @@ describe('LlmService', () => {
|
||||
it('throws NO_ADAPTER for unregistered providers', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect((async () => {
|
||||
for await (const _ of ctx.llm.stream({ provider: 'nope', model: 'any-model', messages: [] })) { /* drain */ }
|
||||
})()).rejects.toThrow('no adapter registered')
|
||||
const stream = ctx.llm.stream({ provider: 'nope', model: 'any-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _ of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
expect(caught).toBeInstanceOf(LlmError)
|
||||
expect((caught as LlmError).code).toBe('NO_ADAPTER')
|
||||
expect((caught as LlmError).message).toContain('no adapter registered')
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
})
|
||||
|
||||
it.each(['done', 'value'] as const)('tags a throwing IteratorResult.%s getter without replacing its Error', async (field) => {
|
||||
const original = new LlmError(`${field} getter failed`, 'RESULT_GETTER_FAILED')
|
||||
const result = field === 'done' ? {} : { done: false }
|
||||
Object.defineProperty(result, field, { get: () => { throw original } })
|
||||
let cleanupLookups = 0
|
||||
const iterator: AsyncIterator<StreamChunk> = {
|
||||
next: () => Promise.resolve(result as unknown as IteratorResult<StreamChunk>),
|
||||
}
|
||||
Object.defineProperty(iterator, 'return', {
|
||||
get: () => {
|
||||
cleanupLookups += 1
|
||||
throw new Error('return getter must not run after iteration fails')
|
||||
},
|
||||
})
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return iterator
|
||||
},
|
||||
}
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
expect(cleanupLookups).toBe(0)
|
||||
})
|
||||
|
||||
it.each(['dispatch', 'iterator'] as const)('tags synchronous adapter %s failures without replacing their Error', async (boundary) => {
|
||||
const original = new LlmError(`${boundary} failed`, 'BOUNDARY_FAILED')
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (boundary === 'dispatch') throw original
|
||||
return { [Symbol.asyncIterator]: () => { throw original } }
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps a nested adapter failure scoped to the nested model call', async () => {
|
||||
const original = new LlmError('nested provider failed', 'NESTED_FAILED')
|
||||
const outer = new RecordingAdapter(SCRIPT)
|
||||
const nested = new ThrowingAdapter(original)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['outer'], outer)
|
||||
ctx.llm.registerAdapter(['nested'], nested)
|
||||
let nestedStream: AsyncIterable<StreamChunk> | undefined
|
||||
ctx.on('llm/stream', (options, next) => {
|
||||
if (options.provider !== 'outer') return next()
|
||||
return (async function* () {
|
||||
nestedStream = ctx.llm.stream({ provider: 'nested', model: 'nested', messages: [] })
|
||||
yield * nestedStream
|
||||
})()
|
||||
})
|
||||
|
||||
const outerStream = ctx.llm.stream({ provider: 'outer', model: 'outer', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of outerStream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(nestedStream).toBeDefined()
|
||||
expect(isLlmAdapterFailure(nestedStream!, caught)).toBe(true)
|
||||
expect(isLlmAdapterFailure(outerStream, caught)).toBe(false)
|
||||
expect(outer.lastOptions).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps call scopes distinct when middleware reuses an iterable', async () => {
|
||||
const firstFailure = new LlmError('first provider failed', 'FIRST_FAILED')
|
||||
const secondFailure = new LlmError('second provider failed', 'SECOND_FAILED')
|
||||
const delegates: AsyncIterable<StreamChunk>[] = []
|
||||
const shared: AsyncIterable<StreamChunk> = {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
const delegate = delegates.shift()
|
||||
if (delegate === undefined) throw new Error('shared stream has no call delegate')
|
||||
return delegate[Symbol.asyncIterator]()
|
||||
},
|
||||
}
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['first'], new ThrowingAdapter(firstFailure))
|
||||
ctx.llm.registerAdapter(['second'], new ThrowingAdapter(secondFailure))
|
||||
ctx.on('llm/stream', (_options, next) => {
|
||||
delegates.push(next())
|
||||
return shared
|
||||
})
|
||||
|
||||
const firstStream = ctx.llm.stream({ provider: 'first', model: 'first', messages: [] })
|
||||
const secondStream = ctx.llm.stream({ provider: 'second', model: 'second', messages: [] })
|
||||
const catchFailure = async (stream: AsyncIterable<StreamChunk>): Promise<unknown> => {
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
return error
|
||||
}
|
||||
return new Error('expected adapter to fail')
|
||||
}
|
||||
|
||||
expect(firstStream).not.toBe(secondStream)
|
||||
const firstCaught = await catchFailure(firstStream)
|
||||
expect(firstCaught).toBe(firstFailure)
|
||||
expect(isLlmAdapterFailure(firstStream, firstCaught)).toBe(true)
|
||||
expect(isLlmAdapterFailure(secondStream, firstCaught)).toBe(false)
|
||||
const secondCaught = await catchFailure(secondStream)
|
||||
expect(secondCaught).toBe(secondFailure)
|
||||
expect(isLlmAdapterFailure(secondStream, secondCaught)).toBe(true)
|
||||
expect(isLlmAdapterFailure(firstStream, secondCaught)).toBe(false)
|
||||
expect(delegates).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('propagates a rejected next promptly without awaiting a non-settling return', async () => {
|
||||
const original = new LlmError('provider failed', 'PROVIDER_FAILED')
|
||||
let cleanupCalls = 0
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return {
|
||||
next: () => Promise.reject(original),
|
||||
return: () => {
|
||||
cleanupCalls += 1
|
||||
return new Promise<IteratorResult<StreamChunk>>(() => {})
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
const failure = (async (): Promise<unknown> => {
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
return error
|
||||
}
|
||||
return new Error('expected adapter iteration to fail')
|
||||
})()
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
const timeout = new Promise<Error>((resolve) => {
|
||||
timer = setTimeout(() => { resolve(new Error('adapter failure did not settle promptly')) }, 100)
|
||||
})
|
||||
const caught = await Promise.race([failure, timeout])
|
||||
if (timer !== undefined) clearTimeout(timer)
|
||||
|
||||
expect(caught).toBe(original)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
expect(cleanupCalls).toBe(0)
|
||||
})
|
||||
|
||||
it('awaits one adapter return on downstream close and leaves its rejection unclassified', async () => {
|
||||
const cleanup = new Error('cleanup failed')
|
||||
let cleanupCalls = 0
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return {
|
||||
next: () => Promise.resolve({ done: false, value: SCRIPT[0]! }),
|
||||
return: () => {
|
||||
cleanupCalls += 1
|
||||
return Promise.reject(cleanup)
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) break
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(cleanup)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(false)
|
||||
expect(cleanupCalls).toBe(1)
|
||||
})
|
||||
|
||||
it('allows downstream close when the adapter iterator has no return method', async () => {
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
return { next: () => Promise.resolve({ done: false, value: SCRIPT[0]! }) }
|
||||
},
|
||||
}
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
let chunks = 0
|
||||
for await (const _chunk of ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })) {
|
||||
chunks += 1
|
||||
break
|
||||
}
|
||||
|
||||
expect(chunks).toBe(1)
|
||||
})
|
||||
|
||||
it('normalizes and tags non-Error adapter failures once', async () => {
|
||||
const adapter = new class extends LlmAdapter {
|
||||
stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
return {
|
||||
[Symbol.asyncIterator](): AsyncIterator<StreamChunk> {
|
||||
// Third-party adapters can reject with arbitrary values.
|
||||
// eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors
|
||||
return { next: () => Promise.reject('plain provider failure') }
|
||||
},
|
||||
}
|
||||
}
|
||||
}()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], adapter)
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBeInstanceOf(HarnessError)
|
||||
expect(caught).toMatchObject({ code: 'UNKNOWN', cause: 'plain provider failure' })
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(true)
|
||||
})
|
||||
|
||||
it('does not tag a failure thrown downstream while consuming adapter output', async () => {
|
||||
const downstream = new Error('consumer failed')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
const stream = ctx.llm.stream({ provider: 'test-model', model: 'test-model', messages: [] })
|
||||
let caught: unknown
|
||||
try {
|
||||
for await (const _chunk of stream) throw downstream
|
||||
} catch (error: unknown) {
|
||||
caught = error
|
||||
}
|
||||
|
||||
expect(caught).toBe(downstream)
|
||||
expect(isLlmAdapterFailure(stream, caught)).toBe(false)
|
||||
expect(isLlmAdapterFailure(new ScriptedAdapter(SCRIPT).stream({
|
||||
provider: 'unbound', model: 'unbound', messages: [],
|
||||
}), caught)).toBe(false)
|
||||
expect(isLlmAdapterFailure(stream, 'consumer failed')).toBe(false)
|
||||
})
|
||||
|
||||
it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => {
|
||||
|
||||
@@ -37,11 +37,10 @@ export type { AgentUnderTest } from './launcher.ts'
|
||||
* (random) session id into a `{{sessionId}}` variable that later steps
|
||||
* reference, since a committed file cannot know the id in advance.
|
||||
*
|
||||
* `promptAndCancel` sends a prompt WITHOUT awaiting its response, waits until
|
||||
* the client observes the first streamed `agent_message_chunk` (so the emitted
|
||||
* frames deterministically precede the cancellation), then cancels the turn —
|
||||
* the only way to exercise a cancel deterministically (a plain `prompt` step
|
||||
* awaits the response, which a cancel/hang scenario would block on forever).
|
||||
* `promptAndCancel` starts a prompt without awaiting completion, waits until
|
||||
* the client observes the selected update (`agent_message_chunk` by default),
|
||||
* then cancels and awaits completion. A named `waitForToolCallUpdate` keeps the
|
||||
* step open for a terminal tool update that may follow the prompt response.
|
||||
*/
|
||||
export type InputStep =
|
||||
| { op: 'initialize'; terminalOutput?: boolean }
|
||||
@@ -49,7 +48,12 @@ export type InputStep =
|
||||
| { op: 'newSessionExpectError'; additionalDirectories?: string[] }
|
||||
| { op: 'prompt'; text: string }
|
||||
| { op: 'promptExpectError'; text: string }
|
||||
| { op: 'promptAndCancel'; text: string }
|
||||
| {
|
||||
op: 'promptAndCancel'
|
||||
text: string
|
||||
afterUpdate?: 'agent_message_chunk' | 'tool_call'
|
||||
waitForToolCallUpdate?: string
|
||||
}
|
||||
| { op: 'cancel' }
|
||||
| { op: 'setConfigOption'; configId: string; value: string }
|
||||
| { op: 'setConfigOptionExpectError'; configId: string; value: string }
|
||||
@@ -342,17 +346,19 @@ async function runStep(
|
||||
case 'promptAndCancel': {
|
||||
const sessionId = getSessionId()
|
||||
if (sessionId === undefined) throw new Error('snapshot-harness: promptAndCancel before newSession')
|
||||
// Dispatch the prompt WITHOUT awaiting (a hang fixture never resolves on
|
||||
// its own). To pin frame order deterministically, wait until the client
|
||||
// has OBSERVED the hang's streamed agent_message_chunk before cancelling —
|
||||
// so those update frames always precede the cancelled prompt response in
|
||||
// the transcript (without this, the late chunk and the response race).
|
||||
// Then cancel and await the prompt, which the bridge settles as
|
||||
// `cancelled` once the abort propagates.
|
||||
// Dispatch without awaiting because the fixture does not settle on its
|
||||
// own. Waiting for the selected update pins it before cancellation and
|
||||
// the cancelled prompt response in the transcript.
|
||||
const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] })
|
||||
await waitForUpdate(u => u.sessionUpdate === 'agent_message_chunk')
|
||||
const afterUpdate = step.afterUpdate ?? 'agent_message_chunk'
|
||||
await waitForUpdate(u => u.sessionUpdate === afterUpdate)
|
||||
// Arm this before cancellation so a fast tool drain cannot outrun the waiter.
|
||||
const toolCallUpdateDone = step.waitForToolCallUpdate === undefined
|
||||
? undefined
|
||||
: waitForUpdate(u => u.sessionUpdate === 'tool_call_update' && u.toolCallId === step.waitForToolCallUpdate)
|
||||
await client.cancel({ sessionId })
|
||||
await promptDone
|
||||
if (toolCallUpdateDone !== undefined) await toolCallUpdateDone
|
||||
return
|
||||
}
|
||||
case 'cancel': {
|
||||
|
||||
@@ -33,6 +33,10 @@ interface Behavior {
|
||||
rejectExtraDirs?: boolean
|
||||
/** How `session/prompt` settles: a clean response, a JSON-RPC error, or a hang until `session/cancel`. */
|
||||
prompt?: 'respond' | 'error' | 'hang-until-cancel'
|
||||
/** Emit a tool call instead of a message chunk before parking a cancellable prompt. */
|
||||
cancelAtToolCall?: boolean
|
||||
/** Emit the parked tool call's terminal update after answering cancellation. */
|
||||
cancelToolCallUpdate?: boolean
|
||||
/** Before responding to a prompt, send a `session/request_permission` request and echo its outcome as a chunk. */
|
||||
permissionProbe?: boolean
|
||||
/** Echo the `DSH_SNAPSHOT_*` env the harness set as a chunk (spec-side env-plumbing assertions). */
|
||||
@@ -126,7 +130,23 @@ async function handlePrompt(id: number | string): Promise<void> {
|
||||
params: { sessionId, update: { sessionUpdate: 'agent_thought_chunk', content: { type: 'text', text: 'mulling' } } },
|
||||
})
|
||||
}
|
||||
chunk('thinking about it')
|
||||
if (behavior.cancelAtToolCall === true) {
|
||||
send({
|
||||
method: 'session/update',
|
||||
params: {
|
||||
sessionId,
|
||||
update: {
|
||||
sessionUpdate: 'tool_call',
|
||||
toolCallId: 'call_fake_1',
|
||||
title: 'fake tool',
|
||||
kind: 'execute',
|
||||
status: 'in_progress',
|
||||
},
|
||||
},
|
||||
})
|
||||
} else {
|
||||
chunk('thinking about it')
|
||||
}
|
||||
if (behavior.echoEnv === true) {
|
||||
chunk(`env:${JSON.stringify({
|
||||
mode: process.env.DSH_SNAPSHOT,
|
||||
@@ -229,6 +249,19 @@ function handleFrame(frame: Record<string, unknown>): void {
|
||||
const parked = parkedPromptId
|
||||
parkedPromptId = null
|
||||
respond(parked, { stopReason: 'cancelled' })
|
||||
if (behavior.cancelToolCallUpdate === true) {
|
||||
send({
|
||||
method: 'session/update',
|
||||
params: {
|
||||
sessionId,
|
||||
update: {
|
||||
sessionUpdate: 'tool_call_update',
|
||||
toolCallId: 'call_fake_1',
|
||||
status: 'failed',
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
return
|
||||
default:
|
||||
|
||||
@@ -334,6 +334,28 @@ describe('runScenario', () => {
|
||||
expect(result.rawStdout.indexOf('thinking about it')).toBeLessThan(result.rawStdout.indexOf('cancelled'))
|
||||
})
|
||||
|
||||
it('promptAndCancel can bracket cancellation with tool-call updates', { timeout: 20_000 }, async () => {
|
||||
const { fixtureFile } = await scenario({
|
||||
prompt: 'hang-until-cancel',
|
||||
cancelAtToolCall: true,
|
||||
cancelToolCallUpdate: true,
|
||||
})
|
||||
const result = await runScenario(
|
||||
{
|
||||
steps: [...boot, {
|
||||
op: 'promptAndCancel',
|
||||
text: 'hang',
|
||||
afterUpdate: 'tool_call',
|
||||
waitForToolCallUpdate: 'call_fake_1',
|
||||
}],
|
||||
},
|
||||
{ agent: AGENT, mode: 'replay', fixtureFile },
|
||||
)
|
||||
expect(result.rawStdout).toContain('"sessionUpdate":"tool_call"')
|
||||
expect(result.rawStdout.indexOf('"sessionUpdate":"tool_call"')).toBeLessThan(result.rawStdout.indexOf('cancelled'))
|
||||
expect(result.rawStdout.indexOf('cancelled')).toBeLessThan(result.rawStdout.indexOf('"sessionUpdate":"tool_call_update"'))
|
||||
})
|
||||
|
||||
it('promptExpectError swallows a model-error response as the expected outcome', { timeout: 20_000 }, async () => {
|
||||
const { fixtureFile } = await scenario({ prompt: 'error' })
|
||||
const result = await runScenario(
|
||||
|
||||
@@ -30,10 +30,12 @@ const scopedSubjectResolvers = Object.freeze({
|
||||
'agent/created': adapt<'agent/created'>(args => args[0]),
|
||||
'agent/disposed': adapt<'agent/disposed'>(args => args[0]),
|
||||
'agent/error': adapt<'agent/error'>(args => args[0]),
|
||||
'agent/post-step': adapt<'agent/post-step'>(args => args[0]),
|
||||
'agent/pre-step': adapt<'agent/pre-step'>(args => args[0]),
|
||||
'agent/prompt-submit': adapt<'agent/prompt-submit'>(args => args[0]),
|
||||
'agent/queued': adapt<'agent/queued'>(args => args[0]),
|
||||
'agent/request': adapt<'agent/request'>(args => args[0]),
|
||||
'agent/request-error': adapt<'agent/request-error'>(args => args[0]),
|
||||
'agent/session-prefix': adapt<'agent/session-prefix'>(args => args[0]),
|
||||
'agent/session-start': adapt<'agent/session-start'>(args => args[0]),
|
||||
'agent/status': adapt<'agent/status'>(args => args[0]),
|
||||
|
||||
@@ -814,7 +814,7 @@ describe('scoped-dispatch invariants', () => {
|
||||
['agent/status', [agent, 'idle']],
|
||||
['agent/queued', [agent, [], { source: { kind: 'user' }, steering: false }]],
|
||||
['agent/session-start', [agent, 'startup']],
|
||||
['agent/pre-step', [agent, 1, 1, '', new AbortController().signal]],
|
||||
['agent/pre-step', [agent, 1, 1, new AbortController().signal]],
|
||||
['agent/prompt-submit', [agent, [], { kind: 'user' }, () => Promise.resolve({ kind: 'allow' })]],
|
||||
['agent/request', [agent, 1, 1, { model: 'm' }, () => Promise.resolve({ model: 'm' })]],
|
||||
['agent/session-prefix', [agent, [], new AbortController().signal, () => Promise.resolve([])]],
|
||||
|
||||
@@ -371,7 +371,7 @@ describe('approval policy (the approval/policy fold)', () => {
|
||||
}
|
||||
|
||||
const preStep = (ctx: Context, agent: Agent): Promise<void> =>
|
||||
ctx.serial('agent/pre-step', agent, 1, 1, '', [], new AbortController().signal)
|
||||
ctx.serial('agent/pre-step', agent, 1, 1, new AbortController().signal)
|
||||
|
||||
/** Append a `request/header` snapshot whose system text is exactly `system`. */
|
||||
function appendHeader(session: Session, system: string): void {
|
||||
|
||||
Reference in New Issue
Block a user