refactor: identify and freeze messages at creation

This commit is contained in:
_Kerman
2026-07-28 13:55:59 +08:00
parent c49c0ba497
commit fbf87e660c
345 changed files with 5220 additions and 2901 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
README.md: 2328188e420df6de60f024982a31d37a858a303e
README.zh.md: 586767a9e790fa68d27673836700fd789ce6a180
README.md: 7c34d5621d6ac644aaac17169f38709147ac1dd5
README.zh.md: 1d2475272640162beab425ac91fc93dd27b53b63

View File

@@ -38,9 +38,11 @@ Reasoning identifiers are opaque adapter-owned strings rather than a core enum.
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use bounded normal retry policy, use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)
### Messages (`message.ts`) and content blocks (`types.ts`)
Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. Assistant messages produced by the loop also carry provider/model provenance and optional adapter-private replay state. Before dispatch, `LlmService` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers. The core block set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it.
`Message` is the shared immutable value used by delivery, durable history, and model requests. Every message has a required `MessageId`, role, content, and typed source from creation onward. `createMessage(input)` mints the identity and returns a detached deep-frozen value; `createUserMessage({ content, source })` fixes the user role; `createAssistantMessage({ content, source })` fixes the assistant role and model source kind; `createToolResultMessage({ callId, content, isError })` fixes the user role and couples the tool source to its result block; `freezeMessage(message)` imports an identity that already exists and never replaces it. Message rewrites preserve the identity and produce another frozen value.
Message content is an array of typed blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. Assistant messages use a model source carrying provider/model provenance and optional adapter-private replay state. Before dispatch, `LlmService` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers. The core block set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it.
Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages.
@@ -55,7 +57,7 @@ Every product adapter sends application identity on provider HTTP requests. `att
### Classes
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and can create an identified, frozen assistant message from them. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks for history.
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) matches its frozen serializable `failure.code`. The payload may also retain validated status, `Retry-After`, and branded provider request id facts; policy remains outside the error.
- `errorChain(value)` — renders a thrown value with its full `cause` chain and AggregateError members for diagnostic surfaces (UI notices, logger lines, durable `turn/end` messages), so transport wrappers like undici's `TypeError: fetch failed` surface the underlying `ECONNREFUSED`/DNS/TLS detail instead of masking it. Rendering only — route on `code`, never by parsing the result.

View File

@@ -38,9 +38,11 @@
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方持有的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据。
- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`。
### 内容块词汇(`types.ts`)
### 消息(`message.ts`)与内容块(`types.ts`)
消息是类型化内容块数组:`text`、`reasoning`、`tool-call`、`tool-result`。联合从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加块类型。loop 产生的 assistant 消息还会携带提供方/模型溯源与可选适配器私有回放状态。dispatch 前,`LlmService` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型/提供方间恢复或转换该状态。核心块集只包含每条已发布路径都支持的块。多模态内容(图像、音频等)没有核心块类型;需要它的功能会通过 map 添加,并一并添加支持它的适配器/UI/压缩实现。
`Message` 是投递、持久历史和模型请求共享的不可变值。每条消息从创建起都必须具有 `MessageId`、角色、内容和带类型的来源。`createMessage(input)` 生成标识,并返回与输入分离且深度冻结的值;`createUserMessage({ content, source })` 固定 user 角色;`createAssistantMessage({ content, source })` 固定 assistant 角色与模型来源类别;`createToolResultMessage({ callId, content, isError })` 固定 user 角色,并将工具来源与其结果块耦合;`freezeMessage(message)` 导入已有标识,绝不将其替换。改写消息时会保留标识,并产生另一个冻结值。
消息内容是类型化内容块数组:`text`、`reasoning`、`tool-call`、`tool-result`。联合从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加块类型。assistant 消息使用模型来源,其中携带提供方/模型溯源与可选适配器私有回放状态。dispatch 前,`LlmService` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型/提供方间恢复或转换该状态。核心块集只包含每条已发布路径都支持的块。多模态内容(图像、音频等)没有核心块类型;需要它的功能会通过 map 添加,并一并添加支持它的适配器/UI/压缩实现。
流式输出是原始 chunk 协议(`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`)。`BlockAssembler` 是将 chunk 组装为块/消息的唯一共享实现。
@@ -55,7 +57,7 @@
### 类
- `LlmAdapter`:提供方适配器的抽象基类。唯一必需方法是 `stream()`。
- `BlockAssembler`:将原始 chunk 逐步组装为完整内容块与 assistant 消息。agent loop 向它提供原始 chunk(同时记录以供回放),并读取已组装块/消息以构建历史。
- `BlockAssembler`:将原始 chunk 逐步组装为完整内容块,并能据此创建带标识且冻结的 assistant 消息。agent loop 向它提供原始 chunk(同时记录以供回放),并读取已组装块以构建历史。
- `HarnessError`:harness 错误分类体系的基类,包含稳定 `code` 字符串(与面向人的 `message` 不同)加 `cause` 链接。它位于所有其他包都导入的叶子包中,因此可以共享单一基类,无需新的依赖边。每包错误(`LlmError`、`ToolArgsError`、`InvariantError` 等)都会扩展它。`isHarnessError(value)` 在 seam 处收窄类型。
- `LlmError`:扩展 `HarnessError`;其稳定 `code` 字符串(`NO_ADAPTER`、`DUPLICATE_ADAPTER` 与 `AUTH`/`RATE_LIMIT` 等适配器 code)与冻结可序列化 `failure.code` 匹配。Payload 还可以保留已验证状态、`Retry-After` 和品牌化提供方请求 id 事实;策略位于错误之外。
- `errorChain(value)`:渲染抛出值的完整 `cause` 链与 AggregateError 成员,供诊断表层使用,包括 UI 通知、logger 行和持久 `turn/end` 消息。因此 undici 的 `TypeError: fetch failed` 等传输包装层会显示底层 `ECONNREFUSED`/DNS/TLS 详细信息,而不是将其遮蔽。该函数只负责渲染:请按 `code` 路由,绝不解析结果。

View File

@@ -8,7 +8,9 @@
import { CallId } from './brand.ts'
import { assertNever } from './never.ts'
import type { ContentBlock, FinishReason, Message, StreamChunk, TokenUsage } from './types.ts'
import { createMessage } from './message.ts'
import type { Message, MessageSource } from './message.ts'
import type { ContentBlock, FinishReason, StreamChunk, TokenUsage } from './types.ts'
interface PartialBlock {
blockType: string
@@ -149,9 +151,10 @@ export class BlockAssembler {
/**
* The assembled assistant message.
* @returns an assistant-role message over `blocks()` (same open-block assembly rules).
* @param source - producer attribution for the assembled message.
* @returns a frozen assistant-role message over `blocks()` (same open-block assembly rules).
*/
message(): Message {
return { role: 'assistant', content: this.blocks() }
message(source: MessageSource = { kind: 'plugin', plugin: 'dsh-llm/assembler' }): Message {
return createMessage({ role: 'assistant', content: this.blocks(), source })
}
}

View File

@@ -12,6 +12,18 @@
import type { Branded } from '@deepseek-ai/dsh-brand'
/** Stable identity carried by one message across inbox, log, and model-request boundaries. */
export type MessageId = Branded<'MessageId'>
/**
* Brand a message identifier.
* @param id - the opaque message identifier.
* @returns the same string, branded; no validation is performed.
*/
export function MessageId(id: string): MessageId {
return id as MessageId
}
/**
* Correlates a model-issued tool call with its result. Provider-issued for
* real adapters; synthesized by mocks/assembler fallbacks.

View File

@@ -13,9 +13,9 @@ import type {
LlmModelInfo,
LlmResolvedModelInfo,
LlmProviderInfo,
Message,
StreamChunk,
} from './types.ts'
import { freezeMessage, type Message } from './message.ts'
import { resolveRetryPolicy } from './retry-policy.ts'
import type { ResolvedRetryPolicy } from './retry-policy.ts'
import type { ProviderRequestId } from './brand.ts'
@@ -30,6 +30,7 @@ export * from './brand.ts'
export * from './never.ts'
export * from './error.ts'
export * from './types.ts'
export * from './message.ts'
export * from './retry-policy.ts'
export { BlockAssembler } from './assembler.ts'
export { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from './call-config.ts'
@@ -50,7 +51,8 @@ declare module 'cordis' {
* process-local {@link markAgentLoopRequest} identity and arrives deep-frozen
* (mutation throws): its content is a pure function of the session log (the
* reconstructability Agent Note), so listeners read it, never rewrite it.
* Hand-built calls own their mutability policy and do not carry that marker.
* Hand-built calls do not carry that marker; their messages already obey
* the immutable creation contract.
* @mode waterfall
*/
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
@@ -457,13 +459,13 @@ export class LlmService extends Service {
/** Remove replay state whose historical route is owned by another adapter. */
private forAdapter(options: GenerateOptions, adapter: LlmAdapter): GenerateOptions {
const messages: Message[] = options.messages.map((message) => {
const provenance = message.provenance
if (message.role !== 'assistant' || provenance?.replayState === undefined) return message
if (this.adapters.get(provenance.provider)?.adapter === adapter) return message
return {
const source = message.source
if (message.role !== 'assistant' || source.kind !== 'model' || source.replayState === undefined) return message
if (this.adapters.get(source.provider)?.adapter === adapter) return message
return freezeMessage({
...message,
provenance: { provider: provenance.provider, model: provenance.model },
}
source: { kind: 'model', provider: source.provider, model: source.model },
})
})
if (messages.every((message, index) => message === options.messages[index])) return options
const filtered = { ...options, messages }

View File

@@ -0,0 +1,159 @@
/** Message value types, identity, and immutable construction helpers. */
import { MessageId, type CallId } from './brand.ts'
import { deepFreeze } from './call-config.ts'
import type { ContentBlock, ToolResultBlock } from './types.ts'
/** Provider ownership and adapter-private replay data for an assistant message. */
export interface AssistantProvenance {
/** Provider route that produced the message. */
provider: string
/** Provider model id that produced the message. */
model: string
/**
* Lossless-JSON adapter state needed to replay the provider response.
* `LlmService` exposes it to a target adapter only when that adapter instance
* currently owns both this historical provider and the target provider.
*/
replayState?: unknown
}
/** Required source of an assistant message produced by a routed model. */
export interface ModelMessageSource extends AssistantProvenance {
kind: 'model'
}
/** Required source of a user-role message carrying one tool result. */
export interface ToolMessageSource {
kind: 'tool'
callId: CallId
}
/**
* Where a message (or injected content) came from.
* Merge-extensible sum type — plugins add their own `kind`s.
*/
export interface MessageSourceMap {
user: { kind: 'user' }
plugin: { kind: 'plugin'; plugin: string }
model: ModelMessageSource
tool: ToolMessageSource
}
/** Any known message source, derived from {@link MessageSourceMap}; switch on `kind` and fall through unknowns (merge-extensible). */
export type MessageSource = MessageSourceMap[keyof MessageSourceMap]
/** One immutable message representation shared by delivery, durable history, and model requests. */
export interface Message {
/** Stable identity preserved across every representation boundary. */
readonly id: MessageId
/** Provider-neutral conversation role. */
readonly role: 'system' | 'user' | 'assistant'
/** Exact model-facing blocks. */
readonly content: ContentBlock[]
/** Required producer provenance. */
readonly source: MessageSource
}
/** A user-role specialization of the one shared message representation. */
export interface UserMessage extends Message {
readonly role: 'user'
}
/** A model-produced assistant specialization of the shared message representation. */
export interface AssistantMessage extends Message {
readonly role: 'assistant'
readonly source: ModelMessageSource
}
/** A tool-result specialization whose model-facing block retains call correlation. */
export interface ToolResultMessage extends Message {
readonly role: 'user'
readonly content: [ToolResultBlock]
readonly source: ToolMessageSource
}
type NewMessage = Omit<Message, 'id'>
type NewUserMessage = Omit<UserMessage, 'id' | 'role'>
type NewAssistantMessage = Omit<AssistantMessage, 'id' | 'role' | 'source'> & {
readonly source: Omit<ModelMessageSource, 'kind'> & { readonly kind?: never }
}
/**
* Detach and deep-freeze a message whose identity already exists.
* @param message - complete message, including its stable identity.
* @returns an immutable snapshot that preserves the identity.
*/
export function freezeMessage<T extends Message>(message: T): T {
return deepFreeze(structuredClone(message))
}
/**
* Create one identified message and freeze it before publication.
* @param input - complete role, content, and source for a new message.
* @returns an immutable message with a fresh stable identity.
*/
export function createMessage<T extends NewMessage>(
input: T & { readonly id?: never },
): T & Pick<Message, 'id'> {
return freezeMessage({
...input,
id: MessageId(crypto.randomUUID()),
})
}
/**
* Create one identified user-role message and freeze it before publication.
* @param input - complete content and source for a new user message.
* @returns an immutable user message with a fresh stable identity.
*/
export function createUserMessage<T extends NewUserMessage>(
input: T & { readonly id?: never; readonly role?: never },
): T & Pick<UserMessage, 'id' | 'role'> {
return createMessage({
...input,
role: 'user',
})
}
/**
* Create one identified model-produced assistant message and freeze it before publication.
* @param input - complete content and model provenance for a new assistant message.
* @returns an immutable assistant message with fixed role/source tags and a fresh stable identity.
*/
export function createAssistantMessage(
input: NewAssistantMessage & { readonly id?: never; readonly role?: never },
): AssistantMessage {
return createMessage({
content: input.content,
role: 'assistant',
source: {
...input.source,
kind: 'model',
},
})
}
/** Input whose acceptance creates one tool-result message. */
export interface ToolResultMessageInput {
readonly callId: CallId
readonly content: ContentBlock[]
readonly isError: boolean
}
/**
* Create and freeze one identified tool-result message.
* @param input - call identity, raw result blocks, and outcome.
* @returns an immutable user-role tool-result message.
*/
export function createToolResultMessage(input: ToolResultMessageInput): ToolResultMessage {
return createUserMessage({
source: { kind: 'tool', callId: input.callId },
content: [{
type: 'tool-result',
toolCallId: input.callId,
content: input.content,
isError: input.isError,
}],
})
}

View File

@@ -6,6 +6,19 @@
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
import type { Message } from './message.ts'
export type {
AssistantMessage,
AssistantProvenance,
Message,
MessageSource,
MessageSourceMap,
ModelMessageSource,
ToolMessageSource,
ToolResultMessage,
UserMessage,
} from './message.ts'
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
export interface LlmFailure {
@@ -67,43 +80,6 @@ export type ContentBlockType = keyof ContentBlockMap
/** Any known content block, derived from {@link ContentBlockMap}; switch on `type` and fall through unknowns (merge-extensible). */
export type ContentBlock = ContentBlockMap[ContentBlockType]
/** Provider ownership and adapter-private replay data for an assistant message. */
export interface AssistantProvenance {
/** Provider route that produced the message. */
provider: string
/** Provider model id that produced the message. */
model: string
/**
* Lossless-JSON adapter state needed to replay the provider response.
* `LlmService` exposes it to a target adapter only when that adapter instance
* currently owns both this historical provider and the target provider.
*/
replayState?: unknown
}
/**
* A single message in a conversation history. Loop-derived assistant messages
* always carry provenance; callers may omit it on hand-built foreign history.
*/
export interface Message {
role: 'system' | 'user' | 'assistant'
content: ContentBlock[]
/** Present only on assistant messages produced by a routed adapter. */
provenance?: AssistantProvenance
}
/**
* Where a message (or injected content) came from.
* Merge-extensible sum type — plugins add their own `kind`s.
*/
export interface MessageSourceMap {
user: { kind: 'user' }
plugin: { kind: 'plugin'; plugin: string }
}
/** Any known message source, derived from {@link MessageSourceMap}; switch on `kind` and fall through unknowns (merge-extensible). */
export type MessageSource = MessageSourceMap[keyof MessageSourceMap]
/**
* Why a model response stopped.
* Merge-extensible so adapters can surface provider-specific reasons.

View File

@@ -0,0 +1,99 @@
import { describe, expect, it } from 'vitest'
import {
CallId,
createAssistantMessage,
createToolResultMessage,
createUserMessage,
freezeMessage,
MessageId,
} from '@deepseek-ai/dsh-llm'
describe('message construction', () => {
it('assigns identity immediately and returns a detached deep-frozen message', () => {
const input = {
content: [{ type: 'text' as const, text: 'original' }],
source: { kind: 'plugin' as const, plugin: 'test' },
}
const message = createUserMessage(input)
expect(message.id).toEqual(expect.any(String))
expect(message.role).toBe('user')
expect(message.id).not.toHaveLength(0)
expect(message).not.toBe(input)
expect(Object.isFrozen(message)).toBe(true)
expect(Object.isFrozen(message.content)).toBe(true)
expect(Object.isFrozen(message.content[0])).toBe(true)
expect(Object.isFrozen(message.source)).toBe(true)
input.content[0]!.text = 'caller mutation'
expect(message.content).toEqual([{ type: 'text', text: 'original' }])
expect(() => {
(message.content[0] as { text: string }).text = 'observer mutation'
}).toThrow()
})
it('freezes an existing identity without minting a replacement', () => {
const id = MessageId('existing')
const input = {
id,
role: 'assistant' as const,
content: [{ type: 'text' as const, text: 'answer' }],
source: { kind: 'model' as const, provider: 'test', model: 'test' },
}
const message = freezeMessage(input)
expect(message).not.toBe(input)
expect(message.id).toBe(id)
expect(Object.isFrozen(message)).toBe(true)
expect(Object.isFrozen(message.content[0])).toBe(true)
})
it('fixes the assistant role and model source kind at creation', () => {
const message = createAssistantMessage({
content: [{ type: 'text', text: 'answer' }],
source: {
provider: 'test-provider',
model: 'test-model',
replayState: { request: 1 },
},
})
expect(message).toMatchObject({
role: 'assistant',
source: {
kind: 'model',
provider: 'test-provider',
model: 'test-model',
replayState: { request: 1 },
},
})
expect(message.id).not.toHaveLength(0)
expect(Object.isFrozen(message)).toBe(true)
expect(Object.isFrozen(message.source)).toBe(true)
})
it('couples tool-result content and provenance to one call identity', () => {
const callId = CallId('call-1')
const message = createToolResultMessage({
callId,
content: [{ type: 'text', text: 'result' }],
isError: false,
})
expect(message).toMatchObject({
role: 'user',
source: { kind: 'tool', callId },
content: [{
type: 'tool-result',
toolCallId: callId,
content: [{ type: 'text', text: 'result' }],
isError: false,
}],
})
expect(message.id).not.toHaveLength(0)
expect(Object.isFrozen(message)).toBe(true)
expect(Object.isFrozen(message.content[0])).toBe(true)
})
})

View File

@@ -15,6 +15,7 @@ import LlmService, {
ReasoningEffortId,
resolveRetryPolicy,
StreamChunk,
createMessage,
} from '@deepseek-ai/dsh-llm'
import type {
LlmModelContext,
@@ -175,6 +176,26 @@ describe('LlmService', () => {
expect(chunks).toEqual(SCRIPT)
})
it('trusts the immutable message creation boundary for direct calls', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new RecordingAdapter(SCRIPT)
ctx.llm.registerAdapter(['test-provider'], adapter)
const message = createMessage({
role: 'user',
content: [{ type: 'text', text: 'hello' }],
source: { kind: 'user' },
})
for await (const _chunk of ctx.llm.stream({
provider: 'test-provider',
model: 'test-model',
messages: [message],
})) { /* drain */ }
expect(adapter.lastOptions?.messages[0]).toBe(message)
})
it('captures provider-owned retry policy at registration and defaults omission', async () => {
const configured = resolveRetryPolicy({ mode: 'always' }, 'test retryPolicy')
const adapter = new class extends ScriptedAdapter {
@@ -1195,15 +1216,18 @@ describe('LlmService', () => {
for await (const _chunk of ctx.llm.stream({
provider: 'target',
model: 'new-model',
messages: [{
messages: [createMessage({
role: 'assistant',
content: [{ type: 'text', text: 'old response' }],
provenance: { provider: 'historical', model: 'old-model', replayState },
}],
source: {
kind: 'model',
...{ provider: 'historical', model: 'old-model', replayState },
},
})],
})) { /* drain */ }
expect(adapter.lastOptions?.messages[0]?.provenance).toEqual({
provider: 'historical', model: 'old-model', replayState,
expect(adapter.lastOptions?.messages[0]?.source).toEqual({
kind: 'model', provider: 'historical', model: 'old-model', replayState,
})
})
@@ -1217,14 +1241,21 @@ describe('LlmService', () => {
for await (const _chunk of ctx.llm.stream({
provider: 'target',
model: 'new-model',
messages: [{
messages: [createMessage({
role: 'assistant',
content: [{ type: 'text', text: 'old response' }],
provenance: { provider: 'historical', model: 'old-model', replayState: { private: 'state' } },
}],
source: {
kind: 'model',
...{ provider: 'historical', model: 'old-model', replayState: { private: 'state' } },
},
})],
})) { /* drain */ }
expect(target.lastOptions?.messages[0]?.provenance).toEqual({ provider: 'historical', model: 'old-model' })
expect(target.lastOptions?.messages[0]?.source).toEqual({
kind: 'model',
provider: 'historical',
model: 'old-model',
})
})
it('preserves immutability while stripping replay state from frozen requests', async () => {
@@ -1236,18 +1267,27 @@ describe('LlmService', () => {
const options = Object.freeze({
provider: 'target',
model: 'new-model',
messages: [{
messages: [createMessage({
role: 'assistant' as const,
content: [{ type: 'text' as const, text: 'old response' }],
provenance: { provider: 'historical', model: 'old-model', replayState: { private: 'state' } },
}],
source: {
kind: 'model',
provider: 'historical',
model: 'old-model',
replayState: { private: 'state' },
},
})],
})
for await (const _chunk of ctx.llm.stream(options)) { /* drain */ }
expect(target.lastOptions).not.toBe(options)
expect(Object.isFrozen(target.lastOptions)).toBe(true)
expect(target.lastOptions?.messages[0]?.provenance).toEqual({ provider: 'historical', model: 'old-model' })
expect(target.lastOptions?.messages[0]?.source).toEqual({
kind: 'model',
provider: 'historical',
model: 'old-model',
})
})
it('creates LlmError with a code for programmatic handling', () => {