simplify(llm): drop unconsumed adapter-change event and assembled call surfaces
The LLM service exposed three call surfaces (stream/streamBlocks/generate) but the only production consumer — the agent loop — uses stream() exclusively, feeding raw chunks through its own BlockAssembler for replay fidelity. Drop the speculative convenience surfaces and the registry-change event that no listener consumed, leaving stream() as the single model-call contract for both production and tests. - Remove LlmService.streamBlocks() and generate(), the llm/generate waterfall, and GenerateResult. - Remove the llm/adapter-change event (declaration + emits) and the listener-throw rollback ordering that existed only to protect it; keep the HMR rollback disposer. - Remove BlockAssembler.flushReady()/flushRemaining()/result() and the flushed cursor — the streaming-flush slice existed only for streamBlocks(). - Adapter tests drive a stream()+BlockAssembler helper (tests/assemble.ts) instead of generate(), exercising the same path production uses. - Land the AGENTS.md "RFCs are proposals, not golden truth" principle and move both RFCs proposed -> implemented. Implements: - docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md - docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md
This commit is contained in:
@@ -4,28 +4,24 @@ Provider-neutral LLM vocabulary and abstract service. This package defines the c
|
||||
|
||||
## Service: `LlmService` (ctx key: `llm`)
|
||||
|
||||
An adapter registry plus streaming / non-streaming call surfaces. Both call surfaces are interceptable via waterfall events.
|
||||
An adapter registry plus a single streaming call surface, interceptable via a waterfall event.
|
||||
|
||||
### Public API
|
||||
|
||||
- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber.
|
||||
- `ctx.llm.models(): string[]` — model names with a registered adapter.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas).
|
||||
- `ctx.llm.streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock>` Stream as completed content blocks (convenience view).
|
||||
- `ctx.llm.generate(options: GenerateOptions): Promise<GenerateResult>` One model call, fully assembled.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
### Events
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
|---|---|---|
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) |
|
||||
| `llm/generate` | waterfall | Intercept/wrap every non-streaming model call |
|
||||
| `llm/adapter-change` | emit | An adapter was registered or unregistered |
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider.
|
||||
- Wrap `llm/stream` or `llm/generate` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -36,8 +32,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
### Classes
|
||||
|
||||
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. Used by the agent loop (raw chunks for replay
|
||||
+ assembled for history) and by `streamBlocks()`/`generate()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response.
|
||||
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
/**
|
||||
* Incremental chunk-to-message assembler. This is the single canonical assembly
|
||||
* algorithm used by both the agent loop and the LLM service convenience views.
|
||||
* algorithm used by the agent loop to build an assistant message from a chunk
|
||||
* stream while logging the raw chunks for replay fidelity.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm/assembler
|
||||
*/
|
||||
|
||||
import { CallId } from './brand.ts'
|
||||
import { assertNever } from './never.ts'
|
||||
import type { ContentBlock, FinishReason, GenerateResult, Message, StreamChunk, TokenUsage } from './types.ts'
|
||||
import type { ContentBlock, FinishReason, Message, StreamChunk, TokenUsage } from './types.ts'
|
||||
|
||||
interface PartialBlock {
|
||||
blockType: string
|
||||
@@ -23,9 +24,8 @@ interface PartialBlock {
|
||||
* Incrementally assembles raw {@link StreamChunk}s into complete
|
||||
* {@link ContentBlock}s and a final assistant {@link Message}.
|
||||
*
|
||||
* This is the single shared assembly implementation: the agent loop feeds it
|
||||
* while logging raw chunks for replay fidelity, and `LlmService.generate()` /
|
||||
* `streamBlocks()` use it to offer assembled views of the same stream.
|
||||
* The agent loop feeds it while logging raw chunks for replay fidelity, then
|
||||
* reads `blocks()` / `message()` / `usage` / `finish` once the stream ends.
|
||||
*
|
||||
* Tolerant of delta-only protocols (no block-start/end); deltas arriving for
|
||||
* an index already closed by `block-end` are ignored (malformed stream) so a
|
||||
@@ -34,7 +34,6 @@ interface PartialBlock {
|
||||
export class BlockAssembler {
|
||||
private partials = new Map<number, PartialBlock>()
|
||||
private order: number[] = []
|
||||
private flushed = 0
|
||||
private _usage: TokenUsage | undefined
|
||||
private _finish: FinishReason | undefined
|
||||
|
||||
@@ -129,44 +128,6 @@ export class BlockAssembler {
|
||||
return this.order.map(index => this.assemble(this.mustGet(index), index))
|
||||
}
|
||||
|
||||
/**
|
||||
* Streaming flush: returns (once) every block that is complete AND has no
|
||||
* incomplete block before it in stream order. Call after each `push()`;
|
||||
* blocks come out strictly in stream order, so a streaming consumer sees
|
||||
* exactly the sequence `blocks()` would produce.
|
||||
*/
|
||||
flushReady(): ContentBlock[] {
|
||||
const ready: ContentBlock[] = []
|
||||
while (this.flushed < this.order.length) {
|
||||
const index = this.order[this.flushed]
|
||||
/* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists in a non-empty array */
|
||||
if (index === undefined) break
|
||||
const partial = this.mustGet(index)
|
||||
if (!partial.block) break
|
||||
ready.push(partial.block)
|
||||
this.flushed += 1
|
||||
}
|
||||
return ready
|
||||
}
|
||||
|
||||
/**
|
||||
* End-of-stream flush: returns (once) all not-yet-flushed blocks, in stream
|
||||
* order, assembling still-open ones from their deltas (delta-only
|
||||
* protocols). After this, `flushReady()` + `flushRemaining()` together have
|
||||
* yielded exactly `blocks()`.
|
||||
*/
|
||||
flushRemaining(): ContentBlock[] {
|
||||
const remaining: ContentBlock[] = []
|
||||
while (this.flushed < this.order.length) {
|
||||
const index = this.order[this.flushed]
|
||||
/* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists */
|
||||
if (index === undefined) break
|
||||
remaining.push(this.assemble(this.mustGet(index), index))
|
||||
this.flushed += 1
|
||||
}
|
||||
return remaining
|
||||
}
|
||||
|
||||
get usage(): TokenUsage | undefined {
|
||||
return this._usage
|
||||
}
|
||||
@@ -179,13 +140,4 @@ export class BlockAssembler {
|
||||
message(): Message {
|
||||
return { role: 'assistant', content: this.blocks() }
|
||||
}
|
||||
|
||||
/** The assembled non-streaming result. */
|
||||
result(): GenerateResult {
|
||||
return {
|
||||
message: this.message(),
|
||||
...this._usage !== undefined ? { usage: this._usage } : {},
|
||||
finish: this.finish,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
/**
|
||||
* LLM service: adapter registry with waterfall-interceptable streaming and
|
||||
* non-streaming call surfaces. Exports the `LlmService` default, the abstract
|
||||
* `LlmAdapter` for provider backends, and `BlockAssembler` for chunk assembly.
|
||||
* LLM service: adapter registry with a waterfall-interceptable streaming call
|
||||
* surface. Exports the `LlmService` default, the abstract `LlmAdapter` for
|
||||
* provider backends, and `BlockAssembler` for chunk assembly.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { ContentBlock, GenerateOptions, GenerateResult, StreamChunk } from './types.ts'
|
||||
import { BlockAssembler } from './assembler.ts'
|
||||
import type { GenerateOptions, StreamChunk } from './types.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
|
||||
export * from './brand.ts'
|
||||
@@ -30,17 +29,6 @@ declare module 'cordis' {
|
||||
* @mode waterfall
|
||||
*/
|
||||
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
|
||||
/**
|
||||
* Waterfall around every non-streaming model call. Bound to the
|
||||
* {@link LlmService}; call `next()` to delegate to the adapter.
|
||||
* @mode waterfall
|
||||
*/
|
||||
'llm/generate'(this: LlmService, options: GenerateOptions, next: () => Promise<GenerateResult>): Promise<GenerateResult>
|
||||
/**
|
||||
* An adapter was registered or unregistered (the model→adapter map changed).
|
||||
* @mode emit
|
||||
*/
|
||||
'llm/adapter-change'(): void
|
||||
}
|
||||
}
|
||||
|
||||
@@ -88,8 +76,7 @@ export class LlmService extends Service {
|
||||
/**
|
||||
* Register an adapter for the given model names. Throws `LlmError` with code
|
||||
* `DUPLICATE_ADAPTER` if any model already has an adapter (all-or-nothing).
|
||||
* Emits `llm/adapter-change` on registration and disposal. Disposed with the
|
||||
* fiber.
|
||||
* Disposed with the fiber.
|
||||
*/
|
||||
registerAdapter(models: string[], adapter: LlmAdapter): () => void {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
@@ -99,17 +86,9 @@ export class LlmService extends Service {
|
||||
}
|
||||
}
|
||||
for (const model of models) this.adapters.set(model, adapter)
|
||||
// Yield the rollback BEFORE emitting the change event: a generator effect
|
||||
// collects each yielded disposer before running the next step, so a
|
||||
// throwing `llm/adapter-change` listener rolls the mutation back instead
|
||||
// of leaking the entry (which would wedge the duplicate check until
|
||||
// restart). The duplicate throws above fire before any mutation, so they
|
||||
// correctly leak nothing.
|
||||
yield () => {
|
||||
for (const model of models) this.adapters.delete(model)
|
||||
this.ctx.emit('llm/adapter-change')
|
||||
}
|
||||
this.ctx.emit('llm/adapter-change')
|
||||
}.bind(this), 'llm.registerAdapter()')
|
||||
// ctx.effect's disposer returns Promise<void>; our disposer API is
|
||||
// synchronous fire-and-forget — discard the (always-resolved) promise.
|
||||
@@ -137,36 +116,6 @@ export class LlmService extends Service {
|
||||
return this.adapter(options.model).stream(options)
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as completed content blocks — a convenience view
|
||||
* for consumers that don't care about token-level deltas. Blocks are
|
||||
* yielded strictly in stream order as soon as they (and everything before
|
||||
* them) complete; blocks left open at end of stream (delta-only protocols)
|
||||
* are assembled and flushed last, so the sequence always equals
|
||||
* `generate()`'s `message.content`.
|
||||
*/
|
||||
async * streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock> {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of this.stream(options)) {
|
||||
assembler.push(chunk)
|
||||
yield * assembler.flushReady()
|
||||
}
|
||||
yield * assembler.flushRemaining()
|
||||
}
|
||||
|
||||
/**
|
||||
* One model call, fully assembled (drains the chunk stream). Dispatches
|
||||
* through the `llm/generate` waterfall (and the inner stream through
|
||||
* `llm/stream`). Same completion guarantees as `streamBlocks()`.
|
||||
*/
|
||||
generate(options: GenerateOptions): Promise<GenerateResult> {
|
||||
return this.ctx.waterfall(this, 'llm/generate', options, async () => {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of this.stream(options)) assembler.push(chunk)
|
||||
return assembler.result()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
|
||||
@@ -193,10 +193,3 @@ export interface GenerateOptions {
|
||||
stop?: string[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** Non-streaming result, assembled from the chunk stream. */
|
||||
export interface GenerateResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
|
||||
@@ -81,36 +81,6 @@ describe('BlockAssembler', () => {
|
||||
expect(() => assembler.blocks()).toThrow('BlockAssembler invariant violated')
|
||||
})
|
||||
|
||||
it('assembles open blocks at end of stream via flushRemaining', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'open' })
|
||||
assembler.push({ type: 'reasoning-delta', index: 1, text: 'thinking' })
|
||||
|
||||
// flushReady returns nothing because index 0 is incomplete and blocking
|
||||
const ready = assembler.flushReady()
|
||||
expect(ready).toEqual([])
|
||||
|
||||
// flushRemaining assembles everything still open
|
||||
const remaining = assembler.flushRemaining()
|
||||
expect(remaining).toEqual([
|
||||
{ type: 'text', text: 'open' },
|
||||
{ type: 'reasoning', text: 'thinking' },
|
||||
])
|
||||
|
||||
// blocks() now matches the flushed view
|
||||
expect(assembler.blocks()).toEqual(remaining)
|
||||
})
|
||||
|
||||
it('result() omits usage key when no usage was received', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'msg' })
|
||||
const result = assembler.result()
|
||||
expect(result.message).toBeDefined()
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
// usage should NOT be present on the object at all
|
||||
expect('usage' in result).toBe(false)
|
||||
})
|
||||
|
||||
it('ignores duplicate block-start for the same index', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'block-start', index: 0, blockType: 'text' })
|
||||
@@ -142,13 +112,11 @@ describe('BlockAssembler', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('includes usage in result() when usage was received', () => {
|
||||
it('exposes usage via the getter when a usage chunk was received', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'msg' })
|
||||
assembler.push({ type: 'usage', usage: { inputTokens: 5, outputTokens: 3 } })
|
||||
const result = assembler.result()
|
||||
expect(result.usage).toEqual({ inputTokens: 5, outputTokens: 3 })
|
||||
expect('usage' in result).toBe(true)
|
||||
expect(assembler.usage).toEqual({ inputTokens: 5, outputTokens: 3 })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -172,25 +140,25 @@ describe('BlockAssembler regressions (property-test findings)', () => {
|
||||
// Found by fast-check (the property-testing RFC): two block-ends at the same index made the
|
||||
// streamed prefix (first block) disagree with final blocks() (second
|
||||
// block). The first close must win — same straggler rule as post-close
|
||||
// deltas — so streaming and one-shot assembly stay identical.
|
||||
// deltas — so the prefix returned incrementally by push() and the final
|
||||
// blocks() stay identical.
|
||||
const chunks: StreamChunk[] = [
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'first' } },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'second' } },
|
||||
]
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed = []
|
||||
const closed = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
const block = streaming.push(chunk)
|
||||
if (block) closed.push(block)
|
||||
}
|
||||
flushed.push(...streaming.flushRemaining())
|
||||
|
||||
const oneShot = new BlockAssembler()
|
||||
for (const chunk of chunks) oneShot.push(chunk)
|
||||
|
||||
expect(flushed).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(closed).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(oneShot.blocks()).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(flushed).toEqual(oneShot.blocks())
|
||||
expect(closed).toEqual(oneShot.blocks())
|
||||
})
|
||||
|
||||
it('push returns undefined for a duplicate block-end (it closed nothing)', () => {
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import fc from 'fast-check'
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
// A small pool of indices so collisions (duplicate-index bugs) are common.
|
||||
@@ -55,38 +55,6 @@ function feed(chunks: StreamChunk[]): BlockAssembler {
|
||||
}
|
||||
|
||||
describe('BlockAssembler properties', () => {
|
||||
it('flushReady() ++ flushRemaining() === blocks(), in order', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed: ContentBlock[] = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
}
|
||||
flushed.push(...streaming.flushRemaining())
|
||||
|
||||
const oneShot = feed(chunks).blocks()
|
||||
expect(flushed).toEqual(oneShot)
|
||||
}))
|
||||
})
|
||||
|
||||
it('streamBlocks-style flush never yields a block before an earlier open one', () => {
|
||||
// flushReady is strict-order: once it stops at an open index, no later
|
||||
// index may be emitted until that one closes. We assert the flushed prefix
|
||||
// is always a prefix of the final blocks() order.
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed: ContentBlock[] = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
}
|
||||
const finalSoFar = streaming.blocks()
|
||||
// Everything flushed mid-stream is a prefix of the full ordered blocks.
|
||||
expect(finalSoFar.slice(0, flushed.length)).toEqual(flushed)
|
||||
}))
|
||||
})
|
||||
|
||||
it('partials map size never exceeds the number of distinct indices seen', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const distinct = new Set<number>()
|
||||
@@ -134,13 +102,9 @@ describe('BlockAssembler properties', () => {
|
||||
|
||||
it('streaming and one-shot assembly agree on usage and finish', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
// Streaming consumer: push + flush as it goes.
|
||||
// Streaming consumer: push as it goes.
|
||||
const streaming = new BlockAssembler()
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
streaming.flushReady()
|
||||
}
|
||||
streaming.flushRemaining()
|
||||
for (const chunk of chunks) streaming.push(chunk)
|
||||
// One-shot consumer: push all, then read.
|
||||
const oneShot = feed(chunks)
|
||||
expect(streaming.usage).toEqual(oneShot.usage)
|
||||
|
||||
@@ -19,24 +19,22 @@ const SCRIPT: StreamChunk[] = [
|
||||
]
|
||||
|
||||
describe('LlmService', () => {
|
||||
it('routes stream() to the registered adapter and generate() assembles it', async () => {
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) chunks.push(chunk)
|
||||
expect(chunks).toHaveLength(3)
|
||||
|
||||
const result = await ctx.llm.generate({ model: 'test-model', messages: [] })
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(chunks).toEqual(SCRIPT)
|
||||
})
|
||||
|
||||
it('throws NO_ADAPTER for unregistered models', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.llm.generate({ model: 'nope', messages: [] })).rejects.toThrow('no adapter registered')
|
||||
await expect((async () => {
|
||||
for await (const _ of ctx.llm.stream({ model: 'nope', messages: [] })) { /* drain */ }
|
||||
})()).rejects.toThrow('no adapter registered')
|
||||
})
|
||||
|
||||
it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => {
|
||||
@@ -71,21 +69,6 @@ describe('LlmService', () => {
|
||||
expect(chunks[0]).toMatchObject({ index: 99 })
|
||||
})
|
||||
|
||||
it('lets llm/generate waterfall listeners intercept and transform the result', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
ctx.on('llm/generate', async function (_options, next) {
|
||||
const result = await next()
|
||||
return { ...result, finish: { kind: 'max-tokens' } as const }
|
||||
})
|
||||
|
||||
const result = await ctx.llm.generate({ model: 'test-model', messages: [] })
|
||||
expect(result.finish).toEqual({ kind: 'max-tokens' })
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
})
|
||||
|
||||
it('creates LlmError with a code for programmatic handling', () => {
|
||||
const err = new LlmError('something went wrong', 'CUSTOM_CODE')
|
||||
expect(err).toBeInstanceOf(Error)
|
||||
@@ -116,20 +99,13 @@ describe('LlmService', () => {
|
||||
expect(isHarnessError('nope')).toBe(false)
|
||||
})
|
||||
|
||||
it('disposes adapter registration on adapter-change event emission', async () => {
|
||||
it('removes the adapter when the returned disposer is called', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
const changes: string[][] = []
|
||||
ctx.on('llm/adapter-change', () => {
|
||||
changes.push([...ctx.llm.models()])
|
||||
})
|
||||
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(changes).toEqual([['m1']])
|
||||
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
dispose()
|
||||
expect(changes).toEqual([['m1'], []])
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
|
||||
@@ -147,25 +123,19 @@ describe('LlmService', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rolls back the adapter entry when an adapter-change listener throws (P1-1)', async () => {
|
||||
it('re-registers a model after its prior registration is disposed', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
// A change listener that throws on the FIRST emit only.
|
||||
let threw = false
|
||||
ctx.on('llm/adapter-change', () => {
|
||||
if (!threw) { threw = true; throw new Error('boom change listener') }
|
||||
})
|
||||
|
||||
// The throwing emit must roll the mutation back, not leak it.
|
||||
expect(() => ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))).toThrow('boom change listener')
|
||||
expect(ctx.llm.models()).toEqual([]) // entry rolled back, not leaked
|
||||
|
||||
// A subsequent listener-free register of the SAME model succeeds and
|
||||
// contributes exactly once (the duplicate check is not wedged).
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
|
||||
// The duplicate check is not wedged: the same model registers cleanly again.
|
||||
const disposeAgain = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
disposeAgain()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user