refactor(llm): drop the image content block until a path can honor it

ImageBlock had no production producer and every consumer dropped it:
the deepseek serializer skipped it, the pi-ai converter skipped it as
unrepresentable, the ACP bridge neither advertises image prompt
capability nor forwards image blocks, and compact-basic charged a flat
85-token estimate and rendered an [image] placeholder. A block
constructed today would silently vanish from the wire — the vocabulary
advertised a capability no path honors, the silent-data-loss shape the
defensive patterns warn against. The only constructors were tests
pinning the skip/estimate branches.

Remove ImageBlock and its ContentBlockMap entry (its cache?: CacheHint
field leaves with it; CacheHint itself and the other two cache? fields
are out of scope). compact-basic loses its explicit image estimate and
placeholder arms (the merge-extensible default arms absorb the case);
the deepseek serializer, pi-ai converter, and ACP codec already handled
image in their default arms, so only their image-naming comments
change. The codec's inbound rejection of ACP-protocol image prompt
content stays — that guards wire content a client can send regardless
of our vocabulary.

Tests that constructed harness image blocks to pin the removed branches
are dropped (the 85-token estimate pin) or retargeted onto plugin-added
block types / other non-text blocks, which the surviving default arms
own. Docs, the type-equiv pastes, and the content-block vocabulary
RFC's block list and multimodal-home consequence are updated in the
same change; the RFC moves to implemented/ and the index is
regenerated. A real multimodal feature reintroduces image via
declaration merging together with the adapter mapping, ACP
advertisement, and compaction pricing that honor it.
This commit is contained in:
Tianyi Cui
2026-07-04 17:21:13 +08:00
parent 226a8b5e4c
commit 2745879132
25 changed files with 97 additions and 107 deletions

View File

@@ -69,8 +69,8 @@ export function turnEndToStopReason(reason: TurnEndReason): StopReason {
* client as message content. Today only `text` maps; `resource_link` is an
* ACP prompt-only input rendered into text by {@link acpPromptToText};
* `reasoning` is surfaced via `agent_thought_chunk`
* streaming rather than as a message block, and `tool-call`/`tool-result`/
* `image` are handled by the tool-call update path or not advertised.
* streaming rather than as a message block, and `tool-call`/`tool-result`
* are handled by the tool-call update path.
*/
export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock | undefined {
switch (block.type) {
@@ -78,7 +78,7 @@ export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock |
return { type: 'text', text: block.text }
// reasoning → streamed as agent_thought_chunk, not a message block
// tool-call / tool-result → the tool_call / tool_call_update path
// image → not advertised
// plugin-added block types → not surfaced
default:
return undefined
}

View File

@@ -1,4 +1,5 @@
import { describe, expect, it } from 'vitest'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { TurnEndReason } from '@deepseek-ai/dsh-session'
import type { ContentBlock as AcpContentBlock } from '@agentclientprotocol/sdk'
import {
@@ -33,9 +34,9 @@ describe('harnessBlockToAcpContent', () => {
expect(harnessBlockToAcpContent({ type: 'text', text: 'hi' })).toEqual({ type: 'text', text: 'hi' })
})
it('returns undefined for non-text blocks (reasoning/tool/image)', () => {
it('returns undefined for non-text blocks (reasoning / plugin-added)', () => {
expect(harnessBlockToAcpContent({ type: 'reasoning', text: 'think' })).toBeUndefined()
expect(harnessBlockToAcpContent({ type: 'image', url: 'https://x/y.png', mimeType: 'image/png' })).toBeUndefined()
expect(harnessBlockToAcpContent({ type: 'chart', data: 'x' } as unknown as ContentBlock)).toBeUndefined()
})
})

View File

@@ -91,7 +91,7 @@ describe('streamSessionEventUpdate', () => {
it('drops non-text tool-result content (text-only)', () => {
const update = updatesFor(evt('tool/result', {
turn: 1, step: 1, callId: CallId('c1'),
content: [{ type: 'image', url: 'https://x/y.png' }],
content: [{ type: 'reasoning', text: 'private' }],
isError: false,
}))[0]
expect((update as { content: unknown[] }).content).toEqual([])