Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(llm): resolve model metadata together

Browse Source
This commit is contained in:
Yichen Jiang
2026-07-26 13:07:27 +08:00
parent baea5018e5
commit 73e7e27799
55 changed files with 607 additions and 460 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
packages/ui/tui/README.md
View File

@@ -10,7 +10,7 @@ This package owns interactive terminal presentation and input only. It injects `
After terminal startup succeeds, the package provides the terminal-local `ctx.tui` extension service. A plugin that injects it can call `openOverlay()` with a component factory and constrained layout options; the host exposes the viewport, semantic theme, display-text escaping, redraw, close, and a lifetime signal, but not the pi-tui tree, terminal, focus controller, or overlay handle. Plugin overlays, the model selector, and user questions share one FIFO modal queue. Each request is an effect of the calling plugin fiber, so unload removes queued work or closes visible work before cleanup settles; terminal shutdown unloads dependents before stopping pi-tui. Overlay state is not logged or replayed. Component code is trusted and may render ANSI styling, but must pass untrusted text through `host.display()`. The [interactive-extension Agent Note](../../../.agents/notes/implemented/architecture/2026-07-22-tui-interactive-extension-service.md) owns the boundary and rejected alternatives.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. The latest logged session title becomes the header subtitle, with `welcome` before a title exists, and the terminal window title becomes `<session title> — <configured title>`. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view compares token-meter pressure with `ctx.llm.resolveModelContext()` for the current route, displays `context unknown` when the adapter has no capacity metadata, and also shows tool-card mode plus the current model and any explicitly selected reasoning effort; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. The latest logged session title becomes the header subtitle, with `welcome` before a title exists, and the terminal window title becomes `<session title> — <configured title>`. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view compares token-meter pressure with `ctx.llm.resolveModelInfo()` context for the current route, displays `context unknown` when the adapter has no capacity metadata, and also shows tool-card mode plus the current model and any explicitly selected reasoning effort; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
An embedding may provide `TuiRuntime.formatCwd` when its logical workspace label differs from the session's host directory. The override changes only the footer label; tools continue to use the session `cwd`.
@@ -22,11 +22,11 @@ When optional `ctx.sessionReferences` is mounted, the same `@` menu also offers
While the agent is running, ordinary editor submissions call `agent.steer()`; otherwise they call `agent.followup()`. A slash at the start of the submitted line enters `ctx.commands` instead: known commands execute directly, unknown commands produce a warning, and neither path automatically reaches the model. A command producer may explicitly schedule agent work; [`dsh-plan-mode`](../../plan/plan-mode/README.md#model-and-human-surfaces) uses that contract for `/plan [message]`. The TUI registers `/help`, `/model`, `/clear`, `/reasoning`, `/tools`, `/redraw`, `/reload`, `/resume`, `/status`, and `/exit` as agent-scoped definitions; every other effective command joins autocomplete and `/help` dynamically, as do `/skill:` completions. A status line above the editor reports the turn phase the TUI derives from session events — waiting for the first token, thinking, responding, or executing tools — with the elapsed time in that phase and the running step total, refreshed each second, and ends with the `Enter sends steering, Esc cancels` hint; while steering messages wait to reach the model it inserts a `N queued ·` badge before the hint that clears as each drains. Ctrl+C or Escape cancels a running turn. Tool cards collapse long bodies into a configurable head/tail preview; Ctrl+O toggles every card between its preview and full output. Ctrl+R toggles reasoning, Ctrl+L redraws, and Ctrl+D exits while idle.
`/model` opens the advisory `ctx.llm` catalog as a keyboard selector: Up/Down moves, Shift+Tab cycles the focused model's adapter-advertised reasoning efforts in display order, Enter selects the model and effort, and Escape closes it. Models without selectable effort metadata ignore Shift+Tab; the selector does not synthesize `off`, clamp a value, or transfer an effort between models. `/model <model>` still selects an unambiguous model id directly, while `/model <provider>/<model>` selects an exact target and uses its adapter default when one exists. The configured target or latest logged request header initializes the selector, and an unlisted current model remains visible because catalogs are advisory. Selection is local to this TUI session. Prompt assembly snapshots the target for one step, replaces `{{provider}}` and `{{model}}`, and applies the same provider/model/reasoning-effort target through `agent/request`; a switch during assembly therefore starts with a later step. The request header durably records targets that reach the model, while an unused selection remains process-local.
`/model` opens the advisory `ctx.llm` catalog as a keyboard selector: Up/Down moves, Shift+Tab cycles the focused model's adapter-advertised reasoning efforts in display order, Enter selects the model and effort, and Escape closes it. Models without selectable effort metadata ignore Shift+Tab; the selector renders the exact advertised list—including `off` when present—and does not synthesize, clamp, or transfer an effort between models. `/model <model>` still selects an unambiguous model id directly, while `/model <provider>/<model>` selects an exact target and uses its adapter default when one exists. The configured target or latest logged request header initializes the selector, and an unlisted current model remains visible because catalogs are advisory. Selection is local to this TUI session. Prompt assembly snapshots the target for one step, replaces `{{provider}}` and `{{model}}`, and applies the same provider/model/reasoning-effort target through `agent/request`; a switch during assembly therefore starts with a later step. The request header durably records targets that reach the model, while an unused selection remains process-local.
`/reload` (EXPERIMENTAL, dev-only) re-reads every file-backed loader config tree and applies the diff to the running app — the HMR watcher's config path, invoked manually; it needs the cordis Loader in the context and degrades to a warning without one, runs only while the agent is idle, and refuses re-entry while a reload is in flight. Module-source hot reload remains watcher-owned. When a `skills` service is mounted, `/skill:<name> [instructions]` loads that skill's instructions into the conversation as a user turn; autocomplete lists the model-invocable skills, and any skill (including a model-disabled one) is loadable by its exact name.
The footer sums the session's reported usage as `↑<uncached input> ↓<output>`, followed by `cache <rate>%` once any input has been billed — the share of billed prompt tokens (uncached input plus cache reads and writes) served from the provider cache, rounded to a percent. It also compares token-meter pressure with `ctx.llm.resolveModelContext()` for the current route (omitting the context share when the adapter has no capacity metadata) and shows the current model and tool-card mode; the right side clips first when the footer is narrow.
The footer sums the session's reported usage as `↑<uncached input> ↓<output>`, followed by `cache <rate>%` once any input has been billed — the share of billed prompt tokens (uncached input plus cache reads and writes) served from the provider cache, rounded to a percent. It also compares token-meter pressure with `ctx.llm.resolveModelInfo()` context for the current route (omitting the context share when the adapter has no capacity metadata) and shows the current model and tool-card mode; the right side clips first when the footer is narrow.
`/status` adds a point-in-time diagnostics card to the transcript and remains available while the agent runs. It reports the session id, title, working directory, selected provider/model, selected reasoning effort or default behavior, reasoning-block visibility, agent state, event/turn/step/tool-call counts, exact input/output/cache token buckets, KV-cache hit rate, token-meter context use and capacity, creation time, and latest event time. Missing titles, models, cache input, or context capacity are labeled instead of inferred. The card is terminal-only and does not duplicate the compact footer.

6
packages/ui/tui/src/index.ts
View File

@@ -630,7 +630,7 @@ async function readModelChoices(
models.push({ provider: provider.id, id: current.model, name: current.model })
}
return Promise.all(models.map(async (model): Promise<ModelChoice> => {
const reasoning = await ctx.llm.resolveModelReasoning(provider.id, model.id)
const reasoning = (await ctx.llm.resolveModelInfo(provider.id, model.id)).reasoning
return {
provider: provider.id,
model: model.id,
@@ -2101,8 +2101,8 @@ export function createTuiChat(
contextWindow = undefined
const resolution = selected === undefined
? Promise.resolve({ kind: 'resolved', contextWindow: undefined } as const)
: ctx.llm.resolveModelContext(selected.provider, selected.model).then(
context => ({ kind: 'resolved', contextWindow: context?.contextWindow } as const),
: ctx.llm.resolveModelInfo(selected.provider, selected.model).then(
info => ({ kind: 'resolved', contextWindow: info.context?.contextWindow } as const),
(error: unknown) => ({ kind: 'error', error } as const),
)
contextResolution = resolution

28
packages/ui/tui/tests/harness.ts
View File

@@ -10,10 +10,9 @@ import AgentRegistry, {
} from '@deepseek-ai/dsh-agent'
import type {
ContentBlock,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
} from '@deepseek-ai/dsh-llm'
import CommandService from '@deepseek-ai/dsh-commands'
import SessionStore, { SessionId, type Session, type SessionHeader } from '@deepseek-ai/dsh-session'
@@ -53,11 +52,10 @@ export interface TuiHarnessOptions {
providers: LlmProviderInfo[]
models: LlmModelInfo[]
listModels?: (provider: string) => Promise<LlmModelInfo[]>
resolveModelContext?: (provider: string, model: string) => Promise<LlmModelContext | undefined>
resolveModelReasoning?: (
resolveModelInfo?: (
provider: string,
model: string,
) => Promise<LlmModelReasoningInfo | undefined>
) => Promise<Pick<LlmResolvedModelInfo, 'context' | 'reasoning'>>
}
/** Provide a fake `sessionPersistence` service so resume surfaces can list sessions. */
sessionPersistence?: {
@@ -128,12 +126,20 @@ export async function createTuiTestHarness<TerminalType extends Terminal, Exit e
return catalog.listModels?.(provider)
?? Promise.resolve(catalog.models.filter(model => model.provider === provider).map(model => ({ ...model })))
},
resolveModelContext(provider: string, model: string) {
return catalog.resolveModelContext?.(provider, model)
?? Promise.resolve({ contextWindow: options.contextWindow ?? 128_000 })
},
resolveModelReasoning(provider: string, model: string) {
return catalog.resolveModelReasoning?.(provider, model) ?? Promise.resolve(undefined)
async resolveModelInfo(provider: string, model: string) {
const advertised = catalog.models.find(candidate =>
candidate.provider === provider && candidate.id === model)
const capabilities = await (catalog.resolveModelInfo?.(provider, model)
?? Promise.resolve({
context: { contextWindow: options.contextWindow ?? 128_000 },
}))
return {
provider,
id: model,
name: advertised?.name ?? model,
...advertised?.description === undefined ? {} : { description: advertised.description },
...capabilities,
}
},
} as never)
}

2
packages/ui/tui/tests/snapshots/model-effort-switching.expected.txt
View File

@@ -26,7 +26,7 @@ buffer
style 8-8 fg=bright-blue
style 38-77 fg=bright-black
style 83-83 fg=bright-blue
15| " │ → deepseek/deepseek-v4-pro DeepSeek V4 Pro — Max │ "
15| " │ → deepseek/deepseek-v4-pro DeepSeek V4 Pro — Off │ "
style 8-8 fg=bright-blue
style 10-64 fg=bright-blue inverse
style 83-83 fg=bright-blue

6
packages/ui/tui/tests/snapshots/model-switching.expected.txt
View File

@@ -8,10 +8,10 @@ buffer
style 10-16 bold
1| " Snapshot agent ready."
style 1-21 fg=bright-black
2| " deepseek-v4-pro max • main-session"
2| " deepseek-v4-pro off • main-session"
style 1-36 dim
3| <blank>
4| " Model selected: deepseek/deepseek-v4-pro. Reasoning effort: Max. New steps will use it. "
4| " Model selected: deepseek/deepseek-v4-pro. Reasoning effort: Off. New steps will use it. "
style 1-87 fg=bright-black
5| "────────────────────────────────────────────────────────────────────────────────────────────"
style 0-91 dim
@@ -19,7 +19,7 @@ buffer
style 1-1 inverse
7| "────────────────────────────────────────────────────────────────────────────────────────────"
style 0-91 dim
8| "deepseek-v4-pro max /workspace/project ↑0 ↓0 0% context tools:collapsed"
8| "deepseek-v4-pro off /workspace/project ↑0 ↓0 0% context tools:collapsed"
style 0-45 dim
style 65-91 dim
9-31| <blank>

17
packages/ui/tui/tests/tui.snapshot.ts
View File

@@ -640,12 +640,16 @@ describe('TUI terminal-state snapshots', () => {
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'DeepSeek V4 Flash' },
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'DeepSeek V4 Pro' },
],
resolveModelReasoning: () => Promise.resolve({
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
resolveModelInfo: () => Promise.resolve({
context: { contextWindow: 128_000 },
reasoning: {
efforts: [
{ id: ReasoningEffortId('off'), name: 'Off' },
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
},
}),
},
}, { columns: 92, rows: 32 })
@@ -657,6 +661,7 @@ describe('TUI terminal-state snapshots', () => {
await renderAfter(harness, () => {
harness.terminal.send('\x1b[B')
harness.terminal.send('\x1b[Z')
harness.terminal.send('\x1b[Z')
})
await checkpoint('model-effort-switching', harness.terminal, { includeScrollback: true })
await renderAfter(harness, () => {

59
packages/ui/tui/tests/tui.spec.ts
View File

@@ -5,7 +5,11 @@ import { describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import { CombinedAutocompleteProvider, type Terminal } from '@earendil-works/pi-tui'
import AgentRegistry, { agentEvents, assembleContextFor, AgentMessageId, type Agent } from '@deepseek-ai/dsh-agent'
import { ReasoningEffortId, type LlmCallConfig } from '@deepseek-ai/dsh-llm'
import {
ReasoningEffortId,
type LlmCallConfig,
type LlmModelReasoningInfo,
} from '@deepseek-ai/dsh-llm'
import { GOAL_CHANGE_VERSION, GoalId, renderGoalChange, type GoalSnapshotChangeMeta } from '@deepseek-ai/dsh-goal'
import CommandService, { type CommandInvocation } from '@deepseek-ai/dsh-commands'
import SessionStore, { SessionId, type JsonValue, type SessionEvent, type SessionHeader, type TurnEndReason } from '@deepseek-ai/dsh-session'
@@ -143,7 +147,11 @@ function provideLlmCatalog(ctx: Context): void {
ctx.provide('llm', {
listProviders: () => [],
listModels: () => Promise.resolve([]),
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: (provider: string, model: string) => Promise.resolve({
provider,
id: model,
name: model,
}),
} as never)
}
@@ -1155,7 +1163,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
expect(result.terminal.output).toContain('restored answer')
expect(result.terminal.output).toContain('write tests')
expect(result.terminal.output).toContain('↑1.3k ↓42')
// Context resolution is async (resolveModelContext); settle before reading.
// Exact model resolution is async; settle before reading.
await tick()
expect(result.terminal.output).toContain('42% context tools:collapsed')
// Narrow terminals clip the right-hand context/tools segment first; the
@@ -1741,7 +1749,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [],
models: [],
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: () => Promise.resolve({}),
},
})
result.terminal.send('/status')
@@ -2288,6 +2296,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
it('opens a keyboard selector and switches the session model without sending slash text to the agent', async () => {
const initialContext = Promise.withResolvers<{ contextWindow: number }>()
let deferInitialContext = true
const result = await setup({
agentOptions: { provider: 'alpha', model: 'a1' },
contextTokens: 50,
@@ -2299,37 +2308,41 @@ describe('pi-tui chat lifecycle and transcript', () => {
{ provider: 'beta', id: 'b1', name: 'Beta One' },
{ provider: 'beta', id: 'shared', name: 'Beta Shared' },
],
resolveModelContext: (provider, model) => provider === 'alpha' && model === 'a1'
? initialContext.promise
: Promise.resolve({ contextWindow: 200 }),
resolveModelReasoning: (provider, model) => {
async resolveModelInfo(provider, model) {
const shouldDeferContext = provider === 'alpha' && model === 'a1' && deferInitialContext
if (shouldDeferContext) deferInitialContext = false
const context = shouldDeferContext
? await initialContext.promise
: { contextWindow: 200 }
let reasoning: LlmModelReasoningInfo | undefined
if (model === 'a1') {
return Promise.resolve({
reasoning = {
efforts: [
{ id: ReasoningEffortId('low'), name: 'Low' },
{ id: ReasoningEffortId('high'), name: 'High' },
],
defaultEffort: ReasoningEffortId('low'),
})
}
if (model === 'b1') {
return Promise.resolve({
}
} else if (model === 'b1') {
reasoning = {
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
})
}
if (provider === 'alpha' && model === 'shared') {
return Promise.resolve({
}
} else if (provider === 'alpha' && model === 'shared') {
reasoning = {
efforts: [
{ id: ReasoningEffortId('standard'), name: 'Standard' },
{ id: ReasoningEffortId('ultra'), name: 'Ultra' },
],
})
}
}
return {
context,
...reasoning === undefined ? {} : { reasoning },
}
return Promise.resolve(undefined)
},
},
})
@@ -2522,7 +2535,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'alpha', name: 'Alpha' }],
models: [{ provider: 'alpha', id: 'a1', name: 'Alpha One' }],
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: () => Promise.resolve({}),
},
})
unset.terminal.send('/model')
@@ -2554,7 +2567,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
listModels: () => Promise.reject(new Error('catalog offline')),
resolveModelContext: () => Promise.reject(new Error('capacity offline')),
resolveModelInfo: () => Promise.reject(new Error('capacity offline')),
},
})
failed.terminal.send('/model')
@@ -2569,7 +2582,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [{ provider: 'deepseek', id: 'model-1', name: 'Model One' }],
resolveModelReasoning: () => Promise.reject(new Error('reasoning metadata offline')),
resolveModelInfo: () => Promise.reject(new Error('reasoning metadata offline')),
},
})
reasoningFailed.terminal.send('/model')
@@ -2619,7 +2632,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
resolveModelContext: () => context.promise,
resolveModelInfo: () => context.promise.then(value => ({ context: value })),
},
})
await contextResult.controller.dispose()