feat(web): show durable token usage and context occupancy in the stats line
The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
This commit is contained in:
Hypatia May
@@ -90,6 +90,8 @@ describe('createFixtureApi', () => {
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
},
|
||||
// No request ran, so pressure is zero and no capacity is known yet.
|
||||
contextPressure: { pressureTokens: 0 },
|
||||
} },
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/client/ui-conversation/README.md
|
||||
README.md: 3973c14f2b8fe746549bb74af85a7a60a7d66aea
|
||||
README.zh.md: a6bb15c4cdd53d05bf28147b97d9d64d1c59da2b
|
||||
README.md: 84051dbe29503bcb20e317247db2ba2026db4528
|
||||
README.zh.md: 36a0983f95dafc55a4fc76a5bc82112fb369efce
|
||||
|
||||
@@ -22,7 +22,7 @@ Per-session UI state for selection and the active view lives in the declared cha
|
||||
|
||||
The composer bar declares session-scoped single seats for `'conversation.input.plan'` (right of the local access-mode control) and `'conversation.input.model'` (immediately before the pending indicator and send/stop button), plus list slots for overlay, dock, left, and right input extensions. Feature packages own each control and its state; ui-conversation supplies placement, the `locked` owner prop, and the standard slot shares. While the `plan` projection's effective target is plan mode, InputBar swaps its textarea placeholder to the plan-task wording, localized through the `command.hint` locale namespace this package registers and shared verbatim with the claimed `/plan` command hint (a host-folded value read through the standard-kit `useProjection`; owner-supplied placeholders win). A pending composer takeover remains mounted when another conversation view is active so the blocked agent can still receive its answer; without a pending interaction, the active-session composer belongs to Chat. The resident no-session shell uses `DisabledInputBar` and therefore dispatches no session-scoped control seats.
|
||||
|
||||
The chat stats line reads full-log billing from the generic `tokenUsage` projection and joins it only at presentation with the connection-local atomic `ConversationSnapshot.modelRequest`; visible nodes supply only the existing turn/step counts. It renders uncached input, output, and cache reads as separate compact values, computes cache hit as `cacheRead / (uncachedInput + cacheRead)` without cache writes, and shows context occupancy only when the same observed request snapshot contains both `contextTokens` and `contextWindow`. Before that request, after reconnect/restore/new subscription, or after a request missing either field, context is labeled unknown rather than queried from the selected model or reconstructed from history. The existing inline stats row remains the sole context UI; the model selector has no circle or accessory.
|
||||
The chat stats line takes its token accounting from two generic token-meter projections read through the standard-kit `useProjection`: `tokenUsage` for full-log billing (cache hit is `cacheRead / (uncachedInput + cacheRead)`, excluding cache writes) and `contextPressure` for context occupancy. Visible nodes supply only the turn and step counts plus the LLM and tool wall times, which are window-scoped facts about what is on screen rather than accounting. A deployment without token-meter drops the token groups, and a route whose adapter advertises no capacity drops the occupancy group instead of rendering a placeholder. Occupancy is deliberately an approximation — its numerator and capacity are independent last-wins projection fields, not one atomic request observation ([rationale](../../llm/token-meter/README.md)). The inline stats row remains the sole context UI; the model selector has no circle or accessory.
|
||||
|
||||
`src/client/` is organized for the future package split: `contract/` is the sole inter-domain shared face (`slots.ts` slot declarations + composed slot props including the tool-row contract, `views.ts` shared primitives, `tool-call-model.ts`); the `skeleton/`, `chat/`, and `toolviews/` (sample registrants) domain directories import contract files and never each other; `apply.ts` is the only assembly point allowed to import all three domains. The `/client` export surface is the contract only — `apply`/`inject`, the two service classes, and the `contract/` type families; implementation components (skeleton, chat rows) and the store factory stay internal and reach the page exclusively through apply's slot registrations (tests take them via the `./src/*` subpath).
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ todo 两个面就是在该形状上的两个注册项，都是普通注册方插
|
||||
|
||||
输入栏为 `'conversation.input.plan'`（位于本地 access 模式控件右侧）和 `'conversation.input.model'`（渲染在 pending 指示器与发送／停止按钮之前）声明会话作用域的单实例 seat，并为 overlay、dock、left 和 right 输入扩展声明列表 slot。各功能包拥有相应控件及其状态；ui-conversation 提供放置位置、`locked` owner prop 和标准 slot share。当 `plan` 投影的有效目标为 plan mode 时，InputBar 将文本框 placeholder 切换为 plan 任务措辞，经本包注册的 `command.hint` locale 命名空间本地化，并与已认领 `/plan` 命令的提示逐字共用同一份文案（经标准套件 `useProjection` 读取的 host 折叠值；owner 提供的 placeholder 优先）。另一个会话视图活跃时，待处理的 composer 接管仍保持挂载，使被阻塞的 agent（智能体）仍能收到回答；没有待处理交互时，活跃会话的 composer 归 Chat 所有。常驻无会话壳使用 `DisabledInputBar`，因此不会分发任何会话作用域的控件 seat。
|
||||
|
||||
聊天统计行从通用 `tokenUsage` 投影读取完整日志计费用量，并且只在展示时把它与连接本地的原子快照 `ConversationSnapshot.modelRequest` 结合；可见节点仅提供既有的轮次和步骤计数。它以相互独立的紧凑值显示未缓存输入、输出与缓存读取，通过 `cacheRead / (uncachedInput + cacheRead)` 计算缓存命中率而不计入缓存写入，并且只有同一份已观测请求快照同时包含 `contextTokens` 与 `contextWindow` 时才显示上下文占用率。在该请求之前、重连／恢复／新订阅之后，或在请求缺少任一字段之后，系统都会把上下文标为「未知」，而不会从所选模型查询或根据历史记录重建。现有的行内统计行仍是唯一的上下文 UI；模型选择器不增加圆环或附属控件。
|
||||
聊天统计行的 token 账目来自经标准套件 `useProjection` 读取的两个通用 token-meter 投影：`tokenUsage` 提供完整日志计费用量（缓存命中率为 `cacheRead / (uncachedInput + cacheRead)`，不计入缓存写入），`contextPressure` 提供上下文占用率。可见节点只提供轮次与步骤计数，以及 LLM 和工具的墙钟时间：这些是关于「屏幕上有什么」的窗口作用域事实，而非账目。未组合 token-meter 的部署会整组省略 token 分组；适配器未公布容量的路由会省略占用率分组，而不是渲染占位文案。占用率是刻意为之的近似值：它的分子与容量是两个相互独立的「后者胜」投影字段，并非同一次请求的原子观测（[原理](../../llm/token-meter/README.md)）。行内统计行仍是唯一的上下文 UI；模型选择器不增加圆环或附属控件。
|
||||
|
||||
`src/client/` 按未来的包拆分组织：`contract/` 是唯一的跨领域共享表层（`slots.ts` slot 声明 + 组合后的 slot props，包括工具行契约、`views.ts` 共享原语、`tool-call-model.ts`）；`skeleton/`、`chat/` 和 `toolviews/`（示例注册方）领域目录只导入 contract 文件，彼此绝不导入；`apply.ts` 是唯一允许导入全部三个领域的组装点。`/client` 导出表层只包含契约：`apply`／`inject`、两个服务类和 `contract/` 类型家族；实现组件（骨架、聊天行）与 store factory 保持内部状态，只能通过 apply 的 slot 注册到达页面（测试通过 `./src/*` 子路径获取它们）。
|
||||
|
||||
|
||||
@@ -218,14 +218,19 @@ describe('small branch tails', () => {
|
||||
})
|
||||
|
||||
it('StatsLine omits the cache-hit segment when no input accounting exists at all', () => {
|
||||
// cacheHitPct is null only when input+cacheRead are both zero (pure
|
||||
// output accounting) — any input makes it a real 0%.
|
||||
// Cache hit is null only when uncached input and cache reads are both zero
|
||||
// (pure output accounting) — any input makes it a real 0%.
|
||||
const snap = {
|
||||
nodes: [{ kind: 'assistant', seq: 1, turn: 1, step: 1, blocks: [], usage: { outputTokens: 10 } }],
|
||||
}
|
||||
const source = { getSnapshot: () => snap, subscribe: () => () => {} }
|
||||
const view = render(
|
||||
<StatsLine useSession={bindSnapshotSelector(source) as unknown as StatsLineProps['useSession']} />,
|
||||
<StatsLine
|
||||
useSession={bindSnapshotSelector(source) as unknown as StatsLineProps['useSession']}
|
||||
useProjection={(key: string) => key === 'tokenUsage'
|
||||
? { uncachedInputTokens: 0, outputTokens: 10, cacheReadTokens: 0, cacheWriteTokens: 0 }
|
||||
: undefined}
|
||||
/>,
|
||||
)
|
||||
expect(view.container.textContent).toBe('1 turns · 1 steps|Input 0 tok · Output 10 tok')
|
||||
})
|
||||
|
||||
@@ -51,7 +51,7 @@ function makeSource(init?: Partial<ConversationSnapshot>) {
|
||||
}
|
||||
|
||||
describe('deriveStats', () => {
|
||||
it('folds turns/steps/token split and cache hit percentage', () => {
|
||||
it('counts turns and steps and never folds node usage into accounting', () => {
|
||||
const stats = deriveStats([
|
||||
assistant(1, 1, { inputTokens: 100, outputTokens: 50, cacheReadTokens: 900 }),
|
||||
assistant(2, 1, { inputTokens: 100, outputTokens: 50 }),
|
||||
@@ -59,12 +59,12 @@ describe('deriveStats', () => {
|
||||
])
|
||||
expect(stats.turns).toBe(2)
|
||||
expect(stats.steps).toBe(3)
|
||||
expect(stats.inputTokens).toBe(1100)
|
||||
expect(stats.outputTokens).toBe(100)
|
||||
expect(stats.cacheHitPct).toBe(82)
|
||||
// Window-scoped by design: the paged window is not an accounting source, so
|
||||
// the fold exposes no token fields at all (billing rides the projection).
|
||||
expect(Object.keys(stats).sort()).toEqual(['llmMs', 'steps', 'toolMs', 'turns'])
|
||||
})
|
||||
|
||||
it('cache hit stays null with no cache accounting; out-of-window tool results ignored', () => {
|
||||
it('ignores tool results with no call time', () => {
|
||||
const tool: ToolResultNode = {
|
||||
kind: 'tool-result', seq: 5, time: 5_000, callId: 'c', call: null, callTime: null, content: [],
|
||||
isError: false, callView: null, resultView: null,
|
||||
@@ -72,7 +72,6 @@ describe('deriveStats', () => {
|
||||
const stats = deriveStats([tool, assistant(1, 1)])
|
||||
expect(stats.steps).toBe(1)
|
||||
expect(stats.toolMs).toBe(0)
|
||||
expect(stats.cacheHitPct).toBeNull()
|
||||
})
|
||||
|
||||
it('sums LLM wall time from assistant timing and tool wall time from call/result pairs', () => {
|
||||
@@ -109,22 +108,71 @@ describe('formatters', () => {
|
||||
})
|
||||
|
||||
describe('StatsLine', () => {
|
||||
function props(source: { getSnapshot(): ConversationSnapshot; subscribe(fn: () => void): () => void }): StatsLineProps {
|
||||
return { useSession: bindSnapshotSelector(source) }
|
||||
const USAGE = { uncachedInputTokens: 10, outputTokens: 5, cacheReadTokens: 90, cacheWriteTokens: 0 }
|
||||
|
||||
/** Stub the projection seat: a key-addressed table of whole values. */
|
||||
function projections(values: Record<string, unknown>): StatsLineProps['useProjection'] {
|
||||
return (key: string) => values[key]
|
||||
}
|
||||
|
||||
function props(
|
||||
source: { getSnapshot(): ConversationSnapshot; subscribe(fn: () => void): () => void },
|
||||
values: Record<string, unknown> = { tokenUsage: USAGE },
|
||||
): StatsLineProps {
|
||||
return { useSession: bindSnapshotSelector(source), useProjection: projections(values) }
|
||||
}
|
||||
|
||||
it('renders the grouped stats row and hides with zero steps', () => {
|
||||
const { source } = makeSource({
|
||||
nodes: [assistant(1, 1, { inputTokens: 10, outputTokens: 5, cacheReadTokens: 90 })],
|
||||
})
|
||||
const { source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
const view = render(<StatsLine {...props(source)} />)
|
||||
// No timing on the fixture: the duration group drops out whole.
|
||||
// No timing on the fixture: the duration group drops out whole. Tokens come
|
||||
// from the projection, so paging the window cannot change them.
|
||||
expect(view.container.textContent).toBe('1 turns · 1 steps|Cache hit 90%|Input 100 tok · Output 5 tok')
|
||||
const empty = makeSource()
|
||||
const emptyView = render(<StatsLine {...props(empty.source)} />)
|
||||
expect(emptyView.container.textContent).toBe('')
|
||||
})
|
||||
|
||||
it('renders context occupancy only when the projection knows a capacity', () => {
|
||||
const { source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
const withCapacity = render(<StatsLine {...props(source, {
|
||||
tokenUsage: USAGE,
|
||||
contextPressure: { pressureTokens: 32_000, contextWindow: 128_000 },
|
||||
})} />)
|
||||
expect(withCapacity.container.textContent).toContain('Context 25% of 128K')
|
||||
// Pressure without capacity has no denominator: the group drops out.
|
||||
const noCapacity = render(<StatsLine {...props(source, {
|
||||
tokenUsage: USAGE,
|
||||
contextPressure: { pressureTokens: 32_000 },
|
||||
})} />)
|
||||
expect(noCapacity.container.textContent).not.toContain('Context')
|
||||
})
|
||||
|
||||
it('clamps occupancy at 100% when pressure exceeds the recorded capacity', () => {
|
||||
// Capacity and pressure are independent last-wins fields, so a model switch
|
||||
// can pair a smaller new window with the previous route's larger prompt.
|
||||
const { source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
const view = render(<StatsLine {...props(source, {
|
||||
tokenUsage: USAGE,
|
||||
contextPressure: { pressureTokens: 300_000, contextWindow: 128_000 },
|
||||
})} />)
|
||||
expect(view.container.textContent).toContain('Context 100% of 128K')
|
||||
})
|
||||
|
||||
it('drops every token group when no projection is composed', () => {
|
||||
const { source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
const view = render(<StatsLine {...props(source, {})} />)
|
||||
expect(view.container.textContent).toBe('1 turns · 1 steps')
|
||||
})
|
||||
|
||||
it('omits cache hit when nothing was billed on the input side', () => {
|
||||
const { source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
const view = render(<StatsLine {...props(source, {
|
||||
tokenUsage: { uncachedInputTokens: 0, outputTokens: 7, cacheReadTokens: 0, cacheWriteTokens: 0 },
|
||||
})} />)
|
||||
expect(view.container.textContent).toBe('1 turns · 1 steps|Input 0 tok · Output 7 tok')
|
||||
})
|
||||
|
||||
it('renders ZERO times during streaming chunk frames (RFC hard acceptance)', () => {
|
||||
const { set, source } = makeSource({ nodes: [assistant(1, 1)] })
|
||||
let renders = 0
|
||||
|
||||
@@ -36,20 +36,24 @@ describe('render branch tails', () => {
|
||||
expect(view.container.querySelector('[data-state="ok"]')).not.toBeNull()
|
||||
})
|
||||
|
||||
it('StatsLine skips usage-less nodes and defaults each absent counter to zero', () => {
|
||||
it('StatsLine counts window nodes but drops every token group without a projection', () => {
|
||||
// Node `usage` is deliberately ignored: billing rides the durable
|
||||
// tokenUsage projection, so an absent projection leaves counts only.
|
||||
const snap = {
|
||||
nodes: [
|
||||
{ kind: 'assistant', seq: 1, turn: 1, step: 1, blocks: [] },
|
||||
{ kind: 'assistant', seq: 2, turn: 1, step: 2, blocks: [], usage: { inputTokens: 4, outputTokens: 6 } },
|
||||
// outputTokens absent: the tokens sum's ?? 0 arm for output.
|
||||
{ kind: 'assistant', seq: 3, turn: 2, step: 1, blocks: [], usage: { inputTokens: 5 } },
|
||||
],
|
||||
}
|
||||
const source = { getSnapshot: () => snap, subscribe: () => () => {} }
|
||||
const view = render(
|
||||
<StatsLine useSession={bindSnapshotSelector(source) as unknown as UseSession<ConversationSnapshot>} />,
|
||||
<StatsLine
|
||||
useSession={bindSnapshotSelector(source) as unknown as UseSession<ConversationSnapshot>}
|
||||
useProjection={() => undefined}
|
||||
/>,
|
||||
)
|
||||
expect(view.container.textContent).toBe('2 turns · 3 steps|Cache hit 0%|Input 9 tok · Output 6 tok')
|
||||
expect(view.container.textContent).toBe('2 turns · 3 steps')
|
||||
})
|
||||
|
||||
it('AssistantMarkdown reasoning as the streaming tail renders the running ring', () => {
|
||||
|
||||
@@ -1969,7 +1969,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'PreparedLlmCall',
|
||||
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n readonly context?: LlmModelContext;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'PreparedReferencedMessage',
|
||||
@@ -2103,6 +2103,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'ReasoningEffortId',
|
||||
declaration: 'export type ReasoningEffortId = Branded<\'ReasoningEffortId\'>;',
|
||||
},
|
||||
{
|
||||
name: 'RequestContext',
|
||||
declaration: 'export interface RequestContext {\n provider: string;\n model: string;\n contextWindow: number;\n}',
|
||||
},
|
||||
{
|
||||
name: 'RequestHeaderReason',
|
||||
declaration: 'export type RequestHeaderReason = \'initial\' | \'resume\' | \'change\';',
|
||||
@@ -2165,7 +2169,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'Session',
|
||||
declaration: 'export class Session {\n get surface(): SessionSurface;\n readonly header: SessionHeader;\n get id(): SessionId;\n readonly firstLiveSeq: number;\n constructor(id: SessionId, seed?: readonly SessionEvent[], header?: SessionHeader);\n get events(): readonly SessionEvent[];\n get seq(): number;\n append<T extends SessionEventType>(type: T, data: SessionEventMap[T], ...opts: T extends SurfaceEventType ? [\n opts: SurfaceIntent\n ] : [\n ]): SessionEvent<T>;\n requestHeader(): EpochHeader | undefined;\n deriveMessages(): Message[];\n deriveEventMessage(event: SessionEvent): Message | null;\n}',
|
||||
declaration: 'export class Session {\n get surface(): SessionSurface;\n readonly header: SessionHeader;\n get id(): SessionId;\n readonly firstLiveSeq: number;\n constructor(id: SessionId, seed?: readonly SessionEvent[], header?: SessionHeader);\n get events(): readonly SessionEvent[];\n get seq(): number;\n append<T extends SessionEventType>(type: T, data: SessionEventMap[T], ...opts: T extends SurfaceEventType ? [\n opts: SurfaceIntent\n ] : [\n ]): SessionEvent<T>;\n requestHeader(): EpochHeader | undefined;\n requestContext(): RequestContext | undefined;\n deriveMessages(): Message[];\n deriveEventMessage(event: SessionEvent): Message | null;\n}',
|
||||
},
|
||||
{
|
||||
name: 'SessionAvailability',
|
||||
@@ -2177,7 +2181,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'SessionEventMap',
|
||||
declaration: 'export interface SessionEventMap {\n \'turn/start\': {\n turn: number;\n trigger: TurnTrigger;\n };\n \'turn/end\': {\n turn: number;\n reason: TurnEndReason;\n };\n \'step/start\': {\n turn: number;\n step: number;\n };\n \'step/end\': {\n turn: number;\n step: number;\n };\n \'user/message\': UserMessage;\n \'assistant/chunk\': {\n turn: number;\n step: number;\n chunk: StreamChunk;\n };\n \'assistant/message\': {\n turn: number;\n step: number;\n message: AssistantMessage;\n usage?: TokenUsage;\n };\n \'tool/call\': {\n turn: number;\n step: number;\n callId: CallId;\n name: string;\n arguments: string;\n };\n \'tool/result\': {\n turn: number;\n step: number;\n message: ToolResultMessage;\n error?: {\n name: string;\n code: string;\n };\n meta?: JsonValue;\n };\n \'steering/message\': {\n turn: number;\n message: UserMessage;\n };\n \'todo/write\': {\n todos: TodoItem[];\n };\n \'request/header\': {\n header: EpochHeader;\n reason: RequestHeaderReason;\n };\n}',
|
||||
declaration: 'export interface SessionEventMap {\n \'turn/start\': {\n turn: number;\n trigger: TurnTrigger;\n };\n \'turn/end\': {\n turn: number;\n reason: TurnEndReason;\n };\n \'step/start\': {\n turn: number;\n step: number;\n };\n \'step/end\': {\n turn: number;\n step: number;\n };\n \'user/message\': UserMessage;\n \'assistant/chunk\': {\n turn: number;\n step: number;\n chunk: StreamChunk;\n };\n \'assistant/message\': {\n turn: number;\n step: number;\n message: AssistantMessage;\n usage?: TokenUsage;\n };\n \'tool/call\': {\n turn: number;\n step: number;\n callId: CallId;\n name: string;\n arguments: string;\n };\n \'tool/result\': {\n turn: number;\n step: number;\n message: ToolResultMessage;\n error?: {\n name: string;\n code: string;\n };\n meta?: JsonValue;\n };\n \'steering/message\': {\n turn: number;\n message: UserMessage;\n };\n \'todo/write\': {\n todos: TodoItem[];\n };\n \'request/header\': {\n header: EpochHeader;\n reason: RequestHeaderReason;\n };\n \'request/context\': RequestContext;\n}',
|
||||
},
|
||||
{
|
||||
name: 'SessionEventMetadataFilter',
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { createUserMessage, LlmError, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelReasoningInfo, LlmResolvedModelInfo } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmModelReasoningInfo, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { Session, SessionId, foldRequestHeader } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
|
||||
@@ -521,3 +521,70 @@ describe('request stability across the loop', () => {
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('request/context capacity records', () => {
|
||||
/** Adapter advertising a per-model capacity, keyed by model id. */
|
||||
function capacityAdapter(windows: Record<string, number>, script: StreamChunk[][]): MockAdapter {
|
||||
return new class extends MockAdapter {
|
||||
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
|
||||
const contextWindow = windows[model]
|
||||
return Promise.resolve({
|
||||
provider,
|
||||
id: model,
|
||||
name: model,
|
||||
...contextWindow === undefined ? {} : { context: { contextWindow } },
|
||||
})
|
||||
}
|
||||
}(script)
|
||||
}
|
||||
|
||||
it('records capacity once and skips it while the route is unchanged', async () => {
|
||||
const adapter = capacityAdapter({ mock: 128_000 }, [textResponse('a'), textResponse('b')])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('capacity-dedup'), { provider: 'mock', model: 'mock' })
|
||||
|
||||
send(agent, 'first')
|
||||
await waitForIdle(ctx, agent)
|
||||
send(agent, 'second')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
const records = agent.session.events.filter(event => event.type === 'request/context')
|
||||
expect(records).toHaveLength(1)
|
||||
expect(records[0]?.data).toEqual({ provider: 'mock', model: 'mock', contextWindow: 128_000 })
|
||||
// Log-only: not a SurfaceEventType, so it can never reach a model request
|
||||
// (the type system rejects a surfaceOp here; the session invariant also
|
||||
// requires the record to sit inside its open turn).
|
||||
expect(agent.session.surface.nodes).not.toContain(records[0]?.seq)
|
||||
})
|
||||
|
||||
it('records a second capacity when the route changes mid-session', async () => {
|
||||
const adapter = capacityAdapter(
|
||||
{ small: 64_000, large: 256_000 },
|
||||
[textResponse('a'), textResponse('b')],
|
||||
)
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('capacity-switch'), { provider: 'mock', model: 'small' })
|
||||
|
||||
send(agent, 'first')
|
||||
await waitForIdle(ctx, agent)
|
||||
ctx.on('agent/request', (subject, _turn, _step, _signal, next) => subject === agent
|
||||
? Promise.resolve({ provider: 'mock', model: 'large' })
|
||||
: next())
|
||||
send(agent, 'second')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(agent.session.events
|
||||
.filter(event => event.type === 'request/context')
|
||||
.map(event => event.data.contextWindow)).toEqual([64_000, 256_000])
|
||||
})
|
||||
|
||||
it('records nothing when the adapter advertises no capacity', async () => {
|
||||
// The absent-capacity path must stay silent rather than log a placeholder:
|
||||
// consumers read "no capacity known" and omit their percentage entirely.
|
||||
const ctx = await harness(new MockAdapter([textResponse('a')]))
|
||||
const agent = ctx.agentLoop.create(SessionId('capacity-absent'), { provider: 'mock', model: 'mock' })
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
expect(agent.session.events.some(event => event.type === 'request/context')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/core/session/README.md
|
||||
README.md: a9b6905dcf2b8ef1f75595e567273f7a3150a412
|
||||
README.zh.md: f1a5e97e32d1ad1abcd6ad96e6c621af9972e989
|
||||
README.md: 861b96c453e677807bded2475fe8e62a74bcd299
|
||||
README.zh.md: 6974a072f5cb32f4e850846bbb02af59cda93303
|
||||
|
||||
@@ -65,6 +65,8 @@ Providers stream token-sized deltas, so a raw log stores hundreds of `assistant/
|
||||
|
||||
`request/header` records a full canonical snapshot of the non-history request envelope with reason `initial`, `resume`, or `change`. `foldRequestHeader()` selects the latest snapshot; legacy delta events and the removed `fallback` reason are rejected. See the [reconstructable-requests Agent Note](../../../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md).
|
||||
|
||||
`request/context` records the registration-bound `contextWindow` of the route a request resolved to, appended inside its step beside `request/header` and only when the provider, model, or capacity differs from the previous record. `session.requestContext()` folds the latest one incrementally, mirroring `requestHeader()`. Capacity stays OUT of `EpochHeader` on purpose: it is adapter metadata describing a route, not an input the request was built from, so it must not enter request reconstruction or header equality — a capacity change is not a header `change`. A route whose adapter advertises no capacity appends nothing.
|
||||
|
||||
A `user/message` stores the complete `UserMessage` directly, including the identity created before routing or prompt admission. It renders its `content` verbatim whether it is a direct human prompt, a synthetic injection, or an admitted goal round; its typed `source` is the only channel that tells them apart and carries any domain-specific durable facts. `assistant/message`, `tool/result`, and `steering/message` likewise store complete message values. Turn execution remains enclosed by `turn/start` and `turn/end`, while an idle injection may append and flush a `user/message` between turns without running the model.
|
||||
|
||||
`tool/result` persists one identified user-role tool-result message, optional internal failure identity, and optional presentation metadata. A tool's successful canonical `value` and human-readable canonical failure message remain execution-local; rendered error content is the replay-authoritative message.
|
||||
|
||||
@@ -65,6 +65,8 @@
|
||||
|
||||
`request/header` 记录非历史请求封装的完整规范快照，其原因为 `initial`、`resume` 或 `change`。`foldRequestHeader()` 选择最新快照；旧版增量事件和已移除的 `fallback` 原因会被拒绝。详见[可重建请求 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md)。
|
||||
|
||||
`request/context` 记录请求所解析到的路由的、绑定注册项的 `contextWindow`，在其所属步骤内紧随 `request/header` 追加，且仅在提供方、模型或容量与上一条记录不同时追加。`session.requestContext()` 以增量方式归并最新一条，与 `requestHeader()` 保持一致。容量刻意不进入 `EpochHeader`：它是描述路由的适配器元数据，不是构建该请求所依据的输入，因此绝不可进入请求重建或请求头相等性判断：容量变化不构成请求头 `change`。适配器不公布容量的路由不追加任何记录。
|
||||
|
||||
`user/message` 会直接存储完整的 `UserMessage`，其中包括路由或提示词准入前创建的标识。无论它是直接人类提示词、合成注入，还是已准入的 Goal Round，都会原样呈现其 `content`；带类型的 `source` 是区分三者的唯一通道，并携带各领域专有的持久事实。`assistant/message`、`tool/result` 和 steering（中途引导）对应的 `steering/message` 也会存储完整的消息值。轮次执行仍由 `turn/start` 与 `turn/end` 包围，而空闲注入可以在轮次之间追加并刷新一条 `user/message`，无需运行模型。
|
||||
|
||||
`tool/result` 持久保存一条带标识、user-role 的工具结果消息，以及可选内部失败标识和可选呈现元数据。工具成功时的规范 `value` 和便于人类阅读的规范失败消息只存在于执行本地；渲染后的错误内容是回放权威消息。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: d343449d1530bf70a3a8c57f883894e29c42d18f
|
||||
README.zh.md: 4dc4a0ca06378116d05fdb4b9b048738930511fd
|
||||
README.md: 1b815cfba8d2ee1dd69e1f00b8a3911c7d9a788c
|
||||
README.zh.md: c08a46a7037dab28c27e1a593aa83fcca08144f3
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: 578728ded9cf51a12abcd70d541404e995028f26
|
||||
README.zh.md: 9d54ddb4792e6c897af7d57a6f8ae98204caa11d
|
||||
README.md: b9bf1dfa253e424e5ec35cd3e7bf0f52af579077
|
||||
README.zh.md: c97fc0b87364dfe9ca46139f0ec82519e191b772
|
||||
|
||||
@@ -21,11 +21,23 @@ The fold tracks full request-header snapshots, step boundaries, surface appends
|
||||
|
||||
Usage accounting sums disjoint input, cache-read, cache-write, and output buckets; reasoning is not added again. Every successful call records an assistant anchor, including content-less calls. An explicit empty provenance list means a known empty provider stream, while absent legacy provenance conservatively treats the durable assistant output as provider output.
|
||||
|
||||
## Session projection
|
||||
## Session projections
|
||||
|
||||
When the composition provides `ctx.sessionProjections`, token-meter registers the `tokenUsage` unit through an optional child fiber. Its client-safe value is the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision.
|
||||
When the composition provides `ctx.sessionProjections`, token-meter registers two units through an optional child fiber.
|
||||
|
||||
The unit uses the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes the key. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
`tokenUsage` carries the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision. The single last-sample slot relies on a session-log ordering property: once a later step reports usage, a legal log never reports usage for an earlier step again.
|
||||
|
||||
`contextPressure` carries `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — and the optional `contextWindow` from the newest `request/context` record. Output is excluded, so the numerator holds still while a turn streams and steps forward when the next request reports its usage.
|
||||
|
||||
Both units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes both keys. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
|
||||
### Context occupancy is an approximation, by design
|
||||
|
||||
`pressureTokens` and `contextWindow` are independent last-wins fields and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's pressure until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now.
|
||||
|
||||
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. The TUI status line has always computed occupancy the same way, dividing a `measure()` total by a separately-resolved capacity for the selected model.
|
||||
|
||||
Making the pair atomic was tried and rejected: it required a transient non-replayable wire frame, which needed lifecycle fencing against cross-stream reordering and left occupancy blank after every reconnect. The [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md) records that comparison. Consumers that need an exact same-boundary figure should call `measure()` at their own request boundary rather than read this projection.
|
||||
|
||||
## Composition
|
||||
|
||||
|
||||
@@ -23,9 +23,21 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
|
||||
|
||||
## 会话投影
|
||||
|
||||
当组合提供 `ctx.sessionProjections` 时，token-meter 会通过一个可选子 fiber 注册 `tokenUsage` 单元。其可安全传给客户端的值包含完整持久日志中的 `uncachedInputTokens`、`outputTokens`、`cacheReadTokens` 和 `cacheWriteTokens`。即使请求随后失败，用量分片仍会计入；同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本，而不是重复计数。推理仍是输出的一个细分项。
|
||||
当组合提供 `ctx.sessionProjections` 时，token-meter 会通过一个可选子 fiber 注册两个单元。
|
||||
|
||||
该单元使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除该键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
`tokenUsage` 携带完整持久日志中的 `uncachedInputTokens`、`outputTokens`、`cacheReadTokens` 和 `cacheWriteTokens`。即使请求随后失败，用量分片仍会计入；同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本，而不是重复计数。推理仍是输出的一个细分项。只保留单个最新样本，依赖的是会话日志的一条顺序性质：一旦某个更晚的步骤报告了用量，合法日志就绝不会再为更早的步骤报告用量。
|
||||
|
||||
`contextPressure` 携带 `pressureTokens`（提供方报告的最新提示词规模，为未缓存输入加缓存读取与写入之和），以及来自最新一条 `request/context` 记录的可选 `contextWindow`。输出不计入其中，因此轮次流式输出期间分子保持不动，等到下一个请求报告用量时才前进。
|
||||
|
||||
两个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这两个键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
|
||||
### 上下文占用率是刻意为之的近似值
|
||||
|
||||
`pressureTokens` 与 `contextWindow` 是两个各自后者胜的独立字段，**不是**对单个请求的一次原子观测。切换模型时，新容量会与上一路由的压力配对，直到下一个请求报告用量为止；而 `pressureTokens` 描述的是最后一个请求，不是此刻的表层。
|
||||
|
||||
这是刻意的选择。占用率百分比是面向用户的参考数字，既不是计费记录，也不是门控输入：harness 中没有任何环节依据它做决策，压缩改为直接读取 `measure()`。TUI 状态行一直以同样的方式计算占用率，即用 `measure()` 总量除以为所选模型单独解析出的容量。
|
||||
|
||||
让这对值保持原子已经尝试过并被否决：它需要一个临时且不可回放的协议帧，进而需要针对跨流重排序的生命周期栅栏，还会让占用率在每次重连后变为空白。[Agent Note（agent 决策记录）](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md)记录了这项对比。需要同一边界精确数字的消费方应在自己的请求边界调用 `measure()`，而不是读取该投影。
|
||||
|
||||
## 组合
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
export type { TokenUsageProjection } from './projection.ts'
|
||||
export type { ContextPressureProjection, TokenUsageProjection } from './projection.ts'
|
||||
|
||||
/** Token-meter plugin configuration; the fixed estimator has no settings. */
|
||||
export type TokenMeterConfig = Record<string, never>
|
||||
|
||||
@@ -147,5 +147,5 @@ ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
|
||||
: { ...state, pressureTokens }
|
||||
},
|
||||
view: state => state,
|
||||
stateVersion: 0,
|
||||
stateVersion: 1,
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@ import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import type { TokenUsageProjection } from '@deepseek-ai/dsh-token-meter/client'
|
||||
import type { ContextPressureProjection, TokenUsageProjection } from '@deepseek-ai/dsh-token-meter/client'
|
||||
|
||||
const ZERO: TokenUsageProjection = {
|
||||
uncachedInputTokens: 0,
|
||||
@@ -220,3 +220,93 @@ describe('tokenUsage session projection', () => {
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
const pressure = (ctx: Context, session: Session): ContextPressureProjection => {
|
||||
const value = ctx.sessionProjections.snapshot(session).values.contextPressure
|
||||
if (value === undefined) throw new Error('contextPressure projection is not registered')
|
||||
return value
|
||||
}
|
||||
|
||||
function recordContext(session: Session, model: string, contextWindow: number): void {
|
||||
session.append('request/context', { provider: 'mock', model, contextWindow })
|
||||
}
|
||||
|
||||
describe('contextPressure session projection', () => {
|
||||
it('serves zero pressure and no capacity for an empty log', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 0 })
|
||||
})
|
||||
|
||||
it('sums prompt-side buckets and excludes response output', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
usageChunk(session, {
|
||||
inputTokens: 100,
|
||||
outputTokens: 4_000,
|
||||
cacheReadTokens: 20,
|
||||
cacheWriteTokens: 5,
|
||||
}, 1, 1)
|
||||
// Output is deliberately absent: occupancy describes the prompt that was
|
||||
// sent, so it holds still while the response streams.
|
||||
expect(pressure(ctx, session).pressureTokens).toBe(125)
|
||||
})
|
||||
|
||||
it('replaces pressure with the newest request rather than accumulating', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
const first = usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
finalUsage(session, { inputTokens: 100, outputTokens: 10 }, 1, 1, [first])
|
||||
startStep(session, 2, 1)
|
||||
usageChunk(session, { inputTokens: 250, outputTokens: 10 }, 2, 1)
|
||||
expect(pressure(ctx, session).pressureTokens).toBe(250)
|
||||
})
|
||||
|
||||
it('carries the newest recorded capacity and replaces it on a model switch', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 64_000 })
|
||||
recordContext(session, 'large', 256_000)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 256_000 })
|
||||
})
|
||||
|
||||
it('pushes no change for unrelated events or a restated capacity', async () => {
|
||||
// The registry gates its change feed on Object.is, so a unit that rebuilt
|
||||
// state for an event it does not care about would push phantom updates.
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
const changed: string[] = []
|
||||
ctx.sessionProjections.onChanged((_session, key) => { changed.push(key) })
|
||||
|
||||
session.append('todo/write', { todos: [] })
|
||||
expect(changed).not.toContain('contextPressure')
|
||||
// A repeated capacity record for the same window is also a no-op.
|
||||
recordContext(session, 'small', 64_000)
|
||||
expect(changed).not.toContain('contextPressure')
|
||||
// A real capacity change still reports.
|
||||
recordContext(session, 'large', 256_000)
|
||||
expect(changed).toContain('contextPressure')
|
||||
})
|
||||
|
||||
it('restores from a JSON checkpoint and unregisters with the token-meter fiber', async () => {
|
||||
const { ctx, session, meterFiber } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 42, outputTokens: 2 }, 1, 1)
|
||||
const checkpoint = JSON.parse(JSON.stringify(
|
||||
ctx.sessionProjections.checkpoint(session),
|
||||
)) as ReturnType<typeof ctx.sessionProjections.checkpoint>
|
||||
|
||||
await meterFiber.dispose()
|
||||
expect(ctx.sessionProjections.snapshot(session).values).not.toHaveProperty('contextPressure')
|
||||
|
||||
await ctx.plugin(TokenMeterService)
|
||||
expect(ctx.sessionProjections.viewCheckpoint(checkpoint).contextPressure).toEqual({
|
||||
pressureTokens: 42,
|
||||
contextWindow: 64_000,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user