fix(web): let the context meter see a compaction

The composer ring, percentage, and `~used / capacity` header read
`contextPressure.pressureTokens`, which moves only when a request reports
usage. Compaction reports none — compact-basic summarizes through a direct
`ctx.llm.stream()` call and appends only its own `compact/*` records plus the
replacement `user/message` — so the meter was frozen across the one action
taken to change it. Driving a real `compactNow` through the agent loop:

    BEFORE compact:  ring=4%  header=~4227/100000  rows=[18, 0, 4365]
    AFTER  compact:  ring=4%  header=~4227/100000  rows=[18, 0,  286]

The composition rows fell 93%; the ring did not move, and would not until an
entire further turn completed. The panel then contradicted itself by more than
an order of magnitude at exactly the moment a reader opens it.

`contextPressure` now also publishes `projectedTokens`: the provider sample
plus the heuristic repricing of everything the surface gained or lost since
that sample, clamped at zero, folded through the shared `surface-fold.ts`. The
sample is stamped before the same event joins the surface, so an
`assistant/message` anchors against the surface its own request carried. Only
the delta is estimated, so the figure stays provider-anchored — the estimator's
CJK and JSON-schema underpricing stays out of the occupancy number — while
reacting the moment content lands or a span is shadowed. Same run after:

    BEFORE compact:  ring=4%  header=~4323/100000  (pressure=4227, projected=4323)
    AFTER  compact:  ring=0%  header=~ 244/100000  (pressure=4227, projected= 244)

`contextOccupancy` prefers the projected figure and falls back to the bare
sample, so a projection restored from a pre-field checkpoint degrades to the
old behavior rather than disappearing. `stateVersion` moves to 3.
This commit is contained in:
Yichen Jiang
2026-08-05 17:00:48 +08:00
parent e62cbe12e4
commit 038699bcb4
20 changed files with 327 additions and 64 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
README.md: dd751b7449fb4f75199af44781ad139f2952e9a4
README.zh.md: 4ebd693f2a6197557ed5f0724020287c37f0a2fc
README.md: dfa56a75ddb239826b68c9d2693911ead15c3144
README.zh.md: 4e3f9e5dec3411746897c607615206e1cba53872

View File

@@ -27,15 +27,17 @@ When the composition provides `ctx.sessionProjections`, token-meter registers th
`tokenUsage` carries the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision. The single last-sample slot relies on a session-log ordering property: once a later step reports usage, a legal log never reports usage for an earlier step again.
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — and optional `contextWindow` from the newest `request/context` record. Pressure stays absent until a provider reports usage; capacity stays absent for a route whose adapter advertises none. Output is excluded, so the numerator holds still while a turn streams and steps forward when the next request reports its usage.
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — optional `projectedTokens`, and optional `contextWindow` from the newest `request/context` record. Both figures stay absent until a provider reports usage; capacity stays absent for a route whose adapter advertises none. Output is excluded, so `pressureTokens` holds still while a turn streams and steps forward when the next request reports its usage.
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays `surface-fold.ts` the same positional fold `measure()` runs — so it equals `measure().surfaceTokens` at every event boundary and compaction shrinks it the way it shrinks the next request. All three figures use the measurement service's fixed heuristic and are estimates: they do not reconcile with the provider-exact `pressureTokens`, and a UI should present them as approximations.
`projectedTokens` is what the NEXT request's prompt would cost: the sample plus the heuristic repricing of everything the surface gained or lost since it was taken, clamped at zero and folded through the same `surface-fold.ts` the measurement service replays. Only the delta is estimated, so the figure stays anchored to the provider while reacting the moment content lands — or a compaction shadows a span. That last case is why the field exists: compaction summarizes through a direct `ctx.llm.stream()` call and appends no usage of its own, so `pressureTokens` alone reports the pre-compaction prompt until an entire further turn completes. Occupancy displays read `projectedTokens`.
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays `surface-fold.ts` — the same positional fold `measure()` runs — so it equals `measure().surfaceTokens` at every event boundary and compaction shrinks it the way it shrinks the next request. All three figures use the measurement service's fixed heuristic and are estimates: they will not sum to `projectedTokens`, whose provider anchor carries exactly the error — CJK text and JSON schemas underprice badly at four characters per token — that the composition rows still contain. Present them as an approximate composition, never as a total.
All three units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes all three keys. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
### Context occupancy is an approximation, by design
`pressureTokens` and `contextWindow` are independent last-wins fields and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's pressure until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now.
The occupancy fields are independent last-wins records and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's sample until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now`projectedTokens` carries that sample forward over the surface's movement, but its anchor is still the older request.
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. The TUI status line has always computed occupancy the same way, dividing a `measure()` total by a separately-resolved capacity for the selected model.

View File

@@ -27,15 +27,17 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
`tokenUsage` 携带完整持久日志中的 `uncachedInputTokens``outputTokens``cacheReadTokens``cacheWriteTokens`。即使请求随后失败,用量分片仍会计入;同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本,而不是重复计数。推理仍是输出的一个细分项。只保留单个最新样本,依赖的是会话日志的一条顺序性质:一旦某个更晚的步骤报告了用量,合法日志就绝不会再为更早的步骤报告用量。
`contextPressure` 携带可选的 `pressureTokens`(提供方报告的最新提示词规模,为未缓存输入加缓存读取与写入之和),以及来自最新一条 `request/context` 记录的可选 `contextWindow`。提供方报告用量前压力保持缺失;路由适配器未公布容量时容量也保持缺失。输出不计入其中,因此轮次流式输出期间分子保持不动,等到下一个请求报告用量时才前进。
`contextPressure` 携带可选的 `pressureTokens`(提供方报告的最新提示词规模,为未缓存输入加缓存读取与写入之和)、可选的 `projectedTokens`,以及来自最新一条 `request/context` 记录的可选 `contextWindow`。提供方报告用量前两个数字都保持缺失;路由适配器未公布容量时容量也保持缺失。输出不计入其中,因此轮次流式输出期间 `pressureTokens` 保持不动,等到下一个请求报告用量时才前进。
`contextBreakdown` 携带启发式的 `systemTokens``toolsTokens``messageTokens`描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价;消息数字重放 `surface-fold.ts`——与 `measure()` 运行的位置折叠是同一份——因此它在每个事件边界上都等于 `measure().surfaceTokens`,压缩会像缩小下一个请求那样缩小它。三个数字都使用测量服务的固定启发式规则,属于估算值:它们不会与提供方精确的 `pressureTokens` 对账UI 应以近似值方式呈现
`projectedTokens` 是「下一个请求的提示词要花多少」:在该样本之上,加上自取样以来表层增减部分的启发式重新计价,下界钳制为零,折叠走的是测量服务重放的同一份 `surface-fold.ts`。只有增量部分是估算的,因此这个数字既锚定在提供方读数上,又能在内容落地——或压缩遮蔽一段区间——的瞬间做出反应。最后这种情况正是该字段存在的理由:压缩通过直连的 `ctx.llm.stream()` 调用生成摘要,自身不追加任何用量,所以仅凭 `pressureTokens` 会一直报告压缩前的提示词规模,直到又跑完一整轮为止。占用率展示读取 `projectedTokens`
`contextBreakdown` 携带启发式的 `systemTokens``toolsTokens``messageTokens`描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价;消息数字重放 `surface-fold.ts`——与 `measure()` 运行的位置折叠是同一份——因此它在每个事件边界上都等于 `measure().surfaceTokens`,压缩会像缩小下一个请求那样缩小它。三个数字都使用测量服务的固定启发式规则,属于估算值:它们加起来不等于 `projectedTokens`——后者的提供方锚点恰好把这些明细行仍然带着的误差排除在外按「4 字符 ≈ 1 token」计价CJK 文本与 JSON schema 会被严重低估)。请把它们当作近似的**组成**呈现,而不是总量。
三个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这三个键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
### 上下文占用率是刻意为之的近似值
`pressureTokens``contextWindow` 是两个各自后者胜的独立字段**不是**对单个请求的一次原子观测。切换模型时,新容量会与上一路由的压力配对,直到下一个请求报告用量为止;而 `pressureTokens` 描述的是最后一个请求,不是此刻的表层。
这些占用率字段各自后者胜、彼此独立**不是**对单个请求的一次原子观测。切换模型时,新容量会与上一路由的样本配对,直到下一个请求报告用量为止;而 `pressureTokens` 描述的是最后一个请求,不是此刻的表层——`projectedTokens` 把该样本沿表层的增减推进到当下,但它的锚点仍然是那个较早的请求
这是刻意的选择。占用率百分比是面向用户的参考数字既不是计费记录也不是门控输入harness 中没有任何环节依据它做决策,压缩改为直接读取 `measure()`。TUI 状态行一直以同样的方式计算占用率,即用 `measure()` 总量除以为所选模型单独解析出的容量。

View File

@@ -20,14 +20,13 @@ export interface TokenUsageProjection {
/**
* Approximate context occupancy for a status display.
*
* The two fields, when present, are deliberately NOT one atomic request
* observation: `pressureTokens` is the newest provider-reported prompt size,
* `contextWindow` the newest recorded route capacity. Switching models can
* therefore pair a fresh capacity with the previous route's pressure until the
* next request reports usage. This is an intentional trade — the value is a
* user-facing reference, not a billing or gating input — and it matches how
* the TUI status line has always computed occupancy. See the token-meter
* README for the full rationale.
* The fields, when present, are deliberately NOT one atomic request
* observation: each is a last-wins record of a different moment. Switching
* models can therefore pair a fresh capacity with the previous route's
* pressure until the next request reports usage. This is an intentional trade
* — the value is a user-facing reference, not a billing or gating input — and
* it matches how the TUI status line has always computed occupancy. See the
* token-meter README for the full rationale.
*/
export interface ContextPressureProjection {
/**
@@ -36,6 +35,15 @@ export interface ContextPressureProjection {
* grow as the current turn streams. Absent until a provider reports usage.
*/
pressureTokens?: number
/**
* What the NEXT request's prompt would cost: {@link pressureTokens} plus the
* heuristic repricing of everything the surface gained or lost since that
* sample. Only the delta is estimated, so the figure stays anchored to the
* provider while still reacting the moment a compaction shadows a span —
* which `pressureTokens` alone cannot do, since compaction reports no usage
* of its own. Absent until a provider reports usage.
*/
projectedTokens?: number
/** Newest recorded route capacity; absent when no adapter advertised one. */
contextWindow?: number
}
@@ -43,10 +51,11 @@ export interface ContextPressureProjection {
/**
* Heuristic composition of the next request's context: what the prompt is
* made of, not what it costs. All three figures use the meter's fixed
* density estimate (they will not sum exactly to the provider-reported
* `pressureTokens`, which is billing-grade and one request behind), and the
* message figure tracks the live surface, so it moves as content is appended
* or compacted while the provider number holds still.
* density estimate, so they will not sum to the provider-anchored
* `projectedTokens`: the estimator systematically underprices CJK text and
* JSON schemas, which is exactly the error the anchoring in
* {@link ContextPressureProjection.projectedTokens} keeps out of the occupancy
* figure. Present these as approximations of composition, never as a total.
*/
export interface ContextBreakdownProjection {
/** Heuristic tokens of the newest request envelope's system prompt; 0 before any request. */

View File

@@ -4,8 +4,12 @@
import { z } from 'zod'
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
import { isSurfaceEvent } from '@deepseek-ai/dsh-session'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
import type { ContextPressureProjection, TokenUsageProjection } from './projection.ts'
import type { TokenSurfaceNode } from './types.ts'
import { foldSurfaceTokens } from './surface-fold.ts'
interface UsageSample {
turn: number
@@ -60,6 +64,7 @@ const projectionSchema = z.object({
// `number | undefined` where the interface declares absent-or-number fields.
const pressureSchema = z.object({
pressureTokens: z.number().int().nonnegative().optional(),
projectedTokens: z.number().int().nonnegative().optional(),
contextWindow: z.number().int().positive().optional(),
}).strict() as unknown as z.ZodType<ContextPressureProjection>
@@ -67,6 +72,29 @@ const pressureSchema = z.object({
const pressureFrom = (usage: TokenUsage): number =>
usage.inputTokens + (usage.cacheReadTokens ?? 0) + (usage.cacheWriteTokens ?? 0)
/** The usage a chunk or finalized message reports for its step, if any. */
const usageOf = (event: SessionEvent): TokenUsage | undefined =>
event.type === 'assistant/chunk' && event.data.chunk.type === 'usage'
? event.data.chunk.usage
: event.type === 'assistant/message'
? event.data.usage
: undefined
/**
* Context-occupancy state: the two independent last-wins records plus the
* priced surface needed to carry the newest sample forward.
*/
interface ContextPressureState {
contextWindow?: number
pressureTokens?: number
/** Priced surface, folded identically to the measurement service's. */
surface: TokenSurfaceNode[]
/** Summed heuristic tokens over {@link surface}. */
surfaceTokens: number
/** {@link surfaceTokens} at the newest usage sample; absent until one lands. */
sampledSurfaceTokens?: number
}
/**
* Token-meter's session projection unit.
*
@@ -115,20 +143,26 @@ ProjectionDefinition<'tokenUsage', TokenUsageState> = {
/**
* Token-meter's context-occupancy projection unit.
*
* Two independent last-wins slots: the newest usage sample supplies the
* Independent last-wins slots: the newest usage sample supplies the provider
* numerator, the newest `request/context` record the denominator. Both are
* whole values, so replay order alone decides the result and no cross-field
* consistency is claimed — the pair is explicitly not one atomic request
* observation (see {@link ContextPressureProjection}).
*
* The numerator is prompt-side only, so it holds still while a turn streams
* and steps forward once the next request reports its usage.
* `pressureTokens` is prompt-side only, so it holds still while a turn streams
* and steps forward once the next request reports its usage. Because nothing
* but a request reports usage, it also cannot see a compaction: the fold
* therefore carries the priced surface alongside it and publishes
* `projectedTokens` — the sample plus the surface's signed movement since it
* was taken — so occupancy answers for the next request rather than the last
* one. A usage sample is stamped BEFORE the same event joins the surface, so
* an `assistant/message` anchors against the surface its own request saw.
*/
export const contextPressureProjectionDefinition:
ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
ProjectionDefinition<'contextPressure', ContextPressureState> = {
key: 'contextPressure',
schema: pressureSchema,
init: () => ({}),
init: () => ({ surface: [], surfaceTokens: 0 }),
apply: (state, event) => {
if (event.type === 'request/context') {
const contextWindow = event.data.contextWindow
@@ -137,17 +171,24 @@ ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
const { contextWindow: _removed, ...withoutContextWindow } = state
return withoutContextWindow
}
const usage = event.type === 'assistant/chunk' && event.data.chunk.type === 'usage'
? event.data.chunk.usage
: event.type === 'assistant/message'
? event.data.usage
: undefined
if (usage === undefined) return state
const pressureTokens = pressureFrom(usage)
return pressureTokens === state.pressureTokens
? state
: { ...state, pressureTokens }
let next = state
const usage = usageOf(event)
if (usage !== undefined) {
const pressureTokens = pressureFrom(usage)
if (pressureTokens !== next.pressureTokens || next.sampledSurfaceTokens !== next.surfaceTokens) {
next = { ...next, pressureTokens, sampledSurfaceTokens: next.surfaceTokens }
}
}
if (!isSurfaceEvent(event)) return next
const fold = foldSurfaceTokens(next.surface, event)
return { ...next, surface: fold.nodes, surfaceTokens: next.surfaceTokens + fold.deltaTokens }
},
view: state => state,
stateVersion: 2,
view: ({ contextWindow, pressureTokens, surfaceTokens, sampledSurfaceTokens }) => ({
...contextWindow === undefined ? {} : { contextWindow },
...pressureTokens === undefined ? {} : { pressureTokens },
...pressureTokens === undefined || sampledSurfaceTokens === undefined
? {}
: { projectedTokens: Math.max(0, pressureTokens + surfaceTokens - sampledSurfaceTokens) },
}),
stateVersion: 3,
}

View File

@@ -235,6 +235,34 @@ function recordContext(session: Session, model: string, contextWindow?: number):
})
}
/** Append one model-visible user turn and return its surface seq. */
function appendUser(session: Session, text: string): number {
return session.append('user/message', createUserMessage({
content: [{ type: 'text', text }],
source: { kind: 'user' },
}), { surfaceOp: 'append' }).seq
}
/** Append one finalized assistant turn carrying its provider usage. */
function appendAssistant(
session: Session,
text: string,
usage: TokenUsage,
turn: number,
step: number,
): number {
return session.append('assistant/message', {
turn,
step,
message: createMessage({
role: 'assistant',
content: [{ type: 'text', text }],
source: { kind: 'model', provider: 'mock', model: 'mock' },
}),
usage,
}, { surfaceOp: 'append', sourceEventSeqs: [] }).seq
}
describe('contextPressure session projection', () => {
it('serves no pressure or capacity for an empty log', async () => {
const { ctx, session } = await harness()
@@ -277,9 +305,13 @@ describe('contextPressure session projection', () => {
startStep(session, 1, 1)
recordContext(session, 'small', 64_000)
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 64_000 })
expect(pressure(ctx, session)).toEqual({
pressureTokens: 100, projectedTokens: 100, contextWindow: 64_000,
})
recordContext(session, 'large', 256_000)
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 256_000 })
expect(pressure(ctx, session)).toEqual({
pressureTokens: 100, projectedTokens: 100, contextWindow: 256_000,
})
})
it('removes an older capacity when the newest route advertises none', async () => {
@@ -288,7 +320,7 @@ describe('contextPressure session projection', () => {
recordContext(session, 'small', 64_000)
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
recordContext(session, 'unknown')
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100 })
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, projectedTokens: 100 })
})
it('pushes no change for unrelated events or a restated capacity', async () => {
@@ -319,7 +351,7 @@ describe('contextPressure session projection', () => {
const checkpoint = JSON.parse(JSON.stringify(
ctx.sessionProjections.checkpoint(session),
)) as ReturnType<typeof ctx.sessionProjections.checkpoint>
expect(checkpoint.contextPressure?.ver).toBe(2)
expect(checkpoint.contextPressure?.ver).toBe(3)
await meterFiber.dispose()
expect(ctx.sessionProjections.snapshot(session).values).not.toHaveProperty('contextPressure')
@@ -327,7 +359,60 @@ describe('contextPressure session projection', () => {
await ctx.plugin(TokenMeterService)
expect(ctx.sessionProjections.viewCheckpoint(checkpoint).contextPressure).toEqual({
pressureTokens: 42,
projectedTokens: 42,
contextWindow: 64_000,
})
})
it('carries the sample forward over surface growth and a compaction', async () => {
const { ctx, session } = await harness()
recordContext(session, 'large', 128_000)
const question = appendUser(session, 'a first question worth a few tokens')
startStep(session, 1, 1)
// The provider prices the prompt its request actually carried; the sample
// must anchor against the surface as of that request, not after the
// assistant message joins it.
const answer = appendAssistant(session, 'an answer of some length', { inputTokens: 900, outputTokens: 20 }, 1, 1)
session.append('step/end', { turn: 1, step: 1 })
const afterTurn = pressure(ctx, session)
expect(afterTurn.pressureTokens).toBe(900)
// The assistant message landed after the sample, so it already shows.
expect(afterTurn.projectedTokens).toBeGreaterThan(900)
const grown = appendUser(session, 'a follow-up question that grows the surface further')
const beforeCompaction = pressure(ctx, session).projectedTokens
expect(beforeCompaction).toBeGreaterThan(afterTurn.projectedTokens!)
// Compaction reports no usage of its own, so `pressureTokens` cannot move;
// the projected figure must shrink anyway — the defect this field fixes.
session.append('user/message', createUserMessage({
content: [{ type: 'text', text: 'summary' }],
source: { kind: 'plugin', plugin: 'test' },
}), {
surfaceOp: { op: 'replace', start: question, end: grown },
sourceEventSeqs: [question, answer, grown],
})
const compacted = pressure(ctx, session)
expect(compacted.pressureTokens).toBe(900)
expect(compacted.projectedTokens).toBeLessThan(beforeCompaction!)
})
it('clamps a projection that heuristic error drove below zero', async () => {
const { ctx, session } = await harness()
recordContext(session, 'large', 128_000)
const question = appendUser(session, 'a question long enough to outprice the sample'.repeat(4))
startStep(session, 1, 1)
// A provider sample far below the heuristic price of what it replaced:
// shadowing that span subtracts more than the sample holds.
appendAssistant(session, 'ok', { inputTokens: 3, outputTokens: 1 }, 1, 1)
session.append('step/end', { turn: 1, step: 1 })
session.append('user/message', createUserMessage({
content: [{ type: 'text', text: '.' }],
source: { kind: 'plugin', plugin: 'test' },
}), {
surfaceOp: { op: 'replace', start: question, end: question },
sourceEventSeqs: [question],
})
expect(pressure(ctx, session).projectedTokens).toBe(0)
})
})