fix(web): let the context meter see a compaction
The composer ring, percentage, and `~used / capacity` header read
`contextPressure.pressureTokens`, which moves only when a request reports
usage. Compaction reports none — compact-basic summarizes through a direct
`ctx.llm.stream()` call and appends only its own `compact/*` records plus the
replacement `user/message` — so the meter was frozen across the one action
taken to change it. Driving a real `compactNow` through the agent loop:
BEFORE compact: ring=4% header=~4227/100000 rows=[18, 0, 4365]
AFTER compact: ring=4% header=~4227/100000 rows=[18, 0, 286]
The composition rows fell 93%; the ring did not move, and would not until an
entire further turn completed. The panel then contradicted itself by more than
an order of magnitude at exactly the moment a reader opens it.
`contextPressure` now also publishes `projectedTokens`: the provider sample
plus the heuristic repricing of everything the surface gained or lost since
that sample, clamped at zero, folded through the shared `surface-fold.ts`. The
sample is stamped before the same event joins the surface, so an
`assistant/message` anchors against the surface its own request carried. Only
the delta is estimated, so the figure stays provider-anchored — the estimator's
CJK and JSON-schema underpricing stays out of the occupancy number — while
reacting the moment content lands or a span is shadowed. Same run after:
BEFORE compact: ring=4% header=~4323/100000 (pressure=4227, projected=4323)
AFTER compact: ring=0% header=~ 244/100000 (pressure=4227, projected= 244)
`contextOccupancy` prefers the projected figure and falls back to the bare
sample, so a projection restored from a pre-field checkpoint degrades to the
old behavior rather than disappearing. `stateVersion` moves to 3.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: dd751b7449fb4f75199af44781ad139f2952e9a4
|
||||
README.zh.md: 4ebd693f2a6197557ed5f0724020287c37f0a2fc
|
||||
README.md: dfa56a75ddb239826b68c9d2693911ead15c3144
|
||||
README.zh.md: 4e3f9e5dec3411746897c607615206e1cba53872
|
||||
|
||||
@@ -27,15 +27,17 @@ When the composition provides `ctx.sessionProjections`, token-meter registers th
|
||||
|
||||
`tokenUsage` carries the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision. The single last-sample slot relies on a session-log ordering property: once a later step reports usage, a legal log never reports usage for an earlier step again.
|
||||
|
||||
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — and optional `contextWindow` from the newest `request/context` record. Pressure stays absent until a provider reports usage; capacity stays absent for a route whose adapter advertises none. Output is excluded, so the numerator holds still while a turn streams and steps forward when the next request reports its usage.
|
||||
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — optional `projectedTokens`, and optional `contextWindow` from the newest `request/context` record. Both figures stay absent until a provider reports usage; capacity stays absent for a route whose adapter advertises none. Output is excluded, so `pressureTokens` holds still while a turn streams and steps forward when the next request reports its usage.
|
||||
|
||||
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays `surface-fold.ts` — the same positional fold `measure()` runs — so it equals `measure().surfaceTokens` at every event boundary and compaction shrinks it the way it shrinks the next request. All three figures use the measurement service's fixed heuristic and are estimates: they do not reconcile with the provider-exact `pressureTokens`, and a UI should present them as approximations.
|
||||
`projectedTokens` is what the NEXT request's prompt would cost: the sample plus the heuristic repricing of everything the surface gained or lost since it was taken, clamped at zero and folded through the same `surface-fold.ts` the measurement service replays. Only the delta is estimated, so the figure stays anchored to the provider while reacting the moment content lands — or a compaction shadows a span. That last case is why the field exists: compaction summarizes through a direct `ctx.llm.stream()` call and appends no usage of its own, so `pressureTokens` alone reports the pre-compaction prompt until an entire further turn completes. Occupancy displays read `projectedTokens`.
|
||||
|
||||
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays `surface-fold.ts` — the same positional fold `measure()` runs — so it equals `measure().surfaceTokens` at every event boundary and compaction shrinks it the way it shrinks the next request. All three figures use the measurement service's fixed heuristic and are estimates: they will not sum to `projectedTokens`, whose provider anchor carries exactly the error — CJK text and JSON schemas underprice badly at four characters per token — that the composition rows still contain. Present them as an approximate composition, never as a total.
|
||||
|
||||
All three units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes all three keys. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
|
||||
### Context occupancy is an approximation, by design
|
||||
|
||||
`pressureTokens` and `contextWindow` are independent last-wins fields and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's pressure until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now.
|
||||
The occupancy fields are independent last-wins records and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's sample until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now — `projectedTokens` carries that sample forward over the surface's movement, but its anchor is still the older request.
|
||||
|
||||
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. The TUI status line has always computed occupancy the same way, dividing a `measure()` total by a separately-resolved capacity for the selected model.
|
||||
|
||||
|
||||
@@ -27,15 +27,17 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
|
||||
|
||||
`tokenUsage` 携带完整持久日志中的 `uncachedInputTokens`、`outputTokens`、`cacheReadTokens` 和 `cacheWriteTokens`。即使请求随后失败,用量分片仍会计入;同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本,而不是重复计数。推理仍是输出的一个细分项。只保留单个最新样本,依赖的是会话日志的一条顺序性质:一旦某个更晚的步骤报告了用量,合法日志就绝不会再为更早的步骤报告用量。
|
||||
|
||||
`contextPressure` 携带可选的 `pressureTokens`(提供方报告的最新提示词规模,为未缓存输入加缓存读取与写入之和),以及来自最新一条 `request/context` 记录的可选 `contextWindow`。提供方报告用量前压力保持缺失;路由适配器未公布容量时容量也保持缺失。输出不计入其中,因此轮次流式输出期间分子保持不动,等到下一个请求报告用量时才前进。
|
||||
`contextPressure` 携带可选的 `pressureTokens`(提供方报告的最新提示词规模,为未缓存输入加缓存读取与写入之和)、可选的 `projectedTokens`,以及来自最新一条 `request/context` 记录的可选 `contextWindow`。提供方报告用量前两个数字都保持缺失;路由适配器未公布容量时容量也保持缺失。输出不计入其中,因此轮次流式输出期间 `pressureTokens` 保持不动,等到下一个请求报告用量时才前进。
|
||||
|
||||
`contextBreakdown` 携带启发式的 `systemTokens`、`toolsTokens` 与 `messageTokens`,描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价;消息数字重放 `surface-fold.ts`——与 `measure()` 运行的位置折叠是同一份——因此它在每个事件边界上都等于 `measure().surfaceTokens`,压缩会像缩小下一个请求那样缩小它。三个数字都使用测量服务的固定启发式规则,属于估算值:它们不会与提供方精确的 `pressureTokens` 对账,UI 应以近似值方式呈现。
|
||||
`projectedTokens` 是「下一个请求的提示词要花多少」:在该样本之上,加上自取样以来表层增减部分的启发式重新计价,下界钳制为零,折叠走的是测量服务重放的同一份 `surface-fold.ts`。只有增量部分是估算的,因此这个数字既锚定在提供方读数上,又能在内容落地——或压缩遮蔽一段区间——的瞬间做出反应。最后这种情况正是该字段存在的理由:压缩通过直连的 `ctx.llm.stream()` 调用生成摘要,自身不追加任何用量,所以仅凭 `pressureTokens` 会一直报告压缩前的提示词规模,直到又跑完一整轮为止。占用率展示读取 `projectedTokens`。
|
||||
|
||||
`contextBreakdown` 携带启发式的 `systemTokens`、`toolsTokens` 与 `messageTokens`,描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价;消息数字重放 `surface-fold.ts`——与 `measure()` 运行的位置折叠是同一份——因此它在每个事件边界上都等于 `measure().surfaceTokens`,压缩会像缩小下一个请求那样缩小它。三个数字都使用测量服务的固定启发式规则,属于估算值:它们加起来不等于 `projectedTokens`——后者的提供方锚点恰好把这些明细行仍然带着的误差排除在外(按「4 字符 ≈ 1 token」计价,CJK 文本与 JSON schema 会被严重低估)。请把它们当作近似的**组成**呈现,而不是总量。
|
||||
|
||||
三个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这三个键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
|
||||
### 上下文占用率是刻意为之的近似值
|
||||
|
||||
`pressureTokens` 与 `contextWindow` 是两个各自后者胜的独立字段,**不是**对单个请求的一次原子观测。切换模型时,新容量会与上一路由的压力配对,直到下一个请求报告用量为止;而 `pressureTokens` 描述的是最后一个请求,不是此刻的表层。
|
||||
这些占用率字段各自后者胜、彼此独立,**不是**对单个请求的一次原子观测。切换模型时,新容量会与上一路由的样本配对,直到下一个请求报告用量为止;而 `pressureTokens` 描述的是最后一个请求,不是此刻的表层——`projectedTokens` 把该样本沿表层的增减推进到当下,但它的锚点仍然是那个较早的请求。
|
||||
|
||||
这是刻意的选择。占用率百分比是面向用户的参考数字,既不是计费记录,也不是门控输入:harness 中没有任何环节依据它做决策,压缩改为直接读取 `measure()`。TUI 状态行一直以同样的方式计算占用率,即用 `measure()` 总量除以为所选模型单独解析出的容量。
|
||||
|
||||
|
||||
@@ -20,14 +20,13 @@ export interface TokenUsageProjection {
|
||||
/**
|
||||
* Approximate context occupancy for a status display.
|
||||
*
|
||||
* The two fields, when present, are deliberately NOT one atomic request
|
||||
* observation: `pressureTokens` is the newest provider-reported prompt size,
|
||||
* `contextWindow` the newest recorded route capacity. Switching models can
|
||||
* therefore pair a fresh capacity with the previous route's pressure until the
|
||||
* next request reports usage. This is an intentional trade — the value is a
|
||||
* user-facing reference, not a billing or gating input — and it matches how
|
||||
* the TUI status line has always computed occupancy. See the token-meter
|
||||
* README for the full rationale.
|
||||
* The fields, when present, are deliberately NOT one atomic request
|
||||
* observation: each is a last-wins record of a different moment. Switching
|
||||
* models can therefore pair a fresh capacity with the previous route's
|
||||
* pressure until the next request reports usage. This is an intentional trade
|
||||
* — the value is a user-facing reference, not a billing or gating input — and
|
||||
* it matches how the TUI status line has always computed occupancy. See the
|
||||
* token-meter README for the full rationale.
|
||||
*/
|
||||
export interface ContextPressureProjection {
|
||||
/**
|
||||
@@ -36,6 +35,15 @@ export interface ContextPressureProjection {
|
||||
* grow as the current turn streams. Absent until a provider reports usage.
|
||||
*/
|
||||
pressureTokens?: number
|
||||
/**
|
||||
* What the NEXT request's prompt would cost: {@link pressureTokens} plus the
|
||||
* heuristic repricing of everything the surface gained or lost since that
|
||||
* sample. Only the delta is estimated, so the figure stays anchored to the
|
||||
* provider while still reacting the moment a compaction shadows a span —
|
||||
* which `pressureTokens` alone cannot do, since compaction reports no usage
|
||||
* of its own. Absent until a provider reports usage.
|
||||
*/
|
||||
projectedTokens?: number
|
||||
/** Newest recorded route capacity; absent when no adapter advertised one. */
|
||||
contextWindow?: number
|
||||
}
|
||||
@@ -43,10 +51,11 @@ export interface ContextPressureProjection {
|
||||
/**
|
||||
* Heuristic composition of the next request's context: what the prompt is
|
||||
* made of, not what it costs. All three figures use the meter's fixed
|
||||
* density estimate (they will not sum exactly to the provider-reported
|
||||
* `pressureTokens`, which is billing-grade and one request behind), and the
|
||||
* message figure tracks the live surface, so it moves as content is appended
|
||||
* or compacted while the provider number holds still.
|
||||
* density estimate, so they will not sum to the provider-anchored
|
||||
* `projectedTokens`: the estimator systematically underprices CJK text and
|
||||
* JSON schemas, which is exactly the error the anchoring in
|
||||
* {@link ContextPressureProjection.projectedTokens} keeps out of the occupancy
|
||||
* figure. Present these as approximations of composition, never as a total.
|
||||
*/
|
||||
export interface ContextBreakdownProjection {
|
||||
/** Heuristic tokens of the newest request envelope's system prompt; 0 before any request. */
|
||||
|
||||
@@ -4,8 +4,12 @@
|
||||
|
||||
import { z } from 'zod'
|
||||
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { isSurfaceEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
|
||||
import type { ContextPressureProjection, TokenUsageProjection } from './projection.ts'
|
||||
import type { TokenSurfaceNode } from './types.ts'
|
||||
import { foldSurfaceTokens } from './surface-fold.ts'
|
||||
|
||||
interface UsageSample {
|
||||
turn: number
|
||||
@@ -60,6 +64,7 @@ const projectionSchema = z.object({
|
||||
// `number | undefined` where the interface declares absent-or-number fields.
|
||||
const pressureSchema = z.object({
|
||||
pressureTokens: z.number().int().nonnegative().optional(),
|
||||
projectedTokens: z.number().int().nonnegative().optional(),
|
||||
contextWindow: z.number().int().positive().optional(),
|
||||
}).strict() as unknown as z.ZodType<ContextPressureProjection>
|
||||
|
||||
@@ -67,6 +72,29 @@ const pressureSchema = z.object({
|
||||
const pressureFrom = (usage: TokenUsage): number =>
|
||||
usage.inputTokens + (usage.cacheReadTokens ?? 0) + (usage.cacheWriteTokens ?? 0)
|
||||
|
||||
/** The usage a chunk or finalized message reports for its step, if any. */
|
||||
const usageOf = (event: SessionEvent): TokenUsage | undefined =>
|
||||
event.type === 'assistant/chunk' && event.data.chunk.type === 'usage'
|
||||
? event.data.chunk.usage
|
||||
: event.type === 'assistant/message'
|
||||
? event.data.usage
|
||||
: undefined
|
||||
|
||||
/**
|
||||
* Context-occupancy state: the two independent last-wins records plus the
|
||||
* priced surface needed to carry the newest sample forward.
|
||||
*/
|
||||
interface ContextPressureState {
|
||||
contextWindow?: number
|
||||
pressureTokens?: number
|
||||
/** Priced surface, folded identically to the measurement service's. */
|
||||
surface: TokenSurfaceNode[]
|
||||
/** Summed heuristic tokens over {@link surface}. */
|
||||
surfaceTokens: number
|
||||
/** {@link surfaceTokens} at the newest usage sample; absent until one lands. */
|
||||
sampledSurfaceTokens?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Token-meter's session projection unit.
|
||||
*
|
||||
@@ -115,20 +143,26 @@ ProjectionDefinition<'tokenUsage', TokenUsageState> = {
|
||||
/**
|
||||
* Token-meter's context-occupancy projection unit.
|
||||
*
|
||||
* Two independent last-wins slots: the newest usage sample supplies the
|
||||
* Independent last-wins slots: the newest usage sample supplies the provider
|
||||
* numerator, the newest `request/context` record the denominator. Both are
|
||||
* whole values, so replay order alone decides the result and no cross-field
|
||||
* consistency is claimed — the pair is explicitly not one atomic request
|
||||
* observation (see {@link ContextPressureProjection}).
|
||||
*
|
||||
* The numerator is prompt-side only, so it holds still while a turn streams
|
||||
* and steps forward once the next request reports its usage.
|
||||
* `pressureTokens` is prompt-side only, so it holds still while a turn streams
|
||||
* and steps forward once the next request reports its usage. Because nothing
|
||||
* but a request reports usage, it also cannot see a compaction: the fold
|
||||
* therefore carries the priced surface alongside it and publishes
|
||||
* `projectedTokens` — the sample plus the surface's signed movement since it
|
||||
* was taken — so occupancy answers for the next request rather than the last
|
||||
* one. A usage sample is stamped BEFORE the same event joins the surface, so
|
||||
* an `assistant/message` anchors against the surface its own request saw.
|
||||
*/
|
||||
export const contextPressureProjectionDefinition:
|
||||
ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
|
||||
ProjectionDefinition<'contextPressure', ContextPressureState> = {
|
||||
key: 'contextPressure',
|
||||
schema: pressureSchema,
|
||||
init: () => ({}),
|
||||
init: () => ({ surface: [], surfaceTokens: 0 }),
|
||||
apply: (state, event) => {
|
||||
if (event.type === 'request/context') {
|
||||
const contextWindow = event.data.contextWindow
|
||||
@@ -137,17 +171,24 @@ ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
|
||||
const { contextWindow: _removed, ...withoutContextWindow } = state
|
||||
return withoutContextWindow
|
||||
}
|
||||
const usage = event.type === 'assistant/chunk' && event.data.chunk.type === 'usage'
|
||||
? event.data.chunk.usage
|
||||
: event.type === 'assistant/message'
|
||||
? event.data.usage
|
||||
: undefined
|
||||
if (usage === undefined) return state
|
||||
const pressureTokens = pressureFrom(usage)
|
||||
return pressureTokens === state.pressureTokens
|
||||
? state
|
||||
: { ...state, pressureTokens }
|
||||
let next = state
|
||||
const usage = usageOf(event)
|
||||
if (usage !== undefined) {
|
||||
const pressureTokens = pressureFrom(usage)
|
||||
if (pressureTokens !== next.pressureTokens || next.sampledSurfaceTokens !== next.surfaceTokens) {
|
||||
next = { ...next, pressureTokens, sampledSurfaceTokens: next.surfaceTokens }
|
||||
}
|
||||
}
|
||||
if (!isSurfaceEvent(event)) return next
|
||||
const fold = foldSurfaceTokens(next.surface, event)
|
||||
return { ...next, surface: fold.nodes, surfaceTokens: next.surfaceTokens + fold.deltaTokens }
|
||||
},
|
||||
view: state => state,
|
||||
stateVersion: 2,
|
||||
view: ({ contextWindow, pressureTokens, surfaceTokens, sampledSurfaceTokens }) => ({
|
||||
...contextWindow === undefined ? {} : { contextWindow },
|
||||
...pressureTokens === undefined ? {} : { pressureTokens },
|
||||
...pressureTokens === undefined || sampledSurfaceTokens === undefined
|
||||
? {}
|
||||
: { projectedTokens: Math.max(0, pressureTokens + surfaceTokens - sampledSurfaceTokens) },
|
||||
}),
|
||||
stateVersion: 3,
|
||||
}
|
||||
|
||||
@@ -235,6 +235,34 @@ function recordContext(session: Session, model: string, contextWindow?: number):
|
||||
})
|
||||
}
|
||||
|
||||
/** Append one model-visible user turn and return its surface seq. */
|
||||
function appendUser(session: Session, text: string): number {
|
||||
return session.append('user/message', createUserMessage({
|
||||
content: [{ type: 'text', text }],
|
||||
source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' }).seq
|
||||
}
|
||||
|
||||
/** Append one finalized assistant turn carrying its provider usage. */
|
||||
function appendAssistant(
|
||||
session: Session,
|
||||
text: string,
|
||||
usage: TokenUsage,
|
||||
turn: number,
|
||||
step: number,
|
||||
): number {
|
||||
return session.append('assistant/message', {
|
||||
turn,
|
||||
step,
|
||||
message: createMessage({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text }],
|
||||
source: { kind: 'model', provider: 'mock', model: 'mock' },
|
||||
}),
|
||||
usage,
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [] }).seq
|
||||
}
|
||||
|
||||
describe('contextPressure session projection', () => {
|
||||
it('serves no pressure or capacity for an empty log', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
@@ -277,9 +305,13 @@ describe('contextPressure session projection', () => {
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 64_000 })
|
||||
expect(pressure(ctx, session)).toEqual({
|
||||
pressureTokens: 100, projectedTokens: 100, contextWindow: 64_000,
|
||||
})
|
||||
recordContext(session, 'large', 256_000)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 256_000 })
|
||||
expect(pressure(ctx, session)).toEqual({
|
||||
pressureTokens: 100, projectedTokens: 100, contextWindow: 256_000,
|
||||
})
|
||||
})
|
||||
|
||||
it('removes an older capacity when the newest route advertises none', async () => {
|
||||
@@ -288,7 +320,7 @@ describe('contextPressure session projection', () => {
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
recordContext(session, 'unknown')
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100 })
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, projectedTokens: 100 })
|
||||
})
|
||||
|
||||
it('pushes no change for unrelated events or a restated capacity', async () => {
|
||||
@@ -319,7 +351,7 @@ describe('contextPressure session projection', () => {
|
||||
const checkpoint = JSON.parse(JSON.stringify(
|
||||
ctx.sessionProjections.checkpoint(session),
|
||||
)) as ReturnType<typeof ctx.sessionProjections.checkpoint>
|
||||
expect(checkpoint.contextPressure?.ver).toBe(2)
|
||||
expect(checkpoint.contextPressure?.ver).toBe(3)
|
||||
|
||||
await meterFiber.dispose()
|
||||
expect(ctx.sessionProjections.snapshot(session).values).not.toHaveProperty('contextPressure')
|
||||
@@ -327,7 +359,60 @@ describe('contextPressure session projection', () => {
|
||||
await ctx.plugin(TokenMeterService)
|
||||
expect(ctx.sessionProjections.viewCheckpoint(checkpoint).contextPressure).toEqual({
|
||||
pressureTokens: 42,
|
||||
projectedTokens: 42,
|
||||
contextWindow: 64_000,
|
||||
})
|
||||
})
|
||||
|
||||
it('carries the sample forward over surface growth and a compaction', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
recordContext(session, 'large', 128_000)
|
||||
const question = appendUser(session, 'a first question worth a few tokens')
|
||||
startStep(session, 1, 1)
|
||||
// The provider prices the prompt its request actually carried; the sample
|
||||
// must anchor against the surface as of that request, not after the
|
||||
// assistant message joins it.
|
||||
const answer = appendAssistant(session, 'an answer of some length', { inputTokens: 900, outputTokens: 20 }, 1, 1)
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const afterTurn = pressure(ctx, session)
|
||||
expect(afterTurn.pressureTokens).toBe(900)
|
||||
// The assistant message landed after the sample, so it already shows.
|
||||
expect(afterTurn.projectedTokens).toBeGreaterThan(900)
|
||||
|
||||
const grown = appendUser(session, 'a follow-up question that grows the surface further')
|
||||
const beforeCompaction = pressure(ctx, session).projectedTokens
|
||||
expect(beforeCompaction).toBeGreaterThan(afterTurn.projectedTokens!)
|
||||
|
||||
// Compaction reports no usage of its own, so `pressureTokens` cannot move;
|
||||
// the projected figure must shrink anyway — the defect this field fixes.
|
||||
session.append('user/message', createUserMessage({
|
||||
content: [{ type: 'text', text: 'summary' }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
}), {
|
||||
surfaceOp: { op: 'replace', start: question, end: grown },
|
||||
sourceEventSeqs: [question, answer, grown],
|
||||
})
|
||||
const compacted = pressure(ctx, session)
|
||||
expect(compacted.pressureTokens).toBe(900)
|
||||
expect(compacted.projectedTokens).toBeLessThan(beforeCompaction!)
|
||||
})
|
||||
|
||||
it('clamps a projection that heuristic error drove below zero', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
recordContext(session, 'large', 128_000)
|
||||
const question = appendUser(session, 'a question long enough to outprice the sample'.repeat(4))
|
||||
startStep(session, 1, 1)
|
||||
// A provider sample far below the heuristic price of what it replaced:
|
||||
// shadowing that span subtracts more than the sample holds.
|
||||
appendAssistant(session, 'ok', { inputTokens: 3, outputTokens: 1 }, 1, 1)
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('user/message', createUserMessage({
|
||||
content: [{ type: 'text', text: '.' }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
}), {
|
||||
surfaceOp: { op: 'replace', start: question, end: question },
|
||||
sourceEventSeqs: [question],
|
||||
})
|
||||
expect(pressure(ctx, session).projectedTokens).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user