refactor(web): project usage and snapshot request context

This commit is contained in:
Hypatia May
2026-07-29 15:27:59 +08:00
parent e37cb23336
commit bf618dabf9
78 changed files with 748 additions and 934 deletions

View File

@@ -1,39 +0,0 @@
# Agent Note: Host-owned Web session metrics
Status: implemented
English | [中文](2026-07-28-host-owned-web-session-metrics.zh.md)
## Problem
A Web stats line derived from the currently loaded conversation nodes is window-dependent under pagination. Compaction can replace visible content without preserving historical usage, while the selected model does not prove that a request used its route or capacity. Cache-write tokens also risk being folded into a cache-hit formula whose denominator has different semantics.
## Decision
The Host owns one session-level metrics projection. It incrementally folds the complete durable event log, keys settled usage by `(turn, step)`, and replaces an earlier usage record for the same key instead of double-counting chunk and message forms. Uncached input, output, cache reads, and cache writes remain four disjoint cumulative buckets. Compaction can change the current prompt surface without erasing historical usage.
Current context pressure is the point-in-time `tokenMeter.measure(session).totalTokens`. Capacity instead belongs to the latest model request attempt observed by the current live mux connection. `LlmService.prepareCall()` retains the context metadata obtained by the exact lookup that also validates reasoning/defaults. After the final provider/model is fixed and the outer `llm/stream` call returns a handle, the loop publishes one contained `agent/model-request` notification. This boundary observes an attempt, not proof of provider I/O: preparation or a synchronous outer waterfall failure emits nothing, while short-circuit handles and later lazy adapter construction, iteration failure, or abort still count.
The tail `session.history` response carries durable usage and pressure, while older pages omit them. Live changes use `session/metrics` mux frames. Both forms carry a durable-log revision and a projection revision; the client accepts only nondecreasing revisions and preserves metrics across older-page prepend.
ApiProxy forwards each notification as a distinct `session/model-request` frame only to mux connections already open when dispatch occurs. It never places the frame in `session.history` or a subscription baseline. The client keeps durable `metrics` and transient `modelRequestContextWindow` as separate snapshot fields, replaces or explicitly clears the capacity on the next observed request, and clears both fields on `session/subscribed`; reconnect, restore, and a new subscription therefore start unknown until another request is observed.
The Web stats line joins the durable projection and live capacity only at presentation. It renders uncached input, output, and cache reads separately, computes cache hit as `cacheRead / (uncachedInput + cacheRead)`, and shows current context as a percentage only when the current connection observed a capacity. Cache writes never enter that percentage. Visible nodes continue to supply only turn and step counts.
## Alternatives considered
**Fold the loaded node window in React.** This cannot survive pagination or compaction and duplicates durable-log semantics in a presentation package.
**Send usage only with raw assistant events.** Reconnect and older-page stitching would still need the client to reconstruct a full-log aggregate, and duplicate usage forms would need protocol-specific repair there.
**Reuse one total-token field for cache hit.** Cache reads, cache writes, and uncached input represent distinct provider accounting buckets; combining them would make the displayed rate misleading.
**Query the selected route before dispatch.** Selection may never produce a request, and a second metadata lookup can race the registration-bound lookup that actually validates and dispatches the call.
**Persist or replay the latest request capacity.** That would make a former request look current on reconnect or restore even though the new connection observed no request. The denominator is deliberately live and opportunistic.
## Consequences
Token totals remain stable across pagination, replay, compaction, and browser reconnect. The client stores a small detached durable projection plus one connection-local denominator instead of scanning the conversation window, and the status row remains readable for large histories through compact number formatting.
The Host performs one incremental log fold per session and schedules durable projection updates only for usage, request-header, or surface-changing events; text and reasoning deltas do not publish metrics. A new connection omits the percentage until it observes a request with context metadata. A later request without metadata clears the denominator, while deployments without a token meter still retain the durable counters and label context unavailable instead of fabricating pressure.

View File

@@ -1,39 +0,0 @@
# Agent Note: Host 拥有的 Web 会话指标
Status: implemented
[English](2026-07-28-host-owned-web-session-metrics.md) | 中文
## 问题
Web 统计行若根据当前加载的会话节点推导指标,其结果会随分页窗口变化。压缩(compaction)可以替换可见内容,却无法保留历史用量;所选模型也不能证明某次请求实际采用了该模型的路由或容量。缓存写入 token 还可能被计入缓存命中率公式,而该公式的分母具有不同语义。
## 决策
Host 拥有一项会话级指标投影。它以增量方式归并完整的持久事件日志,按 `(turn, step)` 标识已结算用量;同一标识再次出现时,会替换较早的用量记录,而不会重复统计分片和消息两种形态。未缓存输入、输出、缓存读取与缓存写入保持为四个彼此独立的累计计数项。压缩可以改变当前提示词表层,但不会抹除历史用量。
当前上下文压力是即时的 `tokenMeter.measure(session).totalTokens`。容量则属于当前实时 mux 连接观察到的最新模型请求尝试。`LlmService.prepareCall()` 会保留同一次精确查询取得的上下文元数据,该查询也负责校验推理设置与默认值。最终提供方/模型确定且外层 `llm/stream` 调用返回句柄后,循环会发布一条 `agent/model-request` 通知,并收容该通知的失败。这个边界观察到的是一次尝试,并不能证明提供方 I/O 已开始:准备阶段或外层 waterfall(瀑布式事件)的同步失败不会发出通知,而短路句柄以及之后的惰性适配器构造、迭代失败或中止仍会计入。
`session.history` 尾页响应携带持久用量与压力,较早页面则省略这两项。实时变更使用 `session/metrics` mux 帧。两种形式都携带持久日志修订号和投影修订号;客户端只接受不减小的修订号,并在向前加载较早页面时保留指标。
ApiProxy 只把每条通知作为独立的 `session/model-request` 帧转发给分派发生时已经打开的 mux 连接。它绝不会把该帧放入 `session.history` 或订阅基线。客户端把持久 `metrics` 与临时 `modelRequestContextWindow` 保存在彼此独立的快照字段中,在观察到下一次请求时替换或显式清除容量,并在收到 `session/subscribed` 时清除这两个字段;因此,重连、恢复和新订阅都会从未知容量开始,直到观察到另一次请求。
Web 统计行只在展示时结合持久投影与实时容量。它分别呈现未缓存输入、输出与缓存读取,通过 `cacheRead / (uncachedInput + cacheRead)` 计算缓存命中率,并且只有当前连接观察到容量时,才把当前上下文显示为该容量的百分比。缓存写入绝不计入缓存命中率。可见节点仍然只提供轮次和步骤计数。
## 备选方案
**在 React 中归并已加载的节点窗口。** 此方案无法跨越分页或压缩保留数据,还会在展示包中重复实现持久日志语义。
**只随原始 assistant 事件发送用量。** 重连和较早页面拼接仍会要求客户端重建完整日志聚合,而且重复的用量形态需要在客户端按协议专门修复。
**为缓存命中率复用单一的 token 总数字段。** 缓存读取、缓存写入与未缓存输入是提供方记账中的不同计数项;将它们合并会使显示的比率产生误导。
**在分派前查询所选路由。** 选择操作可能永远不会产生请求;第二次元数据查询还可能与实际校验并分派调用的、绑定注册项的查询发生竞态。
**持久化或回放最新请求的容量。** 即使新连接没有观察到任何请求,这也会让先前请求在重连或恢复后显得仍然有效。该分母刻意只采用实时且恰好可得的数据。
## 后果
token 总量在分页、回放、压缩和浏览器重连期间保持稳定。客户端存储一项小型、脱耦的持久投影与一个连接本地分母,无需扫描会话窗口;状态行采用紧凑数字格式,因此在较长的历史记录中仍然清晰易读。
Host 为每个会话执行一次增量日志归并,仅为用量事件、请求头事件或表层变更事件调度持久投影更新;文本与推理(reasoning)增量不会发布指标。新连接在观察到带上下文元数据的请求之前不会显示百分比。后续不带元数据的请求会清除该分母;未部署 token 计量器时,系统仍保留持久计数器,并把上下文标示为不可用,而不会虚构压力值。

View File

@@ -1,6 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-host-owned-web-session-metrics.md
2026-07-28-host-owned-web-session-metrics.md: f0a0bdb8ea4c1ba1f983d016c3cfde9c67a0424d
2026-07-28-host-owned-web-session-metrics.zh.md: 4535fd8a4b760aa031c11ab766b6be583b9b740d
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md
2026-07-29-projected-token-usage-and-request-context.md: 3ac8c29f7752828f2d833359293c7b1c513d75ca
2026-07-29-projected-token-usage-and-request-context.zh.md: 02b03d1516a39b7727b214b63a9039623b69c56b

View File

@@ -0,0 +1,45 @@
# Agent Note: Projected token usage and request context
Status: implemented
English | [中文](2026-07-29-projected-token-usage-and-request-context.zh.md)
## Problem
A Web stats line derived from the currently loaded conversation nodes is window-dependent under pagination. Compaction can replace visible content without preserving historical usage. Conversely, context occupancy describes one real request boundary: a selected model is only an intention, and combining token pressure from one moment with capacity resolved for another route creates a false percentage.
These two values therefore have different lifetimes. Provider-reported billing is durable, replayable session state. Request pressure and registration-bound capacity are an opportunistic live observation that must disappear across a connection generation.
## Decision
`@deepseek-ai/dsh-token-meter` registers the generic `tokenUsage` session projection when `ctx.sessionProjections` is present. The projection folds the complete durable log into uncached input, output, cache-read, and cache-write buckets. An `assistant/chunk` usage sample survives a later failed request; an `assistant/message` usage value replaces the earlier value for the same `(turn, step)` instead of being counted twice. Reasoning tokens remain an output subdivision and are not added again. Compaction and surface replacement do not erase earlier billing.
The projection uses the standard projection lifecycle and wire path. History tail baselines, `session/projection` live frames, higher-seq-wins client storage, JSON checkpoints, cache recovery, and unit unload all remain generic. There is no token-specific history field, mux frame, projector, revision counter, or client fence.
`LlmService.prepareCall()` retains context metadata from the exact lookup that also validates reasoning and captures the adapter registration. After the outer stream call returns its handle and before iteration begins, AgentLoop emits one contained `agent/model-request` notification. Preparation or a synchronous outer waterfall failure emits nothing; short-circuit handles and later iterator construction, iteration, or abort failures still count as an observed request attempt.
ApiProxy handles that notification synchronously. It reads `tokenMeter.measure(agent.session).totalTokens` once when the optional service is present and combines the result with the same prepared call's registration-bound `contextWindow`. It broadcasts one atomic `session/model-request` frame containing the route, turn, step, and whichever of `contextTokens` and `contextWindow` are available. Measurement failure omits only the numerator. The frame goes only to mux connections already open at that instant; history, subscription baselines, reconnect, and restore never replay it.
The client stores the complete latest request frame as `ConversationSnapshot.modelRequest`. Every later frame replaces the entire snapshot, so omitted fields clear earlier values. `SessionManager` temporarily holds a pre-instantiation frame, while a new subscription generation, disconnect, or session removal clears both resident and pending values. Model selection alone does not change this snapshot.
The Web `StatsLine` reads `tokenUsage` through the standard `useProjection` hook and reads request telemetry plus visible nodes through `useSession`. It renders uncached input, output, and cache reads separately, computes cache hit as `cacheRead / (uncachedInput + cacheRead)`, and shows context occupancy only when one request snapshot contains both numerator and capacity. Visible nodes continue to supply only turn and step counts. The existing inline text UI is retained; the model selector gains no circle or other accessory.
## Alternatives considered
**A custom session metrics history field and mux frame.** This duplicated the generic projection protocol, cache, recovery, and seq fencing while coupling durable billing to transient request pressure.
**Fold the loaded node window in React.** This cannot survive pagination or compaction and makes a presentation package reconstruct log semantics.
**Publish usage only with final assistant messages.** A request that reports a usage chunk and then fails would lose provider billing.
**Query capacity from the selected model.** Selection may never produce a request, and a second metadata lookup can disagree with the registration-bound lookup used by the actual call.
**Persist or replay the latest request snapshot.** A request from a prior connection would appear current after restore even though no new request was observed.
**Add a context circle beside the model selector.** That placement suggests selected-model state. The existing stats line expresses the request-scoped semantics without introducing a duplicate UI or data path.
## Consequences
Token totals stay stable across pagination, compaction, replay, and reconnect because they are ordinary durable projection state. Context occupancy is deliberately unknown after reconnect until a new real request is observed. Deployments without token-meter or without model capacity still publish the request route and clear stale optional fields instead of fabricating a percentage.
ApiProxy performs one synchronous optional measurement and one frame conversion per observed request. It owns no per-session metrics cache or refresh queue. The browser keeps one generic projection value plus one small connection-local request snapshot, and streaming text deltas do not force the stats line to recompute.

View File

@@ -0,0 +1,45 @@
# Agent Note:token 用量投影与请求上下文
Status: implemented
[English](2026-07-29-projected-token-usage-and-request-context.md) | 中文
## 问题
Web 统计行若根据当前已加载的会话节点推导,其结果会在分页时依赖当前窗口。压缩(compaction)可以替换可见内容,却不保留历史用量。另一方面,上下文占用率描述的是一个真实请求边界:所选模型只代表意图;若把某一时刻的 token 压力与另一路由解析出的容量组合,就会产生虚假百分比。
因此,这两个值具有不同的生命周期。提供方报告的计费用量属于持久、可回放的会话状态。请求压力和与注册项绑定的容量则是恰好可得的实时观测,跨连接代次时必须消失。
## 决策
当 `ctx.sessionProjections` 存在时,`@deepseek-ai/dsh-token-meter` 会注册通用的 `tokenUsage` 会话投影。该投影将完整持久日志归并为未缓存输入、输出、缓存读取和缓存写入四类计数项。即使后续请求失败,`assistant/chunk` 用量样本仍会保留;同一 `(turn, step)` 的 `assistant/message` 用量值会替换先前值,不会重复计数。推理(reasoning)token 仍是输出的细分项,不会再次累加。压缩和表层替换不会抹除先前的计费用量。
该投影使用标准的投影生命周期与协议路径。历史尾页基线、`session/projection` 实时帧、seq 高者胜的客户端存储、JSON 检查点、缓存恢复和单元卸载均保持通用机制。系统没有任何 token 专用的历史字段、mux 帧、投影器、修订计数器或客户端 seq 防护机制。
`LlmService.prepareCall()` 会保留精确查询得到的上下文元数据;同一次查询还会校验推理强度并捕获适配器注册项。外层流调用返回句柄后、开始迭代前,AgentLoop 会发出一条失败受收容的 `agent/model-request` 通知。准备阶段或外层 waterfall(瀑布式事件)的同步失败不会发出通知;短路句柄以及之后的迭代器构造失败、迭代失败或中止仍算作一次已观测的请求尝试。
ApiProxy 会同步处理该通知。当可选服务存在时,它会读取一次 `tokenMeter.measure(agent.session).totalTokens`,并将结果与同一准备完成调用中绑定注册项的 `contextWindow` 合并。它会广播一个原子 `session/model-request` 帧,其中包含路由、轮次、步骤,以及可用的 `contextTokens`/`contextWindow` 字段。测量失败时只省略分子。该帧只发送给当时已经打开的 mux 连接;历史记录、订阅基线、重连和恢复都绝不回放该帧。
客户端将最新的完整请求帧存储为 `ConversationSnapshot.modelRequest`。每个后续帧都会替换整个快照,因此省略字段会清除先前值。`SessionManager` 会临时保存一个实例化前帧;新订阅代次、断开连接或移除会话时,则会同时清除常驻值和待处理值。仅选择模型不会改变该快照。
Web `StatsLine` 通过标准 `useProjection` 钩子读取 `tokenUsage`,并通过 `useSession` 读取请求观测数据与可见节点。它分别显示未缓存输入、输出与缓存读取,通过 `cacheRead / (uncachedInput + cacheRead)` 计算缓存命中率,并且只有同一份请求快照同时包含分子与容量时才显示上下文占用率。可见节点仍只提供轮次和步骤计数。系统保留现有的行内文本 UI;模型选择器不增加圆环或其他附属控件。
## 备选方案
**自定义会话指标历史字段和 mux 帧。** 这会重复实现通用投影协议、缓存、恢复和 seq 防护机制,并将持久计费用量与临时请求压力耦合。
**在 React 中归并已加载的节点窗口。** 此方案无法跨分页或压缩保留数据,还会迫使展示包重建日志语义。
**仅随最终 assistant 消息发布用量。** 如果请求报告一个用量分片后失败,就会丢失提供方计费用量。
**从所选模型查询容量。** 选择操作可能永远不会产生请求;第二次元数据查询还可能与实际调用所用的、绑定注册项的查询不一致。
**持久化或回放最新请求快照。** 即使没有观察到任何新请求,先前连接的请求也会在恢复后显得仍是当前请求。
**在模型选择器旁增加上下文圆环。** 该位置会让人以为这是所选模型的状态。现有统计行可以表达按请求作用域的语义,无需引入重复的 UI 或数据路径。
## 后果
token 总量在分页、压缩、回放和重连期间保持稳定,因为它们属于普通的持久投影状态。重连后,上下文占用率会刻意保持未知,直到系统观察到新的真实请求。未部署 token-meter 或模型不提供容量时,系统仍会发布请求路由,并清除陈旧的可选字段,而不会虚构百分比。
ApiProxy 会为每次已观测请求执行一次可选的同步测量和一次帧转换。它不拥有任何逐会话指标缓存或刷新队列。浏览器只保留一个通用投影值和一个小型连接本地请求快照;流式文本增量不会迫使统计行重新计算。