Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(token-meter): make context occupancy durable projection state

Browse Source 
Replace the transient `session/model-request` mux frame with ordinary durable
session state. Occupancy now rides two last-wins projection fields instead of a
non-replayable frame that needed removal tombstones and cross-stream fencing.

The frame was the only non-replayable class on the mux stream. Because host and
mux are independent SSE streams with no cross-stream order, a request emitted
before a removal could arrive after `host/session-removed`, and a legitimate
request for a new lifecycle reusing the same id could be fenced by a late
removal. Fixing that needed a lifecycle generation on every frame; the frame
itself was the problem.

Removed: the `session/model-request` frame and schema, the `agent/model-request`
core event, the ApiProxy measurement point, the client-side telemetry map and
removal tombstone, and the synthetic `cancelled` open error used to signal
reconnect through the error channel.

Added: `request/context`, a log-only session event recording the
registration-bound capacity of the route a request resolved to, appended beside
`request/header` from the lookup that already prepared the call and skipped when
the route is unchanged. Capacity stays out of `EpochHeader` because it is
adapter metadata about a route, not an input the request was built from, so it
must not join request reconstruction or header equality.

The `contextPressure` projection pairs the newest provider-reported prompt size
with the newest recorded capacity. The two are deliberately not one atomic
request observation: switching models can pair a fresh capacity with the prior
route's pressure until the next request reports usage. The figure is a
user-facing reference, and this matches how the TUI status line has always
computed occupancy.
This commit is contained in:
Hypatia May
2026-07-30 13:53:08 +08:00
parent fdccc58cef
commit 4819210142
62 changed files with 382 additions and 1362 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
packages/client/runtime/tests/client-apply.spec.ts
View File

@@ -102,60 +102,6 @@ describe('runtime client apply', () => {
expect(bench.api.callsOf('session.create')).toHaveLength(1)
})
it('clears connection-local request telemetry after every disconnected generation but not connected', async () => {
const bench = await mount()
const sessions = bench.ctx.get('sessions') as SessionsService
bench.sinks?.onHostEnvelope?.({
rpcId: 'session' as never,
payload: { type: 'host/session-added', blank: true, sessionId: 's-state' } as never,
})
await Promise.resolve()
const session = sessions.binding('s-state' as never)?.session
if (session === undefined) throw new Error('session binding missing')
bench.sinks?.onMuxEnvelope?.({
rpcId: 'request' as never,
payload: {
type: 'session/model-request',
sessionId: 's-state',
turn: 1,
step: 1,
provider: 'test',
model: 'alpha',
contextTokens: 32_000,
contextWindow: 128_000,
} as never,
})
bench.sinks?.onConnected?.()
expect(session.getSnapshot().modelRequest).toMatchObject({
model: 'alpha',
contextTokens: 32_000,
contextWindow: 128_000,
})
bench.sinks?.onDisconnected?.()
expect(session.getSnapshot().modelRequest).toBeNull()
// Every failed generation invokes its own disconnect callback, which
// clears telemetry received before that generation's handshake failed.
bench.sinks?.onMuxEnvelope?.({
rpcId: 'request-2' as never,
payload: {
type: 'session/model-request',
sessionId: 's-state',
turn: 2,
step: 1,
provider: 'test',
model: 'beta',
contextTokens: 48_000,
contextWindow: 256_000,
} as never,
})
expect(session.getSnapshot().modelRequest?.model).toBe('beta')
bench.sinks?.onDisconnected?.()
expect(session.getSnapshot().modelRequest).toBeNull()
})
it('stops the stream loop when the plugin fiber unloads', async () => {
const bench = await mount()
const fiber = [...bench.ctx.registry.values()].find(f => f.name?.includes('client'))