refactor(llm): simplify live request telemetry (round 2)

This commit is contained in:
Hypatia May
2026-07-28 19:03:45 +08:00
parent 35b9c454e5
commit e6ce6abd7d
31 changed files with 159 additions and 187 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/core/agent-loop/README.md
README.md: 39eaafd3abb2faef045c1a2f694d8dffe95c28b0
README.zh.md: f8cc972e957fe95f652d54e859f8e5411288db51
README.md: 6e23d9f543998d5c0266c73197ee52ede51260aa
README.zh.md: cf561a03281f80b7afd5f245752e9ebf70333a8b

View File

@@ -62,7 +62,7 @@ The driver owns one agent for its lifetime and runs inside `ctx.agents.withIniti
Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. The anchor records the assembled content as-is, retains exact chunk provenance (`[]` for a stream with no chunks), and includes usage when available; empty content stays out of derived message history.
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort, materialize its configured default, and retain available context metadata from that same exact-model lookup under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. After the final stream handle is constructed and before adapter iteration, the loop emits one contained live `agent/model-request` notification with turn, step, final provider/model, and optional registration-bound capacity. Preparation or synchronous stream-construction failures emit nothing; later failure or abortion remains an observed dispatch. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort, materialize its configured default, and retain available context metadata from that same exact-model lookup under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. Once the final provider/model is fixed and the outer `llm/stream` call returns a handle, the loop emits one contained live `agent/model-request` notification with turn, step, route, and optional registration-bound capacity. This is an observed Agent-loop attempt, not proof of provider I/O: preparation or a synchronous outer waterfall failure emits nothing, while a short-circuit handle or later lazy adapter construction, failure, or abortion still counts. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and other extension failures close directly. Recovery receives the exact live error, immutable provider facts, immutable prior failures, the immutable retry policy of the adapter registration that served the request, and the turn signal after the failed step closes; the policy is absent if no final adapter served it. A handling listener returns `{ kind: 'retry' }`; the loop closes the failed turn with its error and opens one numbered retry turn without an intervening idle notification. Success clears the consecutive history, and an unhandled failure is terminal. AgentLoop owns one cancellation signal for the current admission or turn. An effective `cancel(cause)` clears pending work unless `keepInbox` is set and cooperatively aborts that signal; idle cancellation is a no-op. Durable `turn/end` records `aborted` for `user` and `parent`, while disposal records `disposed`; undispatched model tool calls receive synthetic `tool/call` and `ABORTED_BEFORE_DISPATCH` result pairs. The cancellation cause changes reporting, not how result context finalized after cancellation is handled. Disposal waits for signal-ignoring work before registry removal. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns the lifecycle and race contract.

View File

@@ -62,7 +62,7 @@ interface Config {
每次提供方调用成功结束时,都会恰好追加一个 `assistant/message` 完成锚点,包括无内容调用和以 `max-tokens` 结束的调用。该锚点原样记录组装后的内容,保留确切的 chunk 溯源(流没有 chunk 时为 `[]`),并在用量可用时包含用量;空内容不会进入派生消息历史。
`agent/request` 返回提供方/模型调用配置后,循环会调用 `ctx.llm.prepareCall()`在活跃轮次信号的控制下校验由适配器持有的推理reasoning强度、填入其配置默认值并从同一次精确模型查询中保留可用的上下文元数据。准备完成的调用会在这次异步解析、`request/header` 日志记录和最终分派期间保留同一项确切的适配器注册,因此 HMR热模块替换不会把某个适配器的能力解析结果与另一适配器的请求混用。最终流句柄构造完成后、适配器开始迭代前,循环会发出一条失败会被收容的实时 `agent/model-request` 通知,其中包含轮次、步骤、最终提供方/模型,以及可选的、与注册项绑定的容量。准备阶段失败或同步流构造失败不会发出通知;之后即使失败或中止,该请求仍视为已观察到的分派。生效配置会在分派前写入日志,因此监听器可以在步骤之间更改推理强度,而不会产生未记录的请求变化。没有已注册适配器的路由会保留原定配置,使 `llm/stream` 监听器可以接管并短路该请求;最终分派仍会以 `NO_ADAPTER` 拒绝未得到处理的路由。新循环实例仅在初始提供方/模型路由与日志路由完全一致时恢复上次的推理强度;路由变化会丢弃由前一模型持有的不透明 ID并单独解析新模型。
`agent/request` 返回提供方/模型调用配置后,循环会调用 `ctx.llm.prepareCall()`在活跃轮次信号的控制下校验由适配器持有的推理reasoning强度、填入其配置默认值并从同一次精确模型查询中保留可用的上下文元数据。准备完成的调用会在这次异步解析、`request/header` 日志记录和最终分派期间保留同一项确切的适配器注册,因此 HMR热模块替换不会把某个适配器的能力解析结果与另一适配器的请求混用。最终提供方/模型确定且外层 `llm/stream` 调用返回句柄后,循环会发出一条失败会被收容的实时 `agent/model-request` 通知,其中包含轮次、步骤、路由,以及可选的、与注册项绑定的容量。这是 agent loop 观察到的一次尝试,并不能证明提供方 I/O 已开始:准备阶段或外层 waterfall瀑布式事件的同步失败不会发出通知而短路句柄或之后的惰性适配器构造、失败或中止仍会计入。生效配置会在分派前写入日志,因此监听器可以在步骤之间更改推理强度,而不会产生未记录的请求变化。没有已注册适配器的路由会保留原定配置,使 `llm/stream` 监听器可以接管并短路该请求;最终分派仍会以 `NO_ADAPTER` 拒绝未得到处理的路由。新循环实例仅在初始提供方/模型路由与日志路由完全一致时恢复上次的推理强度;路由变化会丢弃由前一模型持有的不透明 ID并单独解析新模型。
插件失败会结束当前轮次,而不是结束循环。只有最终适配器分发/迭代失败以及带内的终止错误或中止结束才进入 `agent/request-error`;中间件、结果处理、工具及其他扩展失败会直接关闭轮次。失败步骤关闭后,恢复逻辑会接收确切的实时错误、不可变的提供方事实、不可变的先前失败、为请求提供服务的适配器注册所对应的不可变重试策略,以及轮次信号;如果没有最终适配器为其提供服务,则该策略缺失。处理失败的监听器返回 `{ kind: 'retry' }`循环用其错误关闭失败轮次并在不插入空闲通知的情况下开启一个编号重试轮次。成功会清除连续失败历史未被处理的失败是终态。AgentLoop 为当前接纳或轮次拥有一个取消信号。有效的 `cancel(cause)` 在未设置 `keepInbox` 时清除待处理工作,并以协作方式中止该信号;空闲取消是空操作。持久 `turn/end``user``parent` 记录 `aborted`dispose资源释放则记录 `disposed`;未分发的模型工具调用会收到合成的 `tool/call``ABORTED_BEFORE_DISPATCH` 结果对。取消原因只改变报告方式不改变对取消后已定案结果上下文的处理。dispose 会等待忽略信号的工作完成,然后才从注册表移除。[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)规定生命周期与竞态契约。
@@ -89,7 +89,7 @@ interface Config {
#### Token 影响
每个步骤都会再次计入系统文本与 schema。逐 agent 作用域决定贡献,而权威组装 waterfall(瀑布式事件)可以改变最终请求,并使其监听器负责保持协议连贯。
每个步骤都会再次计入系统文本与 schema。逐 agent 作用域决定贡献,而权威组装 waterfall 可以改变最终请求,并使其监听器负责保持协议连贯。
#### KV Cache 影响

View File

@@ -487,24 +487,21 @@ export class ReactLoopAgent implements Agent {
const assembler = new BlockAssembler()
const chunkSeqs: number[] = []
const onDispatched = (): void => {
emitAgentEvent(
this.loopCtx,
this,
'agent/model-request',
turn,
step,
{
provider: request.provider,
model: request.model,
...preparedCall?.context === undefined
? {}
: { contextWindow: preparedCall.context.contextWindow },
},
)
}
const stream = preparedCall?.stream(request, onDispatched)
?? this.loopCtx.llm.stream(request, onDispatched)
const stream = preparedCall?.stream(request) ?? this.loopCtx.llm.stream(request)
emitAgentEvent(
this.loopCtx,
this,
'agent/model-request',
turn,
step,
{
provider: request.provider,
model: request.model,
...preparedCall?.context === undefined
? {}
: { contextWindow: preparedCall.context.contextWindow },
},
)
try {
for await (const chunk of stream) {
signal.throwIfAborted()

View File

@@ -293,6 +293,7 @@ describe('request stability across the loop', () => {
await ctx.plugin(AgentRegistry)
await ctx.plugin(AgentLoop, { agents: [] })
let observed: GenerateOptions | undefined
let observedRequest: { provider: string; model: string; contextWindow?: number } | undefined
ctx.on('llm/stream', (options) => {
observed = options
return (async function* () {
@@ -303,11 +304,15 @@ describe('request stability across the loop', () => {
provider: 'listener',
model: 'virtual',
})
ctx.on('agent/model-request', (subject, _turn, _step, request) => {
if (subject === agent) observedRequest = { ...request }
})
send(agent, 'go')
await waitForIdle(ctx, agent)
expect(observed).toMatchObject({ provider: 'listener', model: 'virtual' })
expect(observedRequest).toEqual({ provider: 'listener', model: 'virtual' })
expect(agent.session.requestHeader()?.config).toEqual({
provider: 'listener',
model: 'virtual',
@@ -318,7 +323,7 @@ describe('request stability across the loop', () => {
})
})
it('notifies one contained live model-request edge only after successful stream construction', async () => {
it('notifies one contained request attempt after the outer stream handle returns', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
@@ -341,7 +346,9 @@ describe('request stability across the loop', () => {
}
override stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
if (options.model === 'sync-failure') throw new LlmError('construction failed', 'CONSTRUCTION')
if (options.model === 'lazy-sync-failure') {
throw new LlmError('lazy construction failed', 'CONSTRUCTION')
}
if (options.model === 'async-failure') {
return {
[Symbol.asyncIterator]: () => ({
@@ -359,6 +366,22 @@ describe('request stability across the loop', () => {
provider: 'mock',
model: 'capacity',
})
const returnedHandles = new Set<string>()
ctx.on('llm/stream', (options, next) => {
if (options.model === 'outer-failure') {
throw new Error('outer waterfall failed before returning a handle')
}
if (options.model === 'lazy-sync-failure') {
const stream = (async function* () {
yield* next()
})()
returnedHandles.add(options.model)
return stream
}
const stream = next()
returnedHandles.add(options.model)
return stream
})
const observed: {
turn: number
step: number
@@ -366,18 +389,27 @@ describe('request stability across the loop', () => {
model: string
contextWindow?: number
}[] = []
const observedBeforeHandleReturn: string[] = []
ctx.on('agent/model-request', (subject) => {
if (subject === agent) throw new Error('observer failed')
})
ctx.on('agent/model-request', (subject, turn, step, request) => {
if (subject === agent) observed.push({ turn, step, ...request })
if (subject !== agent) return
if (!returnedHandles.has(request.model)) observedBeforeHandleReturn.push(request.model)
observed.push({ turn, step, ...request })
})
ctx.on('agent/request', async (_subject, turn, _step, _signal, next) => ({
...await next(),
model: ['capacity', 'unknown', 'async-failure', 'sync-failure'][turn - 1]!,
model: [
'capacity',
'unknown',
'async-failure',
'lazy-sync-failure',
'outer-failure',
][turn - 1]!,
}))
for (const prompt of ['one', 'two', 'three', 'four']) {
for (const prompt of ['one', 'two', 'three', 'four', 'five']) {
send(agent, prompt)
await waitForIdle(ctx, agent)
}
@@ -402,8 +434,16 @@ describe('request stability across the loop', () => {
provider: 'mock',
model: 'async-failure',
},
{
turn: 4,
step: 1,
provider: 'mock',
model: 'lazy-sync-failure',
},
])
expect(resolutions).toBe(4)
expect(resolutions).toBe(5)
expect(observedBeforeHandleReturn).toEqual([])
expect(returnedHandles.has('outer-failure')).toBe(false)
})
it('a compaction replace rewrites the resend, and the log explains it', async () => {