Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(llm): simplify live request telemetry (round 2)

Browse Source
This commit is contained in:
Hypatia May
2026-07-28 19:03:45 +08:00
parent 35b9c454e5
commit e6ce6abd7d
31 changed files with 159 additions and 187 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/architecture.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/architecture.md
architecture.md: 52072633a0e81afce63c5e162dd1b0af7f6486ca
architecture.zh.md: f0122ece146c17366aa316cfb4ea4196a1db74bd
architecture.md: e9ae1e27d6f1f1170b5da4823f3e988f4e778022
architecture.zh.md: 3a31a2e6694d39f38e9c88de44f59ce7f753871f

4
docs/architecture.md
View File

@@ -92,7 +92,7 @@ forever:
assemble system prompt and tool schemas
snapshot the derived messages (the reconstruction boundary)
'step/start'
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> construct llm/stream (frozen, registration-bound) -> agent/model-request (live, contained) -> iterate
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> obtain outer llm/stream handle (frozen request; prepared calls registration-bound) -> agent/model-request (live, contained attempt) -> iterate
'assistant/chunk'
'assistant/message'
schedule tool calls by ctx.tools.executionMode:
@@ -151,7 +151,7 @@ Log-only events may sit between turns. Owners append through `Session`, flushing
Messages use typed blocks from merge-extensible `ContentBlockMap`; the pattern also types `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`. New blocks coordinate adapters, UI, compaction, token metering, and persistence; replay measurements live in [token-meter.md](core-data-structures/token-meter.md).
Streaming uses raw chunks and `BlockAssembler`. After final-stream construction, the loop emits contained, non-durable, non-replayed `agent/model-request` metadata. Adapters normalize failures; `agent/request-error` may retry. Remote adapters use per-read idle watchdogs. Replay crosses routes only through a shared adapter ([contract](core-data-structures/llm-streaming.md)).
Streaming uses chunks and `BlockAssembler`. When the outer `llm/stream` returns a handle, AgentLoop emits contained, non-durable, non-replayed `agent/model-request` attempt metadata—not proof of provider I/O. `agent/request-error` may retry. Replay crosses routes only through a shared adapter ([contract](core-data-structures/llm-streaming.md)).
## Extension And Composition

4
docs/architecture.zh.md
View File

@@ -92,7 +92,7 @@ forever:
assemble system prompt and tool schemas
snapshot the derived messages (the reconstruction boundary)
'step/start'
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> construct llm/stream (frozen, registration-bound) -> agent/model-request (live, contained) -> iterate
agent/request (config only) -> prepare reasoning/default + context under turn signal -> log request/header -> obtain outer llm/stream handle (frozen request; prepared calls registration-bound) -> agent/model-request (live, contained attempt) -> iterate
'assistant/chunk'
'assistant/message'
schedule tool calls by ctx.tools.executionMode:
@@ -151,7 +151,7 @@ idle inject:
消息使用从可合并扩展的 `ContentBlockMap` 派生的类型化块；同一模式也为 `MessageSource`、`FinishReason`、`TurnTrigger` 和 `TurnEndReason` 定义类型。新增块会协调适配器、UI、压缩、token 计量和持久化；回放计量见 [token-meter.md](core-data-structures/token-meter.md)。
流式输出使用原始分片和 `BlockAssembler`。最终流构造完成后，循环会发出 `agent/model-request` 元数据；该通知的失败会被收容，元数据不会持久化或回放。适配器会规范化故障；`agent/request-error` 可以重试。远程适配器使用逐次读取空闲看门狗。回放仅通过共用适配器跨路由传递（[契约](core-data-structures/llm-streaming.md)）。
流式输出使用分片和 `BlockAssembler`。外层 `llm/stream` 返回句柄时，AgentLoop 会发出 `agent/model-request` 尝试元数据；该通知的失败会被收容，元数据不会持久化或回放，但这并不能证明提供方 I/O 已开始。`agent/request-error` 可以重试。回放仅通过共用适配器跨路由传递（[契约](core-data-structures/llm-streaming.md)）。
## 扩展与组合

12
docs/cordis-catalog/events.md
View File

@@ -166,15 +166,15 @@ Source: [`packages/core/agent/src/types.ts:286`](../../packages/core/agent/src/t
### `agent/model-request` — emit
One model request constructed its final stream handle and is about to iterate it. This live notification is not durable or replayed; failed or aborted iteration still has a dispatch, while preparation and synchronous stream-construction failures do not. Listener failures are contained and cannot affect the request.
One model request obtained its outer `llm/stream` handle and is about to iterate it. This observes an Agent-loop request attempt, not proof that provider I/O began. The notification is live, contained, and not replayed. Preparation or a synchronous outer waterfall failure emits nothing; failures or abortion after the handle returns still count.
```ts cordis-catalog
/**
* One model request constructed its final stream handle and is about to
* iterate it. This live notification is not durable or replayed; failed or
* aborted iteration still has a dispatch, while preparation and
* synchronous stream-construction failures do not. Listener failures are
* contained and cannot affect the request.
* One model request obtained its outer `llm/stream` handle and is about to
* iterate it. This observes an Agent-loop request attempt, not proof that
* provider I/O began. The notification is live, contained, and not replayed.
* Preparation or a synchronous outer waterfall failure emits nothing;
* failures or abortion after the handle returns still count.
* @param agent - the agent dispatching the model request.
* @param turn - the open turn number.
* @param step - the request's step number.

6
docs/cordis-catalog/services.md
View File

@@ -780,16 +780,14 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
* agent-loop request recovery; middleware and nested-call failures remain
* untagged for the outer call.
* @param options - the full request; `options.provider` selects the adapter.
* @param onDispatched - contained Agent-loop notification hook invoked after
* a stream handle is constructed and before its adapter is iterated.
* @returns the chunk stream, possibly wrapped by `llm/stream` listeners.
*/
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
```
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Source: [`packages/llm/llm/src/index.ts:194`](../../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm/src/index.ts:192`](../../packages/llm/llm/src/index.ts)
## `ctx.permission` — `PermissionService`

4
docs/core-data-structures/llm-streaming.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/llm-streaming.md
llm-streaming.md: 89628ebb96a2e8eec5209635cd92859427df4a8d
llm-streaming.zh.md: 9c8fcc1f24b970f3a7cdd7cd08d9ef3b934b4543
llm-streaming.md: 30230d208c582463b3680d760c27f33a71b4cf6f
llm-streaming.zh.md: 32b3485f967b849371f916f6a669b97c97701cc1

6
docs/core-data-structures/llm-streaming.md
View File

@@ -161,7 +161,7 @@ declare class BlockAssembler {
## The seam
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Its optional observer runs after a final stream handle is constructed and before adapter iteration. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, and to retain detached context metadata from that exact lookup. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. AgentLoop observes a request attempt once the outer waterfall returns a stream handle; that limited boundary does not prove a lazy terminal adapter was constructed or began provider I/O. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
* preparation. The request's call-config fields must match {@link config};
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
* @param options - fully assembled request carrying the prepared config.
* @param onDispatched - contained Agent-loop notification hook invoked after
* a stream handle is constructed and before its adapter is iterated.
* @returns the chunk stream, including the `llm/stream` waterfall.
*/
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
```

6
docs/core-data-structures/llm-streaming.zh.md
View File

@@ -161,7 +161,7 @@ declare class BlockAssembler {
## seam
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值，`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份，以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值；字段缺失表示元数据或能力不可用，而不表示目录成员关系无效。解析器会接收可选的取消信号，并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝；直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`，使模型解析、请求头持久记录和分派全程使用同一项注册，并保留来自同一次精确查询的分离上下文元数据。其可选观察器在最终流句柄构造完成后、适配器开始迭代前运行。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
`LlmAdapter` 是提供方 seam：创建子类、实现 `stream()`，再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器；`GenerateOptions.model` 会传给该适配器，无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值，`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考，不是请求白名单：适配器仍是权威，并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份，以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值；字段缺失表示元数据或能力不可用，而不表示目录成员关系无效。解析器会接收可选的取消信号，并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值，因此直接调用也无法绕过对不支持推理强度的拒绝；直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`，使模型解析、请求头持久记录和分派全程使用同一项注册，并保留来自同一次精确查询的分离上下文元数据。适配器查找发生在 `llm/stream` waterfall（瀑布式事件）的终端 continuation，因此 listener 可以在查找前短路调用，或路由一个可变的一次性请求。AgentLoop 在外层 waterfall 返回流句柄时观察到一次请求尝试；这个有限边界不能证明惰性终端适配器已构造完成或开始提供方 I/O。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface（`ctx.llm.stream()`）与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -175,11 +175,9 @@ interface PreparedLlmCall {
* preparation. The request's call-config fields must match {@link config};
* reuse or mismatch fails with `INVALID_PREPARED_CALL`.
* @param options - fully assembled request carrying the prepared config.
* @param onDispatched - contained Agent-loop notification hook invoked after
* a stream handle is constructed and before its adapter is iterated.
* @returns the chunk stream, including the `llm/stream` waterfall.
*/
stream(options: GenerateOptions, onDispatched?: () => void): AsyncIterable<StreamChunk>
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
}
```