Merge remote-tracking branch 'origin/master' into worktree/context-source-cards
# Conflicts: # packages/client/runtime/README.i18n.yaml # packages/client/runtime/README.md # packages/client/runtime/README.zh.md # packages/client/ui-conversation/README.i18n.yaml
This commit is contained in:
creatixchu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/README.md
|
||||
README.md: 66b7beabd73cc3fec7230f209a9da0da48a37c95
|
||||
README.zh.md: 08f561840d1a560cfc9bced6f8757a0b0fc5770a
|
||||
README.md: 92d9fbfa2b8c8db4700562009db49229b2189ab3
|
||||
README.zh.md: 5c6e7aad1db6511bdb660b86e257652128db131f
|
||||
|
||||
@@ -6,10 +6,10 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` |
|
||||
| `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | Exact-provider normal or unbounded request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API adapter (direct fetch + eventsource-parser SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
|
||||
| [`llm/`](llm/README.md) | LLM service and shared streaming vocabulary | `ctx.llm` |
|
||||
| [`token-meter/`](token-meter/README.md) | Replay-aware token measurement | `ctx.tokenMeter` |
|
||||
| [`llm-retry/`](llm-retry/README.md) | Provider-scoped retry policy | listens to `agent/request-error` |
|
||||
| [`llm-deepseek/`](llm-deepseek/README.md) | Direct DeepSeek adapter | registers on `ctx.llm` |
|
||||
| [`llm-pi-ai/`](llm-pi-ai/README.md) | Multi-provider pi-ai adapter | registers on `ctx.llm` |
|
||||
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter supplies retry policy and resolves available exact-model identity, context capacity, and reasoning metadata; the retry executor and token meter remain provider-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
Adapters register provider routes on the seam; retry and token measurement remain separate consumers. The child READMEs own routing, metadata, replay, and provider-wire details; the [LLM architecture decisions](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) own the rationale.
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
# llm/：LLM（大语言模型）能力家族
|
||||
# llm/ — LLM 能力家族
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
LLM seam 及其提供方适配器。接口包（`llm`）拥有抽象服务、内容块词汇和流分片组装器；适配器是在 `ctx.llm` 上注册的具体实现。这些全是**产品**包。
|
||||
LLM（大语言模型）seam 及其提供方适配器。接口包（`llm`）负责抽象服务、内容块词汇和流式分片组装器；适配器是注册到 `ctx.llm` 的具体实现。这些全是**产品**包。
|
||||
|
||||
| 包 | 职责 | ctx key |
|
||||
|---|---|---|
|
||||
| `llm/` | 抽象 LLM 服务 + 内容块词汇 + 分片组装器 | `ctx.llm` |
|
||||
| `token-meter/` | 感知回放的请求 token 与表层 token 测量 | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | 确切提供方的常规或无界请求重试策略 | （监听 `agent/request-error`） |
|
||||
| `llm-deepseek/` | DeepSeek API 适配器，直接使用 fetch + eventsource-parser 和 SSE（Server-Sent Events） | （注册到 `ctx.llm`） |
|
||||
| `llm-pi-ai/` | 通过 `@earendil-works/pi-ai` 实现的多提供方适配器 | （注册到 `ctx.llm`） |
|
||||
| [`llm/`](llm/README.md) | LLM 服务和共享流式词汇 | `ctx.llm` |
|
||||
| [`token-meter/`](token-meter/README.md) | 可感知回放的 token 测量 | `ctx.tokenMeter` |
|
||||
| [`llm-retry/`](llm-retry/README.md) | 提供方作用域的重试策略 | 监听 `agent/request-error` |
|
||||
| [`llm-deepseek/`](llm-deepseek/README.md) | 直接 DeepSeek 适配器 | 注册到 `ctx.llm` |
|
||||
| [`llm-pi-ai/`](llm-pi-ai/README.md) | 多提供方 pi-ai 适配器 | 注册到 `ctx.llm` |
|
||||
|
||||
接口位于 `llm/llm/`；适配器、重试策略和可复用的 token 计量器以扁平结构并列在该分组下。请求按 `provider` 路由，而 `model` 会原样传给选中的适配器。负责该路由的适配器提供重试策略，并解析可用的确切模型身份、上下文容量和推理元数据；重试执行器与 token 计量器仍与提供方无关。新的提供方适配器只需在 `ctx.llm` 上注册一个或多个提供方路由，无需改动消费方。两个已交付实现见[双生 LLM 适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)，测量归属见[回放 token 计量器 Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md)，容量与压缩（compaction）策略归属见[路由模型上下文 Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
适配器在 seam 上注册提供方路由；重试与 token 测量仍是独立消费方。子 README 负责路由、元数据、回放和提供方协议细节；[LLM 架构决策](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md)负责设计原理。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 020aa65073495526be3f32912b7cd06667c52a2e
|
||||
README.zh.md: 0b2c9efd5ec9bc08e21be1966e182a703c5ea405
|
||||
README.md: 0cd265cadb2b2a619613761062ab2cef209bec83
|
||||
README.zh.md: 1883b054277adfd6c3d02b2a76ead9b3f8b0138f
|
||||
|
||||
@@ -40,7 +40,7 @@ The plugin registers the single provider route `deepseek-official` together with
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek-official', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. The adapter default is 1,000,000; pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek-official` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
`maxTokens` is the adapter-configured output cap for conversation requests and defaults to 256,000. Exact-model resolution exposes it as `defaultMaxTokens`; `LlmService` materializes that value into `GenerateOptions.maxTokens` before the agent loop writes `request/header`, so the wire request remains reconstructable. An explicit request or `AgentOptions.maxTokens` value wins and is serialized as `max_tokens`. The adapter does not clamp this request budget against `contextWindow`; deployments with a smaller context or provider output limit must configure a compatible `maxTokens`.
|
||||
`maxTokens` is the adapter-configured output cap for conversation requests and defaults to 256,000. A catalog entry may carry its own `maxTokens`, which wins for that model; an entry without one, and any unlisted pass-through id, resolve to the profile value, so adding a per-model cap changes one model rather than the route. Exact-model resolution exposes the winner as `defaultMaxTokens`; `LlmService` materializes that value into `GenerateOptions.maxTokens` before the agent loop writes `request/header`, so the wire request remains reconstructable. An explicit request or `AgentOptions.maxTokens` value wins and is serialized as `max_tokens`. The adapter does not clamp this request budget against `contextWindow`; deployments with a smaller context or provider output limit must configure a compatible `maxTokens`.
|
||||
|
||||
The same exact-model result exposes ordered `off`, `high`, and `max` efforts under `reasoning` for every pass-through model when deployment policy permits thinking. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header`. `high` and `max` enable thinking and serialize as the official top-level `reasoning_effort`; adapter-owned `off` instead serializes `thinking.type: disabled` and omits `reasoning_effort`. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O.
|
||||
|
||||
@@ -63,7 +63,7 @@ The plugin also declares its route in the configurable-provider directory (`ctx.
|
||||
|
||||
Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. A request whose `GenerateOptions.purpose` is `compaction` (dsh-compact-basic's auxiliary summarization call) additionally carries `x-deepseek-harness-compact: 1`, so the host can separate compaction traffic from conversation requests.
|
||||
|
||||
## Wire-format notes (verified live + against the official docs)
|
||||
## Wire-format notes
|
||||
|
||||
- Streaming only (`stream_options.include_usage` always on). `usage` may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to `[DONE]`, so `usage` always precedes `finish` and nothing follows `finish`.
|
||||
- The adapter-owned `off` effort maps to `thinking: {type: 'disabled'}` and never crosses the wire as `reasoning_effort: 'off'`.
|
||||
@@ -75,10 +75,6 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_<status>` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. A pre-response transport failure (DNS, refused connection, TLS, proxy) throws `TRANSPORT` naming the configured endpoint and chaining the original rejection as `cause`; caller aborts throw `ABORTED`, and the loop's cancellation signal remains authoritative. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks, and a completed stream whose `stop` (or absent) finish opened no content blocks becomes a `finish {kind: 'error'}` with code `EMPTY_RESPONSE` (retried by default policy).
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites run against a local `node:http` mock SSE server (no network), including dynamic `high`/`off`/`max` selection, structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers (next-request base-URL/key pickup, literal precedence, keyless onboarding, last-good snapshots, retry-policy re-registration), and `tests/loader-composition.spec.ts` boots the full chain from a test-only `cordis.yml` through the actual Loader and edits `settings.yaml`/`.env` on disk. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback and a request whose key exists only in a credentials-local document.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### DeepSeek request
|
||||
|
||||
@@ -40,7 +40,7 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
|
||||
|
||||
`contextWindow` 对每个已配置模型都可选，不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek-official', model).context` 先返回精确模型值，再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。适配器默认值为 1,000,000；因此，压力敏感插件可以获得由部署决定的容量，不会将模型 selector 视为权威。为 `deepseek-official` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
`maxTokens` 是适配器为对话请求配置的输出上限，默认值为 256,000。确切模型解析会将其公开为 `defaultMaxTokens`；`LlmService` 会在 agent loop（智能体循环）写入 `request/header` 前，将该值填入 `GenerateOptions.maxTokens`，从而仍可根据持久记录重建协议请求。显式的请求值或 `AgentOptions.maxTokens` 值优先，并会序列化为 `max_tokens`。适配器不会根据 `contextWindow` 自动调低该请求预算；上下文或提供方输出上限较小的部署必须配置与其相容的 `maxTokens`。
|
||||
`maxTokens` 是适配器为对话请求配置的输出上限，默认值为 256,000。Catalog 配置项可以自带 `maxTokens`，它对该模型胜出；不含该上限的配置项以及任何未列出原样传递 id 都解析为 profile 值，因此新增按模型的上限只改变一个模型，而非整条路由。确切模型解析会将胜出值公开为 `defaultMaxTokens`；`LlmService` 会在 agent loop（智能体循环）写入 `request/header` 前，将该值填入 `GenerateOptions.maxTokens`，从而仍可根据持久记录重建协议请求。显式的请求值或 `AgentOptions.maxTokens` 值优先，并会序列化为 `max_tokens`。适配器不会根据 `contextWindow` 自动调低该请求预算；上下文或提供方输出上限较小的部署必须配置与其相容的 `maxTokens`。
|
||||
|
||||
同一确切模型结果会在部署策略允许思考时，为每个原样传递模型在 `reasoning` 下公开有序的 `off`、`high` 和 `max` 推理（reasoning）强度。`reasoningEffort` 选择部署默认值，省略时回退为 `high`。`agent/request` 可以在每个会话步骤替换它；解析后的值会记录在 `request/header`。`high` 和 `max` 会启用思考，并序列化为官方顶层 `reasoning_effort`；适配器持有的 `off` 则序列化为 `thinking.type: disabled`，且省略 `reasoning_effort`。不支持的值会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败。
|
||||
|
||||
@@ -63,7 +63,7 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
|
||||
|
||||
每个请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头，即用于识别 harness 的必需 `User-Agent` 基线（见 [dsh-llm § 应用归因](../llm/README.md#app-attribution-attributionts)）。在该适配器契约（adapter contract）下，直接 DeepSeek 请求与 OpenAI 兼容 gateway 请求都不会获得提供方特定应用归因标头；OpenRouter 应用归因暂缓到未来的显式 OpenRouter 适配器或模式。`GenerateOptions.purpose` 为 `compaction` 的请求（dsh-compact-basic 的辅助摘要调用）还会携带 `x-deepseek-harness-compact: 1`，让宿主可以将压缩流量与会话请求分开。
|
||||
|
||||
## 协议格式说明（已通过实时请求与官方文档验证）
|
||||
## 协议格式说明
|
||||
|
||||
- 只支持流式输出（`stream_options.include_usage` 始终开启）。`usage` 可能附着在 finish 分片上，也可能作为尾随的纯 usage 分片到达；转换器会将两者都延迟到 `[DONE]`，因此 `usage` 始终位于 `finish` 之前，`finish` 之后不会出现任何内容。
|
||||
- 适配器持有的 `off` 推理强度映射为 `thinking: {type: 'disabled'}`，绝不会以 `reasoning_effort: 'off'` 通过协议发送。
|
||||
@@ -75,10 +75,6 @@ harness LLM（大语言模型）seam 的 DeepSeek chat-completions 适配器：
|
||||
|
||||
非 2xx 响应会抛出稳定 code 的 `LlmError`：`AUTH`（401/403）、`QUOTA`（提供方详细信息标识配额、余额或点数耗尽的响应）、`RATE_LIMIT`（其他 429）、`CONTEXT_WINDOW_EXCEEDED`（提供方 code、type 或 message 标识上下文溢出的 400）、`INVALID_REQUEST`（其他 400）、`SERVER`（5xx），其他情况为 `HTTP_<status>`。其可序列化 `failure` 保留 HTTP 状态，以及有效的正 `Retry-After` 秒数／日期延迟和存在时的 `x-request-id` / `x-deepseek-request-id`。响应前传输失败（DNS、连接被拒绝、TLS、proxy）会抛出命名已配置端点的 `TRANSPORT`，并将原始拒绝作为 `cause`；调用方 abort 抛出 `ABORTED`，仍以 loop 的取消信号为准。协议违例抛出 `STREAM_CLOSED`（没有 `[DONE]`）或 `MALFORMED_RESPONSE`（JSON payload 格式错误）。未知协议 `finish_reason`（例如 `content_filter`、`insufficient_system_resource`）会变为 `finish {kind: 'error', failure}` 分片；已完成流如果使用 `stop`（或缺失）finish 但没有开启内容块，就会变为 `finish {kind: 'error'}`，code 为 `EMPTY_RESPONSE`（默认策略会重试）。
|
||||
|
||||
## 测试
|
||||
|
||||
单元套件使用本地 `node:http` mock SSE 服务器（无网络），覆盖动态 `high`／`off`／`max` 选择、结构化 HTTP 事实、格式错误／截断流、调用方 abort、连接失败，以及 idle 超时确实会 abort 实际 body 的证明。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider（下一请求即生效的 base-URL／密钥拾取、字面值优先、无密钥上手、最后可用快照、重试策略重注册），`tests/loader-composition.spec.ts` 则从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起完整链路，并在磁盘上编辑 `settings.yaml`/`.env`。真实 API 覆盖位于 `tests/adapter.e2e.ts`（`pnpm run test:e2e`，需有 key 才会运行）：V4 Flash + V4 Pro，覆盖思考启用／禁用与两种官方 effort 级别，包括思考 + 工具往返与推理回传，以及密钥仅存在于 credentials-local 文档中的请求。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### DeepSeek 请求
|
||||
|
||||
@@ -35,6 +35,8 @@ export interface DeepSeekCatalogModel {
|
||||
description?: string
|
||||
/** Known combined request/response context capacity; omitted when deployment metadata is unavailable. */
|
||||
contextWindow?: number
|
||||
/** Per-request output cap for this model; omission falls back to the profile's {@link DeepSeekConnectionOptions.maxTokens}. */
|
||||
maxTokens?: number
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -181,7 +183,7 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
? { provider, id: model, name: model }
|
||||
: modelInfo(provider, configured),
|
||||
context: { contextWindow },
|
||||
defaultMaxTokens: connection.maxTokens,
|
||||
defaultMaxTokens: configured?.maxTokens ?? connection.maxTokens,
|
||||
...connection.defaults.thinking === 'disabled'
|
||||
? {
|
||||
reasoning: {
|
||||
|
||||
@@ -68,7 +68,7 @@ export interface Config {
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
/** Default thinking effort (default `high`); `off` disables thinking per request. */
|
||||
reasoningEffort?: 'off' | 'high' | 'max'
|
||||
/** Default per-request output cap (default 256,000); explicit request values win. */
|
||||
/** Default per-request output cap (default 256,000); a model's own cap and explicit request values win. */
|
||||
maxTokens?: number
|
||||
/** Positive context capacity used when the selected model has no exact value (default 1,000,000). */
|
||||
defaultContextWindow?: number
|
||||
@@ -85,6 +85,7 @@ const catalogModel: z<DeepSeekCatalogModel> = z.object({
|
||||
name: z.string(),
|
||||
description: z.string(),
|
||||
contextWindow: z.number().step(1).min(1),
|
||||
maxTokens: z.number().step(1).min(1),
|
||||
})
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
@@ -125,6 +126,12 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee
|
||||
`llm-deepseek: catalog model "${model.id}" contextWindow must be a positive integer`,
|
||||
)
|
||||
}
|
||||
if (model.maxTokens !== undefined
|
||||
&& (!Number.isInteger(model.maxTokens) || model.maxTokens <= 0)) {
|
||||
throw new Error(
|
||||
`llm-deepseek: catalog model "${model.id}" maxTokens must be a positive integer`,
|
||||
)
|
||||
}
|
||||
if (seen.has(model.id)) throw new Error(`llm-deepseek: duplicate catalog model "${model.id}"`)
|
||||
seen.add(model.id)
|
||||
return {
|
||||
@@ -132,6 +139,7 @@ function resolveModels(models: readonly DeepSeekCatalogModel[] | undefined): Dee
|
||||
...model.name === undefined ? {} : { name: model.name },
|
||||
...model.description === undefined ? {} : { description: model.description },
|
||||
...model.contextWindow === undefined ? {} : { contextWindow: model.contextWindow },
|
||||
...model.maxTokens === undefined ? {} : { maxTokens: model.maxTokens },
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -793,6 +793,26 @@ describe('plugin registration and config', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it.each([0, 1.5])('rejects a per-model output cap of %s', (maxTokens) => {
|
||||
expect(() => resolveAdapterOptions({ models: [{ id: 'bad-cap', maxTokens }] }))
|
||||
.toThrow(/maxTokens must be a positive integer/)
|
||||
})
|
||||
|
||||
it('prefers a model\'s own output cap over the profile default', async () => {
|
||||
// The profile default stays what an unlisted or uncapped model resolves
|
||||
// to, so adding a per-model cap changes one model rather than the route.
|
||||
const adapter = adapterOf({ maxTokens: 4096, models: [
|
||||
{ id: 'capped', maxTokens: 512 },
|
||||
{ id: 'uncapped' },
|
||||
] })
|
||||
await expect(adapter.resolveModel('deepseek-official', 'capped'))
|
||||
.resolves.toMatchObject({ defaultMaxTokens: 512 })
|
||||
await expect(adapter.resolveModel('deepseek-official', 'uncapped'))
|
||||
.resolves.toMatchObject({ defaultMaxTokens: 4096 })
|
||||
await expect(adapter.resolveModel('deepseek-official', 'not-in-catalog'))
|
||||
.resolves.toMatchObject({ defaultMaxTokens: 4096 })
|
||||
})
|
||||
|
||||
it('rejects invalid context capacity when apply is called directly', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: e8c2682cbb72ca1ac6a5ad6b26bdf63f0695716b
|
||||
README.zh.md: 4175b3a751affa65ac68284c7ead47b1f71b5e15
|
||||
README.md: 75b2136315aed758f18f7fe82afcd4903f4a7b98
|
||||
README.zh.md: ea67250549f1d23d48455fd185283b00183dd538
|
||||
|
||||
@@ -75,10 +75,6 @@ Every request carries the shared attribution header from dsh-llm's `attributionH
|
||||
|
||||
pi-ai installs several provider SDKs and lazy-loads the one selected by the catalog model. The dependency weight is isolated to this opt-in adapter package.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. `tests/dynamic-config.spec.ts` drives real settings-local and credentials-local providers: a settings-born route registers live and drops when the user layer resets, `apiKeyEnv` credentials rotate between requests, and an unknown-provider snapshot keeps the last good profiles. `tests/loader-composition.spec.ts` boots the dormant posture from a test-only `cordis.yml` through the actual Loader and registers its route from an on-disk `settings.yaml` edit. Real-API coverage remains key-gated under `pnpm run test:e2e`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Provider request through pi-ai
|
||||
|
||||
@@ -75,10 +75,6 @@
|
||||
|
||||
pi-ai 会安装多个提供方 SDK，并延迟加载 catalog 模型所选的 SDK。该可选适配器包将依赖体量隔离在自身范围内。
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试使用重定向到本地 mock 服务器的 pi-ai catalog 模型，覆盖提供方／profile 路由、每次适配器调用只发起一个协议请求、idle-timeout 响应终止、调用方 abort、原生 API 选择、端点覆盖、归因、转换、回放状态验证，以及一个适配器实例内的跨提供方／模型回放。`tests/dynamic-config.spec.ts` 驱动真实的 settings-local 与 credentials-local provider：settings 里新生的路由实时完成注册，并在用户层重置时随之移除，`apiKeyEnv` 凭据在两次请求之间轮换，点名未知提供方的快照则保留最后可用 profile。`tests/loader-composition.spec.ts` 从仅测试用的 `cordis.yml` 出发，经真实 Loader 拉起休眠姿态，并从磁盘上的一次 `settings.yaml` 编辑注册出它的路由。真实 API 覆盖仍需 key 才会启用，并通过 `pnpm run test:e2e` 运行。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### 通过 pi-ai 发起的提供方请求
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md
|
||||
README.md: 8de3ea8c9321f04f5af1b0d7ab361f73eaabc822
|
||||
README.zh.md: 978854e9466e271535a10fcea406d0dcb5607285
|
||||
README.md: 23b55a30989cc51d4dd9076b61b6595452b0abd0
|
||||
README.zh.md: 267ef12a87561fd8effef726a781e505225baf03
|
||||
|
||||
@@ -48,6 +48,6 @@ The reconstructed request preserves the prior prefix and is eligible for provide
|
||||
|
||||
- **Agent turns are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
|
||||
- **Always mode retries permanent failures** — authentication, quota, invalid-request, protocol, and unrecoverable context errors continue until success, cancellation, or disposal; deployments own provider-specific cost and latency controls.
|
||||
- **Finite plugin budgets add** — normal mode counts only its configured codes and exact provider policy, while context-overflow compaction owns a separate budget. A future overlapping policy must document and test registration-order behavior.
|
||||
- **Finite plugin budgets add** — normal mode counts only its configured codes and exact provider policy, while context-overflow compaction owns a separate budget. Any overlapping policy must define registration-order behavior.
|
||||
- **Recovery policies compose by waterfall order** — always mode accepts a downstream retry before applying its fallback. A later policy that ignores cancellation and never settles also prevents fallback, turn quiescence, and plugin disposal from completing.
|
||||
- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation.
|
||||
|
||||
@@ -48,6 +48,6 @@
|
||||
|
||||
- **agent 轮次是唯一重试边界**：直接 `ctx.llm.stream()` 消费方仍只尝试一次，因为原始流无法持久地区分各次尝试已经发出的分片。
|
||||
- **always mode 会重试永久性失败**：身份验证、配额、无效请求、协议和无法恢复的上下文错误都会继续重试，直至成功、取消或 dispose；部署负责提供方特定的成本与延迟控制。
|
||||
- **有限插件预算可叠加**：normal mode 只统计已配置 code 和确切提供方策略，上下文溢出压缩（compaction）则拥有独立预算。未来如有重叠策略，必须记录并测试注册顺序行为。
|
||||
- **有限插件预算可叠加**：normal mode 只统计已配置 code 和确切提供方策略，上下文溢出压缩（compaction）则拥有独立预算。任何重叠策略都必须定义注册顺序行为。
|
||||
- **恢复策略按 waterfall 顺序组合**：always mode 会先接受下游重试，再应用自己的回退。后续策略如果忽略取消且永不结算，也会阻止回退、轮次完全停稳和插件 dispose 完成。
|
||||
- **`llm/retry` 记录调度，不是完成**：后续步骤与轮次事件用于确立成功、耗尽或取消。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: 701893b342f9a93a75bec175634b1054f3d17151
|
||||
README.zh.md: 0731e05186bec3f21d1f723c5b94ab7945e4139d
|
||||
README.md: 0935a48a5f5773fbb280bc45e07faaa05c0a4f6e
|
||||
README.zh.md: 83282ab47e6d406cfca30bbd8af40e0c94050504
|
||||
|
||||
@@ -29,13 +29,13 @@ When the composition provides `ctx.sessionProjections`, token-meter registers tw
|
||||
|
||||
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — and optional `contextWindow` from the newest `request/context` record. Pressure stays absent until a provider reports usage; capacity stays absent for a route whose adapter advertises none. Output is excluded, so the numerator holds still while a turn streams and steps forward when the next request reports its usage.
|
||||
|
||||
Both units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes both keys. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
Both units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes both keys. A composition without the projection seam keeps the measurement service's existing behavior.
|
||||
|
||||
### Context occupancy is an approximation, by design
|
||||
|
||||
`pressureTokens` and `contextWindow` are independent last-wins fields and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's pressure until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now.
|
||||
|
||||
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. The TUI status line has always computed occupancy the same way, dividing a `measure()` total by a separately-resolved capacity for the selected model.
|
||||
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. A UI computes occupancy by dividing measured pressure by the separately resolved capacity for the selected model.
|
||||
|
||||
Making the pair atomic was tried and rejected: it required a transient non-replayable wire frame, which needed lifecycle fencing against cross-stream reordering and left occupancy blank after every reconnect. The [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md) records that comparison. Consumers that need an exact same-boundary figure should call `measure()` at their own request boundary rather than read this projection.
|
||||
|
||||
@@ -62,4 +62,3 @@ No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
- **Every measurement clones the current surface** — coherent immutable snapshots make reads O(surface), including below-threshold pressure checks.
|
||||
- **Provider usage is only reusable for an identical canonical envelope** — prompt, prefix, tools, provider, model, or call-config changes deliberately fall back to full heuristic estimation.
|
||||
- **Legacy provenance is conservative** — assistant messages without `sourceEventSeqs` cannot distinguish provider output from listener rewrites, so the fold avoids claiming a known empty or exact chunk stream.
|
||||
- **The TUI and browser fixture retain parallel folds** — `tokenUsage` owns durable session-projection semantics; the TUI keeps its live per-step map because its composition does not mount the generic projection seam, while the browser fixture mirrors the unit for standalone demo data.
|
||||
|
||||
@@ -29,13 +29,13 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
|
||||
|
||||
`contextPressure` 携带可选的 `pressureTokens`（提供方报告的最新提示词规模，为未缓存输入加缓存读取与写入之和），以及来自最新一条 `request/context` 记录的可选 `contextWindow`。提供方报告用量前压力保持缺失；路由适配器未公布容量时容量也保持缺失。输出不计入其中，因此轮次流式输出期间分子保持不动，等到下一个请求报告用量时才前进。
|
||||
|
||||
两个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这两个键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
两个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这两个键。不带投影 seam 的组合会保留测量服务的既有行为。
|
||||
|
||||
### 上下文占用率是刻意为之的近似值
|
||||
|
||||
`pressureTokens` 与 `contextWindow` 是两个各自后者胜的独立字段，**不是**对单个请求的一次原子观测。切换模型时，新容量会与上一路由的压力配对，直到下一个请求报告用量为止；而 `pressureTokens` 描述的是最后一个请求，不是此刻的表层。
|
||||
|
||||
这是刻意的选择。占用率百分比是面向用户的参考数字，既不是计费记录，也不是门控输入：harness 中没有任何环节依据它做决策，压缩改为直接读取 `measure()`。TUI 状态行一直以同样的方式计算占用率，即用 `measure()` 总量除以为所选模型单独解析出的容量。
|
||||
这是刻意的选择。占用率百分比是面向用户的参考数字，既不是计费记录，也不是门控输入：harness 中没有任何环节依据它做决策，压缩改为直接读取 `measure()`。UI 用测得的压力除以为所选模型单独解析出的容量来计算占用率。
|
||||
|
||||
让这对值保持原子已经尝试过并被否决：它需要一个临时且不可回放的协议帧，进而需要针对跨流重排序的生命周期栅栏，还会让占用率在每次重连后变为空白。[Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md)记录了这项对比。需要同一边界精确数字的消费方应在自己的请求边界调用 `measure()`，而不是读取该投影。
|
||||
|
||||
@@ -62,4 +62,3 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
|
||||
- **每次测量都会克隆当前表层**：一致且不可变的快照使读取成为 O(surface)，包括低于阈值的压力检查。
|
||||
- **提供方用量只能为完全相同的规范 envelope 复用**：提示词、前缀、工具、提供方、模型或调用配置变更都会有意回退到完整启发式估算。
|
||||
- **遗留溯源采取保守策略**：没有 `sourceEventSeqs` 的 assistant 消息无法区分提供方输出与 listener 改写，因此 fold 不会声称已知空流或精确分片流。
|
||||
- **TUI 与浏览器 fixture 仍保留并行 fold**：`tokenUsage` 拥有持久会话投影语义；TUI 的组合未挂载通用投影 seam，因此继续维护实时的逐步骤 map，而浏览器 fixture 会为独立 demo 数据镜像该单元。
|
||||
|
||||
@@ -93,8 +93,8 @@ export class TokenMeterService extends Service {
|
||||
super(ctx, 'tokenMeter')
|
||||
validateConfigKeys(config)
|
||||
|
||||
// Projection registration is an optional child: headless and TUI
|
||||
// compositions without the generic registry keep the meter's old shape.
|
||||
// Projection registration is an optional child: compositions without the
|
||||
// generic registry keep the meter's standalone read shape.
|
||||
ctx.inject(['sessionProjections'], (projectionCtx) => {
|
||||
projectionCtx.sessionProjections.register(tokenUsageProjectionDefinition)
|
||||
projectionCtx.sessionProjections.register(contextPressureProjectionDefinition)
|
||||
|
||||
@@ -25,8 +25,7 @@ export interface TokenUsageProjection {
|
||||
* `contextWindow` the newest recorded route capacity. Switching models can
|
||||
* therefore pair a fresh capacity with the previous route's pressure until the
|
||||
* next request reports usage. This is an intentional trade — the value is a
|
||||
* user-facing reference, not a billing or gating input — and it matches how
|
||||
* the TUI status line has always computed occupancy. See the token-meter
|
||||
* user-facing reference, not a billing or gating input. See the token-meter
|
||||
* README for the full rationale.
|
||||
*/
|
||||
export interface ContextPressureProjection {
|
||||
|
||||
Reference in New Issue
Block a user