feat(llm-deepseek): configure max token defaults

This commit is contained in:
Yichen Jiang
2026-07-30 21:04:00 +08:00
parent 2fb90a744e
commit daf70f3660
50 changed files with 430 additions and 95 deletions

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-adapter-owned-max-token-defaults.md
2026-07-30-adapter-owned-max-token-defaults.md: c6fc9f014124607a3e2c3520f9f3b9023b77935f
2026-07-30-adapter-owned-max-token-defaults.zh.md: e0670f43eb5f27c2307c734e01a5b6718c09a67e

View File

@@ -0,0 +1,33 @@
# Agent Note: Adapter-owned max-token defaults
Status: implemented
English | [中文](2026-07-30-adapter-owned-max-token-defaults.zh.md)
## Problem
An LLM adapter could serialize an explicit `GenerateOptions.maxTokens`, but its Cordis configuration could not establish a reconstructable conversation default. Applying a fallback only inside provider serialization would make the wire request differ from the durable `request/header`; putting every provider's default in Agent Loop would instead transfer deployment and model policy into the provider-neutral driver.
## Decision
`LlmResolvedModelInfo.defaultMaxTokens` carries an optional adapter-configured per-request output cap for one exact provider/model route. `LlmService` validates it as a positive safe integer and materializes it into `LlmCallConfig.maxTokens` only when the caller omitted a value. Explicit request or Agent options therefore win without clamping.
The agent loop continues to prepare calls before logging `request/header`, so an adapter default becomes a durable request fact before dispatch. Direct `LlmService.stream()` calls resolve the same default at the final adapter boundary. The field is a request default rather than a hard model output limit; adapters that preserve provider-owned defaults omit it.
The native DeepSeek adapter exposes `maxTokens` in Cordis config with a 256,000-token default and maps the effective value to `max_tokens`. Its default context capacity is 1,000,000 tokens: both built-in V4 entries publish that exact capacity, while configured entries without capacity and unlisted pass-through ids inherit the same adapter-wide fallback.
## Alternatives considered
**Apply the default only in DeepSeek serialization.** Rejected because the provider wire would contain a model-visible value absent from the durable request header.
**Set `AgentOptions.maxTokens` in every shipped application.** Rejected because applications would duplicate adapter deployment policy, direct LLM calls would behave differently, and selecting another provider would retain a DeepSeek-specific cap.
**Represent 256,000 as a hard per-model maximum.** Rejected because the configured value is the desired request budget, not evidence that every configured endpoint rejects larger outputs. Explicit callers remain authoritative.
**Leave the provider default in control.** Rejected for the native DeepSeek deployment because the product requires a stable 256,000-token conversation budget across compatible endpoints.
## Consequences
DeepSeek conversations send `max_tokens: 256000` by default, and the same value appears in the session request header. Deployments can change the adapter default through `llm-deepseek.config.maxTokens`; per-agent and per-request values override it. Other adapters retain their existing behavior until they intentionally publish `defaultMaxTokens`.
The 256,000-token output budget reserves a large part of the one-million-token context on endpoints that pre-allocate requested output. Deployments whose gateway or model supports a smaller budget must lower `maxTokens`; the explicit configuration is preferable to an undocumented provider fallback.

View File

@@ -0,0 +1,33 @@
# Agent Note: 适配器持有的最大 token 默认值
Status: implemented
[English](2026-07-30-adapter-owned-max-token-defaults.md) | 中文
## Problem
LLM(大语言模型)适配器可以序列化显式的 `GenerateOptions.maxTokens`,但无法通过 Cordis 配置建立可重建的对话默认值。仅在提供方序列化中应用回退,会导致协议请求与持久 `request/header` 不一致;若将各提供方默认值都放进 agent loop(智能体循环),则会把部署与模型策略转移到提供方无关的驱动器中。
## Decision
`LlmResolvedModelInfo.defaultMaxTokens` 携带一条确切提供方/模型路由的可选单次请求输出上限,该值由适配器配置。`LlmService` 将其校验为正安全整数,并且仅在调用方省略值时才填入 `LlmCallConfig.maxTokens`。因此,显式请求值或 Agent 选项优先,且不会被自动调整。
agent loop 仍在记录 `request/header` 前准备调用,因此适配器默认值会在分派前成为持久请求事实。直接调用 `LlmService.stream()` 时,也会在最终适配器边界解析同一默认值。该字段是请求默认值,而非模型输出硬上限;保留提供方持有默认值的适配器会省略它。
原生 DeepSeek 适配器在 Cordis 配置中公开 `maxTokens`,默认值为 256,000 token,并将生效值映射为 `max_tokens`。其默认上下文容量为 1,000,000 token:两个内置 V4 配置项均公布这一精确容量;不含容量的已配置项和未列出的原样传递 id 则继承同一个适配器级回退值。
## Alternatives considered
**仅在 DeepSeek 序列化中应用默认值。** 不予采纳,因为提供方协议会包含持久请求 header 中缺失的模型可见值。
**在每个已发布应用中设置 `AgentOptions.maxTokens`。** 不予采纳,因为应用会重复适配器部署策略,直接 LLM 调用的行为将不同,而且选择另一个提供方后仍会保留 DeepSeek 专用上限。
**将 256,000 表示为每模型硬上限。** 不予采纳,因为配置值是所需请求预算,无法证明每个已配置端点都会拒绝更大的输出。显式调用方仍具有最终决定权。
**由提供方默认值控制。** 对原生 DeepSeek 部署不予采纳,因为产品要求各兼容端点都采用稳定的 256,000 token 对话预算。
## Consequences
DeepSeek 对话默认发送 `max_tokens: 256000`,会话请求 header 中也会出现相同的值。部署可以通过 `llm-deepseek.config.maxTokens` 更改适配器默认值;每个 agent 和每次请求的值都会覆盖它。其他适配器会保留现有行为,直至主动公布 `defaultMaxTokens`。
对于预分配请求输出的端点,256,000 token 的输出预算会占用 1,000,000 token 上下文中的很大部分。如果部署使用的 gateway 或模型仅支持较小预算,则必须调低 `maxTokens`;显式配置优于未记录的提供方回退值。