feat(llm-deepseek): configure max token defaults

This commit is contained in:
Yichen Jiang
2026-07-30 21:04:00 +08:00
parent 2fb90a744e
commit daf70f3660
50 changed files with 430 additions and 95 deletions

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-adapter-owned-max-token-defaults.md
2026-07-30-adapter-owned-max-token-defaults.md: c6fc9f014124607a3e2c3520f9f3b9023b77935f
2026-07-30-adapter-owned-max-token-defaults.zh.md: e0670f43eb5f27c2307c734e01a5b6718c09a67e

View File

@@ -0,0 +1,33 @@
# Agent Note: Adapter-owned max-token defaults
Status: implemented
English | [中文](2026-07-30-adapter-owned-max-token-defaults.zh.md)
## Problem
An LLM adapter could serialize an explicit `GenerateOptions.maxTokens`, but its Cordis configuration could not establish a reconstructable conversation default. Applying a fallback only inside provider serialization would make the wire request differ from the durable `request/header`; putting every provider's default in Agent Loop would instead transfer deployment and model policy into the provider-neutral driver.
## Decision
`LlmResolvedModelInfo.defaultMaxTokens` carries an optional adapter-configured per-request output cap for one exact provider/model route. `LlmService` validates it as a positive safe integer and materializes it into `LlmCallConfig.maxTokens` only when the caller omitted a value. Explicit request or Agent options therefore win without clamping.
The agent loop continues to prepare calls before logging `request/header`, so an adapter default becomes a durable request fact before dispatch. Direct `LlmService.stream()` calls resolve the same default at the final adapter boundary. The field is a request default rather than a hard model output limit; adapters that preserve provider-owned defaults omit it.
The native DeepSeek adapter exposes `maxTokens` in Cordis config with a 256,000-token default and maps the effective value to `max_tokens`. Its default context capacity is 1,000,000 tokens: both built-in V4 entries publish that exact capacity, while configured entries without capacity and unlisted pass-through ids inherit the same adapter-wide fallback.
## Alternatives considered
**Apply the default only in DeepSeek serialization.** Rejected because the provider wire would contain a model-visible value absent from the durable request header.
**Set `AgentOptions.maxTokens` in every shipped application.** Rejected because applications would duplicate adapter deployment policy, direct LLM calls would behave differently, and selecting another provider would retain a DeepSeek-specific cap.
**Represent 256,000 as a hard per-model maximum.** Rejected because the configured value is the desired request budget, not evidence that every configured endpoint rejects larger outputs. Explicit callers remain authoritative.
**Leave the provider default in control.** Rejected for the native DeepSeek deployment because the product requires a stable 256,000-token conversation budget across compatible endpoints.
## Consequences
DeepSeek conversations send `max_tokens: 256000` by default, and the same value appears in the session request header. Deployments can change the adapter default through `llm-deepseek.config.maxTokens`; per-agent and per-request values override it. Other adapters retain their existing behavior until they intentionally publish `defaultMaxTokens`.
The 256,000-token output budget reserves a large part of the one-million-token context on endpoints that pre-allocate requested output. Deployments whose gateway or model supports a smaller budget must lower `maxTokens`; the explicit configuration is preferable to an undocumented provider fallback.

View File

@@ -0,0 +1,33 @@
# Agent Note: 适配器持有的最大 token 默认值
Status: implemented
[English](2026-07-30-adapter-owned-max-token-defaults.md) | 中文
## Problem
LLM大语言模型适配器可以序列化显式的 `GenerateOptions.maxTokens`,但无法通过 Cordis 配置建立可重建的对话默认值。仅在提供方序列化中应用回退,会导致协议请求与持久 `request/header` 不一致;若将各提供方默认值都放进 agent loop智能体循环则会把部署与模型策略转移到提供方无关的驱动器中。
## Decision
`LlmResolvedModelInfo.defaultMaxTokens` 携带一条确切提供方/模型路由的可选单次请求输出上限,该值由适配器配置。`LlmService` 将其校验为正安全整数,并且仅在调用方省略值时才填入 `LlmCallConfig.maxTokens`。因此,显式请求值或 Agent 选项优先,且不会被自动调整。
agent loop 仍在记录 `request/header` 前准备调用,因此适配器默认值会在分派前成为持久请求事实。直接调用 `LlmService.stream()` 时,也会在最终适配器边界解析同一默认值。该字段是请求默认值,而非模型输出硬上限;保留提供方持有默认值的适配器会省略它。
原生 DeepSeek 适配器在 Cordis 配置中公开 `maxTokens`,默认值为 256,000 token并将生效值映射为 `max_tokens`。其默认上下文容量为 1,000,000 token两个内置 V4 配置项均公布这一精确容量;不含容量的已配置项和未列出的原样传递 id 则继承同一个适配器级回退值。
## Alternatives considered
**仅在 DeepSeek 序列化中应用默认值。** 不予采纳,因为提供方协议会包含持久请求 header 中缺失的模型可见值。
**在每个已发布应用中设置 `AgentOptions.maxTokens`。** 不予采纳,因为应用会重复适配器部署策略,直接 LLM 调用的行为将不同,而且选择另一个提供方后仍会保留 DeepSeek 专用上限。
**将 256,000 表示为每模型硬上限。** 不予采纳,因为配置值是所需请求预算,无法证明每个已配置端点都会拒绝更大的输出。显式调用方仍具有最终决定权。
**由提供方默认值控制。** 对原生 DeepSeek 部署不予采纳,因为产品要求各兼容端点都采用稳定的 256,000 token 对话预算。
## Consequences
DeepSeek 对话默认发送 `max_tokens: 256000`,会话请求 header 中也会出现相同的值。部署可以通过 `llm-deepseek.config.maxTokens` 更改适配器默认值;每个 agent 和每次请求的值都会覆盖它。其他适配器会保留现有行为,直至主动公布 `defaultMaxTokens`
对于预分配请求输出的端点256,000 token 的输出预算会占用 1,000,000 token 上下文中的很大部分。如果部署使用的 gateway 或模型仅支持较小预算,则必须调低 `maxTokens`;显式配置优于未记录的提供方回退值。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-28-sdk-max-output-tokens.md
2026-07-28-sdk-max-output-tokens.md: 5db48f21892d56addea7b72f73319f9dbfd1e71f
2026-07-28-sdk-max-output-tokens.zh.md: 38b172718716d100726188163ff22be9ea0a7325
2026-07-28-sdk-max-output-tokens.md: 3ba3e226d64b7d3d192d67bd88af463d2b0d9dc5
2026-07-28-sdk-max-output-tokens.zh.md: aec566011d2d7a311b4de509c47ebba383c3b0d7

View File

@@ -12,7 +12,7 @@ The Python and TypeScript SDKs could select a provider and model but could not b
The high-level SDKs expose one optional process-wide output cap: Python names it `max_tokens`, TypeScript names it `maxTokens`, and the shared `initialize` wire payload carries `maxTokens`. The JSON-RPC server rejects values that are not positive safe integers and stores the accepted cap with its provider/model route.
Each SDK-created root Agent receives the cap through `AgentOptions.maxTokens`. Agent Loop places that value in the initial `LlmCallConfig`, logs it in the request header, and reconstructs every dispatched conversation request from that durable header. Omitting the option leaves `maxTokens` absent so the selected provider retains its default.
Each SDK-created root Agent receives the cap through `AgentOptions.maxTokens`. Agent Loop places that value in the initial `LlmCallConfig`; final call preparation preserves the explicit value or materializes an exact-model adapter default, logs the effective cap in the request header, and reconstructs every dispatched conversation request from that durable header. Omitting the SDK option therefore allows the selected adapter or provider route default to apply.
In-process subagents inherit the parent's provider, model, and output cap. An explicit `SubagentStartRequest.agentOptions.maxTokens`, including one configured by `dsh-tool-subagent`, overrides the inherited value for that child and its descendants. Out-of-process providers own the configuration of their separate runtime; `subagent-dsh-sdk` therefore exposes its own optional `maxTokens` and forwards it through that child runtime's SDK handshake.
@@ -20,7 +20,7 @@ Compaction, session-title generation, web search, and other auxiliary calls keep
## Alternatives considered
**Set an adapter environment variable.** This would be DeepSeek-adapter-specific, invisible in the session request header, ineffective for intercepted or alternate adapters, and easy to confuse with a provider default. The cap belongs in provider-neutral request configuration.
**Set only an adapter environment variable.** A serializer-private fallback would be DeepSeek-adapter-specific, invisible in the session request header, ineffective for intercepted or alternate adapters, and easy to confuse with a provider default. Adapter-owned defaults may instead be exposed as exact-model metadata and materialized into provider-neutral request configuration before logging.
**Add `maxTokens` to every `session/prompt`.** Per-turn mutation would enlarge the wire and introduce request-config transitions that callers do not need for the current evaluation use case. A runtime initialization option gives every session in one SDK process the same reproducible budget.

View File

@@ -12,7 +12,7 @@ Python 与 TypeScript SDK 可以选择提供方和模型,却无法限制对话
高层 SDK 公开一个可选的进程级输出上限Python 命名为 `max_tokens`TypeScript 命名为 `maxTokens`,共享的 `initialize` 线载荷使用 `maxTokens`。JSON-RPC 服务端拒绝非正安全整数,并将通过校验的上限与提供方/模型路由一同保存。
每个由 SDK 创建的根 Agent 都通过 `AgentOptions.maxTokens` 获得该上限。Agent Loop 将它放入初始 `LlmCallConfig`记录到请求 header并从该持久化 header 重建每次分派的对话请求。省略该选项时,`maxTokens` 保持缺失,由所选提供方保留默认值。
每个由 SDK 创建的根 Agent 都通过 `AgentOptions.maxTokens` 获得该上限。Agent Loop 将它放入初始 `LlmCallConfig`;最终调用准备会保留显式值,或填入确切模型的适配器默认值,再将生效上限记录到请求 header并从该持久化 header 重建每次分派的对话请求。因此,省略 SDK 选项时会应用所选适配器或提供方路由的默认值。
进程内 subagent 继承父级的提供方、模型和输出上限。显式的 `SubagentStartRequest.agentOptions.maxTokens`(包括通过 `dsh-tool-subagent` 配置的值)会覆盖该子级及其后代的继承值。进程外提供方自行持有其独立运行时的配置;因此 `subagent-dsh-sdk` 公开独立的可选 `maxTokens`,并通过该子运行时自己的 SDK 握手传入。
@@ -20,7 +20,7 @@ Python 与 TypeScript SDK 可以选择提供方和模型,却无法限制对话
## Alternatives considered
**设置适配器环境变量。** 这种方式仅适用于 DeepSeek 适配器,不会出现在会话请求 header 中,对被拦截请求或其他适配器无效,也容易与提供方默认值混淆。该上限属于提供方无关的请求配置。
**设置适配器环境变量。** 序列化器私有回退仅适用于 DeepSeek 适配器,不会出现在会话请求 header 中,对被拦截请求或其他适配器无效,也容易与提供方默认值混淆。适配器持有的默认值可以改为通过确切模型元数据公开,并在记录前填入提供方无关的请求配置。
**在每个 `session/prompt` 上增加 `maxTokens`。** 按轮次修改会扩大线协议,并引入当前评测用例不需要的请求配置转换。运行时初始化选项可让一个 SDK 进程中的每个会话拥有相同、可重现的预算。