feat(llm-deepseek): configure max token defaults
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-adapter-owned-max-token-defaults.md
|
||||
2026-07-30-adapter-owned-max-token-defaults.md: c6fc9f014124607a3e2c3520f9f3b9023b77935f
|
||||
2026-07-30-adapter-owned-max-token-defaults.zh.md: e0670f43eb5f27c2307c734e01a5b6718c09a67e
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: Adapter-owned max-token defaults
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-30-adapter-owned-max-token-defaults.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
An LLM adapter could serialize an explicit `GenerateOptions.maxTokens`, but its Cordis configuration could not establish a reconstructable conversation default. Applying a fallback only inside provider serialization would make the wire request differ from the durable `request/header`; putting every provider's default in Agent Loop would instead transfer deployment and model policy into the provider-neutral driver.
|
||||
|
||||
## Decision
|
||||
|
||||
`LlmResolvedModelInfo.defaultMaxTokens` carries an optional adapter-configured per-request output cap for one exact provider/model route. `LlmService` validates it as a positive safe integer and materializes it into `LlmCallConfig.maxTokens` only when the caller omitted a value. Explicit request or Agent options therefore win without clamping.
|
||||
|
||||
The agent loop continues to prepare calls before logging `request/header`, so an adapter default becomes a durable request fact before dispatch. Direct `LlmService.stream()` calls resolve the same default at the final adapter boundary. The field is a request default rather than a hard model output limit; adapters that preserve provider-owned defaults omit it.
|
||||
|
||||
The native DeepSeek adapter exposes `maxTokens` in Cordis config with a 256,000-token default and maps the effective value to `max_tokens`. Its default context capacity is 1,000,000 tokens: both built-in V4 entries publish that exact capacity, while configured entries without capacity and unlisted pass-through ids inherit the same adapter-wide fallback.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Apply the default only in DeepSeek serialization.** Rejected because the provider wire would contain a model-visible value absent from the durable request header.
|
||||
|
||||
**Set `AgentOptions.maxTokens` in every shipped application.** Rejected because applications would duplicate adapter deployment policy, direct LLM calls would behave differently, and selecting another provider would retain a DeepSeek-specific cap.
|
||||
|
||||
**Represent 256,000 as a hard per-model maximum.** Rejected because the configured value is the desired request budget, not evidence that every configured endpoint rejects larger outputs. Explicit callers remain authoritative.
|
||||
|
||||
**Leave the provider default in control.** Rejected for the native DeepSeek deployment because the product requires a stable 256,000-token conversation budget across compatible endpoints.
|
||||
|
||||
## Consequences
|
||||
|
||||
DeepSeek conversations send `max_tokens: 256000` by default, and the same value appears in the session request header. Deployments can change the adapter default through `llm-deepseek.config.maxTokens`; per-agent and per-request values override it. Other adapters retain their existing behavior until they intentionally publish `defaultMaxTokens`.
|
||||
|
||||
The 256,000-token output budget reserves a large part of the one-million-token context on endpoints that pre-allocate requested output. Deployments whose gateway or model supports a smaller budget must lower `maxTokens`; the explicit configuration is preferable to an undocumented provider fallback.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: 适配器持有的最大 token 默认值
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-30-adapter-owned-max-token-defaults.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
LLM(大语言模型)适配器可以序列化显式的 `GenerateOptions.maxTokens`,但无法通过 Cordis 配置建立可重建的对话默认值。仅在提供方序列化中应用回退,会导致协议请求与持久 `request/header` 不一致;若将各提供方默认值都放进 agent loop(智能体循环),则会把部署与模型策略转移到提供方无关的驱动器中。
|
||||
|
||||
## Decision
|
||||
|
||||
`LlmResolvedModelInfo.defaultMaxTokens` 携带一条确切提供方/模型路由的可选单次请求输出上限,该值由适配器配置。`LlmService` 将其校验为正安全整数,并且仅在调用方省略值时才填入 `LlmCallConfig.maxTokens`。因此,显式请求值或 Agent 选项优先,且不会被自动调整。
|
||||
|
||||
agent loop 仍在记录 `request/header` 前准备调用,因此适配器默认值会在分派前成为持久请求事实。直接调用 `LlmService.stream()` 时,也会在最终适配器边界解析同一默认值。该字段是请求默认值,而非模型输出硬上限;保留提供方持有默认值的适配器会省略它。
|
||||
|
||||
原生 DeepSeek 适配器在 Cordis 配置中公开 `maxTokens`,默认值为 256,000 token,并将生效值映射为 `max_tokens`。其默认上下文容量为 1,000,000 token:两个内置 V4 配置项均公布这一精确容量;不含容量的已配置项和未列出的原样传递 id 则继承同一个适配器级回退值。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**仅在 DeepSeek 序列化中应用默认值。** 不予采纳,因为提供方协议会包含持久请求 header 中缺失的模型可见值。
|
||||
|
||||
**在每个已发布应用中设置 `AgentOptions.maxTokens`。** 不予采纳,因为应用会重复适配器部署策略,直接 LLM 调用的行为将不同,而且选择另一个提供方后仍会保留 DeepSeek 专用上限。
|
||||
|
||||
**将 256,000 表示为每模型硬上限。** 不予采纳,因为配置值是所需请求预算,无法证明每个已配置端点都会拒绝更大的输出。显式调用方仍具有最终决定权。
|
||||
|
||||
**由提供方默认值控制。** 对原生 DeepSeek 部署不予采纳,因为产品要求各兼容端点都采用稳定的 256,000 token 对话预算。
|
||||
|
||||
## Consequences
|
||||
|
||||
DeepSeek 对话默认发送 `max_tokens: 256000`,会话请求 header 中也会出现相同的值。部署可以通过 `llm-deepseek.config.maxTokens` 更改适配器默认值;每个 agent 和每次请求的值都会覆盖它。其他适配器会保留现有行为,直至主动公布 `defaultMaxTokens`。
|
||||
|
||||
对于预分配请求输出的端点,256,000 token 的输出预算会占用 1,000,000 token 上下文中的很大部分。如果部署使用的 gateway 或模型仅支持较小预算,则必须调低 `maxTokens`;显式配置优于未记录的提供方回退值。
|
||||
Reference in New Issue
Block a user