Merge remote-tracking branch 'origin/master' into worktree/web-plugin-config
# Conflicts: # docs/event-producer-consumer.i18n.yaml # docs/event-producer-consumer.md # docs/event-producer-consumer.zh.md # docs/module-graph.i18n.yaml # docs/module-graph.md # docs/module-graph.zh.md # packages/client/ui-conversation/README.i18n.yaml # packages/client/ui-conversation/README.md # packages/client/ui-conversation/README.zh.md # packages/client/ui-conversation/package.json # pnpm-lock.yaml
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-24-workspace-context.md
|
||||
2026-06-24-workspace-context.md: 53a580aee50752b7c6daeff5caa42ba6409c8885
|
||||
2026-06-24-workspace-context.md: e7a3724847b9dc8cfad11e87b2c96a3ef442bcba
|
||||
2026-06-24-workspace-context.zh.md: 39c3f52da10b7299301d10bd8b78330cbae8e19c
|
||||
|
||||
@@ -78,7 +78,7 @@ There is intentionally no watcher. Detection occurs at the next successful struc
|
||||
|
||||
## Consequences
|
||||
|
||||
Workspace guidance is isolated per session and shared by the demo front doors, Web Host, and every tool presentation mode. Initial, nested, and changed instructions are durable and replayable. The generic session/agent context contract carries typed source data through inbox-staged and durably entered user messages without flattening entries.
|
||||
Workspace guidance is isolated per session and shared by the demo entry points, Web Host, and every tool presentation mode. Initial, nested, and changed instructions are durable and replayable. The generic session/agent context contract carries typed source data through inbox-staged and durably entered user messages without flattening entries.
|
||||
|
||||
Repository text remains untrusted input. Lower-authority user-role framing, explicit precedence language, and delimiter escaping reduce risk but do not eliminate prompt injection. Following a candidate symlink to its target widens that surface to off-tree content, so the permission and sandbox layers that confine `ctx.fs` to trusted roots are the boundary that treats workspace files as data rather than authority (the [instruction-symlink follow note](2026-07-21-follow-instruction-symlinks.md) owns the residual risk).
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-harness-level-loop.md
|
||||
2026-07-16-harness-level-loop.md: e84a5738a55988d3b968eac829d9daaf10d7e304
|
||||
2026-07-16-harness-level-loop.zh.md: d7cda56d66d4456ac5c3c7d55bb238e85fcacbc5
|
||||
2026-07-16-harness-level-loop.md: e5dd94fbf76e27f0843666f29622969635ef2fc3
|
||||
2026-07-16-harness-level-loop.zh.md: 773b5ff1f4b07206d0d817c29774435ce24407b4
|
||||
|
||||
@@ -70,7 +70,7 @@ The human UX follows the compact Codex shape in the [public OpenAI Codex TUI dis
|
||||
|
||||
The model receives only `get_goal`, `create_goal`, and `update_goal`. It may create a goal when a direct human request clearly asks for substantial multi-round work, and it may infer that intent in any language. It must not turn routine one-turn work into a goal. Code requires a direct human message in the current live root-agent turn; semantic interpretation remains model judgment. An autonomous goal round may report `complete` or `blocked` for the exact current goal round but cannot edit, pause, resume, or replace the human objective.
|
||||
|
||||
TUI mounts the shared command registry and complete goal stack by default and exposes `/goal` through one producer. ACP mounts the goal domain, model tools, and same-session driver but deliberately omits the human command plane. Every effective registered command is discoverable and invocable through every composed command adapter; a plugin incompatible with an application omits its command producer from that composition rather than relying on registry-level surface masks. The UI-less agent spine is opt-in so one-shot callers do not silently become multi-round operations. The headless CLI and JSON-RPC front doors do not consume the command plane; ordinary human text can still authorize model goal tools when that stack is composed.
|
||||
TUI mounts the shared command registry and complete goal stack by default and exposes `/goal` through one producer. ACP mounts the goal domain, model tools, and same-session driver but deliberately omits the human command plane. Every effective registered command is discoverable and invocable through every composed command adapter; a plugin incompatible with an application omits its command producer from that composition rather than relying on registry-level surface masks. The UI-less agent spine is opt-in so one-shot callers do not silently become multi-round operations. The headless CLI and JSON-RPC entry points do not consume the command plane; ordinary human text can still authorize model goal tools when that stack is composed.
|
||||
|
||||
### Fresh-agent Ralph execution
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ Goal Round 驱动器为每个特定的实时 agent 至多拥有一个待定预
|
||||
|
||||
模型只接收 `get_goal`、`create_goal` 和 `update_goal`。当直接人类请求清楚要求大量多 Round 工作时，模型可以创建目标，并且可以从任何语言推断该意图。它不得把日常单 Turn 工作变成目标。代码要求当前实时根 agent Turn 中有一条人类直接发送的消息；语义解释仍是模型判断。自治目标 Round 可以为确切的当前 Goal Round 报告 `complete` 或 `blocked`，但不能编辑、暂停、恢复或替换人类目标。
|
||||
|
||||
TUI 默认挂载共享命令注册表和完整目标栈，并通过一个生产方暴露 `/goal`。ACP（Agent Client Protocol）挂载目标领域、模型工具和同会话驱动器，但有意省略人类命令平面。每条有效已注册命令都能被每个已组合的命令适配器发现和调用；若插件与某应用不兼容，该应用组合会省略其命令生产方，而不是依赖注册表层面的表面掩码。无 UI 的 agent 主干要求显式选择加入，以免单次调用方静默变成多 Round 操作。无头 CLI（命令行界面）与 JSON-RPC 前端不消费命令平面；挂载目标栈后，普通人类文本仍可授权模型目标工具。
|
||||
TUI 默认挂载共享命令注册表和完整目标栈，并通过一个生产方暴露 `/goal`。ACP（Agent Client Protocol）挂载目标领域、模型工具和同会话驱动器，但有意省略人类命令平面。每条有效已注册命令都能被每个已组合的命令适配器发现和调用；若插件与某应用不兼容，该应用组合会省略其命令生产方，而不是依赖注册表层面的表面掩码。无 UI 的 agent 主干要求显式选择加入，以免单次调用方静默变成多 Round 操作。无头 CLI（命令行界面）与 JSON-RPC 运行入口不消费命令平面；挂载目标栈后，普通人类文本仍可授权模型目标工具。
|
||||
|
||||
### 全新 agent Ralph 执行
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md
|
||||
2026-07-16-persistent-pty-sessions.md: f27d799574e61d92123c9ca4e31c5c2a9d3229b4
|
||||
2026-07-16-persistent-pty-sessions.zh.md: 0c23619faf00794c2c4e7b3f85ef961faddd99e0
|
||||
2026-07-16-persistent-pty-sessions.md: fb9cd06bade7bc357baa738f0d9dd03b7f5b7936
|
||||
2026-07-16-persistent-pty-sessions.zh.md: 55a5848c1ab1e8c2cd3b29f2d4748ea4abbe088c
|
||||
|
||||
@@ -152,7 +152,7 @@ The package ships concise tool guidance explaining persistent state, owner isola
|
||||
|
||||
**Include TUI sequences and BEL handling.** Rejected. The source prototype treats those paths as timing-sensitive and still records unresolved alternate-screen and interaction failures. Line-oriented PTY use proves the core value without making those unverified behaviors foundational.
|
||||
|
||||
**Use an out-of-process daemon immediately.** Rejected for the initial in-process capability because current persistent front doors already keep a Cordis context alive. A daemon becomes justified by cross-process restoration or multi-client attachment, both deferred here.
|
||||
**Use an out-of-process daemon immediately.** Rejected for the initial in-process capability because current long-lived entry points already keep a Cordis context alive. A daemon becomes justified by cross-process restoration or multi-client attachment, both deferred here.
|
||||
|
||||
## Verification
|
||||
|
||||
|
||||
@@ -152,7 +152,7 @@ plugins:
|
||||
|
||||
**包含 TUI sequence 与 BEL 处理。**拒绝。源 prototype 将这些路径视为 timing-sensitive，且仍记录未解决的 alternate-screen 和交互失败。行式 PTY 已能证明核心价值，无需把未经验证的行为放进基础层。
|
||||
|
||||
**立即采用进程外 daemon。**初始的进程内功能不采用，因为当前持久 front door 已能维持 Cordis context。跨进程恢复或多客户端 attach 会让 daemon 变得合理，但两者都已推迟。
|
||||
**立即采用进程外 daemon。**初始的进程内功能不采用，因为当前长驻的运行入口已能维持 Cordis context。跨进程恢复或多客户端 attach 会让 daemon 变得合理，但两者都已推迟。
|
||||
|
||||
## 验证
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-19-human-goal-command.md
|
||||
2026-07-19-human-goal-command.md: 5fdd80f7423b80e84e58f7379130ee59a2e8a723
|
||||
2026-07-19-human-goal-command.zh.md: 89d29497abfc758ff8373d272aa8620ca1bcfcc2
|
||||
2026-07-19-human-goal-command.md: d68e4025a4d37d07211f15ddfc6069bc7c637124
|
||||
2026-07-19-human-goal-command.zh.md: dbda35c731ccdc7aff2d4213a3df8b78070319df
|
||||
|
||||
@@ -68,5 +68,5 @@ The producer suite uses the real command registry, goal service, agent registry,
|
||||
- The portable command contract has no modal editor or confirmation interaction; inline edit and explicit clear are intentional until a general cross-surface interaction primitive exists.
|
||||
- `/goal` does not accept a per-command round cap. Deployment config owns the default, and the authorized model tool can edit a cap after direct human instruction.
|
||||
- TUI renders portable plain text rather than a continuously updated goal status widget. Reconnectable command output and adapter-specific status indicators are deferred.
|
||||
- The ACP automation server, headless CLI, and JSON-RPC front doors do not consume the command registry.
|
||||
- The ACP automation server, headless CLI, and JSON-RPC entry points do not consume the command registry.
|
||||
- The command observes and mutates state but does not certify completion or blockers. Evaluator-backed certification remains deferred to a separate policy layer with an explicit authority and isolation contract.
|
||||
|
||||
@@ -68,5 +68,5 @@ TUI 应用包作出相反的产品选择。它默认让 `goals` 使用所有者
|
||||
- 可移植命令约定没有模态编辑器或确认交互；在出现通用跨界面交互原语之前，行内编辑与明确清除是有意选择。
|
||||
- `/goal` 不接受逐命令 Round 上限。部署配置拥有默认值；得到直接人类指示后，已授权模型工具可以编辑上限。
|
||||
- TUI 渲染可移植纯文本，而不是持续更新的目标状态组件。可重连命令输出和适配器专用状态指示器予以延期。
|
||||
- ACP 自动化服务器、无头 CLI 与 JSON-RPC 前端不消费命令注册表。
|
||||
- ACP 自动化服务器、无头 CLI 与 JSON-RPC 运行入口不消费命令注册表。
|
||||
- 该命令观察并改变状态，但不认证完成或阻塞。基于评估器的认证延期到具有明确权限与隔离约定的独立策略层。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-19-plugin-command-registration.md
|
||||
2026-07-19-plugin-command-registration.md: c8f0f2772a41948e9eb257a16f40194518c568f9
|
||||
2026-07-19-plugin-command-registration.zh.md: a8f7a3d1e7947aeb41344a72106cee55d249010c
|
||||
2026-07-19-plugin-command-registration.md: 76feba84492bd6b245246a6f1fb1e8f68d555684
|
||||
2026-07-19-plugin-command-registration.zh.md: e89cb79ba1e15dfefa527fe7bc37c09c3449f5b4
|
||||
|
||||
@@ -12,7 +12,7 @@ A shared mechanism must remain a UI concern rather than a model tool or agent-lo
|
||||
|
||||
## Decision
|
||||
|
||||
`@deepseek-ai/dsh-commands` in `packages/interaction/commands/` is the product command registry. The TUI app bundle mounts it beside its consuming front door; the [automation-only ACP app](../simplification/2026-07-23-acp-automation-only-protocol.md) and the executor-less, UI-less agent spine omit it. TUI injects the service, while command producers depend only on the registry and any domain they operate.
|
||||
`@deepseek-ai/dsh-commands` in `packages/interaction/commands/` is the product command registry. The TUI app bundle mounts it beside its consuming front end; the [automation-only ACP app](../simplification/2026-07-23-acp-automation-only-protocol.md) and the executor-less, UI-less agent spine omit it. TUI injects the service, while command producers depend only on the registry and any domain they operate.
|
||||
|
||||
### Registry contract
|
||||
|
||||
@@ -50,7 +50,7 @@ TUI tests exercise all migrated built-ins, live plugin discovery, help/autocompl
|
||||
|
||||
- **Keep adapter-local switches** — rejected because optional plugins cannot contribute discovery and behavior without editing the TUI.
|
||||
- **Represent human commands as model tools** — rejected because discovery and direct invocation are human UI behavior; routing through the model adds latency, token cost, and reinterpretation.
|
||||
- **Put the registry in the core agent spine** — rejected because UI-less front doors do not consume it, while TUI can compose it explicitly.
|
||||
- **Put the registry in the core agent spine** — rejected because UI-less entry points do not consume it, while TUI can compose it explicitly.
|
||||
- **Make `dsh-agent-loop` inject commands** — rejected because the loop does not execute or discover human commands. Agent-scoped producers declare the UI dependency in a child plugin instead.
|
||||
- **Attach adapter masks to each definition** — rejected because support is a composition fact, not command-domain state. Every composed adapter exposes a registered command; an incompatible plugin omits registration in that deployment.
|
||||
- **Send unknown slash input to the model** — rejected because typoed or unavailable direct actions must fail predictably rather than change execution planes.
|
||||
@@ -68,4 +68,4 @@ TUI tests exercise all migrated built-ins, live plugin discovery, help/autocompl
|
||||
- Input metadata is limited to an unstructured text hint. Typed forms, argument schemas, and completion providers remain command-owned or require a later registry or consumer extension.
|
||||
- Generic command output is live-only and is not reconstructed after TUI restart.
|
||||
- Registry cancellation stops awaiting immediately, but external work stops only when a handler cooperates with its signal.
|
||||
- The ACP automation server, headless CLI, and JSON-RPC SDK front doors do not expose the command plane; only TUI consumes it.
|
||||
- The ACP automation server, headless CLI, and JSON-RPC SDK entry points do not expose the command plane; only TUI consumes it.
|
||||
|
||||
@@ -12,7 +12,7 @@ TUI 拥有斜杠命令。如果命令名、帮助文本、自动补全、分派
|
||||
|
||||
## 决策
|
||||
|
||||
位于 `packages/interaction/commands/` 的 `@deepseek-ai/dsh-commands` 是产品命令注册表。TUI 应用 bundle（组合包）把它挂载在消费该服务的入口旁；[仅面向自动化的 ACP（Agent Client Protocol）应用](../simplification/2026-07-23-acp-automation-only-protocol.md)和无执行器、无 UI 的智能体 spine（主干）都省略该服务。TUI 注入该服务，命令生产者只依赖注册表及其操作的领域。
|
||||
位于 `packages/interaction/commands/` 的 `@deepseek-ai/dsh-commands` 是产品命令注册表。TUI 应用 bundle（组合包）把它挂载在消费该服务的前端旁；[仅面向自动化的 ACP（Agent Client Protocol）应用](../simplification/2026-07-23-acp-automation-only-protocol.md)和无执行器、无 UI 的智能体 spine（主干）都省略该服务。TUI 注入该服务，命令生产者只依赖注册表及其操作的领域。
|
||||
|
||||
### 注册表约定
|
||||
|
||||
@@ -50,7 +50,7 @@ TUI 测试覆盖全部迁移后的内置命令、实时插件发现、帮助与
|
||||
|
||||
- **保留适配器本地 switch**——不予采纳，因为可选插件无法贡献发现与行为，除非修改 TUI。
|
||||
- **把人类命令表示为模型工具**——不予采纳，因为发现与直接调用属于人类 UI 行为；经由模型路由会增加延迟、token 成本和重新解释。
|
||||
- **把注册表放入核心智能体主干**——不予采纳，因为无 UI 前端不消费它，而 TUI 可以显式组合它。
|
||||
- **把注册表放入核心智能体主干**——不予采纳，因为无 UI 运行入口不消费它，而 TUI 可以显式组合它。
|
||||
- **让 `dsh-agent-loop` 注入 commands**——不予采纳，因为循环不执行也不发现人类命令。智能体作用域生产者改为在子插件中声明 UI 依赖。
|
||||
- **为每个定义附加适配器掩码**——不予采纳，因为支持能力是组合事实，而不是命令领域状态。每个已组合适配器都暴露已注册命令；不兼容插件不会在该部署中注册。
|
||||
- **把未知斜杠输入发送给模型**——不予采纳，因为输入错误或不可用的直接操作必须可预测地失败，而不能改变执行平面。
|
||||
@@ -68,4 +68,4 @@ TUI 测试覆盖全部迁移后的内置命令、实时插件发现、帮助与
|
||||
- 输入元数据仅限非结构化文本提示。类型化表单、参数模式和补全提供器仍由命令拥有，或需要后续注册表或消费方扩展。
|
||||
- 通用命令输出仅实时存在，TUI 重启后不会重建。
|
||||
- 注册表取消会立即停止等待，但外部工作只有在处理器配合信号时才会停止。
|
||||
- ACP 自动化服务器、无头 CLI 与 JSON-RPC SDK 前端不暴露命令平面；只有 TUI 消费它。
|
||||
- ACP 自动化服务器、无头 CLI 与 JSON-RPC SDK 运行入口不暴露命令平面；只有 TUI 消费它。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md
|
||||
2026-07-21-local-instruction-overlay.md: 3c7b2141b0515b5e667be4add6ad765e26c88cd8
|
||||
2026-07-21-local-instruction-overlay.md: fb5f916d426595a80bbbaa4192e4a3975b922b8a
|
||||
2026-07-21-local-instruction-overlay.zh.md: c97ed04607f497d829da0e904c248836252c73f7
|
||||
|
||||
@@ -26,7 +26,7 @@ The base and local candidates in one directory must stay independent across base
|
||||
|
||||
**Keep it opt-in through `instructionFileCandidates`.** Rejected: one directory has a single winner, so a `.local.` name added to that list shadows the base file rather than supplementing it. The packages guidance to keep opt-ins out of shipped defaults is outweighed here by strong prior art and the user-facing expectation that `.local.` files are always read.
|
||||
|
||||
**Default at the product `cordis.yml` level instead of the plugin schema.** Rejected: it would enable `.local.` only for whichever front door remembered to opt in, splitting behavior across TUI/ACP/headless and duplicating a value that belongs beside the existing candidate default.
|
||||
**Default at the product `cordis.yml` level instead of the plugin schema.** Rejected: it would enable `.local.` only for whichever entry point remembered to opt in, splitting behavior across TUI/ACP/headless and duplicating a value that belongs beside the existing candidate default.
|
||||
|
||||
**Reuse the bare directory as the scope key for base and local files.** Rejected: base and local files in one directory would collide in every scope-keyed map, so a change to one would suppress or overwrite the other. A distinct scope key per candidate keeps them independent without widening the persisted metadata shape.
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 2a3925d8b779fed7f8b3330d079fdb3059685fd5
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: be478ece73039201a820d804435610faea4cb3fe
|
||||
@@ -0,0 +1,214 @@
|
||||
# Agent Note: Web multimodal image input and durable attachments
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Before this change, the Web composer accepted only text: `InputBar` received a string draft, `ConversationService.send()` created text content, and the host forwarded that content to the agent. Users could not paste an image, inspect it before sending, submit an image-only prompt, or recover sent images from history.
|
||||
|
||||
This is not only a composer gap. Core needs a durable image content block, providers need explicit modality handling, and the session log must reconstruct everything visible to a model. [The previous image-block removal](../../implemented/simplification/2026-07-04-drop-image-content-block.md) rejected a partial design that could silently lose or flatten images. A browser object URL, local path, provider URL, or base64 payload cannot be canonical session content.
|
||||
|
||||
The [Web client architecture](../../implemented/architecture/2026-07-19-gui-web-client-architecture.md) keeps components pure and per-session composer state in `ctx.conversation`; the [GUI layering and RPC protocol](../../implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md) makes durable events the source of truth for both live rendering and history replay. Image intake, persistence, provider conversion, and rendering therefore need one explicit lifecycle.
|
||||
|
||||
Peer products converge on an attachment rail above the editor, but their storage choices differ. Codex-style paths such as `/var/folders/.../codex-clipboard-*.png` are reasonable intake staging locations, not durable message identities: the operating system may delete them, another host cannot read them, and a resumed session cannot rely on them.
|
||||
|
||||
## Decision
|
||||
|
||||
Pasted or dropped raster images are the Web composer's first consumer of a durable attachment capability. Unsent files remain temporary client-owned draft state. The host validates and durably commits every accepted user image before appending its message event. A provider adapter that produces structured image output must durably commit the output before appending its assistant block. Canonical user and assistant content contains only role-neutral `ImageBlock` references.
|
||||
|
||||
Version one supports PNG, JPEG, WebP, and GIF paste and drag-and-drop, image-only or mixed prompts, historical user and assistant image rendering, and original-image preview on double-click. File picking, generic files, PDF, audio, video, image copying, and a custom context menu remain separate follow-ups.
|
||||
|
||||
### Product behavior
|
||||
|
||||
- Pasting or dropping one or more supported images adds ordered thumbnails above the textarea without inserting placeholder text. Dragging files over the composer highlights the drop target.
|
||||
- The same resident `InputBar` renders the rail in both blank-session Hero and active-session layouts. The rail is hidden when empty and scrolls horizontally instead of widening the composer.
|
||||
- Each approximately 72-by-72-pixel thumbnail has a remove action and opens its original draft image on double-click.
|
||||
- A prompt may contain text and images or images only. Pure text paste remains native browser behavior; mixed clipboard content inserts its text normally while adding its files to the rail, and file-only paste prevents default browser handling. File drops on the composer always prevent browser navigation and report unsupported files locally.
|
||||
- A failed send restores the complete text and image draft without clobbering text or images added while the request was in flight. Removal, successful send, session-scope disposal, rendered-history disposal, and application disposal revoke the object URLs they own.
|
||||
- Historical user and assistant images use one `MessageImage` control. Inline images preserve intrinsic aspect ratio, do not upscale, and stay within a 240-by-240-pixel box.
|
||||
- Double-clicking a message image opens the stored original in a viewport-bounded modal. Escape, the close control, and backdrop activation close it and restore focus.
|
||||
- Version one does not override the browser context menu and provides no explicit image-copy action.
|
||||
|
||||
### Storage lifecycle and ownership
|
||||
|
||||
The persistence boundary is message acceptance, not paste:
|
||||
|
||||
| State | Allowed representation | Durability and ordering |
|
||||
| --- | --- | --- |
|
||||
| Unsent user draft | Browser `File` plus object URL; a native client may use an OS temporary file such as `/var/...` | Temporary and client-owned. It may disappear on reload or process exit and never appears in a session event. |
|
||||
| Accepted user image | Immutable object below `DSH_HOME` plus `ImageAttachmentRef` | The host commits every image before `agent.send()` or `agent.steer()` can append the owning user event. |
|
||||
| Structured model image output | Immutable object below `DSH_HOME` plus `ImageAttachmentRef` | The provider adapter commits the bytes before it emits a completed image block or assistant message event. Temporary URLs, paths, and base64 are forbidden in the event. |
|
||||
|
||||
Each session's `InputMachine` state keeps the ordered runtime-only attachment identifiers alongside the live draft. The framework-owned chat store receives only the draft's plain-text persistence mirror, while `ConversationService` owns the corresponding browser-only `File` and object-URL registry:
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type DraftAttachmentId = Branded<'DraftAttachmentId'>
|
||||
|
||||
interface ChatStoreState {
|
||||
selection: object | null
|
||||
draft: string
|
||||
view: string | null
|
||||
}
|
||||
|
||||
interface InputState {
|
||||
draft: string
|
||||
imageIds: readonly DraftAttachmentId[]
|
||||
}
|
||||
|
||||
interface ComposerAttachment {
|
||||
kind: 'image'
|
||||
id: DraftAttachmentId
|
||||
file: File
|
||||
previewUrl: string
|
||||
}
|
||||
```
|
||||
|
||||
This split uses the session provide channel's input hook and actions as the single subscription path for live composer state while keeping non-serializable browser objects out of persisted JSON. Only the plain-text draft mirror uses `localStorage`; attachment identifiers, browser `File` objects, and object URLs remain scoped to the live session input shell. Unsent images therefore do not survive reload or session-scope disposal. A Workspace switch moves a mixed text-and-image draft only when the destination shell accepts the complete image batch; refusal leaves both parts with the source. A native client may stage input in an OS temporary directory, but it must treat that path exactly like the browser object URL: delete it when no longer needed and copy the bytes into the durable store before message acceptance.
|
||||
|
||||
The local attachment backend resolves an explicit `dshHome`, then `$DSH_HOME`, then `~/.dsh`. It stores content-addressed objects below `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` with owner-only directory and file permissions. On each process's first save for one home, it creates that home and synchronizes every ancestor entry to the filesystem root; existence is not treated as durability because another process may still be between `mkdir` and parent `fsync`. A temporary file is then written, synchronized, atomically published, and made durable with directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) before the service returns a reference. The content digest is encoded in the opaque `sha256:<digest>` identifier. Admission and reads fully decode supported rasters before accepting their format and dimensions, and every read also verifies the digest, byte length, and logged metadata.
|
||||
|
||||
The store performs no automatic deletion in version one. Sent user images and model-generated images remain reachable for history, resume, and fork. Reference-aware garbage collection needs a separate design because an age-only rule can delete data still referenced by a durable session. Deployment byte and pixel limits are admission policy on writes; reads verify the digest and recorded metadata without reapplying current admission limits, so lowering policy does not invalidate older history.
|
||||
|
||||
### Durable content and prompt wire
|
||||
|
||||
The attachment seam exposes immutable image write and verified read operations. The canonical metadata is deliberately narrower than a generic file record:
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type AttachmentId = Branded<'AttachmentId'>
|
||||
|
||||
interface ImageAttachmentRef {
|
||||
attachmentId: AttachmentId
|
||||
mediaType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
|
||||
bytes: number
|
||||
width: number
|
||||
height: number
|
||||
name?: string
|
||||
}
|
||||
|
||||
interface ImageBlock {
|
||||
type: 'image'
|
||||
attachment: ImageAttachmentRef
|
||||
}
|
||||
```
|
||||
|
||||
`ImageBlock` joins the merge-extensible core `ContentBlockMap` and is valid in either user or assistant content. It never carries base64, an object URL, a filesystem path, or a provider-owned locator. This keeps the session event plus immutable object store sufficient to reconstruct the exact model-visible image. The LLM vocabulary therefore has a type-only dependency on the attachment seam; provider runtime dependencies remain adapter-specific.
|
||||
|
||||
The browser cannot mint a durable reference, so `session.prompt` accepts a narrow intake union rather than canonical `ContentBlock[]`:
|
||||
|
||||
```ts
|
||||
export {}
|
||||
|
||||
type PromptInputPart =
|
||||
| { type: 'text'; text: string }
|
||||
| {
|
||||
type: 'image'
|
||||
mediaType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
|
||||
data: string
|
||||
name?: string
|
||||
}
|
||||
```
|
||||
|
||||
Base64 crosses JSON-RPC once and is discarded after persistence. The host validates canonical base64, image count, aggregate bytes, individual bytes, the declared MIME against a fully decoded raster, intrinsic dimensions, and decoded-pixel count. It awaits the seam's storage-free `validateImage` for every batch member before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Storage commits then run in submission order to bound full-raster decoder memory. If a later storage I/O operation fails, the host appends no user event, but an earlier immutable content-addressed object may remain unreferenced; version one leaves cleanup to future reference-aware garbage collection instead of adding destructive rollback to the deduplicated store. Only after every image succeeds does it call the agent with normalized text and durable image blocks in the submitted order. A failure exposes no attachment path or raw bytes.
|
||||
|
||||
`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and rejects invalidated late loads before allocating an object URL so an unmounted session or disposed service cannot repopulate the cache.
|
||||
|
||||
### Model capabilities and provider behavior
|
||||
|
||||
Model catalog entries gain optional merge-extensible input modality declarations. A missing declaration means unknown; a present list without `image` is an explicit negative capability.
|
||||
|
||||
The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Image-bearing prompt admission and model selection share one per-agent serial boundary, and a dequeued prompt remains pending until its durable message event publishes ([ordering decision](../bug-fix/2026-07-29-atomic-web-image-admission.md)); a steering carrier gates from its enqueue until its `steering/message` event publishes, closing the outbox hop that never enters the queued mirror. Selection rejects a text-only target while an image is pending publication or remains in the session's current derived history. Compaction can remove old images and make a later text-only selection valid; idle without publication releases a claimed queued carrier, while steering retained in the outbox stays gated until publication or discard. `session.updateQueue` edits accept text content only, so a queue edit cannot inject an image past this admission boundary. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. The browser rejects unsupported declared image media types before allocating preview URLs, but it does not snapshot deployment limits or model capability: a handshake snapshot cannot represent a session's current target after `session.selectModel`, and deployment policy may change independently. The host validates the complete batch against current byte, count, aggregate, media, dimension, pixel, and routed-model policy before writing any attachment or event; its rejection renders through the composer error strip.
|
||||
|
||||
The Pi-AI adapter is the first visual-input route: it resolves `ctx.attachments` at request time, recursively converts each durable image reference including references nested inside tool results, and emits native image content only for models that declare image input. The shipped composition registers Pi-AI OpenAI and Anthropic routes alongside the text-only default DeepSeek route; selecting the active provider/model remains a host composition or profile concern rather than an image-input CLI feature. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. The hand-written DeepSeek adapter throws typed `UNSUPPORTED_CONTENT` for an image anywhere in the request, including nested tool results. No adapter may flatten or skip an image.
|
||||
|
||||
Core supports structured assistant image blocks, but no current production provider route is certified for image output. Any future output-capable adapter must retrieve provider bytes under bounded size and time policy, validate them through the same attachment service, persist them, and only then publish the atomic `ImageBlock`. A URL in assistant Markdown remains text and is never downloaded automatically.
|
||||
|
||||
Provider-neutral token estimation does not guess visual pricing from image dimensions; provider-reported usage remains authoritative. ACP renders an explicit image marker until that protocol surface gains native image support rather than silently omitting the block.
|
||||
|
||||
Compaction replays the selected conversation prefix, including image references, into the configured summarization route. A visual-capable route resolves those references through its adapter; a text-only route fails explicitly instead of silently dropping the visual context. The synthesized checkpoint remains text-only, and `compact-basic` rejects image summary output with `UNSUPPORTED_CONTENT`.
|
||||
|
||||
### History rendering and original preview
|
||||
|
||||
History folding preserves `ImageBlock` in both user and assistant messages. User images align to the trailing edge above their text; assistant images remain in their original content-block position in the leading narration flow. `MessageImage` derives a stable inline box from recorded dimensions, resolves bytes through the session-authorized loader, uses `object-fit: contain`, and turns a missing or corrupt object into a retryable error control.
|
||||
|
||||
Composer thumbnails and each `MessageImage` own ephemeral original-preview state and invoke the same pure `ImageLightbox`. The modal uses the already resolved original object URL, constrains only display size, focuses its close control, and restores the previous focus target when closed.
|
||||
|
||||
### Limits and trust boundaries
|
||||
|
||||
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 5 MiB per image, 10 images and 20 MiB aggregate image bytes per message, and 40 million intrinsic pixels per image. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (32 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
|
||||
Malformed base64, unsupported or mismatched media, truncated image payloads, excess bytes, excess image count, excess pixels, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser.
|
||||
|
||||
### Package and surface changes
|
||||
|
||||
| Surface | Responsibility |
|
||||
| --- | --- |
|
||||
| `packages/attachment/attachment` | Opaque attachment identifier, image reference, limits, failures, and `ctx.attachments` service. |
|
||||
| `packages/attachment/attachment-local` | Private content-addressed storage, complete raster decoding, integrity verification, and configuration. |
|
||||
| `packages/llm/llm` | Role-neutral `ImageBlock` and input-modality metadata. |
|
||||
| `packages/llm/llm-pi-ai` | Resolve durable supported image input into native provider content. |
|
||||
| `packages/llm/llm-deepseek` | Reject image content explicitly. |
|
||||
| `packages/compact/compact-basic` | Preserve images in summary input and reject non-text checkpoint output explicitly. |
|
||||
| `packages/host/apiproxy` and `packages/bundle/base` | Narrow upload wire, persist-before-event ordering, session-authorized reads, limits and model preflight, plus default profile composition. |
|
||||
| `packages/client/connection` and `packages/client/runtime` | Bounded request buffering, wire types, fixture images, prompt uploads, attachment reads, and durable-reference folding. |
|
||||
| `packages/client/ui-conversation` | Per-session draft images, attachment rail, user and assistant image controls, and original preview. |
|
||||
| `packages/acp/acp` | Explicit fallback rendering for image blocks. |
|
||||
|
||||
The attachment packages form the interface/implementation side of one capability seam. Composer behavior stays in the conversation object layer, provider conversion stays in adapters, and no change is required in `agent-loop`.
|
||||
|
||||
### Implementation
|
||||
|
||||
The implemented slice includes the attachment seam, role-neutral image block, Pi-AI input conversion, DeepSeek rejection, durable host ordering, Web upload/read protocol, current image-limit enforcement, bounded Web request bodies, in-memory draft images, paste/drop rail, user and assistant history rendering, double-click preview, compaction handling, and keyless assembled Web coverage.
|
||||
|
||||
No compatibility shim is required for the pre-release prompt wire; all call sites and fixtures change with the introducing slice.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Keep every intake image in `/var` or another temporary directory
|
||||
|
||||
Temporary storage is appropriate before send, including for a native client that receives clipboard files through the operating system. It is not appropriate after acceptance: cleanup is outside the harness's control, paths are host-specific, and resume or fork can outlive the file. The proposal permits temporary staging but copies accepted bytes into `DSH_HOME` before the event.
|
||||
|
||||
### Persist immediately on paste or drop
|
||||
|
||||
Immediate persistence makes drafts reload-resistant but creates durable objects before a session or message owns them, which requires quota, orphan lifetime, and cleanup policy. Version one keeps the unsent draft temporary and makes send acceptance the durability boundary.
|
||||
|
||||
### Inline base64 in messages and session logs
|
||||
|
||||
This duplicates binary data across RPC, events, history pages, forks, compaction, and browser storage, and invites token accounting to treat encoding text as model text. One immutable object plus small references keeps the durable representation bounded.
|
||||
|
||||
### Use browser object URLs, local paths, or provider URLs as canonical content
|
||||
|
||||
Object URLs expire with the document, local paths are not portable, and provider URLs may expire, track viewers, or expose credentials. They remain temporary transport or preview details only.
|
||||
|
||||
### Use one generic `AttachmentBlock` for images, files, audio, and video
|
||||
|
||||
Composer presentation can use a generic attachment rail, but provider semantics are modality-specific. Images are native multimodal input; PDFs may be provider files or extracted text; video may be native, sampled, or unsupported. A specific `ImageBlock` forces every consumer to handle or reject the modality explicitly.
|
||||
|
||||
### Rely on UI capability checks or silently filter images
|
||||
|
||||
UI state can be stale and does not protect direct SDK, ACP, replay, or uncatalogued model paths. Silent filtering changes user intent. Provider enforcement remains mandatory, while UI checks are optional earlier feedback.
|
||||
|
||||
## Testing
|
||||
|
||||
- Storage tests cover content-addressed deduplication, private permissions, admission failures, corruption/missing-object failures, and reading history after deployment limits are lowered.
|
||||
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, image-admission/model-selection races (queued and steering placements), pending publication, idle release without publication, text-only queue edits, and selection against current derived history after compaction.
|
||||
- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, draft/session-scope/application object-URL cleanup, and a deferred historical read that completes after disposal; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail.
|
||||
- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, recursively nested tool-result images, preserved summary input, and explicit image-output rejection.
|
||||
- A credentialed real-API test sends a PNG through the Anthropic `claude-opus-4-8` route and requires the model to identify its QR code.
|
||||
- The current production adapter set has no certified image-output route; output-provider certification remains outside version one.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Durable storage grows without garbage collection. Version one chooses replay safety over premature deletion.
|
||||
- A missing or corrupt object makes exact model reconstruction fail. Failing loud preserves integrity but may prevent that session from continuing until repaired.
|
||||
- JSON-RPC base64 adds upload memory and roughly one-third encoding overhead. Version-one limits bound it; larger media needs streaming or a binary transport.
|
||||
- Unsent images do not survive reload. Durable drafts need quota and orphan cleanup rather than reusing message storage implicitly.
|
||||
- Original preview decodes more pixels than the inline control displays. Pixel limits, one clicked preview, and object-URL disposal bound but do not eliminate transient browser memory.
|
||||
- Capability metadata may be missing or stale. Host preflight improves feedback, while adapter enforcement remains authoritative.
|
||||
- A future output provider may require authenticated retrieval before an assistant image can complete, adding latency and a new failure point. Persist-before-event ordering favors replay integrity.
|
||||
- File picking, generic files/PDF, audio/video, durable draft staging, image copying, custom context menus, output-provider certification, and reference-aware garbage collection remain independent designs.
|
||||
@@ -0,0 +1,214 @@
|
||||
# Agent Note: Web 多模态图片输入与持久附件
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-22-web-multimodal-image-input-and-durable-attachments.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
在此变更之前，Web 输入区仅接受文本：`InputBar` 接收字符串草稿，`ConversationService.send()` 创建文本内容，宿主再把该内容转发给 agent（智能体）。用户无法粘贴图片、在发送前查看图片、提交仅含图片的提示词，也无法从历史记录中恢复已发送图片。
|
||||
|
||||
这不只是输入区功能缺失。核心层需要持久图片内容块，提供方需要明确处理模态，会话日志则必须重建模型可见的全部内容。[此前移除图片块的决策](../../implemented/simplification/2026-07-04-drop-image-content-block.md)否决了可能静默丢失图片或将其展平的不完整设计。浏览器对象 URL、本地路径、提供方 URL 或 base64 数据都不能成为规范会话内容。
|
||||
|
||||
[Web 客户端架构](../../implemented/architecture/2026-07-19-gui-web-client-architecture.md)要求组件保持纯粹，并将每个会话的输入区状态放在 `ctx.conversation` 中；[GUI 分层与 RPC 协议](../../implemented/architecture/2026-07-19-gui-layering-and-rpc-protocol.md)则要求持久事件成为实时渲染与历史回放的共同真源。因此，图片接收、持久化、提供方转换和渲染需要遵循同一个明确的生命周期。
|
||||
|
||||
同类产品普遍在编辑器上方设置附件栏，但存储方案各不相同。诸如 `/var/folders/.../codex-clipboard-*.png` 的 Codex 式路径适合作为接收输入时的暂存位置，却不能作为持久消息身份：操作系统可能删除文件，另一台宿主无法读取文件，恢复后的会话也不能依赖文件仍然存在。
|
||||
|
||||
## 决策
|
||||
|
||||
粘贴或拖放的光栅图片是 Web 输入区对持久附件能力的首个应用场景。未发送文件仍是由客户端持有的临时草稿状态。宿主在追加相应消息事件前，校验并持久提交每张已接受的用户图片。生成结构化图片输出的提供方适配器在追加相应助手块前，也必须持久提交输出。规范用户内容与助手内容只包含角色无关的 `ImageBlock` 引用。
|
||||
|
||||
第一版支持粘贴和拖放 PNG、JPEG、WebP 与 GIF，支持仅图片或混合提示词，支持渲染历史用户图片与助手图片，并支持双击预览原图。文件选择、通用文件、PDF、音频、视频、图片复制和自定义上下文菜单仍分别作为后续工作。
|
||||
|
||||
### 产品行为
|
||||
|
||||
- 粘贴或拖放一张或多张受支持的图片后，文本框上方会按顺序显示缩略图，但不会插入占位文本。文件拖入输入区时会高亮放置目标。
|
||||
- 同一个常驻 `InputBar` 会在空白会话 Hero 和活跃会话布局中渲染附件栏。附件栏为空时隐藏，通过横向滚动避免撑宽输入区。
|
||||
- 每个缩略图约为 72 × 72 像素，带有移除操作；双击时打开草稿原图。
|
||||
- 提示词可同时包含文本与图片，也可仅包含图片。粘贴纯文本时保持浏览器原生行为；粘贴混合的剪贴板内容时，文本会正常插入，文件则同时添加到附件栏；仅粘贴文件时才阻止浏览器的默认处理。在输入区放置文件时总会阻止浏览器导航，并在本地报告不受支持的文件。
|
||||
- 发送失败时恢复完整的文本与图片草稿，但不会覆盖请求飞行期间新增的文本或图片。移除、发送成功、会话 scope 释放、已渲染历史记录释放和应用释放都会撤销各自持有的对象 URL。
|
||||
- 历史用户图片与助手图片共用一个 `MessageImage` 控件。行内图片保持固有宽高比、不放大，并限制在 240 × 240 像素的边界框内。
|
||||
- 双击消息图片会在不超出视口的模态框中打开存储的原图。按 Escape、激活关闭控件或激活背景区域都会关闭模态框并恢复焦点。
|
||||
- 第一版不覆盖浏览器上下文菜单，也不提供明确的图片复制操作。
|
||||
|
||||
### 存储生命周期与归属
|
||||
|
||||
持久化边界是消息被接受，而不是图片被粘贴：
|
||||
|
||||
| 状态 | 允许的表示 | 持久性与顺序 |
|
||||
| --- | --- | --- |
|
||||
| 未发送的用户草稿 | 浏览器 `File` 加对象 URL；原生客户端可以使用 `/var/...` 等操作系统临时文件 | 临时且由客户端持有。它可能在重载或进程退出后消失，绝不出现在会话事件中。 |
|
||||
| 已接受的用户图片 | `DSH_HOME` 下的不可变对象加 `ImageAttachmentRef` | 在 `agent.send()` 或 `agent.steer()` 能够追加所属用户事件前，宿主提交每张图片。 |
|
||||
| 结构化模型图片输出 | `DSH_HOME` 下的不可变对象加 `ImageAttachmentRef` | 提供方适配器在发出已完成的图片块或助手消息事件前提交字节。事件中禁止出现临时 URL、路径和 base64。 |
|
||||
|
||||
每个会话的 `InputMachine` 状态在实时草稿旁保存仅限运行时的有序附件标识符。框架持有的 chat store 只接收草稿的纯文本持久化镜像，`ConversationService` 则持有相应的浏览器专用 `File` 与对象 URL 注册表：
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type DraftAttachmentId = Branded<'DraftAttachmentId'>
|
||||
|
||||
interface ChatStoreState {
|
||||
selection: object | null
|
||||
draft: string
|
||||
view: string | null
|
||||
}
|
||||
|
||||
interface InputState {
|
||||
draft: string
|
||||
imageIds: readonly DraftAttachmentId[]
|
||||
}
|
||||
|
||||
interface ComposerAttachment {
|
||||
kind: 'image'
|
||||
id: DraftAttachmentId
|
||||
file: File
|
||||
previewUrl: string
|
||||
}
|
||||
```
|
||||
|
||||
这一拆分把会话 provide 通道的输入 hook 与 actions 用作实时输入区状态的唯一订阅路径，同时避免把不可序列化的浏览器对象写进持久 JSON。只有纯文本草稿镜像使用 `localStorage`；附件标识符、浏览器 `File` 对象和对象 URL 都限定在实时会话输入外壳的 scope 内。未发送图片因此无法跨重载或会话 scope 释放保留。切换 Workspace 时，只有目标外壳接受完整图片批次，图文混合草稿才会移动；拒绝时，文本和图片都留在来源外壳。原生客户端可以在操作系统临时目录中暂存输入，但必须像对待浏览器对象 URL 一样对待该路径：不再需要时删除，并在消息被接受前把字节复制进持久存储。
|
||||
|
||||
本地附件后端依次解析显式 `dshHome`、`$DSH_HOME` 和 `~/.dsh`。它把内容寻址对象存储在 `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` 下，并为目录和文件设置仅所有者可访问的权限。每个进程首次为某个 home 保存对象时，都会创建该 home，并逐级同步每个祖先目录项直至文件系统根目录；不能把存在视为持久性，因为另一个进程可能仍处于 `mkdir` 与父目录 `fsync` 之间。随后，服务写入并同步临时文件，再以原子方式发布，并对发布路径执行目录同步使其持久（POSIX；Windows 依赖文件系统元数据日志），之后才返回引用。内容摘要编码在不透明的 `sha256:<digest>` 标识符中。写入准入与读取都会完整解码受支持的光栅图片，之后才接受其格式和尺寸；每次读取还会校验摘要、字节长度和已记录的元数据。
|
||||
|
||||
第一版不对存储执行自动删除。已发送的用户图片和模型生成图片会一直保留，以供历史记录、恢复和 fork 使用。按引用感知的垃圾回收需要单独设计，因为仅按时间清理可能删除仍被持久会话引用的数据。部署的字节和像素限制是写入时的准入策略；读取时会校验摘要和已记录的元数据，但不重新应用当前准入限制，因此收紧策略不会导致旧历史记录失效。
|
||||
|
||||
### 持久内容与提示词协议
|
||||
|
||||
附件服务边界公开不可变图片写入和经过校验的读取操作。规范元数据刻意比通用文件记录更窄：
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type AttachmentId = Branded<'AttachmentId'>
|
||||
|
||||
interface ImageAttachmentRef {
|
||||
attachmentId: AttachmentId
|
||||
mediaType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
|
||||
bytes: number
|
||||
width: number
|
||||
height: number
|
||||
name?: string
|
||||
}
|
||||
|
||||
interface ImageBlock {
|
||||
type: 'image'
|
||||
attachment: ImageAttachmentRef
|
||||
}
|
||||
```
|
||||
|
||||
`ImageBlock` 加入可合并扩展的核心 `ContentBlockMap`，在用户内容和助手内容中都有效。它绝不携带 base64、对象 URL、文件系统路径或提供方持有的定位符。因此，会话事件与不可变对象存储足以共同重建模型可见的确切图片。LLM 词汇由此仅在类型层面依赖附件服务边界；提供方运行时依赖仍由各适配器持有。
|
||||
|
||||
浏览器无法生成持久引用，因此 `session.prompt` 接受范围狭窄的接收联合类型，而不是规范 `ContentBlock[]`：
|
||||
|
||||
```ts
|
||||
export {}
|
||||
|
||||
type PromptInputPart =
|
||||
| { type: 'text'; text: string }
|
||||
| {
|
||||
type: 'image'
|
||||
mediaType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
|
||||
data: string
|
||||
name?: string
|
||||
}
|
||||
```
|
||||
|
||||
Base64 只跨越一次 JSON-RPC，并在持久化后丢弃。宿主会校验规范 base64、图片数量、总字节数、单张图片字节数、声明的 MIME 与完整解码后的光栅图片是否一致、固有尺寸和解码像素数。它会在保存任何成员之前，等待服务边界上不触碰存储的 `validateImage` 完成对每个批次成员的校验，因此一张畸形图片不会把批次中的有效成员留成无引用对象。随后按提交顺序执行存储提交，以限制完整光栅解码器的内存占用。如果后续存储 I/O 操作失败，宿主不会追加用户事件，但先前的不可变内容寻址对象可能保持无引用状态；第一版将清理留给未来按引用感知的垃圾回收，而不向去重存储添加破坏性回滚。只有每张图片都成功后，宿主才会用规范化文本和按提交顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。
|
||||
|
||||
`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符，宿主才提供字节。会话处于渲染状态时，客户端会按会话和附件标识符对加载操作去重；已渲染会话释放时会撤销已解析的 URL，并在分配对象 URL 前拒绝已失效的延迟加载，以免已卸载的会话或已释放的服务重新写入缓存。
|
||||
|
||||
### 模型能力与提供方行为
|
||||
|
||||
模型目录项增加可选且可合并扩展的输入模态声明。缺少声明表示未知；声明存在但不含 `image`，则明确表示不支持图片。
|
||||
|
||||
宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型，并在缺失时依次回退到 agent 选项和宿主默认值；如果该模型明确排除图片输入，宿主会在写入任何附件或事件前拒绝提示词，客户端则恢复草稿。包含图片的提示词准入与模型选择共用一个逐 agent 的串行边界，而且已经出队的提示词在其持久消息事件发布前仍保持待发布状态（[顺序决策](../bug-fix/2026-07-29-atomic-web-image-admission.md)）；steering 载体则从入队起就参与门槛，直到其 `steering/message` 事件发布为止，堵住了从不进入排队镜像的 outbox 窗口。当图片正等待发布或仍存在于会话当前的派生历史中时，模型选择会拒绝纯文本目标。压缩（compaction）可以移除旧图片，使之后选择纯文本目标变得有效；未发布任何事件即转入空闲时，已认领的 queued 载体会被释放，而保留在 outbox 中的 steering 在发布或丢弃前始终受门槛约束。`session.updateQueue` 的编辑只接受文本内容，因此队列编辑无法绕过该准入边界注入图片。能力未知时继续进入适配器强制检查，使未收录的模型标识符仍然可用。浏览器会在分配预览 URL 前拒绝声明不支持的图片媒体类型，但不会为部署限制或模型能力保留快照：握手快照无法表达 `session.selectModel` 之后会话的当前目标，部署策略也可能独立变化。宿主会根据当前的单张字节数、图片数量、总字节数、媒体类型、尺寸、像素数和路由模型策略校验整个批次，再写入任何附件或事件；其拒绝通过 composer 错误条呈现。
|
||||
|
||||
Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachments`，递归转换每个持久图片引用，包括嵌套在工具结果中的引用，并且仅为声明支持图片输入的模型生成提供方原生图片内容。交付的组合会同时注册 Pi-AI OpenAI、Anthropic 路由和仅支持文本的默认 DeepSeek 路由；选择当前提供方／模型仍由宿主组合或配置承担，而不是图片输入 CLI（命令行界面）的功能。在请求时解析服务，可避免 Cordis 加载顺序将可选附件服务的可用性固化。手写 DeepSeek 适配器遇到请求中任何位置的图片时都会抛出类型化的 `UNSUPPORTED_CONTENT` 错误，包括嵌套工具结果中的图片。任何适配器都不得将图片展平或跳过。
|
||||
|
||||
核心层支持结构化助手图片块，但当前没有任何生产提供方路径通过图片输出认证。未来任何支持输出的适配器都必须在有界的大小和时间策略下获取提供方字节，通过同一个附件服务校验并持久化字节，之后才能以原子方式发布 `ImageBlock`。助手 Markdown 中的 URL 仍是文本，绝不自动下载。
|
||||
|
||||
提供方无关的 token 估算不会根据图片尺寸猜测视觉定价；提供方返回的用量仍是权威值。在 ACP（Agent Client Protocol）接口原生支持图片前，ACP 会渲染明确的图片标记，而不是静默省略该块。
|
||||
|
||||
压缩会把选定的会话前缀（包含图片引用）回放到已配置的摘要生成路径中。支持视觉的路径会通过适配器解析这些引用；仅文本路径会明确失败，而不是静默丢弃视觉上下文。合成的检查点仍仅包含文本，`compact-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。
|
||||
|
||||
### 历史渲染与原图预览
|
||||
|
||||
历史记录折叠会在用户消息和助手消息中保留 `ImageBlock`。用户图片在文本上方靠尾端对齐；助手图片则保留在靠前端叙述流中的原内容块位置。`MessageImage` 根据记录的尺寸派生稳定的行内边界框，通过会话授权加载器解析字节，使用 `object-fit: contain`，并将对象缺失或损坏转换为可重试的错误控件。
|
||||
|
||||
输入区缩略图和每个 `MessageImage` 各自持有临时原图预览状态，并调用同一个纯 `ImageLightbox`。模态框使用已经解析的原始对象 URL，只限制显示尺寸；它会聚焦关闭控件，并在关闭时把焦点恢复到先前的目标。
|
||||
|
||||
### 限制与信任边界
|
||||
|
||||
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 5 MiB、每条消息 10 张图片和 20 MiB 图片总字节数，以及每张图片 4,000 万个固有像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限（默认 32 MiB）；如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小，加载就会失败。因此，降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。
|
||||
|
||||
格式错误的 base64、不支持或不匹配的媒体、截断的图片数据、超出字节限制、超出图片数量、超出像素限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段，控制字符会被移除，并且任何本地路径都不会写入日志或返回浏览器。
|
||||
|
||||
### 包与接口变更
|
||||
|
||||
| 接口 | 职责 |
|
||||
| --- | --- |
|
||||
| `packages/attachment/attachment` | 不透明附件标识符、图片引用、限制、错误和 `ctx.attachments` 服务。 |
|
||||
| `packages/attachment/attachment-local` | 私有内容寻址存储、完整光栅解码、完整性校验和配置。 |
|
||||
| `packages/llm/llm` | 角色无关的 `ImageBlock` 和输入模态元数据。 |
|
||||
| `packages/llm/llm-pi-ai` | 将持久且受支持的图片输入解析为提供方原生内容。 |
|
||||
| `packages/llm/llm-deepseek` | 明确拒绝图片内容。 |
|
||||
| `packages/compact/compact-basic` | 在摘要输入中保留图片，并明确拒绝非文本检查点输出。 |
|
||||
| `packages/host/apiproxy` 和 `packages/bundle/base` | 范围狭窄的上传协议、先持久化再追加事件的顺序、会话授权读取、限制和模型前置检查，以及默认 profile 组合。 |
|
||||
| `packages/client/connection` 和 `packages/client/runtime` | 有界请求缓冲、协议类型、fixture（测试前置数据）图片、提示词上传、附件读取和持久引用折叠。 |
|
||||
| `packages/client/ui-conversation` | 每个会话的草稿图片、附件栏、用户与助手图片控件和原图预览。 |
|
||||
| `packages/acp/acp` | 图片块的明确兜底渲染。 |
|
||||
|
||||
附件包（package）构成一个能力服务边界的接口与实现侧。输入区行为留在会话对象层，提供方转换留在适配器中，无需修改 `agent-loop`。
|
||||
|
||||
### 实现
|
||||
|
||||
已实现的范围包括附件服务边界、角色无关的图片块、Pi-AI 输入转换、DeepSeek 拒绝、宿主持久化顺序、Web 上传与读取协议、当前图片限制执行、大小受限的 Web 请求体、内存草稿图片、粘贴与拖放附件栏、用户与助手历史图片渲染、双击预览、压缩处理，以及组装后无需密钥的 Web 覆盖。
|
||||
|
||||
预发布提示词协议不需要兼容包装层；引入相应切片时会同时修改所有调用点和 fixture。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
### 将每张粘贴图片保留在 `/var` 或其他临时目录中
|
||||
|
||||
临时存储适合在发送前使用，也适合通过操作系统接收剪贴板文件的原生客户端。但它不适合在消息被接受后继续使用：清理不在 harness 的控制范围内，路径因宿主而异，恢复或 fork 后的会话也可能比文件存在得更久。提案允许临时暂存，但会在追加事件前将已接受的字节复制进 `DSH_HOME`。
|
||||
|
||||
### 粘贴或拖放后立即持久化
|
||||
|
||||
立即持久化可以让草稿在重载后继续存在，但会在会话或消息持有对象前就创建持久对象，因此必须定义配额、遗留对象生命周期和清理策略。第一版保持未发送草稿为临时状态，并把发送被接受作为持久性边界。
|
||||
|
||||
### 在消息与会话日志中内联 base64
|
||||
|
||||
这种方式会在 RPC、事件、历史分页、fork、压缩和浏览器存储中复制二进制数据，还会诱使 token 计量把编码文本当成模型文本。单一不可变对象配合小型引用，可以让持久表示保持有界。
|
||||
|
||||
### 使用浏览器对象 URL、本地路径或提供方 URL 作为规范内容
|
||||
|
||||
对象 URL 会随文档失效，本地路径不可移植，提供方 URL 则可能过期、跟踪查看者或暴露凭据。它们只能作为临时传输或预览细节存在。
|
||||
|
||||
### 用一个通用 `AttachmentBlock` 表示图片、文件、音频和视频
|
||||
|
||||
输入区展示可以使用通用附件栏，但提供方语义取决于具体模态。图片是原生多模态输入；PDF 可能是提供方文件或提取后的文本；视频可能由模型原生支持、抽帧处理或不受支持。特定的 `ImageBlock` 会迫使每个消费方明确处理或拒绝该模态。
|
||||
|
||||
### 依赖 UI 能力检查或静默过滤图片
|
||||
|
||||
UI 状态可能陈旧，也无法保护直接 SDK、ACP、回放或未收录模型的路径。静默过滤会改变用户意图。提供方强制检查仍是必需项，UI 检查则是可选的提前反馈。
|
||||
|
||||
## 测试
|
||||
|
||||
- 存储测试覆盖内容寻址去重、私有权限、准入失败、对象损坏或缺失时的失败，以及收紧部署限制后读取历史数据。
|
||||
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体、图片准入与模型选择的竞态（排队与 steering 两种放置）、待发布状态、未发布即空闲时的门槛释放、仅文本的队列编辑，以及压缩后依据当前派生历史进行的选择。
|
||||
- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序、草稿、会话作用域和应用层级的对象 URL 清理，以及一项在释放后才完成的延迟历史读取；keyless 的组装后构建产物通道（`apps/web/tests/image-display.snapshot.ts`，`DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`）覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox，以及 composer 粘贴缩略图条。
|
||||
- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、递归嵌套在工具结果中的图片、保留摘要输入，以及明确拒绝图片输出。
|
||||
- 需要凭据的实际 API 测试会通过 Anthropic `claude-opus-4-8` 路径发送一张 PNG，并要求模型识别其中的二维码。
|
||||
- 当前生产适配器集合没有经过认证的图片输出路由；输出提供方认证仍不在第一版范围内。
|
||||
|
||||
## 后果
|
||||
|
||||
- 持久存储会在没有垃圾回收时持续增长。第一版选择回放安全，而不是过早删除。
|
||||
- 对象缺失或损坏会让模型请求无法精确重建。明确失败可以保持完整性，但在修复前可能阻止该会话继续运行。
|
||||
- JSON-RPC base64 会增加上传内存，并带来约三分之一的编码开销。第一版的限制可以约束开销；更大的媒体需要流式传输或二进制传输协议。
|
||||
- 未发送图片无法跨重载保留。持久草稿需要配额和遗留对象清理，而不是隐式复用消息存储。
|
||||
- 原图预览解码的像素多于行内控件显示的像素。像素限制、一次只打开一个预览和对象 URL 释放可以约束但无法消除浏览器瞬时内存占用。
|
||||
- 能力元数据可能缺失或陈旧。宿主前置检查可以改善反馈，适配器强制检查仍是权威结果。
|
||||
- 未来输出提供方可能需要经过身份认证的下载，助手图片才能完成，这会增加延迟与新的故障点。先持久化再追加事件的顺序优先保障回放完整性。
|
||||
- 文件选择、通用文件与 PDF、音频与视频、持久草稿暂存、图片复制、自定义上下文菜单、输出提供方认证和按引用感知的垃圾回收仍是相互独立的设计。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-24-web-session-model-selector.md
|
||||
2026-07-24-web-session-model-selector.md: 78017b14a806f8e609a85e340094cd2a349d47e1
|
||||
2026-07-24-web-session-model-selector.zh.md: 87a342721f5e47f5bcedfafb578eb6916101d2f5
|
||||
2026-07-24-web-session-model-selector.md: e6a96ac62f69a3bd312f61cc920caa259d2dc5b0
|
||||
2026-07-24-web-session-model-selector.zh.md: 5a0245359d69ac6e59a20dc3276b9411c4b25e23
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-07-24-web-session-model-selector.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The Web conversation needs a visible, mutable session model selection sourced from the Host. Copying TUI presentation or hardcoding DeepSeek models in the browser would split model discovery and step-boundary semantics across front doors. A switch made while a response is running also needs one atomic boundary: prompt variables and request routing cannot observe different selections.
|
||||
The Web conversation needs a visible, mutable session model selection sourced from the Host. Copying TUI presentation or hardcoding DeepSeek models in the browser would split model discovery and step-boundary semantics across front ends. A switch made while a response is running also needs one atomic boundary: prompt variables and request routing cannot observe different selections.
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
@@ -6,13 +6,13 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
Web 对话需要一项由 Host 提供、可见且可更改的会话模型选择。如果照搬 TUI 的呈现方式，或在浏览器中硬编码 DeepSeek 模型，就会让模型发现逻辑和步骤边界语义分散到不同前门中。响应运行期间发生的切换还需要一个原子边界：提示词变量与请求路由不能观测到不同的选择。
|
||||
Web 对话需要一项由 Host 提供、可见且可更改的会话模型选择。如果照搬 TUI 的呈现方式，或在浏览器中硬编码 DeepSeek 模型，就会让模型发现逻辑和步骤边界语义分散到不同前端中。响应运行期间发生的切换还需要一个原子边界：提示词变量与请求路由不能观测到不同的选择。
|
||||
|
||||
## 决策
|
||||
|
||||
Web Host 为每个新建或恢复的 Agent 安装 `ModelSelection`。如果会话已经使用过模型，提供方／模型／推理（reasoning）选择来自最新的 `request/header`；否则来自 `ctx.agentDefaultModel`。`session.selectModel` 会赋值会话级选择，提示词组装则将它与请求路由一并捕获，因此运行中步骤发生的切换会应用于下一个组装步骤。下一个实际采用的选择通过完整的 `request/header` 快照持久化；尚未进入请求的选择则仅保存在当前进程中。
|
||||
|
||||
会话 RPC 领域公开 `session.models` 模型目录与 `session.selectModel`。该目录从 LLM（大语言模型）注册表动态构建，并按提供方分组；每个已列出模型的精确元数据还会加入由适配器持有的推理强度 ID、名称、说明和可选默认值。各提供方的目录与精确元数据会按提供方并发加载，且彼此独立失败，因此成功加载的分组仍可与可重试的失败记录一同使用。模型是否位于目录仅供参考：`session.models.current` 独立返回，即使不在任何分组中也仍然可以路由，但提供方停止公布该模型后，Host 不会合成未列出行。两个前门对这一状态给出不同回答：TUI 把未列出的当前模型渲染为独立一行，Web 则显示未设置状态的触发器标签并要求选择替代模型。Web 是编辑目录的 surface，因此缺席的目录行代表一项待作出的选择；TUI 只从现有行中选择。显示未设置标签的 Web composer 仍可以使用当前可路由选择发送消息。精确解析决定提供方／模型组合与显式推理强度是否可用。选择操作通过 `resolveCallConfig` 拒绝不支持的推理强度 ID，并在赋值该选择前具体化适配器配置的默认值。
|
||||
会话 RPC 领域公开 `session.models` 模型目录与 `session.selectModel`。该目录从 LLM（大语言模型）注册表动态构建，并按提供方分组；每个已列出模型的精确元数据还会加入由适配器持有的推理强度 ID、名称、说明和可选默认值。各提供方的目录与精确元数据会按提供方并发加载，且彼此独立失败，因此成功加载的分组仍可与可重试的失败记录一同使用。模型是否位于目录仅供参考：`session.models.current` 独立返回，即使不在任何分组中也仍然可以路由，但提供方停止公布该模型后，Host 不会合成未列出行。两个前端对这一状态给出不同回答：TUI 把未列出的当前模型渲染为独立一行，Web 则显示未设置状态的触发器标签并要求选择替代模型。Web 是编辑目录所在的前端，因此缺席的目录行代表一项待作出的选择；TUI 只从现有行中选择。显示未设置标签的 Web composer 仍可以使用当前可路由选择发送消息。精确解析决定提供方／模型组合与显式推理强度是否可用。选择操作通过 `resolveCallConfig` 拒绝不支持的推理强度 ID，并在赋值该选择前具体化适配器配置的默认值。
|
||||
|
||||
浏览器中的 `ModelService` 为每个实时会话持有一个 `ModelDirectory`。其快照包含当前完整的 `ModelSelection`、分组目录、提供方失败记录、操作错误，以及 `idle`、`loading`、`ready`、`selecting`、`error` 状态。挂载时会预先填充触发器标签，此后每次打开菜单都会刷新目录。目录与选择调用共用操作代次，防止较早响应覆盖较新结果；连接重置会先丢弃当前进程中的投影，再恢复 Host 选择。失败时保留先前的选择和可用分组。
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-06-web-queue-steer-all-gesture.md
|
||||
2026-08-06-web-queue-steer-all-gesture.md: e546f68647dfc9b91ce4699cef4a64694ebc4f76
|
||||
2026-08-06-web-queue-steer-all-gesture.zh.md: fb36852f66a86408af12df59001230b2024eccaf
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: Steer the whole Web queue with an empty-draft Cmd/Ctrl+Enter
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-06-web-queue-steer-all-gesture.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
While a primary session runs, the Web queue accumulates messages the user typed with plain Enter (or queued while the busy-Enter preference was Queue). Flushing them into the current turn required clicking the per-row 插话发送 button once per message; an empty composer draft had no keyboard gesture at all — the input machine rejects empty drafts, so Enter and Cmd/Ctrl+Enter were both no-ops. With several queued messages, steering them one by one is the obvious multi-click friction, and the empty-draft accelerated chord is the natural slot for "steer everything".
|
||||
|
||||
## Decision
|
||||
|
||||
Empty-draft Cmd/Ctrl+Enter now steers every still-pending `queued`-placement inbox row into the running turn, in FIFO order, on a primary session that reports running. The gesture decodes in `InputBar.onKeyDown`: accelerated Enter with a trimmed-empty draft, `running`, no subagent address, and at least one `queued` row calls the new `ComposerKeyboard.steerQueue()` verb instead of `submit()`. `SessionInputShell.steerQueue()` delegates to a hub-wired choreography that re-reads the authoritative `session/queue` snapshot, filters `placement: 'queued'` (pending steering rows are already in the turn), and applies the queue dock's exact strict-steer operation — `session.updateQueue(itemId, { kind: 'steer' })` — sequentially, so FIFO ordering is guaranteed at the host. A `steer-unavailable` (turn closed mid-flush) or `queue-item-not-found` (row claimed meanwhile) converges silently; any other failure surfaces one composer notice (`插话发送失败，请重试。`). No wire, on-disk, or agent-loop change: the host already owns the strict-steer boundary.
|
||||
|
||||
The gesture is strictly the accelerated chord. Plain Enter with an empty draft stays a no-op even under the busy-Enter Steer preference, draft content outranks the queue (accelerated Enter steers only the draft), and idle or subagent sessions keep the existing empty-draft no-op because steering has no live turn to enter.
|
||||
|
||||
The same computed availability gate drives discovery: while the draft is empty, the input is unlocked and not in a transient machine lock, the command menu is closed, an ordinary primary session is running, and at least one row remains `queued`, the textarea placeholder advertises that Cmd/Ctrl+Enter steers all queued messages. An owner-supplied placeholder still takes precedence, and the steer hint deliberately outranks the plan-mode placeholder while available (the gesture genuinely works in that window).
|
||||
|
||||
## Consequences
|
||||
|
||||
One keyboard gesture now replaces N clicks while keeping a single strict-steer path and a single authority for convergence. The per-row button and the gesture are the same host operation, so races and failure semantics stay identical. The gesture and its placeholder share one presentation-layer gate, while the hub re-checks the snapshot at execution time, so the client gate remains advisory and the host remains authoritative.
|
||||
|
||||
## Related
|
||||
|
||||
The per-row 插话发送 action and its strict-steer boundary are owned by [Steer a queued Web message into the active turn](../feature/2026-07-30-web-queue-steer-action.md); this note only adds the whole-queue keyboard gesture on top of that decision.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Intercepting inside the input machine.** Rejected: the machine is queue-agnostic by design (the wiring layer overlays the queue projection) and cannot distinguish the accelerated chord from plain Enter, which must stay a no-op.
|
||||
- **Steering via `session.prompt(mode: 'steer')` per row.** Rejected: that mints new messages instead of transferring the pending occurrences and would split the dock's immutable-message contract; `updateQueue({ kind: 'steer' })` already atomically transfers the exact occurrence.
|
||||
- **Firing all row steers concurrently.** Rejected: arrival order at the host is not guaranteed, and steering order is model-visible; sequential awaits preserve FIFO.
|
||||
- **A new host RPC for steer-all.** Rejected: the existing per-item operation is idempotent enough — each row is one strict steer, and mid-flush closure converges silently — so a protocol change buys nothing.
|
||||
- **A send-button tooltip.** Rejected: the primary button is Stop while an ordinary session is running, which is the only window where the whole-queue gesture is available. The empty-draft placeholder occupies that exact window and can describe the keyboard action directly.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: 空输入时 Cmd/Ctrl+Enter 将 Web 排队消息全部插话
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-06-web-queue-steer-all-gesture.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
主会话运行时，用户用普通 Enter（或在 busy-Enter 偏好为 Queue 时）输入的消息会在 Web 队列里累积。把它们灌进当前轮次需要逐条点击「插话发送」按钮；而输入框草稿为空时没有任何键盘手势——输入机对空草稿直接拒绝，Enter 与 Cmd/Ctrl+Enter 都是空操作。排队消息一多，逐条插话是明显的多点摩擦，空草稿 + 加速 Enter 正是「全部插话」的自然位置。
|
||||
|
||||
## Decision
|
||||
|
||||
空草稿的 Cmd/Ctrl+Enter 现在会把仍在排队（`placement: 'queued'`）的 Inbox 行按 FIFO 顺序全部插话进运行中的轮次，仅限报告 running 的主会话。手势在 `InputBar.onKeyDown` 解码：加速 Enter + 去空白后为空草稿 + `running` + 无 subagent 地址 + 至少一条 `queued` 行时，改走新的 `ComposerKeyboard.steerQueue()` 动词而不是 `submit()`。`SessionInputShell.steerQueue()` 委托给 hub 编排的流程：重新读取权威的 `session/queue` 快照，过滤 `placement: 'queued'`（pending steering 行已经在本轮内），并逐条顺序执行 Queue 面板的严格 steer 操作 `session.updateQueue(itemId, { kind: 'steer' })`，从而在 host 侧保证 FIFO 顺序。`steer-unavailable`（flush 中途轮次关闭）或 `queue-item-not-found`（行已被占用）静默收敛；其他失败弹出一条 composer 通知（「插话发送失败，请重试。」）。无 wire、磁盘或 agent-loop 改动：严格 steer 边界本来就在 host 侧。
|
||||
|
||||
该手势严格限定为加速组合键。空草稿 + 普通 Enter 仍然无操作（即使 busy-Enter 偏好为 Steer）；草稿内容优先于队列（加速 Enter 只插话当前草稿）；idle 或 subagent 会话保持原有空草稿无操作，因为没有可插入的运行中轮次。
|
||||
|
||||
同一套计算得出的可用性门控也负责提示该手势：当草稿为空、输入框未锁定且不处于瞬态机器锁（adjudicating/submitting）、命令菜单未打开、普通主会话正在运行且至少一行仍为 `queued` 时，文本框 placeholder 会提示 Cmd/Ctrl+Enter 将全部排队消息插话发送。owner 提供的 placeholder 仍然优先；可用时 steer 提示会刻意优先于 plan 模式 placeholder（该窗口内手势确实可用）。
|
||||
|
||||
## Consequences
|
||||
|
||||
一个键盘手势替代 N 次点击，同时保持单一严格 steer 路径与单一收敛权威。逐条按钮与手势是同一个 host 操作，竞态与失败语义完全一致。手势及其 placeholder 共用一个呈现层门控；hub 在执行时会重新读取快照，因此客户端门控仍只是建议性的，host 仍是权威。
|
||||
|
||||
## Related
|
||||
|
||||
逐条「插话发送」动作及其严格 steer 边界由 [Steer a queued Web message into the active turn](../feature/2026-07-30-web-queue-steer-action.md) 记录；本笔记只在其之上增加整队列键盘手势。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **在输入机内拦截。** 已拒绝：输入机按设计不感知队列（队列投影由 wiring 层叠加），且无法区分加速 Enter 与必须保持空操作的普通 Enter。
|
||||
- **逐条用 `session.prompt(mode: 'steer')` 插话。** 已拒绝：那会铸造新消息而不是转移 pending 行，破坏 dock 的不可变消息契约；`updateQueue({ kind: 'steer' })` 已经原子地转移了确切的那条。
|
||||
- **并发触发所有行。** 已拒绝：host 到达顺序无法保证，而插话顺序对模型可见；顺序 await 保证 FIFO。
|
||||
- **为 steer-all 新增 host RPC。** 已拒绝：现有逐条操作已足够幂等——每行一次严格 steer，中途关闭静默收敛——协议改动没有收益。
|
||||
- **发送按钮 tooltip。** 已拒绝：普通会话运行时，主按钮是 Stop，这也是整队列手势唯一可用的窗口。空草稿时的 placeholder 恰好在该窗口显示，可以直接说明这项键盘操作。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-07-default-model-follows-the-picker.md
|
||||
2026-08-07-default-model-follows-the-picker.md: ed7e7a424d2cacadea890506fd9150ffdf7a993c
|
||||
2026-08-07-default-model-follows-the-picker.zh.md: 523c6f917dedf21c726ce5631226ba545126c757
|
||||
2026-08-07-default-model-follows-the-picker.md: 2a3ada55486345c0f58f0767bed5a93ecba04b88
|
||||
2026-08-07-default-model-follows-the-picker.zh.md: 08fecc6ec9b177f6424ada3172c67018ac72baea
|
||||
|
||||
@@ -6,13 +6,13 @@ English | [中文](2026-08-07-default-model-follows-the-picker.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
A session model picker and a deployment default are two layers of the same preference. If the picker affects only its addressed session, the next blank session can select a different model with no user-facing way to align the default. If the default lives inside a Host gateway, direct Agent front doors cannot share it without depending on Host or duplicating state.
|
||||
A session model picker and a deployment default are two layers of the same preference. If the picker affects only its addressed session, the next blank session can select a different model with no user-facing way to align the default. If the default lives inside a Host gateway, direct Agent entry points cannot share it without depending on Host or duplicating state.
|
||||
|
||||
Reasoning effort makes the persistence shape significant: a model selection without an effort must clear a stored effort, or the next Agent may apply an effort that its selected model does not accept.
|
||||
|
||||
## Decision
|
||||
|
||||
`AgentDefaultModelService` provides `ctx.agentDefaultModel` and registers `{provider, model, reasoningEffort?}` as the `agent-default-model` Settings section. Its `{provider, model}` composition entry is the base layer and `settings.yaml` supplies the user layer. The service is front-door-neutral, so direct creation and ApiProxy-backed creation share one default ([headless direct core front door](../architecture/2026-08-09-headless-direct-core-front-door.md)). `workspaceRoot` remains ApiProxy config because it is a Host launcher fact rather than model state.
|
||||
`AgentDefaultModelService` provides `ctx.agentDefaultModel` and registers `{provider, model, reasoningEffort?}` as the `agent-default-model` Settings section. Its `{provider, model}` composition entry is the base layer and `settings.yaml` supplies the user layer. The service is entry-point-neutral, so direct creation and ApiProxy-backed creation share one default ([headless direct core entry point](../architecture/2026-08-09-headless-direct-core-entry-point.md)). `workspaceRoot` remains ApiProxy config because it is a Host launcher fact rather than model state.
|
||||
|
||||
`reasoningEffort` belongs to the Settings section but not to the plugin config. Settings layers merge by field, so a configured effort would survive a user selection that omits it. `saveSelection()` instead writes the complete user section; absence therefore clears a stored effort. A deployment-wide effort default belongs to the adapter profile, which resolves it per model.
|
||||
|
||||
|
||||
@@ -6,13 +6,13 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
会话模型选择器与部署默认值是同一项偏好的两个层次。如果选择器只影响其所在会话，下一个空白会话可能选择不同模型，用户却没有途径使默认值与选择器一致。如果默认值位于 Host 网关内部，直接创建 Agent 的前门只有依赖 Host 或复制状态才能共享它。
|
||||
会话模型选择器与部署默认值是同一项偏好的两个层次。如果选择器只影响其所在会话，下一个空白会话可能选择不同模型，用户却没有途径使默认值与选择器一致。如果默认值位于 Host 网关内部，直接创建 Agent 的入口只有依赖 Host 或复制状态才能共享它。
|
||||
|
||||
推理强度使持久化形态成为约定的一部分：不含强度的模型选择必须清除已存强度，否则下一个 Agent 可能会采用所选模型不接受的强度。
|
||||
|
||||
## 决定
|
||||
|
||||
`AgentDefaultModelService` 提供 `ctx.agentDefaultModel`，并把 `{provider, model, reasoningEffort?}` 注册为 `agent-default-model` Settings 分节。其 `{provider, model}` 组合条目是 base 层，`settings.yaml` 提供用户层。该服务不偏向特定前门，因此直接创建与 ApiProxy 支撑的创建共享同一个默认值（[headless 直接 core 前门](../architecture/2026-08-09-headless-direct-core-front-door.md)）。`workspaceRoot` 仍是 ApiProxy 配置，因为它是 Host 启动器事实，而不是模型状态。
|
||||
`AgentDefaultModelService` 提供 `ctx.agentDefaultModel`，并把 `{provider, model, reasoningEffort?}` 注册为 `agent-default-model` Settings 分节。其 `{provider, model}` 组合条目是 base 层，`settings.yaml` 提供用户层。该服务不偏向特定入口，因此直接创建与 ApiProxy 支撑的创建共享同一个默认值（[headless 直接 core 入口](../architecture/2026-08-09-headless-direct-core-entry-point.md)）。`workspaceRoot` 仍是 ApiProxy 配置，因为它是 Host 启动器事实，而不是模型状态。
|
||||
|
||||
`reasoningEffort` 属于 Settings 分节，但不属于插件配置。Settings 层按字段合并，因此已配置的强度会在用户选择省略它时继续存在。`saveSelection()` 写入完整的用户分节；缺席值由此清除已存强度。部署级强度默认值属于适配器 profile，并由它按模型解析。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-08-dsh-run-headless-command.md
|
||||
2026-08-08-dsh-run-headless-command.md: 566eeb5b2a09a0d07d72a68e4a5f449d2822e708
|
||||
2026-08-08-dsh-run-headless-command.zh.md: 177410e783a37026940829da5f81343ddc61cb29
|
||||
2026-08-08-dsh-run-headless-command.md: ed095f4077a23e51bffb647d24eed19ba09e11ed
|
||||
2026-08-08-dsh-run-headless-command.zh.md: 89d54e35573f14786e05d648f2b42891ca27a043
|
||||
|
||||
@@ -22,7 +22,7 @@ dsh run [--profile <name>] [--patch <path>...] <task...>
|
||||
|
||||
`RunInvocation` is a distinct `DshInvocation` member. The generic profile invocation carries no task state and accepts no positional arguments. Both dispatch paths use `runProfile`: profile boot omits `task`, while `run` supplies it. A one-shot profile without `headless-runner` fails through the composed-row check, and profile boot containing that row without a task points to `dsh run --profile <name> "<task>"`.
|
||||
|
||||
The [profile plugin bundle decision](../architecture/2026-08-05-profile-plugin-bundles.md) owns composition. [Headless is a direct core front door](../architecture/2026-08-09-headless-direct-core-front-door.md) owns the execution contract: one fresh persisted Session, final assistant text on stdout, completed/non-completed exit mapping, empty stderr on success, no listening port, and bounded signal shutdown after Agent quiescence and Session flush.
|
||||
The [profile plugin bundle decision](../architecture/2026-08-05-profile-plugin-bundles.md) owns composition. [Headless is a direct core entry point](../architecture/2026-08-09-headless-direct-core-entry-point.md) owns the execution contract: one fresh persisted Session, final assistant text on stdout, completed/non-completed exit mapping, empty stderr on success, no listening port, and bounded signal shutdown after Agent quiescence and Session flush.
|
||||
|
||||
The `run` verb belongs only to one-shot task execution. Application-file launch requires a distinct command name.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ dsh run [--profile <name>] [--patch <path>...] <task...>
|
||||
|
||||
`RunInvocation` 是单独的 `DshInvocation` 成员。通用 profile 调用不携带任务状态，也不接受位置参数。两条分派路径都使用 `runProfile`：profile 启动省略 `task`，而 `run` 提供该字段。缺少 `headless-runner` 的一次性 profile 会触发组合行检查；如果启动的 profile 包含该行却未提供任务，错误会指向 `dsh run --profile <name> "<task>"`。
|
||||
|
||||
[profile 插件组合包决策](../architecture/2026-08-05-profile-plugin-bundles.md)负责组合。[Headless 是直接 core 前门](../architecture/2026-08-09-headless-direct-core-front-door.md)负责执行约定：一个新的持久化会话、stdout 上的最终 assistant 文本、completed／非 completed 的退出状态映射、成功时为空的 stderr、无监听端口，以及 Agent 完全停稳且会话 flush 后的有界信号关闭。
|
||||
[profile 插件组合包决策](../architecture/2026-08-05-profile-plugin-bundles.md)负责组合。[Headless 是直接 core 入口](../architecture/2026-08-09-headless-direct-core-entry-point.md)负责执行约定：一个新的持久化会话、stdout 上的最终 assistant 文本、completed／非 completed 的退出状态映射、成功时为空的 stderr、无监听端口，以及 Agent 完全停稳且会话 flush 后的有界信号关闭。
|
||||
|
||||
`run` 动词只负责一次性任务执行。应用文件启动需要不同的命令名。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-08-user-explicit-skill-invocation.md
|
||||
2026-08-08-user-explicit-skill-invocation.md: 74d9f01f191005db6d3d283a3c56f5ee664447f8
|
||||
2026-08-08-user-explicit-skill-invocation.zh.md: 0f7c9e1261dda796d988c17ddf7199b74be4133e
|
||||
2026-08-08-user-explicit-skill-invocation.md: a7c2c15703af318cb4112f2d3dfda698bc5e3bc2
|
||||
2026-08-08-user-explicit-skill-invocation.zh.md: a8d685e5eb12766bebddebb7f5993579cf08531e
|
||||
|
||||
@@ -10,14 +10,14 @@ A `disable-model-invocation: true` skill is user-only by design: it never enters
|
||||
|
||||
## Decision
|
||||
|
||||
User-explicit invocation is a host-side pre-step injection, uniform for every user-invocable skill and every front end:
|
||||
User-explicit invocation is a host-side pre-step injection, uniform for every user-invocable skill and every entry point:
|
||||
|
||||
- `dsh-tool-skill` registers a second `agent/pre-step` listener (beside its catalog listener, the same seam `workspace-instructions` and the runtime-context snapshot ride): it scans the step's claimed messages for whitespace-bounded `/name` tokens — anywhere in the text, the same word-boundary shape the transcript chip decoration uses — collects first-seen-deduplicated names, loads each through `ctx.skills.get`, checks `isUserInvocable` on the loaded definition (the single lookup that produces what is injected), renders it with the shared `renderSkillContent`, and appends the injections after every other injection of the step: background first (workspace rules, runtime policy, catalog), the material the model must act on last, closest to its answer. Registration order pins the placement — the gesture listener registers before the catalog listener, so the waterfall hands it the catalog-bearing list to extend.
|
||||
- Precision is closed-set matching, exactly like slash commands: `/goal` resolves against the command registry, `/name` against the workspace's user-invocable skill directory; a miss stays ordinary prose, so nothing is ever guessed. Only `source.kind === 'user'` messages are scanned — external text cannot forge a gesture. Paths (`/usr/bin`), fractions (`5/8`), and prefixed tokens (`foo/name`) all break the boundary.
|
||||
- The client keeps the [plain-text-reference decision](../architecture/2026-07-25-web-input-machine-and-slash-pipeline.md): a menu pick lands the literal `/name ` and the prompt ships it verbatim; ui-skill implements no adjudication hooks and no reference codec. `skill.list` (now the domain's only RPC) serves every user-invocable skill with `modelInvocable` so menus mark user-only entries. A name shared with a host command resolves to the command — adjudication claims the line client-side before it becomes a prompt.
|
||||
- The injection is a `user`-role message carrying the `skill-invocation` source (`{ name, form: 'instructions' }`), so `user/message` logging, the context-injection transcript row (labelled with the skill name), and replay all come free; `renderSkillContent` lives in the `dsh-skill` seam, shared verbatim with the `skill` tool result, and the catalog's closing sentence tells the model to follow an injected block instead of re-loading it.
|
||||
|
||||
Peer-product survey (Pi, OpenCode, Claude Code, Kimi Code, Codex, DeepSeek-Reasonix — local checkouts) was unanimous that user-explicit triggering is programmatic injection with zero model participation; the final shape is closest to Codex's core-side `$name` mention scanning, which likewise frees every front end from implementing recognition.
|
||||
Peer-product survey (Pi, OpenCode, Claude Code, Kimi Code, Codex, DeepSeek-Reasonix — local checkouts) was unanimous that user-explicit triggering is programmatic injection with zero model participation; the final shape is closest to Codex's core-side `$name` mention scanning, which likewise frees every entry point from implementing recognition.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -10,14 +10,14 @@ Status: implemented
|
||||
|
||||
## 决策
|
||||
|
||||
用户显式调用是一次宿主侧的 pre-step 注入，对每一个用户可调用的 skill 和每一种前端一致：
|
||||
用户显式调用是一次宿主侧的 pre-step 注入，对每一个用户可调用的 skill 和每一种运行入口一致：
|
||||
|
||||
- `dsh-tool-skill` 注册第二个 `agent/pre-step` 监听器（与其目录监听器并列，也是 `workspace-instructions` 与运行时上下文快照搭乘的同一 seam）：它在该步骤已认领的消息中扫描以空白为界的 `/name` token——文本中任意位置均可，与 transcript（文本记录）chip 装饰所用的词边界形状相同——收集按首见去重的名称，逐个经 `ctx.skills.get` 加载，在已加载定义上检查 `isUserInvocable`（产生注入内容的正是这同一次查找），用共享的 `renderSkillContent` 渲染，并把注入追加在该步骤所有其他注入之后：背景在前（工作区规则、运行时策略、目录），模型必须着手处理的材料在最后、最贴近它的回答。注册顺序钉住了这一位置——手势监听器先于目录监听器注册，因此 waterfall（瀑布式事件）会把携带目录的列表交给它来扩展。
|
||||
- 精确性来自封闭集合匹配，与斜杠命令完全一致：`/goal` 对照命令注册表解析，`/name` 对照工作区的用户可调用 skill 目录解析；未命中即保持为普通行文，因此绝不猜测。只扫描 `source.kind === 'user'` 的消息——外部文本无法伪造手势。路径（`/usr/bin`）、分数（`5/8`）与带前缀的 token（`foo/name`）都会破坏该边界。
|
||||
- 客户端沿用[纯文本引用决策](../architecture/2026-07-25-web-input-machine-and-slash-pipeline.md)：菜单 pick 落下字面文本 `/name `，该文本随提示词原样提交；ui-skill 不实现任何裁决钩子，也没有引用 codec。`skill.list`（现在是该领域唯一的 RPC）提供每一个用户可调用的 skill 并携带 `modelInvocable`，供菜单标出仅限用户的条目。与宿主命令同名的名称解析为命令——客户端会在该行成为提示词之前完成裁决并将其认领。
|
||||
- 注入是一条携带 `skill-invocation` 来源（`{ name, form: 'instructions' }`）的 `user` 角色消息，因此 `user/message` 落账、上下文注入的 transcript 行（以 skill 名称标注）与回放全部免费获得；`renderSkillContent` 位于 `dsh-skill` seam，由注入和 `skill` 工具结果共用，二者内容逐字相同，目录的结尾一句会告诉模型遵循注入块而不是重新加载。
|
||||
|
||||
同类产品调研（Pi、OpenCode、Claude Code、Kimi Code、Codex、DeepSeek-Reasonix——本地检出）一致表明：用户显式触发都是模型零参与的程序化注入；最终形态最接近 Codex 核心侧的 `$name` mention 扫描——它同样让每一种前端免于自行实现识别。
|
||||
同类产品调研（Pi、OpenCode、Claude Code、Kimi Code、Codex、DeepSeek-Reasonix——本地检出）一致表明：用户显式触发都是模型零参与的程序化注入；最终形态最接近 Codex 核心侧的 `$name` mention 扫描——它同样让每一种运行入口免于自行实现识别。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
|
||||
Reference in New Issue
Block a user