docs: reserve seam for complete capabilities
This commit is contained in:
Turtle
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-06-22-fork-child-replay-seed-boundary.md
|
||||
2026-06-22-fork-child-replay-seed-boundary.md: 27db3b768f2503121b0bc8912f9bc0fecb00a10d
|
||||
2026-06-22-fork-child-replay-seed-boundary.zh.md: 9e3f7e59394a5160f5ca5db1c9798b2e05f9d2dc
|
||||
2026-06-22-fork-child-replay-seed-boundary.md: bcbbd279771a2ebe7eee643958ecf09cbd7ca7be
|
||||
2026-06-22-fork-child-replay-seed-boundary.zh.md: ad086880d77a842be8c5dc7378018c92af95e698
|
||||
|
||||
@@ -39,7 +39,7 @@ This closes the routing correctness gap, and two recorded fork scenarios exercis
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Derive the boundary heuristically in `llm-replay`** (the seeded prefix is contiguous parent events ending at the last `turn/end` before the child's first `user/message`). Rejected: a brittle heuristic in the test harness that re-derives a fact the producer already knows. Persisting the boundary at its source (the fork backend) is the "explicit > implicit at package seams" rule applied across the persistence boundary — the reader of a child fixture never has to reconstruct where the inheritance ended.
|
||||
- **Derive the boundary heuristically in `llm-replay`** (the seeded prefix is contiguous parent events ending at the last `turn/end` before the child's first `user/message`). Rejected: a brittle heuristic in the test harness that re-derives a fact the producer already knows. Persisting the boundary at its source (the fork backend) is the "explicit > implicit at package boundaries" rule applied across the persistence boundary — the reader of a child fixture never has to reconstruct where the inheritance ended.
|
||||
- **Pin the format version instead of bumping** (the `SESSION_FORMAT_VERSION = 0` "unstable" stance the event log uses). Rejected for the SQLite *table* layout: `SCHEMA_VERSION` is the monotonic bump-and-reject knob (a small enumerable set of revisions worth telling apart), distinct from the event-vocabulary `version`. Adding a column is precisely the breaking table change it versions, so it bumps.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -39,7 +39,7 @@ subagent 脚本由 [`deriveReplayScript`](../../../../packages/support/llm-repla
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **在 `llm-replay` 中启发式推导边界**（播种前缀是连续的父事件，止于子会话第一条 `user/message` 之前的最后一个 `turn/end`）。否决：在测试 harness 中用脆弱的启发式重新推导一个生产者已经知道的事实。在源头（fork 后端）持久化边界，是「在包 seam 处显式优于隐式」这条规则跨越持久化边界的应用——子会话 fixture（测试前置数据）的读取者永远不需要重建继承在哪里结束。
|
||||
- **在 `llm-replay` 中启发式推导边界**（播种前缀是连续的父事件，止于子会话第一条 `user/message` 之前的最后一个 `turn/end`）。否决：在测试 harness 中用脆弱的启发式重新推导一个生产者已经知道的事实。在源头（fork 后端）持久化边界，是「在包边界处显式优于隐式」这条规则跨越持久化边界的应用——子会话 fixture（测试前置数据）的读取者永远不需要重建继承在哪里结束。
|
||||
- **固定格式版本而不递增**（事件日志使用的 `SESSION_FORMAT_VERSION = 0`「不稳定」姿态）。对 SQLite *表*布局否决：`SCHEMA_VERSION` 是单调递增并拒绝旧版的旋钮（一组小的、值得区分的修订），与事件词汇表的 `version` 不同。新增列正是它所版本化的那种破坏性表变更，因此需要递增。
|
||||
|
||||
## 后果
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md
|
||||
2026-06-22-subagent-snapshot-replay.md: b8fefce5ff27b0cd3cfa2920b137e78cda0d696d
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: e8bd8917a02e02f2ce89105eb64794876653574d
|
||||
2026-06-22-subagent-snapshot-replay.md: 45034c29f608eb5b58f44b80438b801b6224cb82
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: 1b90c1bda854b7cd8cd9eae1e8c9f34328477f15
|
||||
|
||||
@@ -53,6 +53,6 @@ Both replay keyless in the default gate.
|
||||
## Consequences
|
||||
|
||||
- The `TODO(subagent-snapshots)` deferral is resolved: nested-agent transcripts are now a first-class snapshot shape.
|
||||
- `GenerateOptions.sessionId` is a small, honest core-seam addition useful beyond replay (telemetry, request routing).
|
||||
- `GenerateOptions.sessionId` is a small, honest core API addition useful beyond replay (telemetry, request routing).
|
||||
- The `subagent` tool is bound to a single provider, so both children in `subagent-multi` are spawn (fresh). The keying routes by session, not by backend, so it is already correct for fork. The script *derivation* was not: a fork child's log begins with the seeded parent prefix (the parent's `assistant/chunk` events), so deriving its script from the whole log would replay the parent's responses as the child's. That correctness gap is closed by persisting a seed boundary — see [Persist the seed boundary so fork-child replay routes correctly](2026-06-22-fork-child-replay-seed-boundary.md) — and recorded fork + mixed spawn+fork scenarios now exercise both transports through one transcript (see [Record fork and mixed spawn+fork snapshot scenarios](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md)).
|
||||
- Out-of-process (ACP) subagents are a different replay shape entirely (each child is its own PROCESS with its own replay), tracked as `TODO(acp-subagent-replay)` in the PR3 plan.
|
||||
|
||||
@@ -53,6 +53,6 @@ Status: implemented
|
||||
## 后果
|
||||
|
||||
- `TODO(subagent-snapshots)` 延期项已解决：嵌套 agent 的 transcript 现在是快照层的一等形态。
|
||||
- `GenerateOptions.sessionId` 是一个小而诚实的 core-seam 新增，在回放之外同样有用（遥测、请求路由）。
|
||||
- `GenerateOptions.sessionId` 是一个小而诚实的 core API 新增，在回放之外同样有用（遥测、请求路由）。
|
||||
- `subagent` 工具绑定到单一提供方，因此 `subagent-multi` 中的两个子 agent 都是 spawn（全新创建）。键控按会话路由而非按后端路由，因此对 fork 同样正确。但脚本*派生*逻辑此前不正确：fork 子会话的日志以种子化的父前缀（父会话的 `assistant/chunk` 事件）开头，如果从完整日志派生脚本，就会把父 agent 的响应当作子 agent 的来回放。这一正确性缺口通过持久化种子边界来弥合——见[持久化 seed 边界以确保 fork 子会话回放正确路由](2026-06-22-fork-child-replay-seed-boundary.md)——录制的 fork 与混合 spawn+fork 场景现在通过一份 transcript 同时验证两种传输方式（见[记录 fork 与混合 spawn+fork 快照场景](../../archived/testing/2026-06-22-fork-snapshot-scenarios.md)）。
|
||||
- 进程外（ACP（Agent Client Protocol））subagent 是完全不同的回放形态（每个子 agent 是自己的进程、有自己的回放），作为 `TODO(acp-subagent-replay)` 记录在 PR3 计划中。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-22-cross-platform-test-fixtures.md
|
||||
2026-07-22-cross-platform-test-fixtures.md: 6217aabfdbe8f14f869004c8dafb7e19f4b7443a
|
||||
2026-07-22-cross-platform-test-fixtures.zh.md: bb6649a19766967e0c2f70272433c1fa2e93f72a
|
||||
2026-07-22-cross-platform-test-fixtures.md: 9114481543d6cae1661cbed70868eddb2faa09fc
|
||||
2026-07-22-cross-platform-test-fixtures.zh.md: 710ec5887f02b5c3c71d69f16ab726323501ac92
|
||||
|
||||
@@ -30,4 +30,4 @@ Tests for a genuinely POSIX-only primitive use a narrow Windows exclusion on tha
|
||||
|
||||
## Consequences
|
||||
|
||||
Portable fixtures are slightly more explicit because expected paths derive from shared native constants and transport failures enter through a narrow writer seam. Platform-only exclusions require a neighboring cross-platform assertion for the product behavior they support. Windows teardown depends on the host `taskkill` command after graceful protocol shutdown has failed; a successful synchronous result keeps disposal bounded and makes descendant exit observable before cleanup returns, while a failed tree kill remains visible to the disposer.
|
||||
Portable fixtures are slightly more explicit because expected paths derive from shared native constants and transport failures enter through a narrow writer hook. Platform-only exclusions require a neighboring cross-platform assertion for the product behavior they support. Windows teardown depends on the host `taskkill` command after graceful protocol shutdown has failed; a successful synchronous result keeps disposal bounded and makes descendant exit observable before cleanup returns, while a failed tree kill remains visible to the disposer.
|
||||
|
||||
@@ -30,4 +30,4 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
可移植 fixture 需要更显式地构造，因为预期路径要从共享的原生常量派生，传输故障则通过狭窄的写入器 seam 注入。仅适用于特定平台的排除项必须配有相邻的跨平台断言，以继续覆盖相应的产品行为。协议级优雅关停失败后，Windows 上的资源清理依赖宿主的 `taskkill` 命令；命令同步执行成功时，可确保 dispose（资源释放）在有限时间内完成，并确保清理返回前即可观察到后代进程退出；若进程树终止失败，资源释放逻辑仍能观察到该失败。
|
||||
可移植 fixture 需要更显式地构造，因为预期路径要从共享的原生常量派生，传输故障则通过狭窄的写入器钩子注入。仅适用于特定平台的排除项必须配有相邻的跨平台断言，以继续覆盖相应的产品行为。协议级优雅关停失败后，Windows 上的资源清理依赖宿主的 `taskkill` 命令；命令同步执行成功时，可确保 dispose（资源释放）在有限时间内完成，并确保清理返回前即可观察到后代进程退出；若进程树终止失败，资源释放逻辑仍能观察到该失败。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: 095ad9055c8462ecec67609ed9af55df2a2d7adb
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: d32837d0e320df0dd53ef49a50092503328d0d73
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: aa289c9a70623904606e2d06b6492acb3a7d98a3
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: 9e26dcb53f0373ac967c627b95b895385784f925
|
||||
|
||||
@@ -18,7 +18,7 @@ A plain shared-fixture module (the [testing-policy sanctioned shape](../../../..
|
||||
|
||||
`launchWebScaffold()` boots the real web composition from the shipped `apps/cli/config/base.cordis.yml` plus `apps/cli/config/web.cordis.yml` through the vendored Loader's include mechanism — the same tree and mechanism `AppCLIEntry` drives for `dsh web`. Divergences ride include patches over that tree, the ACP `cordis.snapshot.yml` pattern expressed in-process: temp `persistenceRoot`; every host-level `skill-local` root (`dshHome`, `agentsHome`, and `bundledSkillDir`) pinned beneath the temp workspace with watching disabled, because ambient skill catalogs are model-visible input; `workspace-context` disabled (recorded fixtures must not embed this repo's AGENTS.md); `session-title-llm` disabled (its fire-and-forget title call would race the loop for the session's replay cursor); the webserver row pinned to port 0 with the built dist; and in keyless modes `llm-deepseek` disabled. A patch id that stops matching a row fails the boot sweep loudly instead of drifting. The boot runs `chdir`'d to the temp workspace so the api-gateway's `process.cwd()` session default, tool cwds, and fixtures agree; the `dsh web` bin's own glue (argv, profile json, AppCLIEntry) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`. Setup rollback and ordinary close both dispose the Cordis tree before removing the two owned temp roots, attempt every cleanup independently, and report cleanup failures without masking the setup failure.
|
||||
|
||||
Keyless model displacement is the disabled adapter row plus `installLlmReplay` filling the open seam on the settled root ctx in providers-catalog mode — never catch-all: with the adapter row disabled no adapter exists, so catch-all would leave `resolveModelInfo` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert (the published 128k `contextWindow` keeps it inert for small fixtures). The direct install rather than an inserted replay plugin row is deliberate: it returns the `ReplayHandle` the teardown consumption check needs. A scenario with no fixture leaves the seam empty, so a stray stream fails loud with NO_ADAPTER.
|
||||
Keyless model displacement is the disabled adapter row plus `installLlmReplay` filling the empty adapter registry on the settled root ctx in providers-catalog mode — never catch-all: with the adapter row disabled no adapter exists, so catch-all would leave `resolveModelInfo` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert (the published 128k `contextWindow` keeps it inert for small fixtures). The direct install rather than an inserted replay plugin row is deliberate: it returns the `ReplayHandle` the teardown consumption check needs. A scenario with no fixture leaves the registry empty, so a stray stream fails loud with NO_ADAPTER.
|
||||
|
||||
`seedSession()` seeds cold sessions through the real persistence API — a throwaway `Context` mounting `SessionStore` + `SessionPersistenceJsonl` against the host's root, `create()` + `append()`, one `utimes` backdate for deterministic sidebar order (the `semantic-checkpoint.snapshot.ts` precedent) — never raw file writes, so the seeder knows nothing of bucket hashing, filename encoding, or compression, and the host's zstd default needs no boot knob. Seeds are validated at seed time (parseable, ending in `turn/end` — an open final turn would be mutated by resume's crash repair).
|
||||
|
||||
@@ -52,7 +52,7 @@ High-cardinality performance diagnostics use the separate opt-in `apps/web/tests
|
||||
|
||||
## Prior art
|
||||
|
||||
Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot + AI SDK, lobe-chat, open-webui, OpenHands, Chainlit, continue, cline, langfuse, gradio/streamlit; Playwright HAR/route, MSW, Polly/nock, WireMock, aimock). The dominant proven architecture for apps that own their backend is an in-process fake/replay model behind the real backend seam with everything downstream real (LibreChat's `LIBRECHAT_TEST_RUN_HOOK` fake model; ai-chatbot's `MockLanguageModelV3` + `simulateReadableStream`; continue's scripted mock provider classes) — which is what `dsh-llm-replay` already is. Browser-level SSE interception cannot exercise incremental rendering (`route.fulfill` delivers the whole body at once; playwright#33564) and leaves the server SSE stack untested, so projects use it only for edge cases. Chunk pacing as a fixture parameter recurs everywhere (LibreChat 10ms default with slow profiles; ai-chatbot 500ms); real models in CI rot (open-webui's suite grew 120-second timeouts, was disabled, then deleted); sessions are seeded at the persistence layer with controlled timestamps (LibreChat inserts backdated Mongo documents; langfuse seeds its DB). No surveyed project replays a recorded agent-event log through the real backend for UI tests — the closest are provider-level recorded fixtures (aimock) and frontend-level socket history emission (OpenHands MSW) — so the session-log-as-fixture design goes one step beyond prior art along the axis this repo's model-visible ⟺ logged invariant makes natural.
|
||||
Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot + AI SDK, lobe-chat, open-webui, OpenHands, Chainlit, continue, cline, langfuse, gradio/streamlit; Playwright HAR/route, MSW, Polly/nock, WireMock, aimock). The dominant proven architecture for apps that own their backend is an in-process fake/replay model behind the real backend contract with everything downstream real (LibreChat's `LIBRECHAT_TEST_RUN_HOOK` fake model; ai-chatbot's `MockLanguageModelV3` + `simulateReadableStream`; continue's scripted mock provider classes) — which is what `dsh-llm-replay` already is. Browser-level SSE interception cannot exercise incremental rendering (`route.fulfill` delivers the whole body at once; playwright#33564) and leaves the server SSE stack untested, so projects use it only for edge cases. Chunk pacing as a fixture parameter recurs everywhere (LibreChat 10ms default with slow profiles; ai-chatbot 500ms); real models in CI rot (open-webui's suite grew 120-second timeouts, was disabled, then deleted); sessions are seeded at the persistence layer with controlled timestamps (LibreChat inserts backdated Mongo documents; langfuse seeds its DB). No surveyed project replays a recorded agent-event log through the real backend for UI tests — the closest are provider-level recorded fixtures (aimock) and frontend-level socket history emission (OpenHands MSW) — so the session-log-as-fixture design goes one step beyond prior art along the axis this repo's model-visible ⟺ logged invariant makes natural.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -60,7 +60,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
**Mock HTTP provider at `DEEPSEEK_BASE_URL`.** Rejected as the lane's mechanism (kept for the one existing workspace-probe smoke): fixtures become hand-authored OpenAI SSE byte scripts, a second fixture format that drifts from the session-log format the rest of the repo records and replays; the adapter's real HTTP path is with-key e2e's job.
|
||||
|
||||
**Growing the `?fixture` client.** Rejected: tier separation — `FixtureApiClient` exists to test the client shell without a server; everything below the client API seam stays untested by construction.
|
||||
**Growing the `?fixture` client.** Rejected: tier separation — `FixtureApiClient` exists to test the client shell without a server; everything below the client API boundary stays untested by construction.
|
||||
|
||||
**Placeholder `DEEPSEEK_API_KEY` + replay interception instead of disabling the adapter row.** Rejected despite zero composition change and two in-tree precedents: it satisfies `llm-deepseek`'s fail-loud key check with a lie and leaves a dead adapter mounted-but-intercepted; the disabled row (the ACP overlay's move) is honest keylessness and fails loud at the earliest resolvable point.
|
||||
|
||||
@@ -70,7 +70,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
**Spawning the `dsh web` bin with a `DSH_SNAPSHOT` replay branch.** Rejected: it needs a test-only replay branch plus environment plumbing in the shipped CLI. The in-process scaffold already loads the same `apps/cli/config/base.cordis.yml` plus `apps/cli/config/web.cordis.yml`; only argv, profile JSON, and `AppCLIEntry` glue remain outside it, and the keyless CLI smokes cover those paths.
|
||||
|
||||
**Changing the wire protocol for testability.** Rejected: the contract already has a first-class keyless isomorphic seam (`InProcessApiClient(toFetchHandler(api))`), the per-event unbatched SSE is exactly what makes replay observable in a browser, and testing a wire we no longer ship would invert the tier's purpose.
|
||||
**Changing the wire protocol for testability.** Rejected: the contract already has a first-class keyless in-process path (`InProcessApiClient(toFetchHandler(api))`), the per-event unbatched SSE is exactly what makes replay observable in a browser, and testing a wire we no longer ship would invert the tier's purpose.
|
||||
|
||||
**Real-model browser tests as the keyless lane.** Rejected: nondeterministic by construction; the surveyed cautionary case (open-webui) grew unbounded timeouts and was deleted. The with-key W5 smoke stays as the live-model complement.
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
`launchWebScaffold()` 通过 vendored Loader 的 include 机制，从交付的 `apps/cli/config/base.cordis.yml` 与 `apps/cli/config/web.cordis.yml` 启动真实 web 组合——与 `AppCLIEntry` 为 `dsh web` 驱动的是同一棵树、同一套机制。差异全部经 include patch 覆盖在这棵树上，即 ACP `cordis.snapshot.yml` 模式的进程内表达：临时 `persistenceRoot`；每个主机级 `skill-local` 根目录（`dshHome`、`agentsHome` 和 `bundledSkillDir`）都钉在临时工作区下并禁用监听，因为环境 skill（技能）目录是模型可见输入；禁用 `workspace-context`（录制的 fixture 不得嵌入本仓库的 AGENTS.md）；禁用 `session-title-llm`（其发后不管的标题调用会与循环争抢会话的回放游标）；webserver 行钉到端口 0 加已构建 dist；无密钥模式下禁用 `llm-deepseek`。patch 的 id 一旦不再匹配任何行，boot 扫描会大声失败而不是漂移。boot 在临时工作区 `chdir` 下运行，使 api-gateway 的 `process.cwd()` 会话默认值、工具 cwd 与 fixture 一致；`dsh web` bin 自身的胶水（argv、profile json、AppCLIEntry）仍由 `smoke-real.e2e.ts` 中的无密钥 CLI 冒烟把守。初始化回滚和正常关闭都会先对 Cordis 树执行 dispose（资源释放），再删除 scaffold 持有的两个临时根目录；每项清理都会独立尝试，并会报告清理失败而不掩盖初始化失败。
|
||||
|
||||
无密钥的模型替换 = 禁用适配器行的 patch 加 `installLlmReplay` 在停稳的根 ctx 上以提供方目录（providers-catalog）模式填充开放的 seam——绝不用 catch-all：适配器行被禁用后不存在任何适配器，catch-all 会让 `resolveModelInfo` 无路由可走，`compact-basic` 的步后压力检查将步步告警，而不是被可证明地闲置（发布的 128k `contextWindow` 使该路径对小 fixture 保持闲置）。选择直接安装而非插入回放插件行是刻意的：直接安装返回收尾消费检查所需的 `ReplayHandle`。没有 fixture 的场景让 seam 保持空置，任何离群的流式调用都会以 NO_ADAPTER 大声失败。
|
||||
无密钥的模型替换 = 禁用适配器行的 patch 加 `installLlmReplay` 在停稳的根 ctx 上以提供方目录（providers-catalog）模式填充空的适配器注册表——绝不用 catch-all：适配器行被禁用后不存在任何适配器，catch-all 会让 `resolveModelInfo` 无路由可走，`compact-basic` 的步后压力检查将步步告警，而不是被可证明地闲置（发布的 128k `contextWindow` 使该路径对小 fixture 保持闲置）。选择直接安装而非插入回放插件行是刻意的：直接安装返回收尾消费检查所需的 `ReplayHandle`。没有 fixture 的场景让注册表保持空置，任何离群的流式调用都会以 NO_ADAPTER 大声失败。
|
||||
|
||||
`seedSession()` 通过真实持久化 API 播种冷会话——一次性 `Context` 挂载 `SessionStore` + `SessionPersistenceJsonl` 指向 host 的根目录，`create()` + `append()`，一次 `utimes` 回拨保证侧栏顺序确定（`semantic-checkpoint.snapshot.ts` 先例）——绝不裸写文件，因此播种器对桶哈希、文件名编码、压缩一无所知，host 的 zstd 默认值也无需任何启动开关。种子在播种时即校验（可解析、以 `turn/end` 结尾——未闭合的最终轮次会被恢复（resume）的崩溃修复改写）。
|
||||
|
||||
@@ -52,7 +52,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## 业界先例
|
||||
|
||||
调研了 AI 聊天/agent web UI 与 mock 层（LibreChat、vercel/ai-chatbot + AI SDK、lobe-chat、open-webui、OpenHands、Chainlit、continue、cline、langfuse、gradio/streamlit；Playwright HAR/route、MSW、Polly/nock、WireMock、aimock）。自有后端的应用的主流成熟架构是：真实后端 seam 后放一个进程内伪造/回放模型，下游全部真实（LibreChat 的 `LIBRECHAT_TEST_RUN_HOOK` 伪模型；ai-chatbot 的 `MockLanguageModelV3` + `simulateReadableStream`；continue 的脚本化 mock 提供方类）——这正是 `dsh-llm-replay` 已然所是。浏览器层 SSE 拦截无法检验增量渲染（`route.fulfill` 一次性交付整个响应体；playwright#33564），且服务端 SSE 栈完全失测，因此各项目只把它用于边缘用例。分片节奏作为 fixture 参数反复出现（LibreChat 默认 10ms 附慢速档；ai-chatbot 500ms）；CI 里的真实模型会腐烂（open-webui 的套件长出 120 秒超时，先被禁用后被删除）；会话在持久化层以受控时间戳播种（LibreChat 直插回拨时间的 Mongo 文档；langfuse 播种其数据库）。没有任何被调研项目为 UI 测试把录制的 agent 事件日志经真实后端回放——最接近的是提供方层录制 fixture（aimock）与前端层 socket 历史发射（OpenHands MSW）——因此会话日志即 fixture 的设计沿着本仓库「模型可见 ⟺ 已记录」不变式所指的方向比业界先例多走了一步。
|
||||
调研了 AI 聊天/agent web UI 与 mock 层（LibreChat、vercel/ai-chatbot + AI SDK、lobe-chat、open-webui、OpenHands、Chainlit、continue、cline、langfuse、gradio/streamlit；Playwright HAR/route、MSW、Polly/nock、WireMock、aimock）。自有后端的应用的主流成熟架构是：真实后端约定后放一个进程内伪造/回放模型，下游全部真实（LibreChat 的 `LIBRECHAT_TEST_RUN_HOOK` 伪模型；ai-chatbot 的 `MockLanguageModelV3` + `simulateReadableStream`；continue 的脚本化 mock 提供方类）——这正是 `dsh-llm-replay` 已然所是。浏览器层 SSE 拦截无法检验增量渲染（`route.fulfill` 一次性交付整个响应体；playwright#33564），且服务端 SSE 栈完全失测，因此各项目只把它用于边缘用例。分片节奏作为 fixture 参数反复出现（LibreChat 默认 10ms 附慢速档；ai-chatbot 500ms）；CI 里的真实模型会腐烂（open-webui 的套件长出 120 秒超时，先被禁用后被删除）；会话在持久化层以受控时间戳播种（LibreChat 直插回拨时间的 Mongo 文档；langfuse 播种其数据库）。没有任何被调研项目为 UI 测试把录制的 agent 事件日志经真实后端回放——最接近的是提供方层录制 fixture（aimock）与前端层 socket 历史发射（OpenHands MSW）——因此会话日志即 fixture 的设计沿着本仓库「模型可见 ⟺ 已记录」不变式所指的方向比业界先例多走了一步。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
@@ -60,7 +60,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
**`DEEPSEEK_BASE_URL` 处的 mock HTTP 提供方。** 作为本车道机制已否决（仅保留给既有的工作区探针冒烟）：fixture 会变成手写的 OpenAI SSE 字节脚本，一种与仓库其余部分录制回放的会话日志格式渐行渐远的第二 fixture 格式；适配器的真实 HTTP 路径归带密钥 e2e 管。
|
||||
|
||||
**扩展 `?fixture` 客户端。** 已否决：分层纪律——`FixtureApiClient` 的存在意义就是脱离服务器测试客户端 shell；client API seam 以下按构造即失测。
|
||||
**扩展 `?fixture` 客户端。** 已否决：分层纪律——`FixtureApiClient` 的存在意义就是脱离服务器测试客户端 shell；client API 边界以下按构造即失测。
|
||||
|
||||
**用占位 `DEEPSEEK_API_KEY` + 回放拦截替代禁用适配器行。** 尽管零组合改动且树内有两处先例仍被否决：它用谎言满足 `llm-deepseek` 的快速失败密钥检查，还留下一个挂载却被拦截的死适配器；禁用行（ACP overlay 的同款做法）是诚实的无密钥，并在最早可解析点快速失败。
|
||||
|
||||
@@ -70,7 +70,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
**以 `DSH_SNAPSHOT` 回放分支拉起 `dsh web` bin。** 已否决：它需要在交付的 CLI 中增加测试专用回放分支和环境变量管道。进程内 scaffold 已加载同一份 `apps/cli/config/base.cordis.yml` 与 `apps/cli/config/web.cordis.yml`；只剩 argv、profile JSON 和 `AppCLIEntry` 胶水不在其覆盖范围内，而这些路径已由无密钥 CLI 冒烟覆盖。
|
||||
|
||||
**为可测试性改 wire 协议。** 已否决：约定已有第一等的无密钥同构 seam（`InProcessApiClient(toFetchHandler(api))`），逐事件不合批的 SSE 恰是回放在浏览器中可观测的原因，测试一条不再交付的 wire 会颠倒该层的存在意义。
|
||||
**为可测试性改 wire 协议。** 已否决：约定已有第一等的无密钥进程内路径（`InProcessApiClient(toFetchHandler(api))`），逐事件不合批的 SSE 恰是回放在浏览器中可观测的原因，测试一条不再交付的 wire 会颠倒该层的存在意义。
|
||||
|
||||
**以真实模型浏览器测试充当无密钥车道。** 已否决：按构造即不确定；被调研的前车之鉴（open-webui）长出无界超时后被删除。带密钥的 W5 冒烟仍是真实模型侧的补充。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user