ci: isolate web snapshots to consumer lane
This commit is contained in:
imccyu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-20-gui-testing-system.md
|
||||
2026-07-20-gui-testing-system.md: 12699313b6be2aa6e290916979a67364d9d07293
|
||||
2026-07-20-gui-testing-system.zh.md: 34a08feec71c472b5df829c8d93c1ed925f91b53
|
||||
2026-07-20-gui-testing-system.md: 8c6dafb18fc207fc4eac780ba18e108267bc28b1
|
||||
2026-07-20-gui-testing-system.zh.md: 9a0de4bfa8fa2f8de55beef53bedde51649c5d9c
|
||||
|
||||
@@ -34,7 +34,7 @@ Inter-tier discipline: **each tier tests its own layer, upper tiers never re-tes
|
||||
| Baseline | `pnpm run test:gui` | Tier 1+2 vitest (`packages/client packages/host`), seconds-fast, no browser, no server | Casually, after touching any GUI source |
|
||||
| Semantic snapshot | `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot` | Keyless assembled-application semantics plus the repo's transport-specific expected outputs | After a human-visible GUI change; before delivery |
|
||||
| Browser end-to-end | `pnpm run test:web` | Rebuilds the front-end dist first, then runs the tier-3 browser set: the two-level smoke (fixture level + real-host level self-skip) plus the keyless replayed e2e scenarios (`DSH_SNAPSHOT=record`/`refresh` re-record fixtures / rewrite goldens) | After touching the build surface/boot/carriage; before delivery |
|
||||
| Browser expected-output gate | `DSH_SNAPSHOT=replay pnpm run test:web:built` | Reuses CI-built artifacts and compares every committed browser golden without writing | Every Linux pull request and default-branch push |
|
||||
| Browser expected-output gate | `DSH_SNAPSHOT=replay pnpm run test:web:built` | Reuses CI-built artifacts and compares every committed browser golden without writing | Every Linux pull request |
|
||||
| Gate | `pnpm run test:coverage` | The repo-wide gate (host and client GUI packages included, except annotated browser-grade exclusions) | The PR window |
|
||||
|
||||
**Division of labor between the browser scripts and vitest**: Playwright owns browser/carrier black-box regression and long sequential user journeys; ordinary vitest owns data-layer semantics such as reference stability, timing, and wire shapes; snapshot vitest owns stable app-level semantic output through the built composition. These lanes complement each other rather than duplicating assertions.
|
||||
|
||||
@@ -34,7 +34,7 @@ GUI 栈需要考虑多种应用形态，同应用形态内的不同运行环境
|
||||
| 基础 | `pnpm run test:gui` | 1+2 层 vitest（`packages/client packages/host`），秒级、无浏览器无 server | 改 GUI 任意源码后随手跑 |
|
||||
| 语义快照 | `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot` | 无需密钥的组装应用语义，以及仓库按传输形态划分的预期输出 | 用户可见的 GUI 变更后；交付前 |
|
||||
| 浏览器端到端 | `pnpm run test:web` | 先重建前端 dist，再跑 3 层浏览器全集：双级 smoke（fixture 级 + 真 host 级 self-skip）加上无密钥回放 e2e 场景（`DSH_SNAPSHOT=record`/`refresh` 重录 fixture / 重写期望输出） | 改构建面/boot/承载后；交付前 |
|
||||
| 浏览器预期输出门禁 | `DSH_SNAPSHOT=replay pnpm run test:web:built` | 复用 CI 构建的产物，并在不写入的情况下比较每份已提交的浏览器预期输出 | 每个 Linux 拉取请求和默认分支推送 |
|
||||
| 浏览器预期输出门禁 | `DSH_SNAPSHOT=replay pnpm run test:web:built` | 复用 CI 构建的产物，并在不写入的情况下比较每份已提交的浏览器预期输出 | 每个 Linux 拉取请求 |
|
||||
| 门禁 | `pnpm run test:coverage` | 全仓 gate（host 与 client GUI 包均纳入，仅排除带注释的浏览器级例外） | PR 窗口 |
|
||||
|
||||
**浏览器脚本与 vitest 的分工**：Playwright 负责浏览器/承载层黑盒回归和较长的连续用户操作流程；普通 vitest 负责引用稳定性、时序和 wire 结构等数据层语义；快照 vitest 通过构建后的组合负责稳定的应用层语义输出。这些车道彼此互补，而不重复断言。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: 74522e070e9a26e61db02ea4f71c2b205f222037
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: 4ab1e70b3fedb1eae88be457aa0d533c1d193bb1
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: 08227a83bfb45348f85e6c306e0a2cc09a876713
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: e1cdb6e26ac9345161031df10fc0c0a753cb2ba9
|
||||
|
||||
@@ -46,7 +46,7 @@ The lane covers three behavior families. Live-turn scenarios pin ordinary tool e
|
||||
|
||||
### CI stance
|
||||
|
||||
The lane is a required compare-only gate for Linux pull requests under the [browser snapshot CI decision](2026-07-30-web-browser-snapshot-ci-gate.md). The static job publishes `apps/web/dist` with the package build artifacts; the consumer job installs the lockfile-selected Chromium, restores its OS-and-lockfile-keyed cache, and runs the lane with `DSH_SNAPSHOT=replay`. CI never records or refreshes goldens. The default-branch Linux serial jobs run the same comparison and seed the hosted browser cache. Scenarios remain POSIX-oriented and stay outside the Windows and macOS matrices.
|
||||
The lane is a required compare-only gate for Linux pull requests under the [browser snapshot CI decision](2026-07-30-web-browser-snapshot-ci-gate.md). The static job publishes `apps/web/dist` with the package build artifacts; the `node 24 / snapshots and artifacts` consumer job installs the lockfile-selected Chromium, restores its OS-and-lockfile-keyed cache, and runs the lane with `DSH_SNAPSHOT=replay`. CI never records or refreshes goldens. Default-branch serial jobs do not provision Chromium or run the lane. Scenarios remain POSIX-oriented and stay outside the Windows and macOS matrices.
|
||||
|
||||
## Prior art
|
||||
|
||||
@@ -88,4 +88,4 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
## Consequences
|
||||
|
||||
The web surface gains its record-once/replay-forever tier: the real chromium → SSE → apiproxy → loop → tools → persistence chain runs keylessly in ~10-30s, deterministic across repeat runs, with fixtures owned and re-recordable by the lane itself. Costs accepted: every intentional conversation-UI change ends with a keyless `DSH_SNAPSHOT=refresh` (golden churn is reviewed diff, anchors keep semantic green); the aria format is Playwright-owned — the one committed snapshot format the repo does not control — so playwright version bumps must be deliberate bump-and-refresh commits (the dependency floats `^1.49.0` in `apps/web/package.json`; pin exactly if churn bites); replay's first-call-order binding constrains scenarios to one prompting session each, with the consumption assertion as the tripwire; `compact-basic` shares the session's replay cursor and stays inert only under the published 128k catalog window; and required Linux CI pays for Chromium provisioning and the serial browser run so the PR that changes the assembled UI owns its expected-output diff.
|
||||
The web surface gains its record-once/replay-forever tier: the real chromium → SSE → apiproxy → loop → tools → persistence chain runs keylessly in ~10-30s, deterministic across repeat runs, with fixtures owned and re-recordable by the lane itself. Costs accepted: every intentional conversation-UI change ends with a keyless `DSH_SNAPSHOT=refresh` (golden churn is reviewed diff, anchors keep semantic green); the aria format is Playwright-owned — the one committed snapshot format the repo does not control — so playwright version bumps must be deliberate bump-and-refresh commits (the dependency floats `^1.49.0` in `apps/web/package.json`; pin exactly if churn bites); replay's first-call-order binding constrains scenarios to one prompting session each, with the consumption assertion as the tripwire; `compact-basic` shares the session's replay cursor and stays inert only under the published 128k catalog window; and the required consumer job pays for Chromium provisioning and one browser run so the PR that changes the assembled UI owns its expected-output diff.
|
||||
|
||||
@@ -46,7 +46,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
### CI 立场
|
||||
|
||||
根据[浏览器快照 CI 决策](2026-07-30-web-browser-snapshot-ci-gate.md)，该车道是 Linux 拉取请求必需的只比较门禁。static 任务会把 `apps/web/dist` 与包构建产物一同发布；消费方任务安装锁文件选定的 Chromium，恢复以操作系统和锁文件为键的缓存，并用 `DSH_SNAPSHOT=replay` 运行该车道。CI 从不录制或刷新预期输出。默认分支上的 Linux 串行任务执行同样的比较，并填充托管浏览器缓存。场景仍面向 POSIX，并继续置于 Windows 和 macOS 矩阵之外。
|
||||
根据[浏览器快照 CI 决策](2026-07-30-web-browser-snapshot-ci-gate.md)，该车道是 Linux 拉取请求必需的只比较门禁。static 任务会把 `apps/web/dist` 与包构建产物一同发布；`node 24 / snapshots and artifacts` 消费方任务安装锁文件选定的 Chromium，恢复以操作系统和锁文件为键的缓存，并用 `DSH_SNAPSHOT=replay` 运行该车道。CI 从不录制或刷新预期输出。默认分支串行任务不供给 Chromium，也不运行该车道。场景仍面向 POSIX，并继续置于 Windows 和 macOS 矩阵之外。
|
||||
|
||||
## 业界先例
|
||||
|
||||
@@ -88,4 +88,4 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## 后果
|
||||
|
||||
Web 表面获得了录制一次/永久回放的层级：真实 chromium → SSE → apiproxy → 循环 → 工具 → 持久化的链路以约 10-30 秒无密钥运行，重复运行结果确定，fixture 由车道自身持有并可重录。接受的成本：每次有意的会话 UI 变更都以一次无密钥 `DSH_SNAPSHOT=refresh` 收尾（预期输出变动是受评审的 diff，锚断言保住语义绿色）；aria 格式归 Playwright 所有——仓库唯一不受自己控制的提交快照格式——因此 playwright 版本升级必须是刻意的升级加刷新提交（依赖在 `apps/web/package.json` 中浮动为 `^1.49.0`；若变动伤人则改为精确锁定）；回放的首次调用顺序绑定把每个场景限制为至多一个发起提示的会话，消费断言是绊线；`compact-basic` 与会话共享回放游标，仅在发布的 128k 目录窗口下保持闲置；必需的 Linux CI 承担 Chromium 供给与串行浏览器运行的成本，使改动组装后 UI 的 PR（Pull Request）持有相应的预期输出 diff。
|
||||
Web 表面获得了录制一次/永久回放的层级：真实 chromium → SSE → apiproxy → 循环 → 工具 → 持久化的链路以约 10-30 秒无密钥运行，重复运行结果确定，fixture 由车道自身持有并可重录。接受的成本：每次有意的会话 UI 变更都以一次无密钥 `DSH_SNAPSHOT=refresh` 收尾（预期输出变动是受评审的 diff，锚断言保住语义绿色）；aria 格式归 Playwright 所有——仓库唯一不受自己控制的提交快照格式——因此 playwright 版本升级必须是刻意的升级加刷新提交（依赖在 `apps/web/package.json` 中浮动为 `^1.49.0`；若变动伤人则改为精确锁定）；回放的首次调用顺序绑定把每个场景限制为至多一个发起提示的会话，消费断言是绊线；`compact-basic` 与会话共享回放游标，仅在发布的 128k 目录窗口下保持闲置；必需的消费方任务承担 Chromium 供给与一次浏览器运行的成本，使改动组装后 UI 的 PR（Pull Request）持有相应的预期输出 diff。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-30-web-browser-snapshot-ci-gate.md
|
||||
2026-07-30-web-browser-snapshot-ci-gate.md: 71d6bcdbfad11032ea6f8e7d570780f780cfe071
|
||||
2026-07-30-web-browser-snapshot-ci-gate.zh.md: bb28b42c5d2684d29e7e18646d9756b40557eb11
|
||||
2026-07-30-web-browser-snapshot-ci-gate.md: eb7d2fa159cec0c0e11adc28239cf6b3b4f0f593
|
||||
2026-07-30-web-browser-snapshot-ci-gate.zh.md: ff3d49e5b49b4f63e1b1ce52f690f80cdee70b2d
|
||||
|
||||
@@ -12,11 +12,11 @@ The [keyless web browser e2e lane](2026-07-24-web-gui-browser-e2e-lane.md) runs
|
||||
|
||||
For Linux PRs, the `node 24 / snapshots and artifacts` job must run the full web browser replay/compare suite. `scripts/run-gates.ts` registers `test:web:built` as a `ci-consumers` gate and explicitly injects `DSH_SNAPSHOT=replay`; CI never runs in `record` or `refresh` mode, so when the committed goldens disagree with the currently assembled application, the tests fail directly instead of silently rewriting them on the runner and then passing.
|
||||
|
||||
The static CI job already builds all publishable artifacts; it puts `apps/web/dist` and the package `lib/` directories in the built-tree artifact, which the consumer job reuses without rebuilding the entire repository. The consumer job installs Chromium and its system dependencies at the Playwright version in the lockfile, and caches the browser by operating system and the `pnpm-lock.yaml` hash. The default branch's serial Linux job runs the same compare command and populates the default-branch cache, which subsequent PRs can restore directly.
|
||||
The static CI job already builds all publishable artifacts; it puts `apps/web/dist` and the package `lib/` directories in the built-tree artifact, which the consumer job reuses without rebuilding the entire repository. The `node 24 / snapshots and artifacts` consumer job installs Chromium and its system dependencies at the Playwright version in the lockfile, and caches the browser by operating system and the `pnpm-lock.yaml` hash. Default-branch serial jobs do not provision Chromium or run the suite.
|
||||
|
||||
Local `pnpm run test:web` continues to build first and then run the full browser suite; `test:web:built` is the entry point for existing build artifacts. Developers explicitly run `DSH_SNAPSHOT=refresh pnpm run test:web` only after confirming that user-visible output changed intentionally, review every expected-output diff, and then verify again in replay mode that no files are written.
|
||||
|
||||
The gate remains Linux-only: these scenarios target POSIX, and the Windows and macOS serial reference jobs do not run them again. A PR's `all checks passed` verdict already depends on the consumer job, so a browser compare failure blocks the merge without requiring a new branch-protection check name.
|
||||
The gate runs only in the Linux PR consumer job: these scenarios target POSIX, and serial reference jobs do not provision Chromium or run the suite. A PR's `all checks passed` verdict already depends on the consumer job, so a browser compare failure blocks the merge without requiring a new branch-protection check name.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -30,4 +30,4 @@ The gate remains Linux-only: these scenarios target POSIX, and the Windows and m
|
||||
|
||||
## Consequences
|
||||
|
||||
Before merge, every PR proves that the current web assembly matches all committed browser expected outputs, turning a missed refresh from an “unrelated change in a later PR” into a failure in the PR that introduced it. The cost is Chromium provisioning and one serial pass through the browser scenarios in Linux CI; built-artifact reuse and the default-branch browser cache avoid duplicate builds and routine downloads. The gate still makes no claim of cross-platform browser consistency, and if a Playwright/Chromium upgrade changes the ARIA format, the upgrade PR must explicitly refresh the expected outputs and review the churn.
|
||||
Before merge, every PR proves that the current web assembly matches all committed browser expected outputs, turning a missed refresh from an “unrelated change in a later PR” into a failure in the PR that introduced it. The cost is Chromium provisioning and one serial pass through the browser scenarios in the consumer job; built-artifact reuse and the browser cache avoid duplicate builds and downloads on reruns. The gate still makes no claim of cross-platform browser consistency, and if a Playwright/Chromium upgrade changes the ARIA format, the upgrade PR must explicitly refresh the expected outputs and review the churn.
|
||||
|
||||
@@ -12,11 +12,11 @@ Status: implemented
|
||||
|
||||
Linux PR 的 `node 24 / snapshots and artifacts` 必须运行完整 Web 浏览器 replay/compare。`scripts/run-gates.ts` 把 `test:web:built` 作为 `ci-consumers` 的一个 gate，并显式注入 `DSH_SNAPSHOT=replay`；CI 永不以 `record` 或 `refresh` 模式运行，因此提交的 golden 与当前组装应用不一致时测试直接失败，不会在 runner 内静默改写后通过。
|
||||
|
||||
静态 CI job 已经构建全部发布产物；它把 `apps/web/dist` 连同 package `lib/` 放进 built-tree artifact，consumer job 复用该 artifact 而不重复全仓构建。consumer job 按 lockfile 中的 Playwright 版本安装 Chromium 及系统依赖，并以操作系统和 `pnpm-lock.yaml` 哈希缓存浏览器。默认分支的 Linux 串行 job 运行同一 compare 命令并产出默认分支缓存，后续 PR 可直接恢复。
|
||||
静态 CI job 已经构建全部发布产物；它把 `apps/web/dist` 连同 package `lib/` 放进 built-tree artifact，consumer job 复用该 artifact 而不重复全仓构建。`node 24 / snapshots and artifacts` 消费方 job 按 lockfile 中的 Playwright 版本安装 Chromium 及系统依赖，并以操作系统和 `pnpm-lock.yaml` 哈希缓存浏览器。默认分支串行 job 不供给 Chromium，也不运行该套件。
|
||||
|
||||
本地 `pnpm run test:web` 仍先构建再运行浏览器全集；`test:web:built` 是已有构建产物的执行入口。开发者只在确认用户可见输出有意变化后显式运行 `DSH_SNAPSHOT=refresh pnpm run test:web`，评审每一处 expected diff，再以 replay 模式复验不再写文件。
|
||||
|
||||
门禁保持 Linux-only：这些场景面向 POSIX，Windows 与 macOS 的串行参考 job 不重复运行。PR 的 `all checks passed` 已依赖 consumer job，因此浏览器 compare 失败会阻止合并，无需新增 branch-protection check 名称。
|
||||
门禁仅在 Linux PR 消费方 job 中运行：这些场景面向 POSIX，串行参考 job 不供给 Chromium，也不运行该套件。PR 的 `all checks passed` 已依赖 consumer job，因此浏览器 compare 失败会阻止合并，无需新增 branch-protection check 名称。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
@@ -30,4 +30,4 @@ Linux PR 的 `node 24 / snapshots and artifacts` 必须运行完整 Web 浏览
|
||||
|
||||
## 后果
|
||||
|
||||
每个 PR 都在合并前证明当前 Web 组装与所有已提交的浏览器 expected 一致，漏刷从“后续 PR 的无关变化”变成引入 PR 自己的失败。成本是 Linux CI 增加 Chromium 供给和一轮串行浏览器场景；built artifact 复用与默认分支浏览器缓存避免重复构建和常态下载。门禁仍不声称跨平台浏览器一致性，Playwright/Chromium 升级若改变 aria 格式，升级 PR 必须显式 refresh 并评审 churn。
|
||||
每个 PR 都在合并前证明当前 Web 组装与所有已提交的浏览器 expected 一致，漏刷从“后续 PR 的无关变化”变成引入 PR 自己的失败。成本是消费方 job 需要供给 Chromium，并串行运行一轮浏览器场景；built artifact 复用与浏览器缓存避免重跑时重复构建和下载。门禁仍不声称跨平台浏览器一致性，Playwright/Chromium 升级若改变 aria 格式，升级 PR 必须显式 refresh 并评审 churn。
|
||||
|
||||
21
.github/workflows/ci.yml
vendored
21
.github/workflows/ci.yml
vendored
@@ -457,17 +457,9 @@ jobs:
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: ${{ runner.os }}-playwright-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Install Playwright Chromium
|
||||
run: pnpm --filter @deepseek-ai/dsh-frontend exec playwright install --with-deps chromium
|
||||
|
||||
- name: Prepare bubblewrap (unrestrict userns)
|
||||
run: bash scripts/prepare-ci-bubblewrap.sh
|
||||
|
||||
@@ -482,11 +474,6 @@ jobs:
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '1'
|
||||
run: pnpm run check:ci
|
||||
|
||||
- name: Compare web browser snapshots
|
||||
env:
|
||||
DSH_SNAPSHOT: replay
|
||||
run: pnpm run test:web:built
|
||||
|
||||
# Hot-standby drill for the in-house self-hosted pool: every master move
|
||||
# re-runs the complete unsharded aggregate on the persistent 64-core VM,
|
||||
# continuously proving that environment can take over a required lane if
|
||||
@@ -527,9 +514,6 @@ jobs:
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Install Playwright Chromium
|
||||
run: pnpm --filter @deepseek-ai/dsh-frontend exec playwright install --with-deps chromium
|
||||
|
||||
- name: Prepare bubblewrap (unrestrict userns)
|
||||
run: bash scripts/prepare-ci-bubblewrap.sh
|
||||
|
||||
@@ -544,11 +528,6 @@ jobs:
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '1'
|
||||
run: pnpm run check:ci
|
||||
|
||||
- name: Compare web browser snapshots
|
||||
env:
|
||||
DSH_SNAPSHOT: replay
|
||||
run: pnpm run test:web:built
|
||||
|
||||
serial-macos:
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
|
||||
name: serial / macos
|
||||
|
||||
Reference in New Issue
Block a user