Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(web-e2e): rename harness -> scaffold; add interaction coverage

Browse Source 
The shared test module was named harness.ts inside a repo whose product
IS a harness — hopelessly ambiguous. Renamed to scaffold.ts with
launchWebScaffold/WebScaffold; tsconfig plane-split entries, the seam
JSDoc/README mentions, and both Agent Note languages updated.

Both scenarios gain a Playwright interaction step over the settled
transcript (after the golden capture, so committed aria surfaces stay
untouched): replay-round-trip clicks the reasoning fold open/closed over
wire-delivered state; seeded-history expands a read tool row rebuilt from
the cold log and asserts the recorded result text appears (read rows are
expand-in-place — rowExpands routes the click to the inline fold, not the
details column). test:web 30 passed | 1 skipped.
This commit is contained in:
Tianyi Cui
2026-07-24 22:18:19 +08:00
parent 8f97f95d7b
commit 6d3c25f494
9 changed files with 91 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
View File

@@ -12,11 +12,11 @@ The web GUI ships as a real assembled chain — chromium page → client plugin
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay` against the real in-process web assembly, asserting a normalized conversation aria golden plus in-process world state. No new package; the product deltas are the `BootHostOptions.llm` seam and two additive `dsh-llm-replay` surfaces.
### Harness: `apps/web/tests/harness.ts`
### Scaffold: `apps/web/tests/scaffold.ts`
A plain shared-fixture module (the [testing-policy sanctioned shape](../../../../docs/testing.md)), not a package: the gate-worthy logic — replay derivation, session parsing, log scrubbing, persistence — lives in the gated packages `dsh-llm-replay`, `dsh-acp-snapshot`, and `dsh-session-persistence-jsonl`; what remains is boot wiring and browser glue, and chromium-driving code cannot hold per-file 100% coverage on the browserless coverage runners.
`launchWebHarness()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { …, llm: false } })`, `installLlmReplay(host.ctx, { file, providers, paceMs })`, `mountWebPlugins(host.ctx, roster, anchor)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, … })`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing, dist resolution) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md), a ruling this lane's design review explicitly reaffirmed (Loader-izing `dsh web` was declined; it would be its own proposal). Two deliberate assembly divergences from the `dsh web` shell, noted in the harness header: `workspaceContext: false` (recorded fixtures must not embed this repo's AGENTS.md) and `sessionTitleLlm` left at bootHost's disabled default (its fire-and-forget title call would share the session's replay cursor nondeterministically).
`launchWebScaffold()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { …, llm: false } })`, `installLlmReplay(host.ctx, { file, providers, paceMs })`, `mountWebPlugins(host.ctx, roster, anchor)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, … })`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing, dist resolution) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md), a ruling this lane's design review explicitly reaffirmed (Loader-izing `dsh web` was declined; it would be its own proposal). Two deliberate assembly divergences from the `dsh web` shell, noted in the scaffold header: `workspaceContext: false` (recorded fixtures must not embed this repo's AGENTS.md) and `sessionTitleLlm` left at bootHost's disabled default (its fire-and-forget title call would share the session's replay cursor nondeterministically).
The `llm: false` seam is the reviewed resolution of the keyless-boot question: `'deepseek' | false` on `BootHostOptions`, matching the `workspaceContext: Config | false` shape, with `RunningHost.ctx` JSDoc naming "filling a deliberately-open capability seam" as its third sanctioned use. Replay runs in providers-catalog mode with a published `contextWindow` (the TUI `PROVIDERS` shape), never catch-all: with no adapter registered, catch-all would leave `resolveModelContext` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert.
@@ -28,13 +28,13 @@ The barrier stack for a prompted turn, in order: (1) host-side `await agent.when
No single-shot transient-DOM assertions: every hop from replay yield to React commit can coalesce chunks, so sampling `[data-streaming]` is a race by construction. Streaming incrementality is asserted from the persisted `assistant/chunk` events (model-visible ⟺ logged makes the log the authoritative proof). `dsh-llm-replay`'s opt-in `paceMs` (default absent = burst) is a realism knob so the browser observes genuinely incremental SSE; correctness never leans on it, and abort during a pace wait cancels promptly.
Every scenario fails on any pageerror and on the client's connection-loss/gap-repair console warnings: the reconnect machine plus history resync would otherwise self-heal a dead SSE path and the suite would certify a broken wire. Harness `close()` calls the `ReplayHandle.assertConsumed()` teardown check (every recorded script bound, every cursor drained), converting silent underruns and shifted bindings into crisp diagnostics. No vitest retry on the lane; one chromium per file, fresh context per scenario, one host per scenario; viewport pinned; selectors anchor on roles, `data-*` attributes, and visible text.
Every scenario fails on any pageerror and on the client's connection-loss/gap-repair console warnings: the reconnect machine plus history resync would otherwise self-heal a dead SSE path and the suite would certify a broken wire. Scaffold `close()` calls the `ReplayHandle.assertConsumed()` teardown check (every recorded script bound, every cursor drained), converting silent underruns and shifted bindings into crisp diagnostics. No vitest retry on the lane; one chromium per file, fresh context per scenario, one host per scenario; viewport pinned; selectors anchor on roles, `data-*` attributes, and visible text.
### Expected outputs
One committed golden per scenario: a normalized `ariaSnapshot()` of the conversation region (`ui.expected.md`) — uuid/cwd/workspace-basename/duration tokens normalized, captured poll-until-equal at the settled milestone — plus a few role/text anchor assertions that stay green under a semantics-preserving component rewrite while the golden churns reviewably. The aria tree is the mechanization of the client rule "assert what the user would see, never class names". World-state assertions ride `host.ctx` session events inline (which tools ran, `turn/end` completed) instead of a second committed log golden: the persisted-log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence, and re-pinning it here would double refresh cost against the tier discipline. `refresh` is the sole golden writer — a missing golden in replay mode fails with the healing command rather than self-bootstrapping.
The typecheck plane split is structural: `apps/web/tests/{harness,support,replay-round-trip.e2e,seeded-history.e2e}.ts` are host-plane programs (they boot the host spine), so they are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
The typecheck plane split is structural: `apps/web/tests/{scaffold,support,replay-round-trip.e2e,seeded-history.e2e}.ts` are host-plane programs (they boot the host spine), so they are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
### Modes and fixtures
@@ -81,7 +81,7 @@ The lane itself: `pnpm run test:web` runs both scenarios keylessly alongside the
## Deferred
- **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins bootHost's composed prompt/tool schemas (`TODO(web-header-pin)` — the harness `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
- **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins bootHost's composed prompt/tool schemas (`TODO(web-header-pin)` — the scaffold `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
- **CI browser provisioning**: reversal of the no-browser-in-CI ruling, staged criteria above (`TODO(ci-browser)`).
- **Follow-up-prompt-after-resume scenario**: the history/live stitch path over the real wire; add as its own scenario when that code changes or regresses.