@@ -8,50 +8,63 @@ English | [中文](2026-07-18-tui-terminal-state-snapshots.zh.md)
The TUI is a stateful renderer. Its user-visible result depends on ANSI parsing, differential frames, wrapping, scrollback, viewport position, terminal width, focus, cursor state, and each tool's presentation intent. Unit tests that collect `Terminal.write()` fragments can prove event handling, but they cannot prove the final screen a terminal displays. The same screen may also be emitted through different write fragments, so pinning those fragments creates false regressions.
Component-line snapshots stop before ANSI reaches a terminal and miss cursor movement, clearing, styling, overlay composition, and reflow. Raster screenshots include font and platform rendering noise that is unrelated to the TUI contract. The TUI therefore needs a deterministic, reviewable representation of terminal state plus a smaller test at the real process and PTY boundary.
Component-line snapshots stop before ANSI reaches a terminal and miss cursor movement, clearing, styling, overlay composition, and reflow. Raster screenshots include font and platform rendering noise that is unrelated to the TUI contract. A completed flow built by directly appending plausible session events has another blind spot: it proves the renderer accepts those shapes, not that the production agent loop and tool implementations produce them.
The TUI therefore needs a deterministic, reviewable representation of terminal state, recorded model journeys that execute the real downstream stack, and a smaller test at the real process and PTY boundary.
2.`tui.snapshot.ts` mounts the production TUI against a headless terminal emulator and compares semantic terminal-state goldens.
3.`tui-keyless-smoke.e2e.ts` boots the real Loader composition in a PTY, drives a complete scripted conversation through streaming and `ask_user_question`, exits through `/exit`, and verifies terminal teardown. The production coding-agent configuration also retains its banner/exit and startup-failure PTY cases.
2.`packages/ui/tui/tests/tui.snapshot.ts` mounts the production TUI against a headless terminal emulator for transient states that a completed session log cannot retain: in-flight streaming, pending tool calls, overlays, expansion, compaction reflow, errors, and shutdown.
3.`examples/tui-agent/tests/tui.snapshot.ts` replays committed JSONL session logs through the production agent loop and real tools, then compares the resulting semantic terminal state.
4.`examples/tui-agent/tests/tui-keyless-smoke.e2e.ts` boots the real Loader composition in a PTY, drives a scripted conversation through streaming and `ask_user_question`, and verifies startup, input, exit, failure reporting, and terminal restoration.
The package-local `HeadlessTerminal` implements the same pi-tui `Terminal` interface as the process terminal and feeds every ANSI write into the pinned `@xterm/headless` parser. A snapshot waits for pi-tui's synchronized-output end marker before reading state. This makes a checkpoint represent a completed frame rather than a timer-dependent write prefix.
The runnable TUI has its own `examples/tui-agent` leaf beside the readline `coding-agent` and `acp-agent` leaves. It reuses the coding agent's backend and tool composition through an asserted include patch while fixing the shared terminal app to `ui.mode: tui`; TUI snapshots and PTY tests live with that leaf.
Each golden projects terminal state into text: dimensions, active-buffer and viewport coordinates, lifecycle and cursor state, rows, wrap markers, and non-default style ranges. Scroll-heavy cards capture the used buffer; overlays capture the visible viewport. Text and style remain separate so a reviewer can distinguish content changes from presentation changes without decoding ANSI bytes.
### Recorded-session replay
Every checkpoint also enforces theme independence across the complete terminal state: no RGB colors, no palette entries beyond ANSI 0–15, and no explicit background colors. Reverse video remains valid for selection because it uses terminal defaults. The suite owns a closed checkpoint list: its type rejects undeclared names, and its inventory checks reject missing checkpoints and orphaned `.golden.txt` files.
Each example-level scenario directory owns `session.jsonl`, optional child logs `session.<n>.jsonl`, and `terminal.golden.txt`. The primary log supplies user-authored `user/message` prompts and the recorded `assistant/chunk` sequence. `dsh-llm-replay` derives one model-call script per session, binds child logs to fresh child sessions, and is the only mocked boundary. The agent loop, bash and filesystem implementations, Code Mode worker, subagent provider, workflow worker, Cordis tools, presenters, and TUI are production implementations.
The suite rejects a journey when its tool-call sequence differs, an expected event count is missing, a tool result is an error, a turn ends in error, a workflow lifecycle is incomplete, or the live child-session count differs from the fixture set. These assertions prevent an attractive terminal golden from hiding a failed or bypassed production path.
The live-model fixtures use `DSH_SNAPSHOT=record`; record mode rewrites their primary and child JSONL logs and terminal goldens. The deterministic Cordis toolchain keeps an authored complete JSONL script because reliably coercing a live model through five exact tool boundaries and two children is not a stable recording contract. `DSH_SNAPSHOT=refresh` replays every committed script keylessly and rewrites only derived terminal goldens. Plain replay compares without writing, and unknown mode values fail loud.
### Semantic terminal projection
The package-local `HeadlessTerminal` implements the same pi-tui `Terminal` interface as the process terminal and feeds every ANSI write into the pinned `@xterm/headless` parser. Snapshot code waits for synchronized frames to quiesce before reading state, so a checkpoint represents a completed screen rather than a timer-dependent write prefix.
Each golden projects dimensions, active-buffer and viewport coordinates, lifecycle and cursor state, rows, wrap markers, and non-default style ranges into text. Scroll-heavy cards capture the used buffer; overlays capture the visible viewport. Text and style remain separate so a reviewer can distinguish content changes from presentation changes without decoding ANSI bytes.
Every checkpoint enforces theme independence across the complete terminal state: no RGB colors, no palette entries beyond ANSI 0–15, and no explicit background colors. Reverse video remains valid for selection because it uses terminal defaults. Both suites own closed inventories that reject missing scenarios, missing checkpoints, and orphaned golden files.
### Required scenario matrix
| Area | Representative checkpoints | Contract pinned |
| Layer | Scenario | Contract pinned |
|---|---|---|
| Conversation | replay, streaming, completion | Resumed Markdown and reasoning, live deltas, plans, token usage, and max-token completion |
| Code Mode | pending and completed `run_code` | The production Code Mode registry and presenter, source program, captured logs, and result |
| Dynamic workflows | pending and completed `workflow` | The production workflow presenter, metadata, phases, parallel agents, script, and structured result |
| Cordis tools | pending and completed inspect/mount/unmount | The production `cordis_inspect`, `cordis_mount`, and `cordis_unmount` presenters and lifecycle results |
| Advanced tool cards | collapsed and expanded | Terminal, diff, generic, subagent, background-task, and skill card shapes plus output truncation |
| Interaction | question and validation | Constrained multi-select overlay composition, focus, scrolling, selection, and validation errors |
| Surface and layout | before compaction, narrow replacement, wide replacement | Surface replacement removes retired content; resize reflows the surviving surface without resurrection |
| Failure and shutdown | errors/help and disposed terminal | Help and unknown commands, live/turn error de-duplication, interruption, cursor restoration, and terminal stop |
The explicitly model-facing advanced cases use the real `ToolRegistry` configuration and the production Code Mode, workflow, and Cordis tool presenters. Synthetic presenter fixtures are limited to the generic card-shape matrix, where the TUI's input contract is the presenter view itself. Session events remain the driver so replay, streaming, result arrival, surface replacement, and lifecycle ordering exercise the same projection path as production.
The TUI suite is included by `vitest.snapshot.config.ts`, so `pnpm run test:snapshot` compares it keylessly. `pnpm run test:snapshot:refresh` rewrites its derived terminal goldens without contacting a model; `test:snapshot:record` remains meaningful for suites whose transcript source requires recording. Both refresh paths still compare the resulting files in the same run.
| Recorded journey | Multi-turn conversation | Recorded reasoning/text chunks, two input turns, retained history, token totals, and idle editor state |
| Recorded journey | Todo plan | Real `todo_write` execution, result card, and persistent plan rendering |
| Recorded journey | Bash terminal card | Real local executor output, description, exit status, and completed terminal card |
| Recorded journey | Parallel filesystem reads | Two calls from one assistant message, real file contents, ordering, and separate completed cards |
| Recorded journey | Code Mode | Real `run_code` worker execution, two `tool/code-dispatch` events, captured program output, and completed card |
| Recorded journey | Dynamic workflow | Real workflow worker, phase lifecycle, replayed child session, structured return value, and completed card |
| Recorded journey | Cordis dynamic toolchain | Real mount, Code Mode inspect, direct subagent, workflow child, unmount, and all production presenters |
| Transient state | Streaming and pending advanced calls | In-flight reasoning/text plus pending Code Mode, workflow, and Cordis cards that disappear from completed logs |
| Transient state | Cards, interaction, layout, failure, and shutdown | Collapsed/expanded card families, question validation, compaction replacement, resize reflow, help/errors, cursor restoration, and terminal stop |
## Alternatives considered
- **Snapshot raw terminal writes** — rejected because differential rendering may change write boundaries without changing the screen, while cursor and clear sequences are unreadable in review.
- **Snapshot component render lines before terminal output** — rejected because it does not test ANSI parsing, cursor movement, overlays, viewport behavior, or the interaction between independent components in one frame.
- **Snapshot component render lines before terminal output** — rejected because it does not test ANSI parsing, cursor movement, overlays, viewport behavior, or independent components in one frame.
- **Build every completed flow by appending session events** — rejected because a hand-authored event sequence can drift from the agent loop, tool execution, child-session binding, or worker behavior while its presentation test stays green. Direct event construction remains limited to transient renderer states.
- **Reuse ACP stdout goldens as the TUI oracle** — rejected because a recorded model journey is transport-neutral but its presentation is not. TUI scenarios own terminal goldens while using the same JSONL replay vocabulary.
- **Commit raster screenshots** — rejected because fonts, glyph metrics, antialiasing, and host terminal themes make them platform-sensitive and make semantic style changes difficult to review.
- **Use only PTY end-to-end tests** — rejected because raw PTY output is a stream of historical drawing operations, not queryable final state. PTY tests retain the real Loader/input/teardown boundary, while the emulator owns broad state coverage.
- **Copy pi-tui's unpublished virtual-terminal test helper** — rejected because the installed package does not export that helper. A small adapter around the public `@xterm/headless` API keeps the dependency explicit and the projection owned by this package.
## Consequences
-TUI visual regressions produce readable cell-and-style diffs, and the required matrix makes advanced features first-class rather than incidental coverage.
- The test dependency is pinned to the xterm version used by pi-tui. The adapter uses xterm's proposed buffer API, so an xterm upgrade requires rerunning and reviewing the semantic projection.
- The emulator models ANSI terminal state but cannot prove behavior unique to every terminal implementation. The real PTY conversation covers process selection, keyboard input, userinteraction, and teardown without duplicating the full matrix.
- Goldens deliberately encode wrapping and viewport behavior at fixed sizes. Intentional layout changes update them through the keyless refresh command and receive ordinary snapshot review.
-Completed advanced snapshots now fail when the real Code Mode, workflow, subagent, filesystem, bash, or Cordis path breaks, rather than accepting a fabricated result event.
- TUI visual regressions produce readable cell-and-style diffs, while JSONL fixtures retain the exact model chunks that made the production path execute.
- The emulator uses xterm's proposed buffer API. An xterm upgrade requires rerunning and reviewing the semantic projection; terminal-specific behavior still needs the PTY smoke.
- Goldens deliberately encode wrapping and viewport behavior at fixed sizes. Intentional layout changes use keyless refresh, while model-journey changes use record mode and review both JSONL and terminal diffs.
@@ -7,7 +7,7 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
- **Unit** (`pnpm run test`): vitest over `packages|examples/*/tests/**/*.spec.ts`, colocated with what they test. Every registry gets an HMR-safety test (dispose the contributing fiber, assert cleanup). Prefer edge cases, error paths, event ordering, concurrency races, and permanent contract regressions (see `packages/core/agent-loop/tests/contract-regressions.spec.ts`).
- **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped.
- **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e RFC](rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md)).
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless goldens cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized stdout plus the re-persisted log ([ACP snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)); the TUI suite parses real ANSI output into semantic terminal-state goldens and retains a real PTY conversation at the process boundary ([TUI snapshot RFC](rfc/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when the committed transcript or scripted events remain correct; review every golden diff. System-prompt/tool-schema content is pinned by ONE ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless goldens cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized stdout plus the re-persisted log ([ACP snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)). TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state goldens; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot RFC](rfc/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and golden diff. System-prompt/tool-schema content is pinned by ONE ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
## The with-key policy: inference is cheap here
@@ -35,4 +35,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
## When a snapshot test is required
Any change affecting an editor-facing transcript or end-to-end agent UX adds or updates a scenario in the owning snapshot suite, or states in the PR why none applies. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); interactive-terminal presentation uses the semantic TUI matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise.
Any change affecting an editor-facing transcript or end-to-end agent UX adds or updates a scenario in the owning snapshot suite, or states in the PR why none applies. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary). Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise.
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.