test(web): stop the stats line's wall-clock segments from deciding a golden

`StatsLine` renders its LLM, tool-call, and throughput segments only while the
matching measurement exceeds zero, and all three are wall clock taken during
the replay. A machine that finishes a step inside one millisecond drops the
segment a slower one keeps, so a golden recorded what the recording machine's
speed was rather than what the page shows. Goldens across this suite already
disagreed about the LLM segment for that reason, and CI failed on whichever
test landed on a slow enough runner — a different test each run, always the
same one-line difference.

Tokenizing the values was never enough, because presence is what moves. The
normalizer now drops those segments outright, each taking one adjacent
separator so nothing is left holding a dangling separator or a doubled space,
and the recorded goldens are normalized the same way. `TTFT avg` stays: it
gates on a step count the fixture determines.
This commit is contained in:
Yichen Jiang
2026-08-06 15:47:42 +08:00
parent dd36db5434
commit 3e1c63b2ea
16 changed files with 48 additions and 30 deletions

View File

@@ -30,4 +30,4 @@
- text: Select model
- img
- button "Send message" [disabled]
- text: 1 turns · 1 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 0% Input 10 tok · Output 10 tok
- text: 1 turns · 1 steps TTFT avg {{duration}} Cache hit 0% Input 10 tok · Output 10 tok

View File

@@ -36,7 +36,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -46,4 +46,4 @@
- img
- button "7% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 52% Input 17.2K tok · Output 252 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 52% Input 17.2K tok · Output 252 tok

View File

@@ -51,7 +51,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -61,4 +61,4 @@
- img
- button "13% of context used"
- button "Send message" [disabled]
- text: 1 turns · 4 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 77% Input 66.5K tok · Output 312 tok
- text: 1 turns · 4 steps TTFT avg {{duration}} Cache hit 77% Input 66.5K tok · Output 312 tok

View File

@@ -31,7 +31,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -41,4 +41,4 @@
- img
- button "6% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 99% Input 15.7K tok · Output 111 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 99% Input 15.7K tok · Output 111 tok

View File

@@ -23,7 +23,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -33,4 +33,4 @@
- img
- button "6% of context used"
- button "Send message" [disabled]
- text: 1 turns · 1 steps LLM {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 99% Input 7.8K tok · Output 21 tok
- text: 1 turns · 1 steps TTFT avg {{duration}} Cache hit 99% Input 7.8K tok · Output 21 tok

View File

@@ -25,7 +25,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -35,4 +35,4 @@
- img
- button "6% of context used"
- button "Send message" [disabled]
- text: 1 turns · 1 steps LLM {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 99% Input 7.8K tok · Output 79 tok
- text: 1 turns · 1 steps TTFT avg {{duration}} Cache hit 99% Input 7.8K tok · Output 79 tok

View File

@@ -44,4 +44,4 @@
- text: Select model
- img
- button "Send message" [disabled]
- text: 1 turns · 1 steps LLM {{duration}} Input 0 tok · Output 0 tok
- text: 1 turns · 1 steps Input 0 tok · Output 0 tok

View File

@@ -20,7 +20,7 @@
- img
- button "Branch into a new conversation" [disabled]:
- img
- text: Available only on the last message of a completed turn 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: Available only on the last message of a completed turn 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}}
- button "Read a.txt":
- img
- img
@@ -55,4 +55,4 @@
- text: Select model
- img
- button "Send message" [disabled]
- text: 2 turns · 3 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 98% Input 7.8K tok · Output 103 tok
- text: 2 turns · 3 steps TTFT avg {{duration}} Cache hit 98% Input 7.8K tok · Output 103 tok

View File

@@ -36,7 +36,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -46,4 +46,4 @@
- img
- button "4% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 51% Input 10.2K tok · Output 346 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 51% Input 10.2K tok · Output 346 tok

View File

@@ -31,7 +31,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -41,4 +41,4 @@
- img
- button "3% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 95% Input 8.6K tok · Output 180 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 95% Input 8.6K tok · Output 180 tok

View File

@@ -33,7 +33,7 @@
- img
- button "Branch into a new conversation":
- img
- text: 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}}
- button "Context compacted View compaction summary":
- img
- text: Context compacted View compaction summary
@@ -51,4 +51,4 @@
- text: Select model
- img
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 98% Input 15.8K tok · Output 135 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 98% Input 15.8K tok · Output 135 tok

View File

@@ -33,7 +33,7 @@
- img
- button "Branch into a new conversation":
- img
- text: 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}}
- button "Context compacted View compaction summary":
- img
- text: Context compacted View compaction summary
@@ -49,4 +49,4 @@
- text: Select model
- img
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 98% Input 15.8K tok · Output 135 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 98% Input 15.8K tok · Output 135 tok

View File

@@ -37,7 +37,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -47,4 +47,4 @@
- img
- button "6% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 98% Input 15.8K tok · Output 156 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 98% Input 15.8K tok · Output 156 tok

View File

@@ -28,7 +28,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s Now give the same explanation to a human reader. {{clock}}
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} Now give the same explanation to a human reader. {{clock}}
- button "Copy":
- img
- button "Branch into a new conversation" [disabled]:
@@ -43,11 +43,11 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
- button "6% of context used"
- button "Send message" [disabled]
- text: 2 turns · 2 steps LLM {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 99% Input 15.6K tok · Output 158 tok
- text: 2 turns · 2 steps TTFT avg {{duration}} Cache hit 99% Input 15.6K tok · Output 158 tok

View File

@@ -23,7 +23,7 @@
- img
- button "Branch into a new conversation":
- img
- text: {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- text: {{clock}} Ran for {{duration}} TTFT {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
@@ -33,4 +33,4 @@
- img
- button "0% of context used"
- button "Send message" [disabled]
- text: 1 turns · 2 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 0% Input 22 tok · Output 7 tok
- text: 1 turns · 2 steps TTFT avg {{duration}} Cache hit 0% Input 22 tok · Output 7 tok