feat(desktop): trace signal detection + tri-view badges
Adds a small signal-detection layer over trace payloads and surfaces
three kinds of semantic signals inline in the trace tri-view (Timeline
/ Graph / turn footer chips) so a researcher can spot repetitive
tool-call loops, redundant identical calls, and plan-completion beats
without hand-scanning the raw log:
- loop-detected — three or more repeats of the same tool.method
signature inside a window, badge in Timeline gutter
+ colored ring on the Graph node
- redundant-call — identical (tool, arg-hash) call twice without any
state change in between, badge + subdued styling
so it stays advisory not alarmist
- plan-* chips — plan/subplan step-complete beats surfaced above the
assistant body in the turn footer, wired through
the finishTurnContainer tail
Pure rendering; no wire-format changes needed on the runtime side —
signal detection is fixture-driven off the same payload the tri-view
already consumes. Companion RFC L-2 in docs/upstream-ledger.md asks
the runtime to emit these signals natively rather than deriving them
in the shell, so once L-2 lands the shell will consume upstream signals
and this detector becomes a fallback.
Files:
src/renderer/trace-signal-detect.js new (loop/redundant/plan)
src/renderer/trace-tri-view.js +24 (wire detector output)
src/renderer/trace-timeline.js +39 (badge in gutter)
src/renderer/trace-graph.js +30 (colored ring)
src/renderer/renderer.js +81 (finishTurnContainer chips)
src/renderer/index.html +1 (one script tag)
src/renderer/style.css +58 (badge/ring/chip rules)
docs/upstream-ledger.md +121 (append L-2 RFC)
scripts/qa-trace-signals-fixture.mjs new (headless SVG proof)
scripts/qa-trace-signals-shoot.mjs new (CDP live shoot)
docs/trace-signals-shoot/*.html new (3 fixture-driven shots)
test/trace-signal-detect.test.js new (detector unit tests)
test/trace-signal-overlay.test.js new (renderer overlay tests)
Test suite: 1668/1668 pass (17 new). Fixture-driven signals-01-timeline-
loop.html regenerated byte-identical from the merged HEAD.
This commit is contained in:
ZiyaZhang
@@ -0,0 +1,50 @@
|
||||
<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>Signals 01 — Timeline: loop-detected badge</title>
|
||||
<style>
|
||||
body { font: 13px system-ui, sans-serif; margin: 24px; background: #f9fafb; color: #1d1d1f; }
|
||||
h1 { font-size: 15px; margin: 0 0 8px 0; }
|
||||
.note { color: #6b6b70; font-size: 12px; margin: 0 0 16px 0; max-width: 720px; line-height: 1.4; }
|
||||
.stage { background: #fff; padding: 16px; border: 1px solid #e5e5ea; border-radius: 8px; overflow-x: auto; }
|
||||
/* Palette matches src/renderer/style.css tail block for signal badges/rings/chips. */
|
||||
.trace-timeline-svg { font: 11px system-ui, sans-serif; }
|
||||
.trace-timeline-tick { stroke: #d0d0d5; stroke-width: 0.5; stroke-dasharray: 2 3; }
|
||||
.trace-timeline-tick-label { fill: #6b6b70; font-size: 10px; }
|
||||
.trace-timeline-label { fill: #1d1d1f; font-size: 11px; }
|
||||
.trace-timeline-bar.family-step { fill: #b0b4bd; }
|
||||
.trace-timeline-bar.family-llm { fill: #7c3aed; }
|
||||
.trace-timeline-bar.family-tool { fill: #b45309; }
|
||||
.trace-timeline-bar.family-input { fill: #6b6b70; }
|
||||
.trace-timeline-signal-badge { stroke: rgba(0,0,0,0.15); stroke-width: 0.75; }
|
||||
.trace-timeline-signal-badge.sig-error { fill: #dc2626; }
|
||||
.trace-timeline-signal-badge.sig-loop { fill: #ef4444; }
|
||||
.trace-timeline-signal-badge.sig-redundant{ fill: #f59e0b; }
|
||||
.trace-timeline-signal-badge.sig-plan { fill: #2563eb; }
|
||||
.trace-graph-svg { background: #fff; }
|
||||
.trace-graph-node-body { fill: #f3f4f6; stroke: #9ca3af; stroke-width: 1; }
|
||||
.trace-graph-node-body.family-tool { fill: #fde68a; stroke: #b45309; }
|
||||
.trace-graph-node-body.family-step { fill: #e5e7eb; stroke: #6b7280; }
|
||||
.trace-graph-node-body.family-llm { fill: #ddd6fe; stroke: #7c3aed; }
|
||||
.trace-graph-node-glyph { font: 12px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-node-label { font: 10px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-edge { stroke: #9ca3af; stroke-width: 1.2; }
|
||||
.trace-graph-signal-ring { stroke-width: 2.5; fill: none; }
|
||||
.trace-graph-signal-ring.sig-error { stroke: #dc2626; }
|
||||
.trace-graph-signal-ring.sig-loop { stroke: #ef4444; }
|
||||
.trace-graph-signal-ring.sig-plan { stroke: #2563eb; }
|
||||
.assistant-turn { border-left: 3px solid #d0d0d5; padding: 12px; margin: 12px 0; background: #fff; border-radius: 6px; }
|
||||
.turn-body { display: flex; flex-direction: column; gap: 6px; }
|
||||
.text-block { font-size: 13px; color: #1d1d1f; }
|
||||
.tool-row { font-family: ui-monospace, monospace; font-size: 12px; color: #4b5563; }
|
||||
.turn-glyph { display: inline-block; width: 14px; }
|
||||
.turn-signal-chip-row { display: flex; flex-wrap: wrap; gap: 6px; padding: 4px 0 6px 0; align-items: center; }
|
||||
.turn-signal-chip { font-size: 11px; line-height: 1; padding: 3px 8px; border-radius: 10px; border: 1px solid transparent; background: rgba(0,0,0,0.03); color: #1d1d1f; cursor: pointer; font-family: inherit; }
|
||||
.turn-signal-chip::before { content: ''; display: inline-block; width: 6px; height: 6px; border-radius: 50%; margin-right: 6px; vertical-align: 1px; background: currentColor; }
|
||||
.turn-signal-chip.sig-error, .turn-signal-chip.sig-loop { color: #b91c1c; border-color: rgba(239,68,68,0.35); background: rgba(239,68,68,0.06); }
|
||||
.turn-signal-chip.sig-redundant { color: #92400e; border-color: rgba(245,158,11,0.4); background: rgba(245,158,11,0.08); }
|
||||
.turn-signal-chip.sig-plan { color: #1d4ed8; border-color: rgba(37,99,235,0.35); background: rgba(37,99,235,0.06); }
|
||||
</style>
|
||||
</head><body>
|
||||
<h1>Signals 01 — Timeline: loop-detected badge</h1>
|
||||
<p class="note">Three consecutive <code>fs.read</code> calls with identical args. The detector fires <code>loop-detected</code> on the third call; its badge sits in the left gutter, colored red. Signals detected: <code>loop-detected</code>. See docs/upstream-ledger.md L-2.</p>
|
||||
<div class="stage"><div data-total-ms="500" class="trace-timeline"><svg viewBox="0 0 820 110" width="820" height="110" role="img" aria-label="Timeline of 4 rows over 500ms" class="trace-timeline-svg"><g class="trace-timeline-axis"><line x1="220" y1="22" x2="220" y2="106" class="trace-timeline-tick"></line><text x="223" y="12" class="trace-timeline-tick-label">0ms</text><line x1="367" y1="22" x2="367" y2="106" class="trace-timeline-tick"></line><text x="370" y="12" class="trace-timeline-tick-label">125ms</text><line x1="514" y1="22" x2="514" y2="106" class="trace-timeline-tick"></line><text x="517" y="12" class="trace-timeline-tick-label">250ms</text><line x1="661" y1="22" x2="661" y2="106" class="trace-timeline-tick"></line><text x="664" y="12" class="trace-timeline-tick-label">375ms</text><line x1="808" y1="22" x2="808" y2="106" class="trace-timeline-tick"></line><text x="811" y="12" class="trace-timeline-tick-label">500ms</text></g><g data-seq="10" data-kind="step" class="trace-timeline-row family-step depth-0"><text x="8" y="38.96" class="trace-timeline-label"><title>step 1.0 — read main.ts · 500ms</title></text><rect x="220" y="30" width="588" height="10" rx="2" ry="2" class="trace-timeline-bar family-step kind-step"><title>step 1.0 — read main.ts · 500ms · 500ms</title></rect></g><g data-seq="11" data-kind="event" class="trace-timeline-row family-tool depth-1"><text x="20" y="56.96" class="trace-timeline-label"><title>tool: fs.read</title></text><rect x="278.8" y="48" width="35.27999999999997" height="10" rx="2" ry="2" class="trace-timeline-bar family-tool kind-event"><title>tool: fs.read · 30ms</title></rect></g><g data-seq="12" data-kind="event" class="trace-timeline-row family-tool depth-1"><text x="20" y="74.96" class="trace-timeline-label"><title>tool: fs.read</title></text><rect x="396.4" y="66" width="35.27999999999997" height="10" rx="2" ry="2" class="trace-timeline-bar family-tool kind-event"><title>tool: fs.read · 30ms</title></rect></g><g data-seq="13" data-kind="event" class="trace-timeline-row family-tool depth-1"><text x="20" y="92.96" class="trace-timeline-label"><title>tool: fs.read</title></text><rect x="514" y="84" width="35.27999999999997" height="10" rx="2" ry="2" class="trace-timeline-bar family-tool kind-event"><title>tool: fs.read · 30ms</title></rect></g><g class="trace-timeline-badges"><circle cx="210" cy="89" r="4" class="trace-timeline-signal-badge sig-loop"><title>Loop detected · 3 × fs.read · prior seq 11, 12 · (heuristic)</title></circle></g></svg></div></div>
|
||||
</body></html>
|
||||
@@ -0,0 +1,50 @@
|
||||
<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>Signals 02 — Graph: tool-error ring</title>
|
||||
<style>
|
||||
body { font: 13px system-ui, sans-serif; margin: 24px; background: #f9fafb; color: #1d1d1f; }
|
||||
h1 { font-size: 15px; margin: 0 0 8px 0; }
|
||||
.note { color: #6b6b70; font-size: 12px; margin: 0 0 16px 0; max-width: 720px; line-height: 1.4; }
|
||||
.stage { background: #fff; padding: 16px; border: 1px solid #e5e5ea; border-radius: 8px; overflow-x: auto; }
|
||||
/* Palette matches src/renderer/style.css tail block for signal badges/rings/chips. */
|
||||
.trace-timeline-svg { font: 11px system-ui, sans-serif; }
|
||||
.trace-timeline-tick { stroke: #d0d0d5; stroke-width: 0.5; stroke-dasharray: 2 3; }
|
||||
.trace-timeline-tick-label { fill: #6b6b70; font-size: 10px; }
|
||||
.trace-timeline-label { fill: #1d1d1f; font-size: 11px; }
|
||||
.trace-timeline-bar.family-step { fill: #b0b4bd; }
|
||||
.trace-timeline-bar.family-llm { fill: #7c3aed; }
|
||||
.trace-timeline-bar.family-tool { fill: #b45309; }
|
||||
.trace-timeline-bar.family-input { fill: #6b6b70; }
|
||||
.trace-timeline-signal-badge { stroke: rgba(0,0,0,0.15); stroke-width: 0.75; }
|
||||
.trace-timeline-signal-badge.sig-error { fill: #dc2626; }
|
||||
.trace-timeline-signal-badge.sig-loop { fill: #ef4444; }
|
||||
.trace-timeline-signal-badge.sig-redundant{ fill: #f59e0b; }
|
||||
.trace-timeline-signal-badge.sig-plan { fill: #2563eb; }
|
||||
.trace-graph-svg { background: #fff; }
|
||||
.trace-graph-node-body { fill: #f3f4f6; stroke: #9ca3af; stroke-width: 1; }
|
||||
.trace-graph-node-body.family-tool { fill: #fde68a; stroke: #b45309; }
|
||||
.trace-graph-node-body.family-step { fill: #e5e7eb; stroke: #6b7280; }
|
||||
.trace-graph-node-body.family-llm { fill: #ddd6fe; stroke: #7c3aed; }
|
||||
.trace-graph-node-glyph { font: 12px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-node-label { font: 10px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-edge { stroke: #9ca3af; stroke-width: 1.2; }
|
||||
.trace-graph-signal-ring { stroke-width: 2.5; fill: none; }
|
||||
.trace-graph-signal-ring.sig-error { stroke: #dc2626; }
|
||||
.trace-graph-signal-ring.sig-loop { stroke: #ef4444; }
|
||||
.trace-graph-signal-ring.sig-plan { stroke: #2563eb; }
|
||||
.assistant-turn { border-left: 3px solid #d0d0d5; padding: 12px; margin: 12px 0; background: #fff; border-radius: 6px; }
|
||||
.turn-body { display: flex; flex-direction: column; gap: 6px; }
|
||||
.text-block { font-size: 13px; color: #1d1d1f; }
|
||||
.tool-row { font-family: ui-monospace, monospace; font-size: 12px; color: #4b5563; }
|
||||
.turn-glyph { display: inline-block; width: 14px; }
|
||||
.turn-signal-chip-row { display: flex; flex-wrap: wrap; gap: 6px; padding: 4px 0 6px 0; align-items: center; }
|
||||
.turn-signal-chip { font-size: 11px; line-height: 1; padding: 3px 8px; border-radius: 10px; border: 1px solid transparent; background: rgba(0,0,0,0.03); color: #1d1d1f; cursor: pointer; font-family: inherit; }
|
||||
.turn-signal-chip::before { content: ''; display: inline-block; width: 6px; height: 6px; border-radius: 50%; margin-right: 6px; vertical-align: 1px; background: currentColor; }
|
||||
.turn-signal-chip.sig-error, .turn-signal-chip.sig-loop { color: #b91c1c; border-color: rgba(239,68,68,0.35); background: rgba(239,68,68,0.06); }
|
||||
.turn-signal-chip.sig-redundant { color: #92400e; border-color: rgba(245,158,11,0.4); background: rgba(245,158,11,0.08); }
|
||||
.turn-signal-chip.sig-plan { color: #1d4ed8; border-color: rgba(37,99,235,0.35); background: rgba(37,99,235,0.06); }
|
||||
</style>
|
||||
</head><body>
|
||||
<h1>Signals 02 — Graph: tool-error ring</h1>
|
||||
<p class="note">A failing <code>bash</code> call followed by a successful retry. The detector emits <code>tool-error</code> on the failing call's seq and <code>plan-restart</code> on the retry. The Graph node for the failing call gets a red outer ring; the retry node gets a blue ring. Signals detected: <code>tool-error, tool-error, plan-restart</code>.</p>
|
||||
<div class="stage"><div data-layout-mode="bfs-layered-fallback" class="trace-graph"><svg viewBox="0 0 494 200" width="494" height="200" role="img" aria-label="Agent graph: 3 nodes, 2 edges" class="trace-graph-svg"><g class="trace-graph-edges"><path d="M44,44 C107,44 121,44 184,44" fill="none" class="trace-graph-edge kind-seq"></path><path d="M184,44 C247,44 261,44 324,44" fill="none" class="trace-graph-edge kind-seq"></path></g><g class="trace-graph-nodes"><g data-seq="10" transform="translate(44, 44)" class="trace-graph-node family-step status-ok"><circle cx="0" cy="0" r="14" class="trace-graph-node-body family-step"></circle><text x="0" y="4" text-anchor="middle" class="trace-graph-node-glyph">●</text><text x="0" y="28" text-anchor="middle" class="trace-graph-node-label">step 1.0</text><title>step 1.0 · step · 500ms · seq 10</title></g><g data-seq="11" transform="translate(184, 44)" class="trace-graph-node family-tool status-ok"><circle cx="0" cy="0" r="14" class="trace-graph-node-body family-tool"></circle><circle cx="0" cy="0" r="18" fill="none" class="trace-graph-signal-ring sig-error"><title>Tool error · bash: ENOENT</title></circle><text x="0" y="4" text-anchor="middle" class="trace-graph-node-glyph">▲</text><text x="0" y="28" text-anchor="middle" class="trace-graph-node-label">bash</text><title>bash · tool · 30ms · seq 11</title></g><g data-seq="13" transform="translate(324, 44)" class="trace-graph-node family-tool status-ok"><circle cx="0" cy="0" r="14" class="trace-graph-node-body family-tool"></circle><circle cx="0" cy="0" r="18" fill="none" class="trace-graph-signal-ring sig-plan-restart"><title>Plan restart · retry bash after seq 12 · (heuristic)</title></circle><text x="0" y="4" text-anchor="middle" class="trace-graph-node-glyph">▲</text><text x="0" y="28" text-anchor="middle" class="trace-graph-node-label">bash</text><title>bash · tool · 30ms · seq 13</title></g></g></svg></div></div>
|
||||
</body></html>
|
||||
@@ -0,0 +1,63 @@
|
||||
<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>Signals 03 — Turn container: signal chip row</title>
|
||||
<style>
|
||||
body { font: 13px system-ui, sans-serif; margin: 24px; background: #f9fafb; color: #1d1d1f; }
|
||||
h1 { font-size: 15px; margin: 0 0 8px 0; }
|
||||
.note { color: #6b6b70; font-size: 12px; margin: 0 0 16px 0; max-width: 720px; line-height: 1.4; }
|
||||
.stage { background: #fff; padding: 16px; border: 1px solid #e5e5ea; border-radius: 8px; overflow-x: auto; }
|
||||
/* Palette matches src/renderer/style.css tail block for signal badges/rings/chips. */
|
||||
.trace-timeline-svg { font: 11px system-ui, sans-serif; }
|
||||
.trace-timeline-tick { stroke: #d0d0d5; stroke-width: 0.5; stroke-dasharray: 2 3; }
|
||||
.trace-timeline-tick-label { fill: #6b6b70; font-size: 10px; }
|
||||
.trace-timeline-label { fill: #1d1d1f; font-size: 11px; }
|
||||
.trace-timeline-bar.family-step { fill: #b0b4bd; }
|
||||
.trace-timeline-bar.family-llm { fill: #7c3aed; }
|
||||
.trace-timeline-bar.family-tool { fill: #b45309; }
|
||||
.trace-timeline-bar.family-input { fill: #6b6b70; }
|
||||
.trace-timeline-signal-badge { stroke: rgba(0,0,0,0.15); stroke-width: 0.75; }
|
||||
.trace-timeline-signal-badge.sig-error { fill: #dc2626; }
|
||||
.trace-timeline-signal-badge.sig-loop { fill: #ef4444; }
|
||||
.trace-timeline-signal-badge.sig-redundant{ fill: #f59e0b; }
|
||||
.trace-timeline-signal-badge.sig-plan { fill: #2563eb; }
|
||||
.trace-graph-svg { background: #fff; }
|
||||
.trace-graph-node-body { fill: #f3f4f6; stroke: #9ca3af; stroke-width: 1; }
|
||||
.trace-graph-node-body.family-tool { fill: #fde68a; stroke: #b45309; }
|
||||
.trace-graph-node-body.family-step { fill: #e5e7eb; stroke: #6b7280; }
|
||||
.trace-graph-node-body.family-llm { fill: #ddd6fe; stroke: #7c3aed; }
|
||||
.trace-graph-node-glyph { font: 12px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-node-label { font: 10px system-ui, sans-serif; fill: #1d1d1f; }
|
||||
.trace-graph-edge { stroke: #9ca3af; stroke-width: 1.2; }
|
||||
.trace-graph-signal-ring { stroke-width: 2.5; fill: none; }
|
||||
.trace-graph-signal-ring.sig-error { stroke: #dc2626; }
|
||||
.trace-graph-signal-ring.sig-loop { stroke: #ef4444; }
|
||||
.trace-graph-signal-ring.sig-plan { stroke: #2563eb; }
|
||||
.assistant-turn { border-left: 3px solid #d0d0d5; padding: 12px; margin: 12px 0; background: #fff; border-radius: 6px; }
|
||||
.turn-body { display: flex; flex-direction: column; gap: 6px; }
|
||||
.text-block { font-size: 13px; color: #1d1d1f; }
|
||||
.tool-row { font-family: ui-monospace, monospace; font-size: 12px; color: #4b5563; }
|
||||
.turn-glyph { display: inline-block; width: 14px; }
|
||||
.turn-signal-chip-row { display: flex; flex-wrap: wrap; gap: 6px; padding: 4px 0 6px 0; align-items: center; }
|
||||
.turn-signal-chip { font-size: 11px; line-height: 1; padding: 3px 8px; border-radius: 10px; border: 1px solid transparent; background: rgba(0,0,0,0.03); color: #1d1d1f; cursor: pointer; font-family: inherit; }
|
||||
.turn-signal-chip::before { content: ''; display: inline-block; width: 6px; height: 6px; border-radius: 50%; margin-right: 6px; vertical-align: 1px; background: currentColor; }
|
||||
.turn-signal-chip.sig-error, .turn-signal-chip.sig-loop { color: #b91c1c; border-color: rgba(239,68,68,0.35); background: rgba(239,68,68,0.06); }
|
||||
.turn-signal-chip.sig-redundant { color: #92400e; border-color: rgba(245,158,11,0.4); background: rgba(245,158,11,0.08); }
|
||||
.turn-signal-chip.sig-plan { color: #1d4ed8; border-color: rgba(37,99,235,0.35); background: rgba(37,99,235,0.06); }
|
||||
</style>
|
||||
</head><body>
|
||||
<h1>Signals 03 — Turn container: signal chip row</h1>
|
||||
<p class="note">The chip row sits above the assistant body. Each chip covers one signal kind detected in this turn (loop / redundant / plan-update). Clicking a chip auto-opens the trace drawer for a drill-in. Signals detected in this turn: <code>plan-update, loop-detected, redundant-call</code>.</p>
|
||||
<div class="stage">
|
||||
<section class="assistant-turn">
|
||||
<div class="turn-body">
|
||||
<div class="turn-signal-chip-row"><button class="turn-signal-chip sig-plan" title="Plan update · “Here is the new plan: 1. read main.ts
|
||||
2. edit imports
|
||||
3. verify.” · (heuristic)">Plan update</button><button class="turn-signal-chip sig-loop" title="Loop detected · 3 × fs.read · prior seq 3, 4 · (heuristic)">Loop detected</button><button class="turn-signal-chip sig-redundant" title="Redundant call · fs.read, matches seq 5 · (heuristic)">Redundant call</button></div>
|
||||
<div class="text-block">Here is the new plan: 1. read main.ts, 2. edit imports, 3. verify.</div>
|
||||
<div class="tool-row"><span class="turn-glyph">▸</span>fs.read(main.ts)</div>
|
||||
<div class="tool-row"><span class="turn-glyph">▸</span>fs.read(main.ts)</div>
|
||||
<div class="tool-row"><span class="turn-glyph">▸</span>fs.read(main.ts)</div>
|
||||
<div class="tool-row"><span class="turn-glyph">▸</span>bash(ls)</div>
|
||||
<div class="tool-row"><span class="turn-glyph">▸</span>fs.read(main.ts)</div>
|
||||
</div>
|
||||
</section></div>
|
||||
</body></html>
|
||||
@@ -79,3 +79,124 @@ so there's no coupling between land order.
|
||||
Future entries append below. Keep the numbering monotone (L-2, L-3…) so a
|
||||
cross-repo reference like "see upstream ledger L-1" stays stable.
|
||||
-->
|
||||
|
||||
## L-2 Runtime should emit semantic trace signals (loop / redundant / plan-*)
|
||||
|
||||
**Symptom.** A researcher watching a session can't see at a glance where
|
||||
the interesting things happened: the agent got stuck in a tool-call loop,
|
||||
called the same tool with the same args twice within a few turns,
|
||||
mid-turn rewrote its plan, or restarted after a tool error. These are the
|
||||
first four things a debugger wants highlighted, and today the trace tri-view
|
||||
+ main assistant flow show every step at equal visual weight. The Tree
|
||||
column has an ✗ glyph for tool errors — that's the only pre-existing signal.
|
||||
|
||||
**Root cause.** The runtime wire has no dedicated "signal" event type. All
|
||||
diagnostic annotations that today's UI could show — loop detection,
|
||||
redundant-call detection, plan updates, plan restarts, ordinary tool errors
|
||||
raised to signal status — are inferable from the flat event stream but not
|
||||
themselves emitted. Concretely:
|
||||
|
||||
- `packages/core/agent-loop/src/loop.ts` — the loop sees every tool/call
|
||||
and every tool/result but never emits a `trace/signal` derived from them.
|
||||
- `packages/core/tools/src/index.ts` — tool descriptors don't declare
|
||||
loop/retry guardrails that could feed a signal emission.
|
||||
- `packages/core/planner/` (or the equivalent — grep for `plan` in
|
||||
`deepseek-harness-dev/packages/core/`) — plan updates are internal state,
|
||||
not observable on the wire.
|
||||
|
||||
The desktop renderer therefore has no signal to render.
|
||||
|
||||
**Local workaround.** New `src/renderer/trace-signal-detect.js` is a
|
||||
heuristic detector run over `meta.cachedEvents`. It emits five signal kinds:
|
||||
|
||||
- `loop-detected` — ≥ N consecutive same-tool + same-args-prefix calls (N=3)
|
||||
- `redundant-call` — same (name, args-prefix) reappearing within an 8-seq
|
||||
window with at least one different call in between
|
||||
- `plan-update` — assistant/message text matching "new/revised/updated
|
||||
plan", "here's the plan", or a two-line numbered-list intro
|
||||
- `plan-restart` — same tool re-invoked after a `tool/result` `ok:false`
|
||||
- `tool-error` — surfaced from the already-visible `ok:false` result, but
|
||||
also stamped on the matching call seq so the Graph node (which absorbs
|
||||
the result into the call) has a place to hang the badge
|
||||
|
||||
Signals are then rendered as:
|
||||
|
||||
- Timeline bars: colored dots to the left of each affected row
|
||||
(`.trace-timeline-signal-badge` in `style.css`)
|
||||
- Graph nodes: outer ring around the node
|
||||
(`.trace-graph-signal-ring`, highest-priority signal wins the color)
|
||||
- Assistant turn container: a chip row above the body
|
||||
(`.turn-signal-chip-row` / `.turn-signal-chip`), one chip per signal
|
||||
kind observed in that turn, clicking a chip auto-opens the trace drawer
|
||||
|
||||
The detector already special-cases wire-emitted signals: any event whose
|
||||
`type === 'trace/signal'` is passed through verbatim (marked `source: 'wire'`)
|
||||
and the heuristic scan skips seqs already covered by wire signals. This
|
||||
means the shape is forward-compatible: once upstream lands the fix, the
|
||||
runtime's signals win and the heuristic scan becomes dead code without
|
||||
requiring a renderer change.
|
||||
|
||||
Spot the workaround in code review by:
|
||||
|
||||
- `src/renderer/trace-signal-detect.js` — the entire file
|
||||
- `src/renderer/trace-timeline.js` — the `options.signals` badge loop
|
||||
(around the "Signal badges" comment)
|
||||
- `src/renderer/trace-graph.js` — the "Signal ring" block inside
|
||||
`renderGraph`'s node loop
|
||||
- `src/renderer/trace-tri-view.js` — the `_computeSignals(records)` call
|
||||
passing a `bySeq` map to both Timeline and Graph
|
||||
- `src/renderer/renderer.js` — `applyTurnSignalChips(sessionId, section)`
|
||||
called at the end of `finishTurnContainer`
|
||||
- `src/renderer/style.css` — the `.trace-timeline-signal-badge` /
|
||||
`.trace-graph-signal-ring` / `.turn-signal-chip*` blocks at the tail
|
||||
|
||||
**Upstream fix (needed).** Emit `trace/signal` events from the runtime with
|
||||
this shape:
|
||||
|
||||
```
|
||||
{
|
||||
type: 'trace/signal',
|
||||
seq: <number>, // seq the signal decorates (often the tool/call seq)
|
||||
time: <number>,
|
||||
data: {
|
||||
signal: 'loop-detected'|'redundant-call'|'plan-update'|'plan-restart'|'tool-error',
|
||||
// signal-specific fields; the renderer uses these for tooltip content:
|
||||
name?: string, // for tool-family signals: which tool
|
||||
argsKey?: string, // 80-char args prefix
|
||||
run?: number, // loop-detected: number of consecutive matches
|
||||
priorSeqs?: number[], // loop-detected: earlier calls in the run
|
||||
priorSeq?: number, // redundant-call / plan-restart pointer
|
||||
priorErrorSeq?: number,
|
||||
snippet?: string, // plan-update: first 80 chars of the plan text
|
||||
error?: string, // tool-error
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Suggested seams (grep upstream to confirm exact `packages/…` paths — the
|
||||
comments below cite the same file family as L-1):
|
||||
|
||||
1. **Loop/redundant detection** — add a small ring buffer of recent
|
||||
`tool/call` (name, argsKey) pairs inside `agent-loop`'s emit path. When
|
||||
the buffer trips the threshold, emit `trace/signal` before the offending
|
||||
`tool/call` reaches the wire so the signal precedes the call in seq order.
|
||||
Threshold defaults (loopN=3, window=8) can be config-flagged.
|
||||
2. **Plan updates** — plumbed from the planner / plan-summary side. Ideal
|
||||
shape: a `plan/updated` event upstream, with `trace-signal` derived from
|
||||
it. If the planner state is internal, at minimum emit the "assistant
|
||||
drafted a new plan" fact when it happens (the heuristic detector proves
|
||||
the text is recoverable, but the wire truth is upstream).
|
||||
3. **Plan restarts / tool errors** — pair `tool/result` `ok:false` with the
|
||||
next same-tool `tool/call` and emit the paired signal at emit time.
|
||||
|
||||
The detector-side dedup already covers the "signal already came from the
|
||||
wire" case, so upstream can ship these one at a time without a big-bang
|
||||
change: each signal kind lands, its heuristic branch becomes dead code, and
|
||||
eventually `trace-signal-detect.js` reduces to a pass-through for the wire
|
||||
signals.
|
||||
|
||||
Once all five signal kinds land upstream, the renderer-side detector
|
||||
becomes pure pass-through (roughly 20 lines) — a small doc-only PR at
|
||||
that point removes the heuristic scan entirely. Until then, the tri-view
|
||||
tabs read as "here's what the runtime is showing you" (wire signals) and
|
||||
"here's what the shell inferred" (heuristic, tooltipped as such).
|
||||
|
||||
Reference in New Issue
Block a user