feat(desktop): DSH Electron desktop shell — harness internals visualized

Minimal Electron shell over the DSH JSON-RPC runtime — a first-look at
what a ChatGPT.app-style host on top of the DeepSeek Harness looks
like, with the harness's normally-invisible internals (trace timeline,
context surface, subagent tree, compaction, plugin registry, rubrics)
brought forward as first-class UI surfaces so plugin authors and
researchers can see what the agent is actually doing.

Runs against three keyless-to-live profiles (stdio-echo works on
master out of the box; daemon-echo / daemon-vibe-echo activate once
the daemon-demo lands; stdio-deepseek and daemon-vibe hit the real
DeepSeek API when you supply a key). HARNESS_DEV auto-resolves to the
in-repo runtime when this shell ships under examples/desktop/, so a
fresh clone launches without config; env DSH_DEV_ROOT overrides for
custom layouts, and a sibling deepseek-harness-dev/ checkout is the
original dev workflow.

Cold-clone gate (P0 fixes for first-time-clone usability):
- HARNESS_DEV: 3-candidate resolver (env → walk-up in-repo marker →
  sibling), unit-tested via mock fs so ordering is locked without
  needing either real layout on disk.
- config yml leaves rewritten at assemble time so the sibling-clone
  paths (../../deepseek-harness-dev/examples/echo-agent/…) become
  the in-repo paths (../../echo-agent/…) in the released tree —
  source yml stays usable for local dev, released tree ships a
  working shape.
- pnpm-workspace.yaml allowBuilds.electron = true (was placeholder).
- missing-key card in stdio-deepseek offers a one-click switch to
  stdio-echo (the keyless profile that works on master) rather than
  daemon-echo (blocked on the not-yet-shipped daemon-demo).
- assemble-oss-release.sh rewrites the source-side breadcrumb name
  'dsh-desktop-demo' → 'dsh-desktop' for the released package.json.

FOUC guard on the onboarding gate (41fc5df carried) keeps the
first-launch splash from flashing before the runtime probe finishes.

Test suite (1634 tests in source, 3990 in the runtime repo) covers
resolver ordering, renderer classifiers, trace timeline shape,
compaction diff rendering, rubric parity, and the missing-key
onboarding paths.
This commit is contained in:
ZiyaZhang
2026-07-18 12:59:34 -07:00
parent 67053d1cf6
commit e8f5c0b51b
378 changed files with 104431 additions and 0 deletions

View File

@@ -0,0 +1,22 @@
---
name: bug-fix
group: fix-optimize
template: fixed
executor: llm-judge
version: v1
description: Evaluate a bug-fix trajectory — reproduction, minimality, tests, regressions, explanation.
---
## Checklist
- Reproduces the reported failure with a minimal case before patching
- Patch is minimal: no unrelated cleanups, formatting, or refactors
- Regression test added or existing test updated to cover the fix
- No obvious regressions in adjacent behavior (spot-check callers)
- Written explanation states the root cause, not just the symptom
## Notes
- Prefer patches that add a regression test in the same PR.
- Deduct when the model conflates symptom (crash) with cause (bad input).
- Deduct when the model rewrites more than the failing branch.

View File

@@ -0,0 +1,23 @@
---
name: code-review
group: se-process
template: code-review
executor: llm-judge
version: v1
description: Reviewer rubric — evaluates a code-review trajectory on style, correctness, and test coverage.
---
## Checklist
- Identifies incorrect behavior (bugs, edge cases) present in the diff
- Style feedback is specific to project conventions, not generic
- Suggests test coverage where the diff adds untested code paths
- Prioritizes correctness over style when both are present
- Tone is direct without being hostile; asks questions when uncertain
## Notes
- LLM-as-judge for now; the code-executor variant would run project
linters and compare their output against the review comments.
- Deduct when reviewer misses a real bug the diff introduces.
- Deduct when reviewer flags a style-only issue as correctness-critical.

View File

@@ -0,0 +1,25 @@
---
name: correctness-score
group: fix-optimize
template: fixed
executor: llm-judge
version: v1
description: Continuous 0-1 correctness score — real-valued continuous primitive.
---
## Dimensions
- correctness :: continuous :: 0-1 :: Correctness
## Checklist
- Overall correctness on the trajectory's stated goal (0 = wrong, 1 = correct)
## Notes
- Continuous primitive: judge model emits a real number in [0, 1].
- Export carries `dim_types.correctness = { type: 'continuous', min: 0, max: 1 }`
so downstream consumers can treat this as a probability rather than a
discrete class.
- Small integer ranges (like 1-5) render as button rows; the 0-1 range
renders as a numeric input for a real-valued score.

View File

@@ -0,0 +1,24 @@
---
name: intent-triage
group: interaction-reasoning
template: fixed
executor: llm-judge
version: v1
description: Categorical verdict on how well the model triaged user intent — enum categorical primitive.
---
## Dimensions
- verdict :: categorical :: bad,ok,good :: Triage verdict
## Checklist
- Categorical judgment on intent triage quality: bad · ok · good
## Notes
- Categorical primitive: judge emits one of {bad, ok, good}.
- Export preserves the enum text (`verdict: 'good'`) rather than mapping
to an integer index — matches LangSmith's Feedback tab where a
categorical value renders as its string label.
- Renders as a three-button pill row; keyboard 1/2/3 selects.

View File

@@ -0,0 +1,25 @@
---
name: multi-turn-feedback
group: interaction-reasoning
template: multi-turn
executor: llm-judge
version: v1
description: Score each assistant turn against the 5 fixed multi-turn dimensions; each dim 1-5 relative to the immediately preceding user feedback.
---
## Checklist
- Feedback understanding — the model correctly parsed the user's ask
- Fix effectiveness — the response actually addressed the previous feedback
- No regression — behavior that was already good stayed good
- Over-correction — the model changed only what the feedback asked for
- Convergence — the turn is moving toward a stable answer
## Notes
- One score per dimension per assistant turn, on a 1–5 scale.
- The pinned prior-user feedback is the anchor: every dim scores this turn
relative to that specific prior message, not the whole trajectory.
- Stage-2 of the RL plan uses this rubric alongside the stage-1 task rubric
— this one grades feedback response quality, the other grades base
task quality.

View File

@@ -0,0 +1,23 @@
---
name: passes-bench
group: se-process
template: fixed
executor: llm-judge
version: v1
description: Boolean pass/fail on a bench probe — two-state boolean primitive.
---
## Dimensions
- passes :: boolean :: pass/fail :: Passes bench probe
## Checklist
- Did the trajectory pass the associated bench probe?
## Notes
- Boolean primitive: two-state (pass / fail).
- Export carries `passes: true` or `passes: false`; downstream RL loop can
treat this as a hard label for reward-shaping.
- Renders as a two-button toggle; keyboard 1 = true, 2 = false.

View File

@@ -0,0 +1,24 @@
---
name: svg-generation
group: interaction-reasoning
template: per-prompt
executor: llm-judge
version: v1
description: Per-prompt rubric — judge model composes SVG-specific criteria before scoring, comparing to a reference image described in the prompt.
---
## Checklist
- The generated SVG renders without XML errors
- Shape count and composition match the prompt intent
- Colors and gradient use match the reference (or spec)
- Text elements (if any) match the requested strings
- Prompt-specific criteria (added by the judge model per task)
## Notes
- Per-prompt template: the judge model receives the prompt + reference image
description, then extends the fixed checklist with 2–4 prompt-specific
items before scoring. Output preserves the {resolved, score, reason}
block for DSBench compatibility.
- Deduct heavily when the SVG is a stub (empty viewBox, no shapes).