Files

Tianyi Cui 27f5f84e3b docs: address Codex review of the RFC reorg

- Fix two root-AGENTS.md cross-links that the depth bump left pointing at the
  new docs/AGENTS.md instead of the root file they cite (capability-seams,
  optional-code-mode). These resolved on disk so verify-md-links passed — the
  gate checks existence, not which file you meant; corrected to ../../../.
- Broaden verify-md-links scope to .agents/skills/**/*.md: this PR rewrote the
  dsh-code-review skill's links into the RFC tree, but the skill dir was outside
  the gate, so a broken skill link would have passed silently.
- Percent-decode the path component before the existence check, so a valid
  encoded relative target (My%20File.md) is not falsely reported broken; a
  malformed escape (%zz) is reported broken rather than crashing the gate.
- Drop the merged property-testing RFC's "nightly CI job 100x" claim: that line
  came from the original proposal, not the accepted decision, and CI has only
  push/pull_request triggers — note it as possible future work instead.

doc-sync (incl. verify-md-links over 58 files), doc-typecheck, lint pass.

2026-06-18 02:41:19 +08:00

3.3 KiB

Raw Blame History

RFC: Property-based testing for protocol-shaped code

Status: implemented (proposed 2026-06-11, accepted 2026-06-14)

Merges the original proposal and the decision record for one topic. It found a real BlockAssembler duplicate-block-end bug on first run.

Context

Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion, inbox scheduling — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for. The motivating evidence: a streamBlocks ordering bug once survived 100% line coverage of the happy paths. Per-file 100% coverage proves every line ran, not that every interleaving is correct.

Decision

Adopt fast-check (a root devDependency) with one tests/properties.spec.ts per protocol-shaped package, generators tuned for realistic-but-adversarial inputs (not uniform noise) and numRuns kept so the suite stays well under ~10s locally. Failures print a reproducible seed. (The original proposal also sketched a nightly CI job running 100× the iterations; that was not shipped — the property suite runs only in the normal push/pull_request CI, and a scheduled high-iteration job remains possible future work.)

dsh-llm / BlockAssembler: arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: flushReady()+flushRemaining() ≡ blocks() in order; the streamed prefix is always a prefix of final blocks(); partial count ≤ distinct indices; re-assembly idempotent.
dsh-session: arbitrary event logs. Invariants: deriveMessages deterministic; replay-from-seed identical; seq strictly monotonic; non-message events never affect derived history; derived content is decoupled from the log.
dsh-tools: arbitrary SchemaSpec. Invariants: JSON Schema required equals the required:true keys at every level; conversion total; and the composition with runtime arg validation — generated args satisfying a spec pass validateArgs, and targeted corruptions (dropped required key, non-object top level) are rejected. This closes the validator/InferArgs drift risk.
dsh-agent-loop: arbitrary send schedules against a never-exhausting adapter, driven through the agent/status settle signal (no wall-clock sleeps). Invariants: no message lost; turn numbers strictly increase; status transitions stay on the legal machine.

Consequences

Generator quality is the value lever — the generators bias toward small index pools and short strings so collisions and interleavings are common.
It already paid off: the BlockAssembler stream found a real bug — a duplicate block-end at the same index overwrote an already-flushed block, so the streamed prefix disagreed with final blocks(). Fixed (first close wins, matching the existing straggler rule) with a dedicated regression test.
A property flake from a timeout is a finding, not something to retry away. The loop properties are deterministic by construction (settle on agent/status), so a hang is a real defect.
Property tests supplement, not replace, the example tests that pin specific branches for the 100%-coverage gate.

3.3 KiB Raw Blame History Unescape Escape

RFC: Property-based testing for protocol-shaped code

Context

Decision

Consequences

3.3 KiB

Raw Blame History