docs(rfc): classify RFCs by kind via path-encoded subdirectories

Add a second axis to every RFC — its class (feature, bug-fix,
simplification, architecture, process, testing) — encoded in the path
as docs/rfc/{lifecycle}/{class}/file.md. The folder is the label, so
the closed set is enforced by structure rather than a parsed field.

Two new doc-sync gates back it:
- verify-rfc-classification: every RFC sits in a valid class folder and
  the README index lists it under the matching lifecycle→class heading.
- verify-doc-refs: every docs/*.md path cited in a packages|examples TS
  comment resolves — closes a drift class verify-md-links can't see, and
  catches the four comment refs this reorg moved.

The README gains a Classification section explaining the taxonomy and
per-class index sub-sections. A self-referential process RFC records why
the scheme is path-encoded and gated.
This commit is contained in:
Tianyi Cui
2026-06-20 22:29:45 +08:00
parent 906705e353
commit 605587e79c
92 changed files with 673 additions and 293 deletions

View File

@@ -0,0 +1,25 @@
# RFC: Deterministic tests, the replay invariant fixture, and race stress
Status: proposed
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
## Problem
Several loop tests synchronize with `setTimeout(30)` sleeps — flakiness debt that wastes agent cycles on retries and can mask ordering bugs. Separately, our core architectural promise (any session log replays to identical derived history) is asserted in two tests but is cheap to assert *everywhere*. And the inbox wakeup race was verified by hand exactly once; nothing re-verifies it continuously.
## Proposal
Three measures:
1. **No wall-clock sleeps in tests.** Replace `setTimeout(N)` waits with event-driven waits (the existing `waitForIdle` pattern, extended to `waitForStatus`, `waitForEvent(n)`) or vitest fake timers where time itself is under test. Enforce with a lint rule banning `setTimeout` in `packages/*/tests` outside an allowlisted helper module.
2. **Universal replay fixture.** A shared test helper wraps the loop harness so that after every test, the agent's session log is replayed into a fresh Session and `deriveMessages()` equality is asserted automatically. The invariant then gets checked hundreds of times per CI run across every scenario the suite produces, not twice.
3. **Nightly race stress.** A CI job running the agent-loop and inbox suites with `vitest --repeat=200` (and `--shuffle`) to flush scheduling-dependent failures; any flake found is a bug to fix, never a retry.
## Plan
Land 1 and 2 together (they touch the same helpers); add the nightly job after the suite is sleep-free so repeats are fast.
## Risks
Fake timers interact subtly with Promise scheduling in the loop — prefer event-driven waits; reserve fake timers for timer-service behavior itself.

View File

@@ -0,0 +1,28 @@
# RFC: Mutation testing as the coverage counterweight
Status: proposed
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
## Problem
The per-file 100% coverage gate ([the quality-gates decision](../../implemented/process/2026-06-11-quality-gates.md)) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs.
## Proposal
Stryker (`@stryker-mutator/vitest-runner`) over `packages/*/src`:
- **PR-scoped incremental runs** (changed files only) as a CI job — fast enough to gate merges once tuned.
- **Nightly full runs** with a tracked mutation score; start by recording, then set the threshold at the observed baseline and ratchet upward (same policy as coverage: thresholds only ever tighten).
- Surviving mutants are work items: an agent picks a survivor, writes the killing test, repeats — a well-shaped autonomous loop.
- Equivalent mutants (provably behavior-preserving) get annotated exclusions with reasons, mirroring the `/* v8 ignore */` policy.
## Plan
1. Add Stryker config scoped to one package (llm — smallest, most algorithmic) and measure runtime.
2. Expand to all packages; record baseline scores in the config.
3. Wire the nightly job; add the incremental PR job once runtime is acceptable.
## Risks
Runtime: mutation testing is expensive; per-file 100% coverage helps (every mutant is at least reached). If PR-scoped runs stay too slow, keep them nightly-only and rely on the score ratchet.

View File

@@ -0,0 +1,31 @@
# RFC: Use `session.jsonl` as the only snapshot session-log artifact
Status: proposed
## Problem
Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.golden.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.golden.jsonl`. In the current fixtures, the normalized recorded log and normalized golden are identical for the ordinary recorded scenarios.
Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.golden.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string, "status"?: number }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario.
## Proposal
Remove the `session.golden.jsonl` concept entirely. Every scenario has at most one committed session-log artifact, `session.jsonl`:
- For recorded scenarios, `session.jsonl` remains the raw harvested log. Replay still derives model chunks from it, and the snapshot test compares the replay run's normalized persisted log against normalized `session.jsonl`.
- For authored override scenarios, `replay.override.json` drives model behavior and `session.jsonl` holds the expected produced session log. The replay adapter ignores the fixture for model chunks when the override exists, so the same file can be the expected log without affecting replay behavior.
- For no-model scenarios, `session.jsonl` can stay as the minimal fixture needed to boot `llm-replay`; no session-log comparison is needed unless the scenario creates a persisted session.
Stdout goldens remain unchanged; they are the editor-facing projection and are not redundant with the session fixture.
## Acceptance criteria
- `session.golden.jsonl` disappears from the snapshot harness, fixtures, orphan guards, and docs.
- The snapshot test derives the expected session log from `session.jsonl` for every model scenario.
- Authored sidecar scenarios commit their expected produced log in `session.jsonl`; `replay.override.json` remains the model-behavior override.
- Orphan-fixture guards understand which files are required by scenario kind.
- The [ACP snapshot tests RFC](../../implemented/testing/2026-06-19-acp-snapshot-tests.md) is updated to describe the reduced fixture set.
## What we give up
Reviewers lose one artifact name that made the expected persisted log visually separate from the replay fixture. The stdout golden still protects the editor transcript, and comparing replay output to `session.jsonl` preserves the loop/persistence regression check without duplicating files.