docs: replace vague provenance prose with recorded facts

This commit is contained in:
Turtle
2026-08-09 15:35:02 +08:00
parent 8c124f84b6
commit 9704749b01
380 changed files with 946 additions and 874 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/proposed/process/2026-07-13-human-review-skill-maintenance.md
2026-07-13-human-review-skill-maintenance.md: 76391bd110b7a86b194a9340ffcf7cc4602d1d2e
2026-07-13-human-review-skill-maintenance.zh.md: 8ebff759cf29fa7773e4b883f83c231f88e153d2
2026-07-13-human-review-skill-maintenance.md: b741c8b595fcd36613cb246af483863d1e056697
2026-07-13-human-review-skill-maintenance.zh.md: 1c00577fdb08c30b1fff794c08da20ec38891de1

View File

@@ -10,13 +10,13 @@ The `dsh-code-review` skill records failure modes that require reviewer judgment
## Proposal
Periodic out-of-repo maintenance. A private tool, kept on the skill maintainer's machine rather than committed to this repository, runs against a clean full-history checkout at refreshed `origin/master`. The intended scheduler runs daily with a two-UTC-day overlap; manual runs accept another `--since` duration or repeated `--pr` arguments for an explicit set. The scan is idempotent against the current skill and stores no repository cursor. The only repository file changed by promotion is [.agents/skills/dsh-code-review/SKILL.md](../../../skills/dsh-code-review/SKILL.md); the draft PR carries a provenance summary so reviewers can audit the source feedback and adoption evidence without the private adapter logs.
Periodic out-of-repo maintenance. A private tool, kept on the skill maintainer's machine rather than committed to this repository, runs against a clean full-history checkout at refreshed `origin/master`. The intended scheduler runs daily with a two-UTC-day overlap; manual runs accept another `--since` duration or repeated `--pr` arguments for an explicit set. The scan is idempotent against the current skill and stores no repository cursor. The only repository file changed by promotion is [.agents/skills/dsh-code-review/SKILL.md](../../../skills/dsh-code-review/SKILL.md); the draft PR lists the source feedback URLs or IDs and adoption evidence without exposing the private adapter logs.
```mermaid
flowchart TD
A["Maintainer or scheduler runs the tool on origin/master"] --> B["List PRs merged in the overlap window"]
B --> C["Collect pre-merge User feedback and final PR evidence"]
C --> D["Two reviewers verify provenance and adoption"]
C --> D["Two reviewers verify the author and adoption"]
D --> E{"Both confirm human-authored and adopted?"}
E -- "No" --> F["Exclude or retain as unresolved"]
E -- "Yes" --> G["Two reviewers classify against the current skill"]
@@ -38,7 +38,7 @@ Each feedback item carries a stable source ID and bounded change evidence. When
### Dual-reviewer classification and drafting
Two independently configured reviewer adapters classify every eligible item by provenance (`human-authored`, `forwarded-automation`, or `unclear`) and adoption (`adopted`, `rejected`, or `unclear`). Only matching `human-authored` plus `adopted` verdicts proceed. The adopted set then receives a second independent classification against the current skill: candidate, already covered, implementation-specific, or not feedback. A singleton may qualify; recurrence is not required. Disagreement receives one bounded re-evaluation and remains visible in run artifacts if unresolved. A single batch whose adapter output fails schema or id validation is failed closed at the batch level — every feedback item in it is marked unclear and routed to `excluded` — rather than aborting the whole run; the offending raw output is preserved under the run's private artifacts for debugging. If either adapter returns no valid result for any nonempty batch in an operation, the run exits non-zero and emits a failure record instead of reporting “no candidate.”
Two independently configured reviewer adapters classify who authored every eligible item (`human-authored`, `forwarded-automation`, or `unclear`) and whether the change adopted it (`adopted`, `rejected`, or `unclear`). Only matching `human-authored` plus `adopted` verdicts proceed. The adopted set then receives a second independent classification against the current skill: candidate, already covered, implementation-specific, or not feedback. A singleton may qualify; recurrence is not required. Disagreement receives one bounded re-evaluation and remains visible in run artifacts if unresolved. A single batch whose adapter output fails schema or id validation is failed closed at the batch level — every feedback item in it is marked unclear and routed to `excluded` — rather than aborting the whole run; the offending raw output is preserved under the run's private artifacts for debugging. If either adapter returns no valid result for any nonempty batch in an operation, the run exits non-zero and emits a failure record instead of reporting “no candidate.”
The primary adapter drafts from structured agreed guidance, never raw review text. It remains tool-free and read-only by adapter-author contract: it returns complete candidate file content, which the tool validates before writing the sole target. Both adapters then review the same complete skill diff; blocking findings return to a bounded revision loop, and both must approve the same revision. The tool rejects staged changes and edits outside the target skill both before running the documentation and lint gates and again before reporting success, so a gate or concurrent process that adds another path cannot slip through. It restores its own write on failure using best-effort compare-and-swap so a concurrent maintainer edit is not overwritten. On success it saves a candidate bundle containing the source `origin/master` commit, source skill blob ID, reviewed diff, complete candidate, source feedback IDs and URLs, landed evidence ranges, adapter verdicts, and gate results; it never commits, pushes, opens, or merges a PR.
@@ -48,21 +48,21 @@ Each private executable receives a byte-bounded, versioned JSON request on stdin
### Promotion contract
The promote helper starts from a clean checkout at refreshed `origin/master` and refuses to apply a candidate when the current skill blob differs from the bundle's recorded source blob. The operator then reruns the maintenance analysis or manually rebases the diff and repeats the candidate review; the helper never replaces a newer `SKILL.md` with stale complete-file output. After applying a current candidate, it opens a draft PR whose body lists the source feedback URLs or IDs, the landed commit range used as adoption evidence, the originating run, gate results, and any operator edits. Raw adapter prompts and responses remain private, but repository reviewers receive enough provenance to judge whether each proposed rule follows from adopted human feedback.
The promote helper starts from a clean checkout at refreshed `origin/master` and refuses to apply a candidate when the current skill blob differs from the bundle's recorded source blob. The operator then reruns the maintenance analysis or manually rebases the diff and repeats the candidate review; the helper never replaces a newer `SKILL.md` with stale complete-file output. After applying a current candidate, it opens a draft PR whose body lists the source feedback URLs or IDs, the landed commit range used as adoption evidence, the originating run, gate results, and any operator edits. Raw adapter prompts and responses remain private, but repository reviewers receive those concrete inputs so they can judge whether each proposed rule follows from adopted human feedback.
### Where the mechanism lives
The tool source, adapter binaries, provider credentials, and intended daily scheduler are kept private to the maintainer's machine rather than committed to this repository. This document specifies the protocol; the reference implementation is private infrastructure. The mechanism serves a single skill maintained by a single operator, so the ongoing cost of vetting mechanism edits through repository review outweighs any provenance benefit. If the mechanism is ever handed off to a second maintainer, that handoff is a follow-up Agent Note that revises this decision — the operator doc at [docs/cookbook/maintaining-dsh-code-review.md](../../../../docs/cookbook/maintaining-dsh-code-review.md) is the entry point for anyone taking over.
The tool source, adapter binaries, provider credentials, and intended daily scheduler are kept private to the maintainer's machine rather than committed to this repository. This document specifies the protocol; the reference implementation is private infrastructure. The mechanism serves a single skill maintained by a single operator, so the ongoing cost of vetting mechanism edits through repository review outweighs the benefit of committing the tool and its history. If the mechanism is ever handed off to a second maintainer, that handoff is a follow-up Agent Note that revises this decision — the operator doc at [docs/cookbook/maintaining-dsh-code-review.md](../../../../docs/cookbook/maintaining-dsh-code-review.md) is the entry point for anyone taking over.
## Alternatives considered
- **Ship the tool inside this repository.** Rejected for a single-maintainer scope: repository maintenance overhead (typecheck, lint, coverage, cross-cutting refactors) would exceed the value of committed provenance. Retained option for a later handoff.
- **Ship the tool inside this repository.** Rejected for a single-maintainer scope: repository maintenance overhead (typecheck, lint, coverage, cross-cutting refactors) would exceed the value of committed source and review history. Retained option for a later handoff.
- **Record every feedback-time PR head** — rejected: it improves causal isolation but requires a continuously running observer, durable event state, retries, and force-push reconciliation. Periodic maintenance uses reviewed-commit evidence where available and fails closed on broader whole-PR evidence.
- **Persist a processed-PR cursor** — rejected: an overlapping time-window scan is cheap and naturally idempotent against the current skill, while cursor state creates recovery and missed-event problems.
- **Run on every new comment** — rejected: review waves produce many related comments and lack the final artifact needed to judge adoption.
- **Treat merge or thread resolution as adoption** — rejected: a PR can merge with rejected, superseded, or intentionally unresolved feedback.
- **Create or merge repository changes automatically** — rejected: the tool first needs a track record of useful periodic output. The maintainer inspects and promotes the local diff through normal repository review.
- **Learn from bot findings that were fixed** — rejected: the source contract is human review feedback. Actor type is filtered before analysis, and human accounts forwarding automated findings are excluded by provenance review.
- **Learn from bot findings that were fixed** — rejected: the source contract is human review feedback. Author type is filtered before analysis, and human accounts forwarding automated findings are excluded by the author check.
- **Use one reviewer as author and final judge** — rejected: independent verdicts expose unsupported generalization before it reaches the skill.
## Acceptance criteria
@@ -73,7 +73,7 @@ Promotion from `proposed/` to `implemented/` requires all of the following to be
- Both reviewer adapters are independently configured (distinct providers or models) and complete an analyze / adopt / review pass without user intervention. **Observed on 2026-07-15:** distinct primary/secondary adapters completed adoption + analysis in ~8 minutes; batch fail-closed handled one adapter id-hallucination without aborting the run.
- A scheduler triggers the tool without an interactive terminal, and a candidate diff (or a "no candidate" record) reaches the operator through a durable notification channel.
- A controlled acquisition case advances the target branch with a feedback-matching change after the feedback baseline; the reviewer evidence excludes that target-only change while retaining a later PR-owned change.
- The promote helper rejects a candidate after the source skill changes, and a current candidate opens a draft PR with the provenance summary defined above.
- The promote helper rejects a candidate after the source skill changes, and a current candidate opens a draft PR with the source feedback IDs, adopted commit range, originating run, checks, and operator edits defined above.
- At least one candidate diff produced by this workflow is inspected by the operator and promoted to `master` through a normal repository PR review. That PR is the evidence that the workflow can turn adopted feedback into shipped skill guidance.
## Risks

View File

@@ -10,13 +10,13 @@ Status: proposed
## 提案
在仓库外定期维护。一项保存在 skill 维护者机器上、而不提交到本仓库的私有工具,会针对刷新至 `origin/master` 的干净完整历史 checkout 运行。预期的调度器每天运行,并使用两天的 UTC 重叠窗口;手动运行可以通过另一个 `--since` 时长或重复的 `--pr` 参数指定显式集合。扫描相对于当前 skill 是幂等的,不存储仓库游标。推广时唯一会变更的仓库文件是 [.agents/skills/dsh-code-review/SKILL.md](../../../skills/dsh-code-review/SKILL.md)draft PRPull Request携带来源概述,因此评审人无需私有适配器日志,也能审计源反馈与采纳证据
在仓库外定期维护。一项保存在 skill 维护者机器上、而不提交到本仓库的私有工具,会针对刷新至 `origin/master` 的干净完整历史 checkout 运行。预期的调度器每天运行,并使用两天的 UTC 重叠窗口;手动运行可以通过另一个 `--since` 时长或重复的 `--pr` 参数指定显式集合。扫描相对于当前 skill 是幂等的,不存储仓库游标。推广时唯一会变更的仓库文件是 [.agents/skills/dsh-code-review/SKILL.md](../../../skills/dsh-code-review/SKILL.md)draft PRPull Request列出源反馈 URL 或 ID 和采纳证据,不会公开私有适配器日志
```mermaid
flowchart TD
A["Maintainer or scheduler runs the tool on origin/master"] --> B["List PRs merged in the overlap window"]
B --> C["Collect pre-merge User feedback and final PR evidence"]
C --> D["Two reviewers verify provenance and adoption"]
C --> D["Two reviewers verify the author and adoption"]
D --> E{"Both confirm human-authored and adopted?"}
E -- "No" --> F["Exclude or retain as unresolved"]
E -- "Yes" --> G["Two reviewers classify against the current skill"]
@@ -38,7 +38,7 @@ flowchart TD
### 双评审人分类与起草
两个独立配置的评审适配器,会从来源`human-authored``forwarded-automation``unclear`和采纳情况`adopted``rejected``unclear`两个维度,对每个符合条件的条目进行分类。只有两个适配器都判定为 `human-authored``adopted` 的条目才会继续。采纳集合随后会针对当前 skill 接受第二次独立分类:候选项、已经覆盖、实现专用或并非反馈。单个条目即可符合要求,不要求重复出现。意见分歧会得到一次有界的重新评估;如果仍未解决,则继续保留在运行产物中。单个批次的适配器输出如果未通过 schema 或 id 校验,系统会在批次层按不采纳处理:其中的每条反馈都标记为 unclear 并路由到 `excluded`,而不是中止整次运行;有问题的原始输出会保存在该次运行的私有产物中,供调试使用。如果任一适配器在某项操作的任何非空批次中都没有返回有效结果,运行会以非零状态退出并发出失败记录,而不会报告「没有候选项」。
两个独立配置的评审适配器,会对每个符合条件的条目分类:谁编写了它`human-authored``forwarded-automation``unclear`,以及变更是否采纳了它`adopted``rejected``unclear`)。只有两个适配器都判定为 `human-authored``adopted` 的条目才会继续。采纳集合随后会针对当前 skill 接受第二次独立分类:候选项、已经覆盖、实现专用或并非反馈。单个条目即可符合要求,不要求重复出现。意见分歧会得到一次有界的重新评估;如果仍未解决,则继续保留在运行产物中。单个批次的适配器输出如果未通过 schema 或 id 校验,系统会在批次层按不采纳处理:其中的每条反馈都标记为 unclear 并路由到 `excluded`,而不是中止整次运行;有问题的原始输出会保存在该次运行的私有产物中,供调试使用。如果任一适配器在某项操作的任何非空批次中都没有返回有效结果,运行会以非零状态退出并发出失败记录,而不会报告「没有候选项」。
主适配器只根据结构化的共同指引起草,绝不接收原始评审文本。根据适配器作者的约定,它保持无工具且只读:返回完整的候选文件内容,由工具校验后写入唯一目标。两个适配器随后评审同一份完整 skill diff阻塞性问题会进入有界修订循环而且两者必须批准同一版修订。工具会在运行文档和 lint 检查前,以及报告成功前,再次拒绝暂存改动和目标 skill 之外的编辑,因此检查或并发进程无法通过添加其他路径混入。失败时,工具使用尽力而为的比较并交换回滚自身写入,以免覆盖维护者的并发编辑。成功时,它保存一份候选资料包,其中包含源 `origin/master` commit、源 skill blob ID、已评审 diff、完整候选文件、源反馈 ID 与 URL、落地证据范围、适配器判定和检查结果它绝不提交、推送、打开或合并 PR。
@@ -48,21 +48,21 @@ flowchart TD
### 推广约定
推广辅助工具从刷新至 `origin/master` 的干净 checkout 开始;当前 skill blob 与资料包中记录的源 blob 不同时,它拒绝应用候选项。运维方随后重新运行维护分析,或手动把 diff 变基后重新评审候选项;辅助工具绝不会用陈旧的完整文件输出替换较新的 `SKILL.md`。应用仍然有效的候选项后,它会打开 draft PR其正文列出源反馈 URL 或 ID、用作采纳证据的落地 commit 范围、来源运行、检查结果和任何运维方编辑。原始适配器提示词与响应保持私有,但仓库评审人会获得足够的来源信息,以判断每条提议规则是否确实来自已采纳的人类反馈。
推广辅助工具从刷新至 `origin/master` 的干净 checkout 开始;当前 skill blob 与资料包中记录的源 blob 不同时,它拒绝应用候选项。运维方随后重新运行维护分析,或手动把 diff 变基后重新评审候选项;辅助工具绝不会用陈旧的完整文件输出替换较新的 `SKILL.md`。应用仍然有效的候选项后,它会打开 draft PR其正文列出源反馈 URL 或 ID、用作采纳证据的落地 commit 范围、发起该候选项的运行、检查结果和任何运维方编辑。原始适配器提示词与响应保持私有,但仓库评审人会获得这些具体输入,从而判断每条提议规则是否确实来自已采纳的人类反馈。
### 机制所在位置
工具源码、适配器二进制文件、提供方凭据和预期的每日调度器保存在维护者机器上,不会提交到本仓库。本文规定协议,参考实现属于私有基础设施。该机制只服务于由单个运维方维护的一项 skill因此持续通过仓库评审审查机制改动的成本高于来源可追溯性的收益。如果该机制将来移交给第二位维护者,移交工作需要一篇后续 Agent Note 来修订本决策;任何接手者都应从运维文档 [docs/cookbook/maintaining-dsh-code-review.md](../../../../docs/cookbook/maintaining-dsh-code-review.md) 入手。
工具源码、适配器二进制文件、提供方凭据和预期的每日调度器保存在维护者机器上,不会提交到本仓库。本文规定协议,参考实现属于私有基础设施。该机制只服务于由单个运维方维护的一项 skill因此持续通过仓库评审审查机制改动的成本高于提交工具及其历史的收益。如果该机制将来移交给第二位维护者,移交工作需要一篇后续 Agent Note 来修订本决策;任何接手者都应从运维文档 [docs/cookbook/maintaining-dsh-code-review.md](../../../../docs/cookbook/maintaining-dsh-code-review.md) 入手。
## 考虑过的替代方案
- **把工具放入本仓库。** 对单维护者作用域不予采纳仓库维护开销类型检查、lint、覆盖率与横切重构会超过已提交来源信息的价值。未来移交时仍可重新考虑。
- **把工具放入本仓库。** 对单维护者作用域不予采纳仓库维护开销类型检查、lint、覆盖率与横切重构会超过已提交源码和评审历史的价值。未来移交时仍可重新考虑。
- **记录每条反馈产生时的 PR head**:不予采纳,因为这需要持续运行的观察器、持久事件状态、重试和强制推送协调。定期维护会在可用时使用经过评审的 commit 证据,并在整 PR 证据范围过宽、无法确认时直接排除。
- **持久化已处理 PR 游标**:不予采纳,因为带重叠的时间窗口扫描成本低廉,并且相对于当前 skill 天然幂等;游标状态反而带来恢复和漏事件问题。
- **每次新评论都运行**:不予采纳,因为一轮评审会产生许多相关评论,并且缺少判断采纳情况所需的最终产物。
- **把合并或讨论串解决视为采纳**:不予采纳,因为 PR 可能在反馈被拒绝、被取代或刻意不解决的情况下合并。
- **自动创建或合并仓库改动**:不予采纳,因为工具首先需要通过有用的定期输出积累可信记录。维护者检查并通过普通仓库评审推广本地 diff。
- **从已经修复的 bot 问题中学习**:不予采纳,因为来源约定限定为人类评审反馈。系统会在分析前按 actor 类型过滤,并通过来源评审排除转发自动化问题的人类账号。
- **从已经修复的 bot 问题中学习**:不予采纳,因为来源约定限定为人类评审反馈。系统会在分析前按作者类型过滤,并通过作者检查排除转发自动化问题的人类账号。
- **让同一个评审人既当作者又作最终裁决**:不予采纳,因为独立判定能在不受支持的概括进入 skill 之前暴露问题。
## 验收标准
@@ -73,7 +73,7 @@ flowchart TD
- 两个评审适配器独立配置(不同的提供方或模型),无需用户干预即可完成 analyzeadoptreview 流程。**2026-07-15 已观察:** 不同的主/次适配器约用 8 分钟完成采纳与分析;一次适配器 id 幻觉由批次级保守失败机制处理,没有中止整次运行。
- 调度器在没有交互式终端的情况下触发工具,并通过持久通知通道把候选 diff或「没有候选项」记录送达运维方。
- 一个受控采集场景会在反馈基线之后,向目标分支加入与反馈匹配的变更;评审证据排除该目标分支专有变更,同时保留后续由 PR 自身加入的变更。
- 源 skill 改变后,推广辅助工具会拒绝候选项;仍然有效的候选项会按照上文定义的来源概述打开 draft PR。
- 源 skill 改变后,推广辅助工具会拒绝候选项;仍然有效的候选项会打开 draft PR,其中包含上文定义的源反馈 ID、已采纳 commit 范围、发起该候选项的运行、检查和运维方编辑
- 该工作流生成的至少一个候选 diff 会由运维方检查,并通过普通仓库 PR 评审推广到 `master`。该 PR 用以证明工作流能够把已采纳反馈转化为已交付的 skill 指引。
## 风险