Merge branch 'master' into worktree/llm-mock-fault-server
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-19-acp-snapshot-tests.md: 43900632c4e5e3904c2d0e3b75f2a3fe3b3a50cc
|
||||
2026-06-19-acp-snapshot-tests.zh.md: a4613b19001afe87c1f6bdb61196fabbfa947a4c
|
||||
2026-06-19-acp-snapshot-tests.md: b4cda8f32fe7a84a977bcbdbe5db0671cb9a7083
|
||||
2026-06-19-acp-snapshot-tests.zh.md: 5337c3852b524af4e8c556e93ec80084b30a6d0b
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-06-19-acp-snapshot-tests.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Unit tests do not exercise the complete ACP subprocess transcript, while real-API tests are nondeterministic and key-gated. Editor-facing `session/update` output can therefore regress despite green unit coverage, as the [default-export postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) demonstrated.
|
||||
Unit tests do not exercise the complete assembled-agent subprocess or its ACP automation wire, while real-API tests are nondeterministic and key-gated. Loader wiring, backend behavior, and protocol output can therefore regress despite green unit coverage, as the [default-export postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) demonstrated.
|
||||
|
||||
The blocker for a full-transcript test is the model: the agent's output is driven by a non-deterministic LLM, and a key-gated test that hits the real API on every run is neither deterministic nor CI-runnable. We want the fidelity of a real run with the determinism of a fixture.
|
||||
|
||||
@@ -52,10 +52,10 @@ Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with
|
||||
|
||||
A snapshot run asserts **two** normalized surfaces, because the harness's external surfaces are distinct:
|
||||
|
||||
1. The **stdout transcript** — the framed `session/update` JSON-RPC the editor sees. Catches regressions in the ACP bridge's event→update translation (`streamSessionEventUpdate`). Compared against a committed `stdout.expected.jsonl`.
|
||||
1. The **stdout transcript** — the framed ACP JSON-RPC responses and committed-message updates an automation client receives. It catches regressions in the transport contract and is compared against a committed `stdout.expected.jsonl`.
|
||||
2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt text is scrubbed; one scenario per header class pins readable prompt and tool content as described in the [header-pinning Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar.
|
||||
|
||||
The surfaces are complementary: stdout covers bridge projection, while JSONL covers loop, tool, and boundary structure that the projection omits.
|
||||
The surfaces are complementary: stdout covers the minimal automation wire, while JSONL covers loop, tool, and boundary structure that the wire intentionally omits.
|
||||
|
||||
Normalization replaces session, cwd, protocol-id, timestamp, path, and process volatility while preserving deterministic sequence numbers. Scenarios constrain real bash use to stable commands. The stdout expected output remains wire-shaped JSONL and every raw line must parse as JSON. Vitest updates only the stdout expected output; normalized session equality never overwrites the replay fixture.
|
||||
|
||||
@@ -79,6 +79,6 @@ Tool determinism comes from a generated cwd, scrubbed environment, fresh non-log
|
||||
|
||||
## Consequences
|
||||
|
||||
The new tier adds reviewed per-scenario input, session, stdout, optional override, and optional workspace fixtures. Workspace seeds are copied into the generated cwd for both record and replay. In return the tier provides deterministic keyless transcript coverage through the real Loader and tool composition. The subprocess, input, workspace, normalization, and replay harness can support examples beyond ACP.
|
||||
The tier adds reviewed per-scenario input, session, stdout, optional override, and optional workspace fixtures. Workspace seeds are copied into the generated cwd for both record and replay. In return the tier provides deterministic keyless coverage through the real Loader and tool composition. Most retained scenarios exercise the assembled backend rather than ACP; the [automation-only ACP decision](../simplification/2026-07-23-acp-automation-only-protocol.md#snapshot-boundary) keeps that corpus here and defers any move to a transport-neutral headless suite as an independent testing change (the suite-level FIXME marks it).
|
||||
|
||||
This Agent Note relates to but does not supersede the [proposed determinism Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md): that proposal's "universal replay fixture" re-derives session *message history* after every test (an internal-consistency invariant), whereas snapshot tests pin the *external protocol output*. They are complementary — one guards the event-sourcing invariant, the other guards the editor-facing contract.
|
||||
This Agent Note relates to but does not supersede the [proposed determinism Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md): that proposal's "universal replay fixture" re-derives session *message history* after every test (an internal-consistency invariant), whereas these snapshots pin assembled behavior plus the external automation output. They are complementary until the backend corpus moves off ACP.
|
||||
|
||||
@@ -6,9 +6,9 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
单元测试不会覆盖完整的 ACP(Agent Client Protocol)子进程 transcript(文本记录),而真实 API 测试不具确定性且受密钥门控。因此,即使单元覆盖率为绿色,面向编辑器的 `session/update` 输出仍可能回归,[默认导出事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)已经证明了这一点。
|
||||
单元测试不会覆盖组装后的完整 agent(智能体)子进程及其 ACP(Agent Client Protocol)自动化线协议,而真实 API 测试不具确定性且受密钥门控。因此,即使单元覆盖率为绿色,Loader 接线、后端行为和协议输出仍可能回归,[默认导出事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)已经证明了这一点。
|
||||
|
||||
全 transcript 测试的阻塞因素在于模型:agent(智能体)的输出由非确定性的 LLM(大语言模型)驱动,而每次运行都命中真实 API 的密钥门控测试既不确定也无法在 CI 中运行。我们需要真实运行的保真度与 fixture(测试前置数据)的确定性兼得。
|
||||
全 transcript(文本记录)测试的阻塞因素在于模型:agent 的输出由非确定性的 LLM(大语言模型)驱动,而每次运行都命中真实 API 的密钥门控测试既不确定也无法在 CI 中运行。我们需要真实运行的保真度与 fixture(测试前置数据)的确定性兼得。
|
||||
|
||||
本 Agent Note(agent 决策记录)记下了新增第三层测试——**快照测试**——的决策,以及让它具备确定性、在 CI 中无需密钥、且维护成本低廉的设计选择。
|
||||
|
||||
@@ -52,10 +52,10 @@ Status: implemented
|
||||
|
||||
快照运行断言**两个**归一化后的表面,因为 harness 的外部表面是不同的:
|
||||
|
||||
1. **stdout transcript**——编辑器看到的、经过 framing 的 `session/update` JSON-RPC。用于捕获 ACP bridge 中事件→更新转换(`streamSessionEventUpdate`)的回归。与已提交的 `stdout.expected.jsonl` 比较。
|
||||
1. **stdout transcript**——自动化客户端收到的、经过 framing 的 ACP JSON-RPC 响应与已提交的消息更新。它捕获传输契约的回归,与已提交的 `stdout.expected.jsonl` 比较。
|
||||
2. **重新持久化的会话 JSONL**,经过规范化后与 `session.jsonl` 比较。同一 fixture 同时作为重放来源和预期日志。提示词文本会被清理;按照[请求头固定 Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md)所述,每种请求头类别由一个场景固定可读提示词与工具内容。Override 场景仅从其 sidecar 派生模型行为。
|
||||
|
||||
两个表面互补:stdout 覆盖 bridge 投影,JSONL 覆盖投影所省略的 loop、工具和 boundary 结构。
|
||||
两个表面互补:stdout 覆盖精简的自动化线协议,JSONL 覆盖线协议有意省略的 loop、工具和 boundary 结构。
|
||||
|
||||
规范化会替换会话、cwd、协议 id、时间戳、路径和进程易变值,同时保留确定性序号。场景把真实 bash 使用限制在稳定命令上。stdout 预期输出仍是线协议形状的 JSONL,每个原始行都必须可解析为 JSON。Vitest 只更新 stdout 预期输出;规范化会话相等性检查从不覆盖重放 fixture。
|
||||
|
||||
@@ -79,6 +79,6 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
新测试层为每个场景增加经过评审的输入、会话、stdout、可选 override 和可选 workspace fixture。记录与重放都会把 workspace seed 复制到生成的 cwd。作为回报,该层通过真实 Loader 和工具组合提供确定性的无密钥 transcript 覆盖。子进程、输入、workspace、规范化和重放 harness 也可以支持 ACP 之外的示例。
|
||||
该测试层为每个场景增加经过评审的输入、会话、stdout、可选 override 和可选 workspace fixture。记录与重放都会把 workspace seed 复制到生成的 cwd。作为回报,该层通过真实 Loader 和工具组合提供确定性的无密钥覆盖。保留下来的大多数场景测试的是组装后的后端而非 ACP;[仅面向自动化的 ACP 决策](../simplification/2026-07-23-acp-automation-only-protocol.md#snapshot-boundary)将该语料保留在此处,并把向传输无关 headless 套件的任何迁移推迟为一项独立的测试变更(套件级 FIXME 标记了这一点)。
|
||||
|
||||
本 Agent Note 与[拟议的确定性 Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md)相关,但不取代它:该提案的“通用重放 fixture”在每次测试后重新派生会话*消息历史*(内部一致性不变量),而快照测试固定*外部协议输出*。两者相互补充——一个守护事件溯源不变量,另一个守护面向编辑器的契约。
|
||||
本 Agent Note 与[拟议的确定性 Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md)相关,但不取代它:该提案的“通用重放 fixture”在每次测试后重新派生会话*消息历史*(内部一致性不变量),而这些快照固定组装后的行为与外部自动化输出。在后端语料迁出 ACP 之前,两者相互补充。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-19-real-api-e2e-ci.md: a9289980ada8ab6390b5daa488f07ec258db1bd5
|
||||
2026-06-19-real-api-e2e-ci.zh.md: dcb5759d26e7680d392a4855665b2eb88551ba79
|
||||
2026-06-19-real-api-e2e-ci.md: 935664fd01df4844ee19be7b4f2f297ebf5bd29b
|
||||
2026-06-19-real-api-e2e-ci.zh.md: 9c10614f5a7e6b38f6850b29fab87d0e09806c5f
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-06-19-real-api-e2e-ci.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The harness leans hard on real-API tests by policy: [docs/testing.md](../../../../docs/testing.md) argues that a no-key suite proves the plumbing but not the product, and the [ACP inject postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) is the standing proof — 178 keyless tests stayed green while a real editor session crashed instantly. The real-API e2e suite (`pnpm run test:e2e`, the `*.e2e.ts` files) exists precisely to close that gap: it drives the agent against the live DeepSeek API — real model calls, real bash tools, multi-turn, resume, ACP-over-stdio.
|
||||
The harness leans hard on real-API tests by policy: [docs/testing.md](../../../../docs/testing.md) argues that a no-key suite proves the plumbing but not the product, and the [ACP inject postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) is the standing proof — 178 keyless tests stayed green while a real ACP client session crashed instantly. The real-API e2e suite (`pnpm run test:e2e`, the `*.e2e.ts` files) exists precisely to close that gap: it drives the agent against the live DeepSeek API — real model calls, real bash tools, multi-turn, resume, ACP-over-stdio.
|
||||
|
||||
The default gate ([.github/workflows/ci.yml](../../../../.github/workflows/ci.yml)) is deliberately keyless: it carries no secret and runs for forks. `test:e2e` self-skips without a key (`describe.skipIf(!process.env.DEEPSEEK_API_KEY)`), so adding it there would report green without exercising the real suite. A separate secret-bearing workflow is required to make real-API coverage a merge signal.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
根据策略,harness 高度依赖真实 API 测试:[docs/testing.md](../../../../docs/testing.md) 指出,无密钥套件证明的是管线,而非产品;[ACP(Agent Client Protocol)inject 事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)则是常设证据——178 项无密钥测试保持绿色时,真实编辑器会话却立即崩溃。真实 API e2e 套件(`pnpm run test:e2e`,即 `*.e2e.ts` 文件)的存在正是为了弥合这一缺口:它针对实时 DeepSeek API 驱动 agent(智能体)——真实模型调用、真实 bash 工具、多轮次、恢复、ACP-over-stdio。
|
||||
根据策略,harness 高度依赖真实 API 测试:[docs/testing.md](../../../../docs/testing.md) 指出,无密钥套件证明的是管线,而非产品;[ACP(Agent Client Protocol)inject 事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)则是常设证据——178 项无密钥测试保持绿色时,真实 ACP 客户端会话却立即崩溃。真实 API e2e 套件(`pnpm run test:e2e`,即 `*.e2e.ts` 文件)的存在正是为了弥合这一缺口:它针对实时 DeepSeek API 驱动 agent(智能体)——真实模型调用、真实 bash 工具、多轮次、恢复、ACP-over-stdio。
|
||||
|
||||
默认门禁([.github/workflows/ci.yml](../../../../.github/workflows/ci.yml))刻意无密钥:不携带 secret,可供 fork 运行。`test:e2e` 在无密钥时自动跳过(`describe.skipIf(!process.env.DEEPSEEK_API_KEY)`),因此将其加入该工作流只会报绿而不会真正执行真实套件。要让真实 API 覆盖率成为合并信号,需要一个独立的、携带 secret 的工作流。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-22-subagent-snapshot-replay.md: 4aad8b6ddd3e4f8b8ad5565e10b263953d81ea31
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: 96890bc98a64357d9785b8081c8ba123bb3c716e
|
||||
2026-06-22-subagent-snapshot-replay.md: 8cd7bc86e07af9ed274c18574b575b9070854e88
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: eae78129405fedd03c2c579845c07c6e5694cc30
|
||||
|
||||
@@ -6,12 +6,12 @@ English | [中文](2026-06-22-subagent-snapshot-replay.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized stdout transcript + re-persisted session log against committed expected outputs. It is the only tier that exercises the full editor-facing transcript end to end.
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized automation wire + re-persisted session log against committed expected outputs. Most scenarios exercise assembled backend behavior through that real process boundary.
|
||||
|
||||
It was built for ONE session per process, and that assumption is wired into two places:
|
||||
|
||||
- **`dsh-llm-replay` keyed nothing.** It served the Nth `llm/stream` call the Nth recorded entry from a single global cursor. With a parent agent AND an in-process subagent both streaming on one context, the calls interleave and the single cursor hands the child the parent's script (and vice versa).
|
||||
- **The harness harvested one log.** `findSessionLog` walked the sessions root and returned the FIRST `.jsonl` it found. A subagent runs as a second `Session` with its own log in the same cwd bucket, so the child's transcript was silently dropped.
|
||||
- **The harness harvested one log.** `findSessionLog` walked the sessions root and returned the FIRST `.jsonl` it found. A subagent runs as a second `Session` with its own log, so the child's transcript was silently dropped.
|
||||
|
||||
This was the `TODO(subagent-snapshots)` deferral recorded in the [subagent seam Agent Note](../feature/2026-06-21-subagent-capability-seam.md): the in-process backends (PR2) shipped with unit + e2e coverage, but the full-transcript snapshot tier could not express a nested-agent shape until this infrastructure landed. This Agent Note is that stacked follow-up.
|
||||
|
||||
@@ -39,7 +39,7 @@ The alternative considered and rejected was a **call-ordered merge of the parent
|
||||
|
||||
### 3. The harness harvests every log, primary-first
|
||||
|
||||
`harvestSessionLogs` collects every `.jsonl` across every cwd bucket under the sessions root (the JSONL backend puts a parent and its same-cwd child in the same bucket), parses each header, and orders them primary-first: the top-level session (no `parentSession`) leads, then each child by ascending `createdAt`. `RunResult.sessionLogs` is the plural result; the spec writes each back to its fixture on record (`session.jsonl` + `session.<n>.jsonl`) and diffs each harvested log against its fixture on replay. The normalizer already accepted plural session ids and collapses any stray UUID, so no normalizer change was needed.
|
||||
`harvestSessionLogs` recursively collects every fixed `session.jsonl` transcript under the sessions root (the JSONL backend gives each parent and child its own project/session directory), parses each header, and orders them primary-first: the top-level session (no `parentSession`) leads, then each child by ascending `createdAt`. `RunResult.sessionLogs` is the plural result; the spec writes each back to its fixture on record (`session.jsonl` + `session.<n>.jsonl`) and diffs each harvested log against its fixture on replay. The normalizer already accepted plural session ids and collapses any stray UUID, so no normalizer change was needed.
|
||||
|
||||
### 4. Scenarios
|
||||
|
||||
|
||||
@@ -6,12 +6,12 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
快照层(`pnpm run test:snapshot`)会启动真实 `acp-agent` 子进程,通过 [`dsh-llm-replay`](../../../../packages/support/llm-replay) 重放已记录会话,并将规范化 stdout transcript(文本记录)+ 重新持久化的会话日志与已提交预期输出进行 diff。它是唯一端到端覆盖完整面向编辑器 transcript 的测试层。
|
||||
快照层(`pnpm run test:snapshot`)会启动真实 `acp-agent` 子进程,通过 [`dsh-llm-replay`](../../../../packages/support/llm-replay) 重放已记录会话,并将规范化后的自动化线协议 + 重新持久化的会话日志与已提交预期输出进行 diff。大多数场景通过这条真实进程边界测试组装后的后端行为。
|
||||
|
||||
该层最初为每个进程只有一个会话而构建,这一假设硬编码在两处:
|
||||
|
||||
- **`dsh-llm-replay` 没有做任何键控。** 它用一个全局游标,将第 N 次 `llm/stream` 调用对应到单一录制序列的第 N 条。当父 agent(智能体)和一个进程内 subagent 在同一个上下文上同时流式输出时,调用交错,单一游标会把子 agent 的脚本发给父 agent(反之亦然)。
|
||||
- **harness 只收集一份日志。** `findSessionLog` 遍历 sessions 根目录,返回找到的第一个 `.jsonl`。subagent 作为第二个 `Session` 运行,在同一个 cwd bucket 下有自己的日志,因此子 agent 的 transcript 被静默丢弃。
|
||||
- **harness 只收集一份日志。** `findSessionLog` 遍历 sessions 根目录,返回找到的第一个 `.jsonl`。subagent 作为第二个 `Session` 运行并拥有自己的日志,因此子 agent 的 transcript(文本记录)被静默丢弃。
|
||||
|
||||
这就是 [subagent seam Agent Note(agent 决策记录)](../feature/2026-06-21-subagent-capability-seam.md)中通过 `TODO(subagent-snapshots)` 推迟的工作:进程内后端(PR2)落地时已有单元 + e2e 覆盖,但在这套基础设施落地前,完整 transcript 快照层无法表达嵌套 agent 形状。本 Agent Note 就是该堆叠式后续工作。
|
||||
|
||||
@@ -39,7 +39,7 @@ Status: implemented
|
||||
|
||||
### 3. harness 收集所有日志,主会话优先
|
||||
|
||||
`harvestSessionLogs` 收集 sessions 根目录下每个 cwd bucket 中的所有 `.jsonl`(JSONL 后端将父会话与同 cwd 的子会话放在同一个 bucket),解析各自的 header,并按主会话优先排序:顶层会话(无 `parentSession`)在前,各子会话按 `createdAt` 升序排列。`RunResult.sessionLogs` 是复数结果;spec 在录制时将每份日志写回对应 fixture(`session.jsonl` + `session.<n>.jsonl`),在回放时将每份收集到的日志与其 fixture 做 diff。归一化器已支持复数会话 id 并会折叠任何游离 UUID,因此无需修改归一化器。
|
||||
`harvestSessionLogs` 递归收集 sessions 根目录下所有固定命名为 `session.jsonl` 的 transcript(JSONL 后端为每个父会话和子会话分别提供独立的项目/会话目录),解析各自的 header,并按主会话优先排序:顶层会话(无 `parentSession`)在前,各子会话按 `createdAt` 升序排列。`RunResult.sessionLogs` 是复数结果;spec 在录制时将每份日志写回对应 fixture(`session.jsonl` + `session.<n>.jsonl`),在回放时将每份收集到的日志与其 fixture 做 diff。归一化器已支持复数会话 id 并会折叠任何游离 UUID,因此无需修改归一化器。
|
||||
|
||||
### 4. 场景
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-04-hook-snapshot-matrix.md: ceb91a70e1f9582cf2a7cb4cec0ba8aaf5699b8e
|
||||
2026-07-04-hook-snapshot-matrix.zh.md: 250995dd1aae566aa5b9d40a14ed281b14fbfe0b
|
||||
2026-07-04-hook-snapshot-matrix.md: 98f9db27fd8afaaa99c9985dff4d148ef4eb926b
|
||||
2026-07-04-hook-snapshot-matrix.zh.md: 40b9ee84ad7232ac806096375e8da42bd02bfddf
|
||||
|
||||
@@ -8,7 +8,7 @@ English | [中文](2026-07-04-hook-snapshot-matrix.zh.md)
|
||||
|
||||
The hook bridges — [`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude) (7 Claude Code hook points) and [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex) (5 Codex points) — map external hook commands onto the harness interception seams. They carry deep unit and coverage-spec coverage (every decision arm, every payload dialect, driven against a mocked seam) plus one key-gated e2e (`hooks.e2e.ts`, a live `PreToolUse` block). But the full-transcript snapshot tier — the one net that boots the real `acp-agent` subprocess, replays a recorded session keyless, and diffs the normalized ACP stdout + re-persisted log against committed expected outputs — covered exactly ONE hook: a Claude `UserPromptSubmit` block (`hook-cc-promptsubmit-block`).
|
||||
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction, rendered exactly as an editor sees it. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction through the automation wire and persisted log. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Status: implemented
|
||||
|
||||
钩子 bridge——[`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude)(7 个 Claude Code 钩子点)和 [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex)(5 个 Codex 点)——把外部钩子命令映射到 harness 拦截 seam。它们有深入的单元与覆盖率规格覆盖(每个决策分支、每种 payload dialect,针对 mock seam 驱动),外加一个受密钥门控的 e2e(`hooks.e2e.ts`,实时 `PreToolUse` 阻止)。但完整 transcript(文本记录)快照层——会启动真实 `acp-agent` 子进程、无需密钥重放已记录会话,并将规范化 ACP(Agent Client Protocol)stdout + 重新持久化日志与已提交预期输出进行 diff 的那张网——只覆盖了一个钩子:Claude `UserPromptSubmit` 阻止(`hook-cc-promptsubmit-block`)。
|
||||
|
||||
这正是 mock 单元测试在结构上无法替代的层级:它验证的是真实 bridge 将真实钩子进程的结果翻译到真实 seam 决策,再到真实 agent loop(智能体循环)的反应,渲染结果与编辑器看到的完全一致。一个 bridge 翻译或 loop 结构的回归,即使让所有单元测试保持绿色,也会在除那一个钩子点之外的所有点上逃逸;而对于 Codex bridge,ACP 示例甚至没有加载它,因此没有任何 Codex 钩子能端到端触发。
|
||||
这正是 mock 单元测试在结构上无法替代的层级:它验证的是真实 bridge 将真实钩子进程的结果翻译到真实 seam 决策,再经由自动化线协议和持久化日志检验真实 agent loop(智能体循环)的反应。一个 bridge 翻译或 loop 结构的回归,即使让所有单元测试保持绿色,也会在除那一个钩子点之外的所有点上逃逸;而对于 Codex bridge,ACP 示例甚至没有加载它,因此没有任何 Codex 钩子能端到端触发。
|
||||
|
||||
## 决策
|
||||
|
||||
|
||||
Reference in New Issue
Block a user