test: trim PostToolUse snapshot retries
This commit is contained in:
@@ -29,6 +29,8 @@ Thirteen scenarios under `examples/acp-agent/tests/snapshots/`, naming `hook-<di
|
||||
|
||||
Each hook command emits only FIXED LITERAL strings (no timestamps/pids/`$RANDOM`/cwd echoes); the snapshot normalizer scrubs the one volatile field a `hook/result` carries (`durationMs`). The `Stop` scenarios self-limit with a marker file (`.stop_fired`) so the force-continue does not loop — the `stop_hook_active` loop-guard is still a bridge `TODO`, so an unconditional Stop hook would force-continue every step.
|
||||
|
||||
The `PostToolUse` block scenarios also self-limit: their prompts require exactly one tool call and then a verbatim report of whatever result the hook supplies. Each golden therefore pins one blocked call and the model's reaction without recording repeated retry cycles.
|
||||
|
||||
### Three hook points are deliberately NOT snapshotted
|
||||
|
||||
Discovered while building the matrix, and documented here because the omission is a decision, not an oversight:
|
||||
@@ -41,7 +43,7 @@ The matrix therefore covers every hook point that has a DETERMINISTIC, OBSERVABL
|
||||
## Consequences
|
||||
|
||||
- Every bridge seam mapping with an observable transcript is now guarded at the full-transcript tier, in the real app, for both dialects — including the Codex bridge, which had no end-to-end coverage at all. Recorded goldens capture the model's real reaction to a denied/blocked/force-continued turn, which a hand-authored transcript could only guess at.
|
||||
- The block scenarios are keyless (no model turn); the rest replay keyless from recorded fixtures. `pnpm run test:snapshot:record` regenerates the recorded fixtures from the live API and self-skips without a key like every recorded scenario.
|
||||
- The `UserPromptSubmit` block scenarios are authored keylessly (no model turn); the rest replay keylessly from recorded fixtures. `pnpm run test:snapshot:record` regenerates the recorded fixtures from the live API and self-skips without a key like every recorded scenario.
|
||||
- The prove-red discipline holds: tampering a hook config's output (e.g. changing a deny reason) turns its scenario red on replay — the hook process runs FOR REAL during replay (only the model is replayed), so the golden guards the actual hook→seam→loop path, not a mock of it.
|
||||
- The `acp-agent` demo now loads a Codex bridge it will usually no-op (no `codex-hooks.json` in a typical project), which is the intended fail-soft behavior, not a cost.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user