feat(tools): validate model-generated tool args at the boundary (RFC 005 pt 1)

defineTool now runs validateArgs against the SchemaSpec before execute, so a
malformed model call returns a self-correctable isError result listing the
violations instead of reaching the typed body untyped-in-practice. The
validator mirrors schemaSpecToJsonSchema semantics exactly (required from
required:true only, extra keys allowed, default not applied, object/array
without properties/items only type-checks, enum membership).

tool-bash's hand-rolled type/required checks (carrying the TODO(RFC 005)
stopgap note) are slimmed to just the value constraints the DSL can't express
(non-empty strings, positive timeout). Graduates RFC 005 pt 1 to ADR 0011.
This commit is contained in:
Tianyi Cui
2026-06-13 23:00:42 +08:00
parent 39b3db4b9c
commit 36a30180b8
10 changed files with 325 additions and 44 deletions

View File

@@ -0,0 +1,20 @@
# ADR 0011: Runtime arg validation at the model boundary
Status: accepted (2026-06-13)
## Context
`defineTool` (ADR 0005) gives tool authors a typed `execute(args)` via the `InferArgs<S>` mapping. But that type is a compile-time claim about a value that arrives at runtime as model-generated JSON: nothing forced the model to honor the schema, so a malformed call — missing a required key, a string where a number was declared, an enum value outside the set — reached `execute` typed-in-name-only. The tool body then either crashed on the bad shape (a generic stack trace the model can't act on) or, worse, silently misbehaved. Meanwhile the converter already encodes the exact structure a validator would need to walk.
## Decision
`validateArgs(spec, args): string[]` interprets a `SchemaSpec` over a runtime value, returning human-readable violations (empty = valid), and is total (never throws). `defineTool` runs it before the typed body; on violations it throws `ToolArgsError` (`code: 'INVALID_ARGS'`, message listing the violations), which the registry's existing execute-waterfall catch turns into an `isError` result the model reads and self-corrects from.
The validator mirrors `schemaSpecToJsonSchema` semantics exactly — same structure walked, same rules: top level must be a non-array object; required keys come only from `required: true`; extra keys are allowed (no `additionalProperties: false`); `default` is not applied; an `object`/`array` prop without `properties`/`items` only type-checks; `enum` is membership. Raw-registered (MCP) tools are not touched — they validate their own input.
## Consequences
- The model gets actionable feedback on its own malformed calls instead of an opaque crash, closing the gap between `InferArgs`'s promise and runtime reality.
- The validator and `InferArgs` must stay in agreement; that drift risk is closed by a property test (RFC 001) generating args that satisfy `InferArgs` and asserting they pass `validateArgs`.
- `ToolArgsError` is a plain `Error` with a `code` field for now; if a harness-wide error taxonomy lands it becomes a subclass without changing callers that read `.message`.
- Validation cost is negligible next to a model call.

View File

@@ -22,3 +22,4 @@ Do NOT write an ADR for: a mechanical or local choice (a variable name, a one-fi
| [0008](0008-tsdown-over-dumble.md) | tsdown for JS bundling instead of dumble | accepted |
| [0009](0009-capability-seams.md) | Capability seams — interface / implementation / consumer split | accepted |
| [0010](0010-twin-llm-adapters.md) | Two LLM adapters as a design-verification twin | accepted |
| [0011](0011-runtime-arg-validation.md) | Runtime arg validation at the model boundary | accepted |

View File

@@ -1,6 +1,6 @@
# RFC 005: Runtime validation at the model boundary, error taxonomy, dev-mode invariants
Status: proposed
Status: partially implemented — part 1 (arg validation) → [ADR 0011](../adr/0011-runtime-arg-validation.md); parts 2-3 in progress
## Problem

View File

@@ -8,7 +8,7 @@ Proposals for substantial future work — reviewed before implementation, unlike
| [002](002-mutation-testing.md) | Mutation testing as the coverage counterweight | proposed |
| [003](003-deterministic-and-stress-testing.md) | Deterministic tests + replay invariant fixture + race stress | proposed |
| [004](004-architectural-conformance.md) | Architectural rules: dependency-cruiser, adapter conformance kit | proposed |
| [005](005-runtime-validation-and-error-taxonomy.md) | Runtime arg validation, structured error taxonomy, dev-mode invariants | proposed |
| [005](005-runtime-validation-and-error-taxonomy.md) | Runtime arg validation, structured error taxonomy, dev-mode invariants | partially implemented |
| [006](006-doc-sync-and-api-reports.md) | Doc-sync enforcement and API extractor reports | proposed |
| [007](007-supply-chain-and-vendor-drift.md) | Supply chain checks and vendor drift verification | proposed |
| [008](008-immutable-public-surfaces.md) | Deep-readonly public surfaces | proposed |