feat: unify JSON value schema DSL

This commit is contained in:
Tianyi Cui
2026-07-21 01:11:55 +08:00
parent 9a5c81f9e5
commit 8500974fd4
62 changed files with 1929 additions and 1179 deletions

View File

@@ -8,7 +8,7 @@ Tool parameters must reach the model as standard JSON Schema while giving tool a
## Decision
A small custom DSL in dsh-tools: `SchemaSpec` (per-property specs with `required: true` booleans), type-level `InferArgs<S>` mapping a spec to the argument type (required keys non-optional, others genuinely optional via `?`), a runtime `schemaSpecToJsonSchema()` converter, and `defineTool()` tying them together. Raw JSON-Schema `ToolDefinition`s remain accepted by `ToolRegistry.register()` — that's how MCP-sourced tools arrive.
This decision is superseded by the [unified JSON-value schema DSL](2026-07-20-unified-json-value-schema-dsl.md), which retains the small authoring surface while making parameters and typed values share one vocabulary. `ParameterSchemaSpec` keeps per-property `required: true`; `InferArgs<S>` maps required keys to non-optional properties; `parameterSchemaSpecToJsonSchema()` compiles the implicit open object root; and `defineTool()` ties inference, compilation, and validation together. Raw JSON-Schema `ToolDefinition`s remain accepted by `ToolRegistry.register()` for MCP and other external tools.
## Alternatives considered
@@ -17,5 +17,5 @@ A small custom DSL in dsh-tools: `SchemaSpec` (per-property specs with `required
## Consequences
- First-party tool authors get zero-cast typed args; the type gymnastics cost stays inside the core package (sanctioned by the AGENTS.md type-safety policy).
- The DSL is deliberately small (string/number/boolean/object/array, enum, default, nested properties/items). Gaps vs full JSON Schema (unions, formats, constraints) are accepted until real tools demand them.
- The owning unified note defines the current nodes, literal constraints, unions, JSON-value boundary, and object-openness rules.
- The `InferArgs` mapping is regression-tested at the type level after an early optionality bug.

View File

@@ -4,13 +4,13 @@ Status: implemented
## Problem
`defineTool` ([the custom schema DSL](2026-06-11-custom-schema-dsl.md)) gives tool authors a typed `execute(args)` via the `InferArgs<S>` mapping. But that type is a compile-time claim about a value that arrives at runtime as model-generated JSON: nothing forced the model to honor the schema, so a malformed call — missing a required key, a string where a number was declared, an enum value outside the set — reached `execute` typed-in-name-only. The tool body then either crashed on the bad shape (a generic stack trace the model can't act on) or, worse, silently misbehaved. Meanwhile the converter already encodes the exact structure a validator would need to walk.
`defineTool` ([the unified schema DSL](2026-07-20-unified-json-value-schema-dsl.md)) gives tool authors a typed `execute(args)` via the `InferArgs<S>` mapping. But that type is a compile-time claim about a value that arrives at runtime as model-generated JSON: nothing forced the model to honor the schema, so a malformed call — missing a required key, a string where a number was declared, or a literal outside the declared set — reached `execute` typed-in-name-only. The tool body then either crashed on the bad shape or silently misbehaved.
## Decision
`validateArgs(spec, args): string[]` interprets a `SchemaSpec` over a runtime value, returning human-readable violations (empty = valid), and is total (never throws). `defineTool` runs it before the typed body; on violations it throws `ToolArgsError` (`code: 'INVALID_ARGS'`, message listing the violations), which the registry's existing execute-waterfall catch turns into an `isError` result the model reads and self-corrects from.
`validateArgs(spec, args): string[]` compiles a `ParameterSchemaSpec` and delegates to the shared `validateJsonSchemaValue()` walker, returning human-readable violations for a well-formed declaration. `defineTool` snapshots the compiled parameter schema at definition time and runs that validation before the typed body; violations throw `ToolArgsError` (`INVALID_ARGS`), which the registry returns as an error result the model can correct.
The validator mirrors `schemaSpecToJsonSchema` semantics exactly — same structure walked, same rules: top level must be a non-array object; required keys come only from `required: true`; extra keys are allowed (no `additionalProperties: false`); `default` is not applied; an `object`/`array` prop without `properties`/`items` only type-checks; `enum` is membership. Raw-registered (MCP) tools are not touched — they validate their own input.
The validator and compiler therefore share exact semantics: the implicit parameter root is an open object; required keys come only from `required: true`; defaults remain annotations; explicit nested objects honor their declared openness; arrays recurse through `items`; scalar literal constraints are type-correct; and `oneOf` accepts exactly one matching branch. Raw-registered tools own their input validation.
## Consequences

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-20-unified-json-value-schema-dsl.md: ab7bb268407ac26230283172ebb291de80412bf2
2026-07-20-unified-json-value-schema-dsl.zh.md: 479699fc2d58666b86861f0ea1db907ae4957dc5

View File

@@ -0,0 +1,32 @@
# Agent Note: Unified JSON-value schema DSL
Status: implemented
English | [中文](2026-07-20-unified-json-value-schema-dsl.zh.md)
## Problem
Tool parameters used a small author DSL while subagent/workflow structured output used a separate raw JSON Schema subset and validator. The two vocabularies disagreed about roots, scalar constraints, and validation, so a typed canonical tool-output contract would either duplicate both paths again or accept schemas that some projection could not enforce.
## Decision
`dsh-tools` owns one JSON-value schema vocabulary with two representations. `ValueSchemaSpec` is the author form for any JSON root; `ParameterSchemaSpec` is its implicit object-property-map form with per-property `required: true`. `JsonSchemaNode` is the raw wire form. Both support string, finite number, integer, boolean, null, array, object, type-correct scalar `enum`/`const`, and exact-one `oneOf`; `{ type: 'json' }` is author-only sugar for an annotation-only unconstrained raw node.
An explicit author object must declare `additionalProperties: true | false`. The implicit parameter root and raw JSON Schema preserve the standard open default. `InferValue<S>` and `InferArgs<P>` derive TypeScript values from the same declarations that `valueSchemaSpecToJsonSchema()` and `parameterSchemaSpecToJsonSchema()` compile. `assertSupportedJsonSchema()` rejects unsupported or misplaced keywords, and `validateJsonSchemaValue()` enforces the accepted subset against the lossless `JsonValue` boundary: no `undefined`, negative zero, non-finite numbers, sparse arrays, cycles, exotic objects, functions, symbols, or other coercive values.
Object-rooting is a consumer rule rather than a vocabulary restriction. Subagent and workflow caller-defined structured outputs use `assertObjectJsonSchema()` and `ObjectJsonSchema`; tool outputs may use any root. Dynamic Cordis registrations rebuild realm-foreign schemas into host-owned JSON, preserve raw-wrapper openness, and require direct-DSL object openness before calling the same compiler.
## Alternatives considered
- **Keep separate parameter and structured-output schema systems:** rejected because every added output construct would require parallel inference, compilation, validation, and code-generation changes with no useful ownership boundary.
- **Adopt full JSON Schema or Ajv:** rejected because the harness must fail on every construct it cannot project into its generated SDK and validators; accepting a larger language would make enforcement and model guidance dishonest.
- **Make every object implicitly open or closed:** rejected because either choice hides a consequential author decision. Only the legacy-shaped implicit parameter root and external raw schema retain an intentional default.
- **Define `oneOf` as first-match:** rejected because branch ordering would change validation semantics and allow overlapping branches to hide ambiguous values.
## Consequences
- Parameter validation, output validation, schema-to-TypeScript generation, subagent/workflow guards, and dynamic registration share one enforced vocabulary.
- Output declarations can infer object, array, scalar, or null roots; subagent/workflow structured outputs remain object-rooted at their existing seams.
- Explicit object openness and type-correct literal constraints make malformed declarations fail during authoring or registration rather than during a later model call.
- Raw tools may still register broader JSON Schema directly, but unified code generation treats unsupported schemas as unknown instead of pretending to enforce them.
- Runtime and compile-time tests cover every root, exact-one overlap/no-match behavior, raw open defaults, explicit openness, lossy JSON values, and inference.

View File

@@ -0,0 +1,32 @@
# Agent Note统一 JSON 值 schema DSL
Status: implemented
[English](2026-07-20-unified-json-value-schema-dsl.md) | 中文
## 问题
工具参数使用一套精简的作者侧 schema DSLsubagent工作流的结构化输出则使用另一套原始 JSON Schema 子集和校验器。两套词汇在根类型、标量约束和校验方式上并不一致;如果继续沿用这种划分,类型化的规范工具输出契约要么还需重复实现两条路径,要么只能接受部分投影无法强制执行的 schema。
## 决策
`dsh-tools` 以两种表示形式统一管理一套 JSON 值 schema 词汇。`ValueSchemaSpec` 是可描述任意 JSON 根类型的作者侧形式;`ParameterSchemaSpec` 是其隐式对象属性映射形式,每个属性可标记 `required: true``JsonSchemaNode` 是原始协议表示。两种形式都支持字符串、有限数值、整数、布尔值、null、数组、对象、类型正确的标量 `enum``const`,以及要求恰好匹配一个分支的 `oneOf``{ type: 'json' }` 仅是作者侧语法糖,会编译为仅含注解、不施加约束的原始节点。
显式的作者侧对象必须声明 `additionalProperties: true | false`。隐式参数根对象和原始 JSON Schema 保留标准的默认开放语义。`InferValue<S>``InferArgs<P>` 根据同一份声明推导 TypeScript 值,`valueSchemaSpecToJsonSchema()``parameterSchemaSpecToJsonSchema()` 也将这些声明编译为 JSON Schema。`assertSupportedJsonSchema()` 会拒绝不受支持或位置错误的关键字;`validateJsonSchemaValue()` 则以无损 `JsonValue` 边界校验受支持的子集,不允许 `undefined`、负零、非有限数、稀疏数组、循环引用、非普通对象、函数、symbol 及其他需要强制转换的值。
对象根限制属于消费方规则,不属于 schema 词汇本身。subagent 和工作流中由调用方定义的结构化输出通过 `assertObjectJsonSchema()``ObjectJsonSchema` 保持对象根限制;工具输出可以使用任意根类型。动态 Cordis 注册会把跨 JavaScript 运行域传入的 schema 重建为当前运行时持有的 JSON保留原始包装层的默认开放语义并要求直接使用 DSL 声明的对象明确选择开放方式,然后再调用同一编译器。
## 备选方案
- **保留两套独立的参数与结构化输出 schema 系统:**不予采纳。每新增一种输出结构,都必须分别修改类型推导、编译、校验和代码生成,而这种重复并未形成有意义的职责边界。
- **采用完整 JSON Schema 或 Ajv**不予采纳。harness 必须拒绝所有无法投影到生成 SDK 和校验器中的结构;如果接受更大的语言子集,强制执行能力和模型指引就会与事实不符。
- **让所有对象默认开放或默认封闭:**不予采纳。这两种选择都会隐藏一项影响重大的作者决策。只有保持旧有形态的隐式参数根对象和外部原始 schema 才有意保留默认值。
- **把 `oneOf` 定义为首个匹配分支:**不予采纳。这样一来,分支顺序会改变校验语义,重叠分支也会掩盖值的歧义。
## 影响
- 参数校验、输出校验、schema 到 TypeScript 的代码生成、subagent工作流门禁和动态注册共用一套强制执行的词汇。
- 输出声明可以推导对象、数组、标量或 null 根类型subagent工作流的结构化输出仍在其现有服务边界保持对象根限制。
- 显式的对象开放方式和类型正确的字面量约束会让格式错误的声明在编写或注册阶段快速失败,而不是拖到后续模型调用时才失败。
- 原始工具仍可直接注册范围更广的 JSON Schema但统一代码生成会把不受支持的 schema 视为未知类型,不会假装自己能够强制执行。
- 运行时和编译期测试覆盖所有根类型、恰好匹配一个分支时的重叠/无匹配行为、原始 schema 的默认开放语义、显式开放方式、有损 JSON 值和类型推导。

View File

@@ -44,7 +44,7 @@ A `workflow` tool mirroring `dsh-tool-subagent`'s synchronous shape: start, awai
An output schema makes a schema-valid committed capture mandatory for successful child completion. The scoped runtime presents the capture tool and instruction, commits only a successful final outcome—including the enclosing `run_code` outcome for an SDK call—denies later side effects after capture becomes pending, and stops the child without another model step after commit. A validation failure remains a retryable tool error; clean completion without a committed capture settles as an error.
`StructuredOutputSchema` is the raw enforceable JSON-Schema subset in `dsh-tools` (single-string `type`, `properties`/`required`/`additionalProperties`, `items`, scalar `enum`/`const`), and unsupported keywords fail loudly because that wire data becomes the capture tool's parameters verbatim. The [agent-scope runtime-design Agent Note](../architecture/2026-07-12-agent-scope-runtime-design.md#structured-output-commits-only-authoritative-outcomes) owns the assembly, commit, guard, and terminal-stop correctness algorithms.
`ObjectJsonSchema` is the object-rooted consumer view of the unified enforceable raw JSON Schema subset in `dsh-tools`; unsupported keywords fail loudly because that wire data becomes the capture tool's parameters verbatim. The [unified JSON-value schema Agent Note](../architecture/2026-07-20-unified-json-value-schema-dsl.md) owns the vocabulary and validation semantics, while the [agent-scope runtime-design Agent Note](../architecture/2026-07-12-agent-scope-runtime-design.md#structured-output-commits-only-authoritative-outcomes) owns the assembly, commit, guard, and terminal-stop algorithms.
## Testing
@@ -68,7 +68,7 @@ Worker-side logic runs through an in-process `MessageChannel` so V8 coverage mea
- **Background execution as the default** (CC's shape): deferred; foreground-synchronous matches `dsh-tool-subagent`'s cut, and background semantics should be designed ONCE across bash/subagent/workflow rather than per-tool.
- **Workflow-layer JSON parsing for `agent({schema})`**: duplicating a seam concern at one consumer while the seam's capability flag stayed dishonestly `false`.
- **Meta embedded in the script as `export const meta = {...}`** (CC's exact format): keeps scripts self-contained and CC scripts drop-in, but obtaining meta requires evaluating model-written text on the host. Even an empty timed vm context cannot bound script-controlled getters when the host reads the resulting object. A JSON parameter removes the scanner, evaluation, and host-spin hole; the cost is that a CC script's meta header must move into the parameter (the body stays drop-in).
- **`SchemaSpec` as the outputSchema type**: the author-facing DSL cannot express what arrives as data and cannot be validated against without conversion loss.
- **`ValueSchemaSpec` as the `outputSchema` wire type**: the author form now has equivalent vocabulary, but a workflow supplies realm-foreign raw JSON Schema data; pretending that runtime data is a trusted author declaration would skip the raw-schema assertion boundary.
- **A schema-object library (zod, or the repo's schemastery) for the structured-output subset**: the schema is wire data — plain JSON that crosses the vm realm boundary in `agent({schema})` and lands verbatim in the forced tool's parameters — exactly where live schema objects cannot sit; consuming raw JSON Schema at runtime would need a third-party converter on top (zod core only emits JSON Schema, not the reverse), and it would put a second schema language beside schemastery's config role.
- **ajv for value validation**: it validates FULL JSON Schema, so the subset gate — the module's actual point, since every accepted keyword must be one the harness enforces — would remain hand-written regardless; it compiles validators through `new Function`; and it would be dsh-tools' first runtime dependency, all to replace the ~70-line value walker while the path-qualified, every-violation error reporting stays custom either way.
- **Provider JSON mode instead of the capture tool:** it guarantees valid JSON, not schema conformance, and its interaction with tool calling is unclear. The capture tool preserves in-turn validation retries. Provider-side strict tool schemas can later narrow the accepted subset without changing this design.

View File

@@ -32,7 +32,7 @@ Sandbox globals are deliberately small: a tagged write-through `console` (`[cord
Mount code crosses the vm boundary through three controls. Dual-realm `instanceof` recognizes both host and vm objects. `harness.defineTool` normalizes results into host-realm JSON and validates the `ToolExecuteReturn` shape before logging. The mounted plugin receives a whitelist context façade, not a raw or pass-through `Context`; framework plumbing and context-valued returns are rejected. Service reads require a declared `inject`, preserving Cordis activation and unload semantics. `ctx.tools.get` exposes only the schema view, so mounted code cannot bypass `ToolRegistry.execute` by calling a definition directly.
The boundary normalizes unambiguous JSON-Schema forms into `SchemaSpec`, including object wrappers, `integer`, and optional fields. Invalid vocabulary fails with the accepted alternatives. Parse, TypeScript, missing-return, Node-API, and duplicate-tool errors include the relevant source line or corrective contract without narrating implementation internals.
The boundary normalizes unambiguous JSON-Schema forms into `ParameterSchemaSpec`, preserving `integer`, raw object openness, and required arrays. Direct DSL object nodes must declare `additionalProperties`; invalid vocabulary fails with the accepted alternatives. Parse, TypeScript, missing-return, Node-API, and duplicate-tool errors include the relevant source line or corrective contract without narrating implementation internals.
### The dynamic group and mount lifecycle
@@ -60,7 +60,7 @@ Model-visible ⟺ logged holds with no new session event type: a mount or unmoun
| Dimension | Structured per-capability tools | Single `cordis_mount` |
|---|---|---|
| Schema correctness | `parameters` is still a model-written JSON object needing SchemaSpec validation, merely one step earlier | The same validation runs at the sandbox boundary, with the same instructive errors |
| Schema correctness | `parameters` is still model-written JSON needing unified-schema validation, merely one step earlier | The same validation runs at the sandbox boundary, with the same instructive errors |
| The code field | An `execute` body is still model-written JS in a vm; the realm and service-call correctness problems are unchanged | One sandbox, one normalization path, one guarded registration |
| Capability coverage | Tools only; listeners, services, `inject` relations each need another structured tool — a surface that grows without bound | One vocabulary (a cordis plugin) covers every effect, present and future |
| Cross-mount composition | Not expressible in a tool-registration payload | Native `provide`/`inject`, ordinary cordis semantics |

View File

@@ -19,7 +19,7 @@ The scoping line was not picked top-down; it was discovered by testing candidate
The rule that settled the remaining cases: ***the type you write, hold, or receive is core; the machinery that types it, renders it, or persists it is a sub-page detail.*** Worked through:
- A data structure is **core** if it flows through the agent-loop spine — the loop holds, derives, streams, or logs it on every turn regardless of which plugins load (`Message`, `StreamChunk`, `SessionEvent`, the `Agent` handle) — **or** it is the single headline type a plugin author writes against a pipeline (`ToolDefinition`).
- `ToolDefinition` is core (it is what every tool author writes) **even though the loop never holds one** — authoring-importance overrides the strict flows-through-spine rule for this one headline type. But its typing machinery — the `SchemaSpec`/`InferArgs` DSL — is a sub-page detail (you write a `ToolDefinition`; the type-level machinery that types it you do not). That is the spine-vs-seam line made sharp.
- `ToolDefinition` is core (it is what every tool author writes) **even though the loop never holds one** — authoring-importance overrides the strict flows-through-spine rule for this one headline type. But its typing machinery — `ValueSchemaSpec`, `ParameterSchemaSpec`, `InferValue`, and `InferArgs` — is a sub-page detail. That is the spine-vs-seam line made sharp.
- `ToolSchema` is core (it is a field of `GenerateOptions`, the model request that flows through every step) even though it is conceptually part of the tool pipeline — *flows through the spine* wins over *conceptual home* when they conflict.
- The tool-presentation vocabulary (`ToolCallView`/`ToolResultView`, …), the `SessionPersistence` durability seam, and bash vocabulary are sub-pages.

View File

@@ -14,7 +14,7 @@ Adopt `fast-check` (a root devDependency) with one `tests/properties.spec.ts` pe
- **dsh-llm / BlockAssembler:** arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: `blocks()` count ≤ distinct indices seen; re-assembly idempotent (`blocks()` is stable across repeated calls and `message().content` mirrors it); `blocks()` never throws and yields only valid content-block tags; `finish` reflects the last `finish` chunk, defaulting to `{kind:'stop'}` when none arrives.
- **dsh-session:** arbitrary event logs. Invariants: `deriveMessages` deterministic; replay-from-seed identical; seq strictly monotonic; non-message events never affect derived history; derived content is decoupled from the log.
- **dsh-tools:** arbitrary `SchemaSpec`. Invariants: JSON Schema `required` equals the `required:true` keys at every level; conversion total; **and the composition with [runtime arg validation](../architecture/2026-06-11-runtime-arg-validation.md)** — generated args satisfying a spec pass `validateArgs`, and targeted corruptions (dropped required key, non-object top level) are rejected. This closes the validator/`InferArgs` drift risk.
- **dsh-tools:** arbitrary `ParameterSchemaSpec`. Invariants: JSON Schema `required` equals the `required:true` keys at every level; conversion is total for valid declarations; **and the composition with [runtime arg validation](../architecture/2026-06-11-runtime-arg-validation.md)** — generated args satisfying a spec pass `validateArgs`, and targeted corruptions (dropped required key, non-object top level) are rejected. Focused cases cover every value root, exact-one overlap/no-match, explicit openness, raw defaults, and lossy JSON. This closes the compiler/validator/`InferArgs` drift risk.
- **dsh-agent-loop:** arbitrary send schedules against a never-exhausting adapter, driven through the `agent/status` settle signal (no wall-clock sleeps). Invariants: no message lost; turn numbers strictly increase; status transitions stay on the legal machine.
## Consequences

View File

@@ -9,7 +9,7 @@ The [subagent seam](../../implemented/feature/2026-06-21-subagent-capability-sea
- **`outputSchema`/`structured` and `toolFilter`** (`SubagentCapabilities`, `SubagentStartRequest`, `SubagentResult` in `packages/subagent/subagent/src/types.ts`): at the decision point, every real provider declared `outputSchema: false, toolFilter: false` (`packages/subagent/subagent-spawn/src/index.ts`, `packages/subagent/subagent-fork/src/index.ts`, `packages/subagent/subagent-acp/src/index.ts`); the sole production `ctx.subagents.start` caller (`packages/subagent/tool-subagent/src/index.ts`) built `{ prompt, parent, signal?, agentOptions? }` and structurally could not set either; `structured` appeared only in the scripted test fixture. The service's capability check carried two assert rows whose only exercisers were the rejection tests.
- **`SubagentRun.sendMessage` / `SubagentRun.resume`** (same file): implemented by NO provider — not even the mock; the spawn spec asserts their *absence*.
The only reason `dsh-subagent` depends on `dsh-tools` at all is `outputSchema`'s `SchemaSpec` type. Three subsequent subagent workstreams (per-session snapshot replay, the fork seed boundary, the ACP backend) landed around this surface without growing a single consumer.
The only reason `dsh-subagent` depended on `dsh-tools` at the decision point was `outputSchema`'s schema type (now `ObjectJsonSchema`). Three subsequent subagent workstreams (per-session snapshot replay, the fork seed boundary, the ACP backend) landed around this surface without growing a single consumer.
## Proposal