fix(tools): correct the trim-order claim and check the Literal escape dependency
trim and escape commute for every input, so the new whitespace test does
not pin their order: UNPRINTABLE and LONE_SURROGATE are disjoint from the
set trim() strips, and both escapes emit plain non-whitespace ASCII,
leaving the leading and trailing whitespace runs byte-identical. State that instead of the false causal clause.
pyScalar's Literal path escapes nothing itself -- JSON.stringify is what
keeps it parseable, covering NUL and, under ES2019 well-formed
stringification, unpaired surrogates. Record the dependency and turn it
into a checked invariant. Pin the docstring emission site for a lone
surrogate too, mirroring the NUL case.
Two docstring corrections: describe's caller enumeration omitted the
synthetic { description } wrapper docLines builds, and "special in
statement position" does not describe `_`, which is special in a match
pattern. Both keep the conclusion they support.
Note which of the two table guards fires depends on the entry point.
This commit is contained in:
@@ -27,9 +27,11 @@ const IDENTIFIER = /^[A-Za-z_][A-Za-z0-9_]*$/
|
||||
* class-syntax `TypedDict` field. Such a tool renders under subscript access
|
||||
* and such an object degrades to ``dict[str, Any]`` — the model still reaches
|
||||
* every tool and field without collisions.
|
||||
* Soft keywords (``match``, ``case``, ``type``, ``_``) are deliberately
|
||||
* ABSENT: they are only special in statement position, so ``match: str`` as a
|
||||
* field and ``async def match(...)`` as a method are both legal, and including
|
||||
* Soft keywords (``match``, ``case``, ``type``, ``_`` — the language
|
||||
* reference's whole set) are deliberately ABSENT: each is special in exactly
|
||||
* one syntactic position — a statement head, or a ``match`` pattern for ``_``
|
||||
* — so ``match: str`` as a field and ``async def match(...)`` as a method are
|
||||
* both legal, and including
|
||||
* them would needlessly degrade common search/regex tool fields to
|
||||
* ``dict[str, Any]``. Underscore-leading names are handled separately, not
|
||||
* here: a non-dunder ``__token`` name-mangles, a dunder present on
|
||||
@@ -113,8 +115,9 @@ const LONE_SURROGATE = /[\ud800-\udfff]/gu
|
||||
/**
|
||||
* The collapsed one-line `description` of a schema node (byte-stable across
|
||||
* formatting churn), or `undefined` when the node carries none. Every caller
|
||||
* passes an object (validated property nodes, or the ToolSdkSchema itself),
|
||||
* so only the description field needs guarding. A description that collapses
|
||||
* passes an object — a validated property node, the `ToolSdkSchema` itself, or
|
||||
* the `{ description }` wrapper {@link docLines} synthesizes — so only the
|
||||
* description field needs guarding. A description that collapses
|
||||
* to nothing (empty, or whitespace only) is `undefined` too: it documents the
|
||||
* node no better than an absent one, and emitting it would leave an empty
|
||||
* `"""` docstring or a bare `# ` line in the SDK. Only ECMAScript whitespace
|
||||
@@ -257,6 +260,15 @@ function childClassName(base: string, segment: string): string {
|
||||
* representable as a JavaScript number, so the SDK would document a value no
|
||||
* program can pass. The TS flavor needs no counterpart: its literal is re-read
|
||||
* by a JS parser back into the same double.
|
||||
*
|
||||
* `JSON.stringify` is also what keeps this path's output parseable, and it is
|
||||
* the only thing that does: it escapes both code points CPython refuses in
|
||||
* source — NUL among the C0 controls, and unpaired surrogates under ES2019
|
||||
* well-formed stringification, which the engines range guarantees. The
|
||||
* `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE}
|
||||
* because nothing quotes it. DEL and the C1 controls do reach a `Literal[...]`
|
||||
* raw — legal but invisible, byte-for-byte as in the TS flavor; escaping them
|
||||
* is a both-flavors change.
|
||||
*/
|
||||
function pyScalar(value: JsonSchemaScalar): string {
|
||||
if (value === true) return 'True'
|
||||
|
||||
@@ -51,6 +51,15 @@ describe('jsonSchemaToPy', () => {
|
||||
expect(jsonSchemaToPy({ type: 'string', enum: [] })).toBe('Any')
|
||||
})
|
||||
|
||||
it('leans on JSON.stringify to keep a Literal parseable', () => {
|
||||
// The two code points CPython refuses in source reach this path as well,
|
||||
// and nothing here escapes them itself — `JSON.stringify` does, NUL as a
|
||||
// C0 control and a lone surrogate under ES2019 well-formed stringification.
|
||||
// Python decodes both escapes back to the value the schema declared.
|
||||
expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`)
|
||||
expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`)
|
||||
})
|
||||
|
||||
it('emits exact digits for a beyond-safe-range integer literal', () => {
|
||||
// Python integers are arbitrary-precision, so the emitted digits ARE the
|
||||
// value the model programs against. `String(2 ** 60)` prints the rounded
|
||||
@@ -750,7 +759,9 @@ describe('renderToolsSdkPy', () => {
|
||||
// so the block stays parseable with the code point intact.
|
||||
expect(renderToolsSdkPy([described('zero\u200bwidth')])).toContain('"""zero\u200bwidth"""')
|
||||
// Whitespace around a surviving control character is not an absent
|
||||
// description: the escape runs before the trim, so what is left is visible.
|
||||
// description. The escape's output is non-whitespace ASCII and the escaped
|
||||
// sets are disjoint from what `trim()` strips, so the two operations touch
|
||||
// different characters and their order is unobservable.
|
||||
expect(renderToolsSdkPy([described(' \u0085 ')])).toContain(String.raw`# \x85`)
|
||||
})
|
||||
|
||||
@@ -764,6 +775,7 @@ describe('renderToolsSdkPy', () => {
|
||||
const high = renderToolsSdkPy([described('a\ud800b')])
|
||||
expect(high).not.toContain('\ud800')
|
||||
expect(high).toContain(String.raw`# a\ud800b`)
|
||||
expect(high).toContain(String.raw`"""a\\ud800b"""`)
|
||||
// A lone LOW surrogate is just as unencodable, and `\xNN` reaches neither.
|
||||
expect(renderToolsSdkPy([described('a\udfffb')])).toContain(String.raw`# a\udfffb`)
|
||||
// A well-formed pair is ONE astral code point, not two surrogates — the
|
||||
|
||||
Reference in New Issue
Block a user