docs(tools): record the case-mapping read point and narrow the class-name quantifier
`camelCase`'s `toUpperCase()` is a fourth reader of the engine's Unicode tables, on a table distinct from XID membership and with a wider window: a tool named U+019B passes `isBareIdentifier` and compiles as `async def` on CPython 3.9.6, but Node maps the head to U+A7DC and the declared `class Args` fails there with `invalid non-printable character`. Record it alongside the three XID read points in the renderer docs and in the note's CPython-floor obligation, and pin the derivation with a test. Correct three over-quantified sentences: a `camelCase`-derived class name is evaluated for every tool but only reaches emitted text when some object shape in the schema declares a `TypedDict`. Attribute the `str.isidentifier()` equivalence to `IDENTIFIER` rather than to the predicate, which is deliberately stricter, and restore the antecedent the mode qualification dropped.
This commit is contained in:
@@ -31,8 +31,8 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u
|
||||
*
|
||||
* Python identifiers are not ASCII: `路径` is as legal a field name as `path`,
|
||||
* and rejecting it would degrade the whole enclosing object, dropping every
|
||||
* field's name, requiredness, and type — which under `mode: 'code'` is the
|
||||
* model's only source for them.
|
||||
* field's name, requiredness, and type — information whose only source under
|
||||
* `mode: 'code'` is this generated text.
|
||||
*
|
||||
* NFKC stability is a second and separate condition, because CPython
|
||||
* normalizes identifiers at compile time while JSON keys are compared as
|
||||
@@ -41,9 +41,11 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u
|
||||
* that normalize together would collapse into one declaration. Those names
|
||||
* take the subscript path, which carries their exact bytes.
|
||||
*
|
||||
* The equivalence to `str.isidentifier()` was measured across 21 samples with
|
||||
* zero divergence, on Node 22.23.1 against CPython 3.9.6 — the halves the two
|
||||
* conditions are proxies for, both tested by that run.
|
||||
* `IDENTIFIER`'s equivalence to `str.isidentifier()` was measured across 21
|
||||
* samples with zero divergence, on Node 22.23.1 against CPython 3.9.6. The
|
||||
* predicate as a whole is deliberately stricter than `isidentifier()`, which
|
||||
* does not test NFKC stability: `'field'.isidentifier()` is True and this
|
||||
* returns false.
|
||||
*
|
||||
* Both conditions are evaluated against the ENGINE's Unicode tables, and the
|
||||
* two sides are versioned independently — `\p{XID_Start}`/`\p{XID_Continue}`
|
||||
@@ -62,14 +64,22 @@ const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u
|
||||
* condition reduces to the same skew, since normalization stability guarantees
|
||||
* an assigned character's normalization never changes afterwards.
|
||||
*
|
||||
* This predicate is not the only reader of those tables. {@link camelCase}
|
||||
* reads them too, through its split set and its head test, and its output is
|
||||
* emitted for EVERY tool — including one this predicate rejected, whose
|
||||
* `TypedDict` is still declared and named. A tool named `zz-\u{1E4D0}x` never
|
||||
* This predicate is not the only reader of engine tables. {@link camelCase}
|
||||
* reads them at three further points — its split set, its head test, and its
|
||||
* `toUpperCase()` case mapping — and this predicate's verdict gates none of
|
||||
* them: a class name derived there reaches emitted text whenever any object
|
||||
* shape in the tool's schema declares a `TypedDict`, including for a tool this
|
||||
* predicate rejected. A tool named `zz-\u{1E4D0}x` with such parameters never
|
||||
* reaches the skew here (the `-` rejects it outright) yet emits
|
||||
* `class Zz\u{1E4D0}xArgs`, which that same 3.9.6 refuses. Closing the
|
||||
* exposure therefore covers all three read points, not this predicate alone;
|
||||
* it needs the target interpreter's version, which the backend reporting
|
||||
* `class Zz\u{1E4D0}xArgs`, which that same 3.9.6 refuses. The case mapping is
|
||||
* a separate table rather than an XID membership test, and it fails on names
|
||||
* both conditions above accept: `\u{019B}` is XID_Start and NFKC-stable, so
|
||||
* this predicate accepts it and `async def \u{019B}` compiles on 3.9.6, but
|
||||
* Node uppercases it to `\u{A7DC}` — unassigned in that CPython, whose own
|
||||
* `.upper()` is the identity here — and the declared `class \u{A7DC}Args`
|
||||
* fails with `invalid non-printable character U+A7DC`. Closing the exposure
|
||||
* therefore covers all four read points, not this predicate alone; it needs
|
||||
* the target interpreter's version, which the backend reporting
|
||||
* `language: 'python'` owns and which is unpublished on this base, so the note
|
||||
* records it as that PR's decision.
|
||||
*
|
||||
@@ -237,10 +247,13 @@ function docLines(description: unknown, indent: number): string[] {
|
||||
* normalizing only the un-prefixed part would emit a name CPython compiles to
|
||||
* a different symbol. The second call is idempotent on the un-prefixed arm.
|
||||
*
|
||||
* The split set and the head test read the engine's Unicode tables, so this
|
||||
* function carries the same version skew {@link isBareIdentifier} documents,
|
||||
* by an independent path: a class name derived here is emitted for every tool,
|
||||
* including one the predicate rejected.
|
||||
* The split set, the head test, and `toUpperCase()` all read the engine's
|
||||
* Unicode tables, so this function carries the same version skew
|
||||
* {@link isBareIdentifier} documents, by paths independent of it: a class name
|
||||
* derived here reaches emitted text whenever any object shape in the tool's
|
||||
* schema declares a `TypedDict`, and the predicate's verdict on the tool name
|
||||
* does not gate that. The case mapping is the one that can fail on a name the
|
||||
* predicate accepted; the worked example is there.
|
||||
* @param raw - the schema field or tool name to derive from.
|
||||
* @returns a class-name segment safe to emit.
|
||||
*/
|
||||
|
||||
@@ -472,6 +472,27 @@ describe('renderToolsSdkPy', () => {
|
||||
expect(text).not.toContain('async def find')
|
||||
})
|
||||
|
||||
it('derives a class name through the case-mapping table, independently of the bare-name predicate', () => {
|
||||
// The head capitalization reads a table `isBareIdentifier` never consults,
|
||||
// so the class-name path can carry a character the predicate cleared. ƛ
|
||||
// (U+019B) is XID_Start and NFKC-stable, so the method is emitted bare;
|
||||
// the head maps to (U+A7DC), a code point the engine's tables assign and
|
||||
// an older interpreter's do not. This pins which table produced the name,
|
||||
// so a change to the mapping step shows up here rather than only in a
|
||||
// downstream Python parse.
|
||||
const text = renderToolsSdkPy([
|
||||
{
|
||||
name: 'ƛ',
|
||||
description: 'Lambda with stroke.',
|
||||
parameters: { type: 'object', additionalProperties: false, properties: { q: { type: 'string' } }, required: ['q'] },
|
||||
output: { type: 'string' },
|
||||
},
|
||||
])
|
||||
expect(text).toContain('async def ƛ(self, args: Args) -> str:')
|
||||
expect(text).toContain('class Args(TypedDict):')
|
||||
expect(text).not.toContain('class ƛArgs')
|
||||
})
|
||||
|
||||
it('drops a surrogate half rather than cutting a pair when capping an astral class-name base', () => {
|
||||
// Class-name bases are capped by `slice`, which counts UTF-16 code units,
|
||||
// so a boundary landing inside an astral pair would leave a lone high
|
||||
|
||||
Reference in New Issue
Block a user