docs(tools): close the NEL half of the raw pass-through and reflow
Four non-blocking review suggestions, all prose plus one assertion. `UNPRINTABLE`'s new sentence named three characters but only two raw-reach points, leaving "and NEL?" open; it now says all three reach text through `pyScalar`, and how the description path handles each. `pyScalar`'s raw-pass-through list already covered NEL under "the C1 controls", and the test now pins it alongside LS and PS, so the docstring's claim has a mechanical check for every character it names. The test title said "paragraph separators" for a pair whose first member is LINE SEPARATOR. Two docstring paragraphs are reflowed to the file's ~80 columns after the earlier inserts left short lines. The note's CPython-floor obligation gains a second axis: the `typing` names the block spells (`TypedDict` 3.8, `NotRequired` 3.11, `A | B` annotations 3.10) are definition-time evaluation floors, not parse floors, so the floor PR does not read "parseable on the supported range" as "executable on it".
This commit is contained in:
@@ -172,7 +172,9 @@ interface RenderState {
|
||||
* string at run time but do not end a physical line in source — measured on
|
||||
* CPython 3.9.6 and 3.12.13, each accepted in both positions with the value
|
||||
* round-tripping — so they are safe raw wherever they reach emitted text
|
||||
* unescaped, which for LS and PS is {@link pyScalar}'s `JSON.stringify`.
|
||||
* unescaped, which for all three is {@link pyScalar}'s `JSON.stringify`: the
|
||||
* `description` path escapes NEL under the class above and folds LS and PS in
|
||||
* {@link describe}'s `\s+` collapse, both of them being ECMAScript `\s`.
|
||||
*/
|
||||
const UNPRINTABLE = /[\u0000-\u0008\u000e-\u001f\u007f-\u009f]/g
|
||||
|
||||
@@ -404,24 +406,23 @@ function childClassName(base: string, segment: string): string {
|
||||
* code point CPython refuses anywhere in source — NUL among the C0 controls,
|
||||
* and the whole D800–DFFF unpaired-surrogate block, escaped under ES2019
|
||||
* well-formed stringification, which the engines range guarantees — and the
|
||||
* ones that break this line in particular,
|
||||
* a bare `"` closing the literal early, a trailing odd backslash eating the
|
||||
* closing quote, and a bare LF/CR ending it before its terminator. The
|
||||
* `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE}
|
||||
* because nothing quotes it, and folds newlines in {@link describe}.
|
||||
* ones that break this line in particular, a bare `"` closing the literal
|
||||
* early, a trailing odd backslash eating the closing quote, and a bare LF/CR
|
||||
* ending it before its terminator. The `description` path carries
|
||||
* {@link UNPRINTABLE} and {@link LONE_SURROGATE} because nothing quotes it,
|
||||
* and folds newlines in {@link describe}.
|
||||
*
|
||||
* That leans on a coincidence worth naming: every escape `JSON.stringify` can
|
||||
* emit (`\"`, `\\`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`) is also a Python
|
||||
* escape denoting the same character, so the emitted `Literal[...]` both
|
||||
* parses and decodes back to the value the schema declared. DEL, the C1
|
||||
* controls, and LS/PS (U+2028/U+2029) do reach it raw — legal but invisible,
|
||||
* byte-for-byte as in the TS flavor; escaping them is a both-flavors change.
|
||||
* LS and PS are legal here for the reason {@link UNPRINTABLE} records: they
|
||||
* are `str.splitlines()` boundaries, not tokenizer line terminators. The
|
||||
* subscript tool-name
|
||||
* comment quotes its name through its own call to the same `JSON.stringify`,
|
||||
* never through this function, and inherits both halves — escapes and
|
||||
* pass-throughs alike.
|
||||
* controls (NEL among them), and LS/PS (U+2028/U+2029) do reach it raw —
|
||||
* legal but invisible, byte-for-byte as in the TS flavor; escaping them is a
|
||||
* both-flavors change. Those last three are legal here for the reason
|
||||
* {@link UNPRINTABLE} records: they are `str.splitlines()` boundaries, not
|
||||
* tokenizer line terminators. The subscript tool-name comment quotes its name
|
||||
* through its own call to the same `JSON.stringify`, never through this
|
||||
* function, and inherits both halves — escapes and pass-throughs alike.
|
||||
*/
|
||||
function pyScalar(value: JsonSchemaScalar): string {
|
||||
if (value === true) return 'True'
|
||||
|
||||
@@ -67,17 +67,20 @@ describe('jsonSchemaToPy', () => {
|
||||
expect(jsonSchemaToPy({ type: 'string', const: 'ends\\' })).toBe(String.raw`Literal["ends\\"]`)
|
||||
})
|
||||
|
||||
it('passes the paragraph separators through raw, which CPython does not treat as line terminators', () => {
|
||||
// `JSON.stringify` escapes LF and CR but not LS/PS (U+2028/U+2029), which
|
||||
// is safe here and not by accident: they are `str.splitlines()` boundaries,
|
||||
// not tokenizer line terminators, so they end neither a string literal nor
|
||||
// a `#` comment — measured on CPython 3.9.6 and 3.12.13. Pinning the raw
|
||||
// form keeps a later "escape them for symmetry with LF" change from
|
||||
// landing as a silent both-flavors divergence from `ts-types`.
|
||||
// Escapes below — the two forms denote the same bytes, and neither
|
||||
// character has a visible width.
|
||||
it('passes the line and paragraph separators through raw, which CPython does not treat as line terminators', () => {
|
||||
// `JSON.stringify` escapes LF and CR but not NEL (U+0085), LS (U+2028), or
|
||||
// PS (U+2029), which is safe here and not by accident: those three are
|
||||
// `str.splitlines()` boundaries, not tokenizer line terminators, so they
|
||||
// end neither a string literal nor a `#` comment — measured on CPython
|
||||
// 3.9.6 and 3.12.13. Pinning the raw form keeps a later "escape them for
|
||||
// symmetry with LF" change from landing as a silent both-flavors
|
||||
// divergence from `ts-types`. Escapes below — the two forms denote the
|
||||
// same bytes, and none of the three has a visible width.
|
||||
expect(jsonSchemaToPy({ type: 'string', const: 'a\u2028b' })).toBe('Literal["a\u2028b"]')
|
||||
expect(jsonSchemaToPy({ type: 'string', enum: ['a\u2029b'] })).toBe('Literal["a\u2029b"]')
|
||||
// NEL is inside `UNPRINTABLE`'s class, so the description path escapes it;
|
||||
// this is the one route that carries it raw.
|
||||
expect(jsonSchemaToPy({ type: 'string', const: 'a\u0085b' })).toBe('Literal["a\u0085b"]')
|
||||
})
|
||||
|
||||
it('emits exact digits for a beyond-safe-range integer literal', () => {
|
||||
|
||||
Reference in New Issue
Block a user