docs: equal-authority pairing with sidecar consistency records

Redesign per review: neither language is canonical. A pair is three sibling files — foo.md, foo.zh.md, foo.i18n.yaml — and either language may be authored first (a Chinese-first RFC is as legitimate as an English-first one). The sidecar record holds the FULL git blob hash of both sides as of the last confirmed-consistent state, replacing the in-file one-directional fingerprint; editing either side without re-confirming the pair goes red. New --write mode re-records a pair after both sides are brought in line, making the confirmation a reviewable yaml diff. Pairs merge whole (completeness enforced). - gate rewritten around pair anchors (union of .zh.md and .i18n.yaml remnants) so half-deleted pairs are caught from either side; red/green proven for en-only edit, zh-only edit, missing record, and a record for an excluded file - verify-rfc-classification now skips .zh.md counterparts (same RFC, indexed via its English filename; the pairing gate owns consistency) - docs/i18n/README.md + translation-rules.md reframed bidirectionally (terminology table binds both directions; typography section governs the Chinese side); zh counterparts updated; skill workflow updated - RFC amended to the shipped design, records the English-canonical in-file-fingerprint model as considered-and-revised; RFC translated (docs/rfc/.../2026-07-02-bilingual-docs-and-pairing-gate.zh.md) and added to the required frontier - generated docs stay excluded with the follow-up recorded: teach the generators to emit Chinese, then de-list
2026-07-03 07:41:24 -07:00
parent bb10931478
commit ec05295a0c
18 changed files with 307 additions and 171 deletions
--- a/scripts/translation-pairing.manifest.json
+++ b/scripts/translation-pairing.manifest.json
@@ -3,7 +3,8 @@
    "README.md",
    "docs/development.md",
    "docs/i18n/README.md",
-    "docs/i18n/translation-rules.md"
+    "docs/i18n/translation-rules.md",
+    "docs/rfc/implemented/process/2026-07-02-bilingual-docs-and-pairing-gate.md"
  ],
  "excluded": [
    "docs/AGENTS.md",
--- a/scripts/verify-rfc-classification.ts
+++ b/scripts/verify-rfc-classification.ts
@@ -68,6 +68,9 @@ for (const lifecycle of LIFECYCLES) {
    const segs = match.split('/')
    // Allowlisted file directly at the lifecycle root (e.g. implemented/AGENTS.md).
    if (segs.length === 2 && ROOT_ALLOWLIST.has(segs[1] ?? '')) continue
+    // A Chinese counterpart (foo.zh.md, docs/i18n/README.md) is the SAME RFC,
+    // indexed via its English filename; the pairing gate owns its consistency.
+    if (match.endsWith('.zh.md')) continue
    const cls = segs[1]
    const base = segs[2]
    if (segs.length !== 3 || cls === undefined || base === undefined) {
--- a/scripts/verify-translation-pairing.ts
+++ b/scripts/verify-translation-pairing.ts
@@ -1,41 +1,50 @@
 /**
 * Doc-sync gate: enforce the bilingual pairing contract (docs/i18n/README.md).
- * English is canonical; the translation of `foo.md` is a sibling `foo.zh.md`
- * whose FIRST line fingerprints the English source it was translated from:
+ * English and Chinese carry EQUAL authority — either language may be authored
+ * first — so consistency is recorded per pair in a sidecar metadata file,
+ * `foo.i18n.yaml`, holding the full git blob hash of BOTH files as of the last
+ * time a human confirmed the two say the same thing:
 *
- *   <!-- i18n-source: docs/foo.md@<first 12 hex of git blob hash> -->
+ *   foo.md: <40-hex blob hash>
+ *   foo.zh.md: <40-hex blob hash>
 *
 * The gate checks, mechanically, the checkable half of the contract:
 *
- *   1. Every English file in the manifest's `required` list has a `.zh.md`
- *      sibling (the enforcement frontier — grows batch by batch).
- *   2. Every EXISTING `.zh.md`, required or not, is sound: its source exists
- *      (no orphans), its fingerprint equals the source's current blob hash
- *      (no stale translations), both sides carry the language-switcher link,
- *      and its structural signature matches the source one to one — heading
+ *   1. Every file in the manifest's `required` list has a COMPLETE pair
+ *      (the enforcement frontier — grows batch by batch).
+ *   2. Every pair that exists at all is complete and consistent: all three
+ *      files present (a `.zh.md` or a `.i18n.yaml` without its counterparts
+ *      is an error — pairs merge whole, never half), each side's current
+ *      blob hash equals the recorded one (an edit to EITHER side without a
+ *      re-confirmed counterpart goes red), both sides carry the language
+ *      switcher, and the structural signatures match one to one — heading
 *      depths in order, fenced code blocks VERBATIM (info string + content),
 *      table column counts, list kinds, and every link target except the
 *      switcher itself.
 *   3. `excluded` files (generated docs, agent instructions, the bilingual
- *      terminology table) have no `.zh.md` at all.
+ *      terminology table) have no `.zh.md` and no `.i18n.yaml` at all.
 *
- * What it deliberately does NOT check is translation quality: a green gate
- * means the pair is fresh and structurally sound, not that the Chinese is
- * faithful — accuracy, terminology, and tone are the human reviewer's half
- * of the contract (docs/i18n/translation-rules.md).
+ * What it deliberately does NOT check is translation quality or which side
+ * is "right": a green gate means the pair was confirmed consistent at these
+ * exact contents, not that the confirmation was sound — accuracy,
+ * terminology, and tone are the human reviewer's half of the contract
+ * (docs/i18n/translation-rules.md).
 *
- * The fingerprint is a git BLOB hash, not a commit hash, so a translation
- * updated in the same PR as its English source verifies without any history
- * lookup: staleness is a pure content comparison, computed here directly
- * (sha1 of `blob <size>\0<content>`) without spawning git.
+ * Blob hashes, not commit hashes, so a pair edited in the same PR verifies
+ * without any history lookup: consistency is a pure content comparison,
+ * computed here directly (sha1 of `blob <size>\0<content>`) without spawning
+ * git. The recorded hash also recovers the last-confirmed text of either
+ * side (`git cat-file -p <hash>`) for diff-based minimal updates.
 *
- * Run: `tsx scripts/verify-translation-pairing.ts` — or with `--list` to print
- * the translation state (missing/stale/ok) of every in-scope document as a
- * work list; `--list` always exits 0.
+ * Run: `tsx scripts/verify-translation-pairing.ts` — or with `--list` to
+ * print the pairing state of every in-scope document as a work list (always
+ * exits 0), or with `--write` to (re)record both hashes for every complete
+ * pair after you have brought the two sides back in line (the resulting
+ * yaml diff is the reviewable act of confirming consistency).
 */

 import { createHash } from 'node:crypto'
-import { existsSync, readFileSync } from 'node:fs'
+import { existsSync, readFileSync, writeFileSync } from 'node:fs'
 import { basename, join, resolve } from 'node:path'
 import { glob } from 'node:fs/promises'
 import { fromMarkdown } from 'mdast-util-from-markdown'
@@ -45,9 +54,10 @@ import type { Nodes } from 'mdast'

 const root = resolve(import.meta.dirname, '..')
 const listMode = process.argv.includes('--list')
+const writeMode = process.argv.includes('--write')

 /** Scope of the bilingual contract: the root README and the docs tree. */
-const SCOPE_PATTERNS = ['README.md', 'README.zh.md', 'docs/**/*.md']
+const SCOPE_PATTERNS = ['README.md', 'README.zh.md', 'README.i18n.yaml', 'docs/**/*.md', 'docs/**/*.i18n.yaml']

 /** The enforcement frontier and the never-paired set (docs/i18n/README.md § Scope). */
 interface Manifest {
@@ -56,9 +66,6 @@ interface Manifest {
 }
 const manifest = JSON.parse(readFileSync(join(root, 'scripts/translation-pairing.manifest.json'), 'utf8')) as Manifest

-/** First line of a translation: fingerprint of the English source it renders. */
-const FINGERPRINT = /^<!-- i18n-source: (?<path>\S+)@(?<hash>[0-9a-f]{12}) -->$/
-
 /**
 * An excluded entry ending in `/` excludes the whole directory. The trailing
 * slash IS the path boundary — `docs/tool-catalog/` cannot prefix-match a
@@ -69,18 +76,50 @@ function isExcluded(file: string): boolean {
  return manifest.excluded.some(entry => (entry.endsWith('/') ? file.startsWith(entry) : file === entry))
 }

-/** Git blob hash (what `git hash-object` prints), truncated to 12 hex digits. */
+/** Full git blob hash (what `git hash-object` prints). */
 function blobHash(content: Buffer): string {
  const hash = createHash('sha1')
  hash.update(`blob ${content.byteLength}\0`)
  hash.update(content)
-  return hash.digest('hex').slice(0, 12)
+  return hash.digest('hex')
+}
+
+/** The three paths of a pair, derived from the English-file path. */
+function pairPaths(source: string): { zh: string; meta: string } {
+  return { zh: source.replace(/\.md$/, '.zh.md'), meta: source.replace(/\.md$/, '.i18n.yaml') }
+}
+
+const META_LINE = /^([^:#]+\.md): ([0-9a-f]{40})$/
+
+/** Parse a `foo.i18n.yaml` consistency record: basename → recorded blob hash. */
+function parseMeta(content: string): Map<string, string> | undefined {
+  const out = new Map<string, string>()
+  for (const line of content.split('\n')) {
+    if (line === '' || line.startsWith('#')) continue
+    const match = META_LINE.exec(line)
+    if (!match?.[1] || !match[2]) return undefined
+    out.set(match[1], match[2])
+  }
+  return out
+}
+
+/** Render a `foo.i18n.yaml` consistency record. */
+function renderMeta(source: string, sourceHash: string, zh: string, zhHash: string): string {
+  return [
+    '# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each',
+    '# side as of the last confirmed-consistent state. Both languages carry equal authority;',
+    '# after editing either side, bring the other along and re-record with:',
+    '#   pnpm run verify-translation-pairing --write',
+    `${basename(source)}: ${sourceHash}`,
+    `${basename(zh)}: ${zhHash}`,
+    '',
+  ].join('\n')
 }

 /**
- * The structural signature a translation must reproduce from its source, as
- * ordered sequences so a swap or a level change is caught, not just a count
- * change. Prose is deliberately absent: the gate checks shape, never wording.
+ * The structural signature the two sides must share, as ordered sequences so
+ * a swap or a level change is caught, not just a count change. Prose is
+ * deliberately absent: the gate checks shape, never wording.
 */
 interface Signature {
  /** Heading depths in document order (h2 → 2). */
@@ -157,7 +196,7 @@ function signatureDiff(source: Signature, zh: Signature): string[] {
    const length = Math.max(s.length, z.length)
    for (let i = 0; i < length; i++) {
      if (s[i] !== z[i]) {
-        out.push(`${field} #${i + 1} diverges from the source: source has ${show(s[i])}, translation has ${show(z[i])}`)
+        out.push(`${field} #${i + 1} diverges between the pair: ${show(s[i])} vs ${show(z[i])}`)
        break
      }
    }
@@ -169,16 +208,34 @@ function parse(content: string): Nodes {
  return fromMarkdown(content, { extensions: [gfm()], mdastExtensions: [gfmFromMarkdown()] })
 }

-// Enumerate the scope once, split into sources and translations.
+// Enumerate the scope once.
 const files = new Set<string>()
 for (const pattern of SCOPE_PATTERNS) {
  for await (const match of glob(pattern, { cwd: root })) files.add(match)
 }
 const translations = [...files].filter(f => f.endsWith('.zh.md')).sort()
-const sources = [...files].filter(f => !f.endsWith('.zh.md')).sort()
+const metas = [...files].filter(f => f.endsWith('.i18n.yaml')).sort()
+const sources = [...files].filter(f => f.endsWith('.md') && !f.endsWith('.zh.md')).sort()
+
+// --write: (re)record both hashes for every complete pair, creating missing records.
+if (writeMode) {
+  let written = 0
+  for (const source of sources) {
+    if (isExcluded(source)) continue
+    const { zh, meta } = pairPaths(source)
+    if (!existsSync(join(root, zh))) continue
+    const record = renderMeta(source, blobHash(readFileSync(join(root, source))), zh, blobHash(readFileSync(join(root, zh))))
+    if (existsSync(join(root, meta)) && readFileSync(join(root, meta), 'utf8') === record) continue
+    writeFileSync(join(root, meta), record)
+    console.log(`verify-translation-pairing: recorded ${meta}`)
+    written++
+  }
+  console.log(`verify-translation-pairing: ${written} record(s) written; run the check to validate the pairs.`)
+  process.exit(0)
+}

 const errors: string[] = []
-const state = new Map<string, 'ok' | 'stale' | 'missing'>()
+const state = new Map<string, 'ok' | 'out-of-sync' | 'missing'>()

 // 1. Required pairs exist.
 for (const req of manifest.required) {
@@ -186,82 +243,90 @@ for (const req of manifest.required) {
    errors.push(`${req}: listed in translation-pairing.manifest.json \`required\` but the file does not exist`)
    continue
  }
-  const zh = req.replace(/\.md$/, '.zh.md')
+  const { zh } = pairPaths(req)
  if (!existsSync(join(root, zh))) {
    errors.push(`${req}: required to have a translation, but ${zh} does not exist`)
    state.set(req, 'missing')
  }
 }

-// 2. Every existing translation is sound.
-for (const zh of translations) {
-  const source = zh.replace(/\.zh\.md$/, '.md')
-  const sourceAbs = join(root, source)
-  if (!existsSync(sourceAbs)) {
-    errors.push(`${zh}: orphan — its English source ${source} does not exist (delete or rename the translation alongside its source)`)
-    continue
-  }
+// 2. Every pair that exists at all is complete and consistent. Anchor on the
+// union of .zh.md files and .i18n.yaml records so a half-deleted pair is
+// caught from either remnant.
+const pairAnchors = new Set<string>()
+for (const zh of translations) pairAnchors.add(zh.replace(/\.zh\.md$/, '.md'))
+for (const meta of metas) pairAnchors.add(meta.replace(/\.i18n\.yaml$/, '.md'))
+
+for (const source of [...pairAnchors].sort()) {
+  const { zh, meta } = pairPaths(source)
+  const have = { source: existsSync(join(root, source)), zh: existsSync(join(root, zh)), meta: existsSync(join(root, meta)) }
+
  if (isExcluded(source)) {
-    errors.push(`${zh}: ${source} is excluded from pairing (generated or bilingual-by-construction); this translation must not exist`)
+    if (have.zh) errors.push(`${zh}: ${source} is excluded from pairing (generated or bilingual-by-construction); this translation must not exist`)
+    if (have.meta) errors.push(`${meta}: ${source} is excluded from pairing; this consistency record must not exist`)
+    continue
+  }
+  const missing = Object.entries(have).filter(([, ok]) => !ok).map(([k]) => (k === 'source' ? source : k === 'zh' ? zh : meta))
+  if (missing.length > 0) {
+    errors.push(`${source}: incomplete pair — missing ${missing.join(', ')} (pairs merge whole: both languages plus the .i18n.yaml record)`)
    continue
  }

-  const zhContent = readFileSync(join(root, zh), 'utf8')
-  const firstLine = zhContent.split('\n', 1)[0] ?? ''
-  const match = FINGERPRINT.exec(firstLine)
-  if (!match?.groups) {
-    errors.push(`${zh}: first line is not an i18n-source fingerprint (expected \`<!-- i18n-source: ${source}@<12-hex> -->\`, got \`${firstLine.slice(0, 60)}\`)`)
-    continue
-  }
-  if (match.groups['path'] !== source) {
-    errors.push(`${zh}: fingerprint names ${match.groups['path']} but the sibling source is ${source}`)
+  const sourceContent = readFileSync(join(root, source))
+  const zhContent = readFileSync(join(root, zh))
+  const record = parseMeta(readFileSync(join(root, meta), 'utf8'))
+  if (!record || record.size !== 2 || !record.has(basename(source)) || !record.has(basename(zh))) {
+    errors.push(`${meta}: malformed consistency record (expected exactly \`${basename(source)}: <40-hex>\` and \`${basename(zh)}: <40-hex>\`)`)
    continue
  }

-  const sourceContent = readFileSync(sourceAbs)
-  const current = blobHash(sourceContent)
-  if (match.groups['hash'] !== current) {
-    errors.push(`${zh}: stale — fingerprint ${match.groups['hash']} but ${source} is now ${current} (update the translation, then re-fingerprint)`)
-    state.set(source, 'stale')
+  let consistent = true
+  for (const [file, content] of [[source, sourceContent], [zh, zhContent]] as const) {
+    const current = blobHash(content)
+    if (record.get(basename(file)) !== current) {
+      errors.push(`${file}: out of sync — content no longer matches the pair's last confirmed-consistent state in ${meta} (bring the other side along, then re-record with --write)`)
+      consistent = false
+    }
+  }
+  if (!consistent) {
+    state.set(source, 'out-of-sync')
    continue
  }

-  const zhTree = parse(zhContent)
  const sourceTree = parse(sourceContent.toString('utf8'))
+  const zhTree = parse(zhContent.toString('utf8'))
  if (!linksTo(zhTree, basename(source))) {
    errors.push(`${zh}: missing language switcher — no link to ${basename(source)}`)
  }
  if (!linksTo(sourceTree, basename(zh))) {
    errors.push(`${source}: missing language switcher — no link back to ${basename(zh)}`)
  }
-  const sourceSig = signatureOf(sourceTree, basename(zh))
-  const zhSig = signatureOf(zhTree, basename(source))
-  for (const divergence of signatureDiff(sourceSig, zhSig)) {
-    errors.push(`${zh}: ${divergence}`)
+  for (const divergence of signatureDiff(signatureOf(sourceTree, basename(zh)), signatureOf(zhTree, basename(source)))) {
+    errors.push(`${source} ↔ ${zh}: ${divergence}`)
  }
  if (!state.has(source)) state.set(source, 'ok')
 }

-// Complete the state map for --list: any in-scope, non-excluded source with no translation yet is backlog.
+// Complete the state map for --list: any in-scope, non-excluded document with no pair yet is backlog.
 for (const source of sources) {
  if (!isExcluded(source) && !state.has(source)) state.set(source, 'missing')
 }

 if (listMode) {
-  const order = { stale: 0, missing: 1, ok: 2 } as const
+  const order = { 'out-of-sync': 0, missing: 1, ok: 2 } as const
  const rows = [...state.entries()].sort((a, b) => order[a[1]] - order[b[1]] || a[0].localeCompare(b[0]))
  for (const [file, status] of rows) {
    const required = manifest.required.includes(file)
-    console.log(`${status.padEnd(7)} ${file}${status === 'missing' ? (required ? '  (required)' : '  (backlog)') : ''}`)
+    console.log(`${status.padEnd(11)} ${file}${status === 'missing' ? (required ? '  (required)' : '  (backlog)') : ''}`)
  }
-  const counts = { ok: 0, stale: 0, missing: 0 }
+  const counts = { 'ok': 0, 'out-of-sync': 0, 'missing': 0 }
  for (const status of state.values()) counts[status]++
-  console.log(`verify-translation-pairing: ${counts.ok} ok, ${counts.stale} stale, ${counts.missing} missing (of ${state.size} in scope)`)
+  console.log(`verify-translation-pairing: ${counts.ok} ok, ${counts['out-of-sync']} out-of-sync, ${counts.missing} missing (of ${state.size} in scope)`)
  process.exit(0)
 }

 if (errors.length === 0) {
-  console.log(`verify-translation-pairing: ${translations.length} translation(s) checked against ${manifest.required.length} required pair(s), all sound.`)
+  console.log(`verify-translation-pairing: ${pairAnchors.size} pair(s) checked against ${manifest.required.length} required, all consistent.`)
  process.exit(0)
 }