Merge origin/master into worktree/ci-under-minute
This commit is contained in:
Tianyi Cui
@@ -20,7 +20,7 @@ The build gate makes the hook self-contained from a clean worktree. `publint`, `
|
||||
|
||||
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and calls publint's supported API against an in-memory view of each manifest's declared publication files plus npm's mandatory metadata files. That keeps unpublished workspace files invisible to publint without a package-manager subprocess per package. A worker pool sized from `availableParallelism()` bounds parallel file loading and linting; `DSH_PUBLINT_CONCURRENCY` can cap or raise it, and results print in deterministic package order.
|
||||
|
||||
The aggregate package scripts remain the source of truth for ad hoc local runs. The scheduler is a parallel execution plan over their member gates, not a replacement vocabulary.
|
||||
The per-gate package scripts remain the vocabulary for ad hoc local runs. `hygiene` stays an aggregate `&&` chain the scheduler mirrors, while `doc-sync` has since moved its member list into the scheduler itself ([doc-sync through the gate scheduler](2026-07-21-doc-sync-through-gate-scheduler.md)).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-21-doc-sync-through-gate-scheduler.md: b79df2dd7d3515cb0434ac672f7f87c3271d900b
|
||||
2026-07-21-doc-sync-through-gate-scheduler.zh.md: 9395244e3c7700166ad87c49219073210c66bc7e
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note: doc-sync through the gate scheduler
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-21-doc-sync-through-gate-scheduler.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`pnpm run doc-sync` was a `&&` chain of 24 `pnpm run` subcommands. Each link paid a full pnpm wrapper start (workspace resolution, script lookup, tsx boot) before its script ran; measured on a development host, the 24 script bodies together finish in about 34 seconds while the chained form takes around 3 minutes, and the wrapper stall reproduces on local disk, so every developer and CI lane pays it, not just network-filesystem checkouts. The chain also ran serially even though the member gates are read-only and independent, and it silently drifted from [scripts/run-gates.ts](../../../../scripts/run-gates.ts): `verify-cordis-api` joined the chain when the runtime API catalog landed but was never added to `docSyncLeafGates`, so CI and pre-push never enforced that catalog's freshness.
|
||||
|
||||
## Decision
|
||||
|
||||
`doc-sync` in `package.json` now delegates to the existing bounded scheduler — `tsx scripts/run-gates.ts doc-sync` — the same way `check:pre-push` and the `check:ci:*` scripts already do ([parallel pre-push gates](2026-07-06-parallel-pre-push-gates.md), [parallel GitHub CI gates](2026-07-06-parallel-github-ci-gates.md)). The new `doc-sync` mode expands to exactly `docSyncLeafGates()`, making the leaf list in `run-gates.ts` the single source of truth for the member set; the chain that could drift from it is gone. Like `pre-push`, the mode caps default concurrency at four workers because several doc gates each build a full `ts.Program`; `DSH_GATE_CONCURRENCY` still overrides.
|
||||
|
||||
The drift this consolidation surfaced is fixed in the same change: `docSyncLeafGates` gains the missing `verify-cordis-api` leaf, so CI and pre-push now gate the generated runtime API catalog alongside the other generated docs.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the `&&` chain and only fix the missing leaf** — repairs today's drift but keeps two member lists that will drift again, and keeps the 24 serial pnpm wrapper starts.
|
||||
- **A dedicated `scripts/doc-sync.ts` importing each verify module in one process** — saves even the per-gate tsx boot, but requires refactoring all 24 scripts from run-at-import to callable entry points and loses the scheduler's per-gate timing, isolation, and failure grouping; the wrapper start the scheduler already avoids is the dominant cost.
|
||||
- **Shell loop over `tsx scripts/*.ts`** — avoids pnpm wrapper starts cheaply but adds a second execution vocabulary next to the scheduler CI already uses, with none of its scheduling or reporting.
|
||||
|
||||
## Consequences
|
||||
|
||||
One `pnpm run doc-sync` now costs one pnpm wrapper start plus the slowest dependency chain of member gates instead of 24 wrapper starts plus the sum of all members. Adding a doc gate is one edit in `docSyncLeafGates` (plus the package script itself for ad hoc runs); `package.json` keeps the per-gate `verify-*` scripts as the vocabulary for running one gate by hand. The scheduler prints per-gate timing, so a slow doc-sync points at the gate that dominates. `pnpm run doc-sync` output is now interleaved scheduler output rather than sequential per-command output; anything parsing that output must key on the `run-gates:` summary lines.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note: doc-sync 走门禁调度器
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-21-doc-sync-through-gate-scheduler.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`pnpm run doc-sync` 原本是把 24 个 `pnpm run` 子命令用 `&&` 串起来的链。每一环都要先付一次完整的 pnpm 包装层启动（workspace 解析、脚本查找、tsx 启动）才轮到脚本本体；在开发机上实测，24 个脚本本体合计约 34 秒即可跑完，而链式形态耗时约 3 分钟，且包装层的停顿在本地磁盘上同样复现，因此每位开发者和每条 CI 车道都在付这笔开销，并非只有网络文件系统上的检出受影响。这条链还是串行执行的，尽管各成员门禁只读且相互独立；它也在悄悄偏离 [scripts/run-gates.ts](../../../../scripts/run-gates.ts)：运行时 API 目录落地时 `verify-cordis-api` 加入了链，却从未加进 `docSyncLeafGates`，导致 CI 和 pre-push 从未把关该目录的新鲜度。
|
||||
|
||||
## 决策
|
||||
|
||||
`package.json` 中的 `doc-sync` 现在委托给既有的有界调度器——`tsx scripts/run-gates.ts doc-sync`——与 `check:pre-push` 和各 `check:ci:*` 脚本的做法一致（[并行 pre-push 门禁](2026-07-06-parallel-pre-push-gates.md)、[并行 GitHub CI 门禁](2026-07-06-parallel-github-ci-gates.md)）。新增的 `doc-sync` 模式恰好展开为 `docSyncLeafGates()`，使 `run-gates.ts` 里的叶子列表成为成员集合的唯一真源；那条可能与之漂移的链不复存在。与 `pre-push` 一样，该模式把默认并发上限设为四个 worker，因为多个文档门禁各自要构建完整的 `ts.Program`；`DSH_GATE_CONCURRENCY` 仍可覆盖。
|
||||
|
||||
这次整合暴露出的漂移在同一变更中修复：`docSyncLeafGates` 补上缺失的 `verify-cordis-api` 叶子，CI 和 pre-push 从此与其他生成文档一起把关生成的运行时 API 目录。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **保留 `&&` 链，只补缺失的叶子**——能修好今天的漂移，但保留了两份还会再漂移的成员列表，也保留了 24 次串行的 pnpm 包装层启动。
|
||||
- **专门的 `scripts/doc-sync.ts` 在单进程内 import 各校验模块**——连每个门禁的 tsx 启动也能省掉，但需要把全部 24 个脚本从 import 即执行改造成可调用入口，还会失去调度器的按门禁计时、隔离和失败分组；而调度器已经避免的包装层启动才是开销的大头。
|
||||
- **用 shell 循环跑 `tsx scripts/*.ts`**——以低成本避开 pnpm 包装层启动，却在 CI 已经使用的调度器旁边增加了第二套执行词汇，且没有它的任何调度与报告能力。
|
||||
|
||||
## 结果
|
||||
|
||||
一次 `pnpm run doc-sync` 的成本从 24 次包装层启动加全部成员之和，变为一次包装层启动加成员门禁中最慢的依赖链。新增文档门禁只需在 `docSyncLeafGates` 改一处（外加 package script 本身以便手工单独运行）；`package.json` 保留各 `verify-*` 脚本作为手工运行单个门禁的词汇。调度器输出按门禁计时，doc-sync 变慢时能直接指向占大头的门禁。`pnpm run doc-sync` 的输出从逐命令顺序输出变为调度器的交错输出；解析该输出的工具必须以 `run-gates:` 摘要行为准。
|
||||
Reference in New Issue
Block a user