From 5773ef9b0c9cb32b4bb879594d10ae8332399608 Mon Sep 17 00:00:00 2001 From: Turtle Date: Wed, 22 Jul 2026 10:55:20 +0800 Subject: [PATCH] fix(gates): bound pre-push test gate vitest workers --- .../2026-07-06-parallel-pre-push-gates.md | 8 ++++++++ scripts/run-gates.ts | 16 +++++++++++++++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md b/.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md index 710c37cb3a..8c1c35dccb 100644 --- a/.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md +++ b/.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md @@ -10,6 +10,8 @@ Flattening those members directly into `lefthook.yml` solves the local hook only `publint` has the same shape one level lower. Each package is linted independently against its own manifest and built output, but the runner loops through every package in order. On this repo that makes one package-publication gate consume time proportional to the number of packages even though the checks do not share mutable state. +The unit-suite gate is the sharpest instance of the parallel runner's own pressure. At vitest's all-core default it oversubscribed the machine against its concurrent siblings — build, snapshot, and the doc leaves — and the resulting CPU starvation blew the 5s per-test timeout on subprocess-spawning tests (the hooks bridges spawn shells; the sandbox probe `spawnSync`-es a launcher) with a shifting victim set from run to run. + ## Decision [lefthook.yml](../../../../lefthook.yml) keeps one pre-push job named `full check` and runs `pnpm run check:pre-push`. That package script delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), the same bounded scheduler CI uses. @@ -18,6 +20,8 @@ The `pre-push` mode expands into leaf gates for the unit suite, snapshot suite, The build gate makes the hook self-contained from a clean worktree. `publint`, `verify-node-next-types`, and the pre-push form of `doc-typecheck` wait for that build output, while source-only gates continue in parallel. +The unit-suite (`test`) gate runs `vitest` with a pool bounded to half the available cores by default; `DSH_TEST_MAX_WORKERS` overrides it, mirroring the coverage gate's `DSH_COVERAGE_MAX_WORKERS`. The bound lives only in the `pre-push` mode's gate; CI runs the coverage gate instead, so it never touches CI timing. + [scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages//` and runs `publint` with a worker pool sized from `availableParallelism()`. `DSH_PUBLINT_CONCURRENCY` can cap or raise the worker count for local machines and CI runners with different resource profiles. Results are buffered per package and printed in deterministic package order, so parallel execution does not scramble each package's log block. The per-gate package scripts remain the vocabulary for ad hoc local runs. `hygiene` stays an aggregate `&&` chain the scheduler mirrors, while `doc-sync` has since moved its member list into the scheduler itself ([doc-sync through the gate scheduler](2026-07-21-doc-sync-through-gate-scheduler.md)). @@ -30,6 +34,8 @@ The per-gate package scripts remain the vocabulary for ad hoc local runs. `hygie - **Background subcommands inside shell scripts** - can parallelize work, but it loses lefthook's job names, per-job timing, and failure grouping, and makes signal handling harder to reason about. - **Declare one publint lefthook job per package** - exposes maximum parallelism, but it turns the hook into a hand-maintained package inventory that drifts exactly when new packages are added. - **Run publint with unbounded concurrency** - minimizes elapsed time on small machines only by gambling with process count, memory pressure, package tarball creation, and readable logs. +- **Leave the `test` gate at vitest's all-core default** - matches a standalone `pnpm run test`, but under the pre-push runner it overlaps three sibling gates and oversubscribes the machine, so subprocess-spawning tests intermittently blow their 5s timeout; bounding the pool trades a slower isolated `test` gate for a stable one. +- **Raise the per-test timeout instead of bounding workers** - would cover the vitest-level timeouts, but the sandbox probe's own 5s `spawnSync` budget is a real-time product default the test asserts against, not a vitest timeout, so only lowering the concurrent process count keeps it green. ## Consequences @@ -38,3 +44,5 @@ The hook's critical path becomes the slowest real gate instead of the sum of hid The hook file stays short, and the duplicated member list lives in [scripts/run-gates.ts](../../../../scripts/run-gates.ts), where CI and pre-push can share it. The cost is a custom scheduler script instead of pure lefthook configuration, plus a build in the local pre-push path. `publint-all.ts` becomes asynchronous code and buffers command output instead of inheriting stdio live. The payoff is package-level parallelism with stable output order and one environment variable for resource tuning. + +The bounded `test` gate runs slower in isolation than a full-machine `vitest run` but no longer starves its siblings, so the hook stops producing spurious per-test timeout failures at its default concurrency. A machine shared with other heavy processes can still spike past saturation beyond this hook's control; `DSH_TEST_MAX_WORKERS` and `DSH_GATE_CONCURRENCY` let a contributor tighten the footprint further when that happens. diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 2de30b76a7..823fc91828 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -195,7 +195,7 @@ function gatesForMode(selected: Mode): Gate[] { return [ pnpmScript('runtime-closure', 'verify-runtime-closure', { label: 'runtime closure' }), pnpmScript('cordis-config', 'verify-cordis-config', { label: 'Cordis config' }), - pnpmScript('test', 'test'), + prePushTestGate(), pnpmScript('duplication', 'duplication'), snapshotGate(), pnpmScript('build', 'build'), @@ -294,6 +294,20 @@ function coverageGate(): Gate { }) } +// The pre-push runner overlaps this all-core vitest gate with sibling gates (build, snapshot, +// doc leaves). Bound its pool to half the cores by default (>=2 workers) so it does not +// oversubscribe them; DSH_TEST_MAX_WORKERS overrides it, mirroring coverageGate's +// DSH_COVERAGE_MAX_WORKERS. Pre-push only — CI runs the coverage gate — so the bound never +// touches CI timing. Failure mode and rationale in the parallel pre-push gates Agent Note +// (.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md). +function prePushTestGate(): Gate { + const override = positiveIntArg('DSH_TEST_MAX_WORKERS', '--maxWorkers') + const workers = override.length > 0 + ? override + : [`--maxWorkers=${Math.max(2, Math.floor(availableParallelism() / 2))}`] + return pnpmExec('test', ['vitest', 'run', ...workers], { label: 'test' }) +} + // The snapshot suite boots the example bins in `lib` mode (built artifact under plain Node, // plugins via real exports) — CI and pre-push already build, so they exercise what ships rather // than the tsx/source path dev uses. It therefore waits on `build`.