ci: pre-wire admin-only failover from hosted pools to the in-house pool

The three required Linux jobs now resolve their pool through the
DSH_CI_FAILOVER repository variable. Unset, everything runs exactly as
today on the hosted enterprise pools. Setting it to 'selfhosted'
(repo-admin-only, not PR-editable, no merge required — a merge would be
deadlocked behind the failing checks themselves) retargets all three
onto the vm-backup pool, halves the coverage worker bound and snapshot
concurrency for the shared VM, and skips the hosted-path cache restores.

Adds a bilingual failover runbook (switch, capacity via the four
registered spare instances, switch-back, trust boundary) and links it
from the topology note. The push-triggered standby lane remains the
continuous proof that the failover target works.
This commit is contained in:
Chinesezjc
2026-07-26 00:34:53 +08:00
parent 310a387b14
commit 0fd6dc8924
7 changed files with 111 additions and 9 deletions

View File

@@ -30,9 +30,22 @@ jobs:
# Three enterprise jobs isolate coverage, static analysis, and the
# build-backed consumer tail. The static job publishes its exact build so
# consumers do not repeat the longest part of their critical path.
#
# FAILOVER: each Linux enterprise job resolves its pool through the
# DSH_CI_FAILOVER repository variable. Unset (normal), the expressions
# pick the hosted enterprise pools below. Setting the variable to
# 'selfhosted' (repo Settings → Actions → Variables; admin-only, not
# PR-editable, no merge required) retargets all three onto the in-house
# vm-backup pool and re-running the failed jobs is the entire switch —
# see .agents/notes/implemented/process/ci-failover-runbook.md. The
# in-house pool's readiness is re-proven on every master push by the
# serial-linux-selfhosted standby lane below.
node-24:
if: github.event_name == 'pull_request'
runs-on: dsh-enterprise-ubuntu-latest-32core-test
runs-on: >-
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|| 'dsh-enterprise-ubuntu-latest-32core-test' }}
name: node 24 / static
env:
DSH_GATE_CONCURRENCY: '8'
@@ -77,17 +90,28 @@ jobs:
node-24-coverage:
if: github.event_name == 'pull_request'
runs-on: dsh-enterprise-ubuntu-24-04-32core-test
runs-on: >-
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|| 'dsh-enterprise-ubuntu-24-04-32core-test' }}
name: node 24 / coverage
env:
DSH_COVERAGE_MAX_WORKERS: '24'
# Failover halves the worker bound: the hosted 32-core runner is
# exclusive to one job, but the failover pool shares one 64-core VM
# across four runner instances, and the timing-sensitive process
# suites have documented aggregate-contention failures.
DSH_COVERAGE_MAX_WORKERS: ${{ vars.DSH_CI_FAILOVER == 'selfhosted' && '12' || '24' }}
DSH_GATE_CONCURRENCY: '8'
steps:
- uses: actions/checkout@v6
with:
persist-credentials: false
# Skipped under failover: the self-hosted VM's persistent pnpm store
# serves warm installs directly, and this hosted-path restore would
# spend ~52 s pulling ~180 MB into a path pnpm never reads there.
- uses: actions/cache/restore@v4
if: vars.DSH_CI_FAILOVER != 'selfhosted'
with:
path: /home/runner/.local/share/pnpm/store/v11
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
@@ -118,7 +142,10 @@ jobs:
node-24-consumers:
needs: node-24
if: github.event_name == 'pull_request'
runs-on: dsh-enterprise-ubuntu-latest-32core-test
runs-on: >-
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|| 'dsh-enterprise-ubuntu-latest-32core-test' }}
name: node 24 / snapshots and artifacts
env:
DSH_ESLINT_CACHE: '1'
@@ -126,7 +153,8 @@ jobs:
DSH_GATE_CONCURRENCY: '8'
DSH_NODE_COMPAT_SKIP_TYPECHECK: '1'
DSH_PUBLINT_CONCURRENCY: '8'
DSH_SNAPSHOT_MAX_CONCURRENCY: '32'
# Failover halves snapshot concurrency for the shared 64-core VM.
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER == 'selfhosted' && '16' || '32' }}
steps:
- uses: actions/checkout@v6
with:
@@ -140,7 +168,9 @@ jobs:
- name: Restore built tree
run: tar -xzf "$RUNNER_TEMP/node-24-built-tree.tar.gz"
# Skipped under failover — see the coverage lane's identical rationale.
- uses: actions/cache/restore@v4
if: vars.DSH_CI_FAILOVER != 'selfhosted'
with:
path: /home/runner/.local/share/pnpm/store/v11
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}