ci: pre-wire admin-only failover from hosted pools to the in-house pool
The three required Linux jobs now resolve their pool through the DSH_CI_FAILOVER repository variable. Unset, everything runs exactly as today on the hosted enterprise pools. Setting it to 'selfhosted' (repo-admin-only, not PR-editable, no merge required — a merge would be deadlocked behind the failing checks themselves) retargets all three onto the vm-backup pool, halves the coverage worker bound and snapshot concurrency for the shared VM, and skips the hosted-path cache restores. Adds a bilingual failover runbook (switch, capacity via the four registered spare instances, switch-back, trust boundary) and links it from the topology note. The push-triggered standby lane remains the continuous proof that the failover target works.
This commit is contained in:
40
.github/workflows/ci.yml
vendored
40
.github/workflows/ci.yml
vendored
@@ -30,9 +30,22 @@ jobs:
|
||||
# Three enterprise jobs isolate coverage, static analysis, and the
|
||||
# build-backed consumer tail. The static job publishes its exact build so
|
||||
# consumers do not repeat the longest part of their critical path.
|
||||
#
|
||||
# FAILOVER: each Linux enterprise job resolves its pool through the
|
||||
# DSH_CI_FAILOVER repository variable. Unset (normal), the expressions
|
||||
# pick the hosted enterprise pools below. Setting the variable to
|
||||
# 'selfhosted' (repo Settings → Actions → Variables; admin-only, not
|
||||
# PR-editable, no merge required) retargets all three onto the in-house
|
||||
# vm-backup pool and re-running the failed jobs is the entire switch —
|
||||
# see .agents/notes/implemented/process/ci-failover-runbook.md. The
|
||||
# in-house pool's readiness is re-proven on every master push by the
|
||||
# serial-linux-selfhosted standby lane below.
|
||||
node-24:
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: dsh-enterprise-ubuntu-latest-32core-test
|
||||
runs-on: >-
|
||||
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
|
||||
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|
||||
|| 'dsh-enterprise-ubuntu-latest-32core-test' }}
|
||||
name: node 24 / static
|
||||
env:
|
||||
DSH_GATE_CONCURRENCY: '8'
|
||||
@@ -77,17 +90,28 @@ jobs:
|
||||
|
||||
node-24-coverage:
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: dsh-enterprise-ubuntu-24-04-32core-test
|
||||
runs-on: >-
|
||||
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
|
||||
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|
||||
|| 'dsh-enterprise-ubuntu-24-04-32core-test' }}
|
||||
name: node 24 / coverage
|
||||
env:
|
||||
DSH_COVERAGE_MAX_WORKERS: '24'
|
||||
# Failover halves the worker bound: the hosted 32-core runner is
|
||||
# exclusive to one job, but the failover pool shares one 64-core VM
|
||||
# across four runner instances, and the timing-sensitive process
|
||||
# suites have documented aggregate-contention failures.
|
||||
DSH_COVERAGE_MAX_WORKERS: ${{ vars.DSH_CI_FAILOVER == 'selfhosted' && '12' || '24' }}
|
||||
DSH_GATE_CONCURRENCY: '8'
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
# Skipped under failover: the self-hosted VM's persistent pnpm store
|
||||
# serves warm installs directly, and this hosted-path restore would
|
||||
# spend ~52 s pulling ~180 MB into a path pnpm never reads there.
|
||||
- uses: actions/cache/restore@v4
|
||||
if: vars.DSH_CI_FAILOVER != 'selfhosted'
|
||||
with:
|
||||
path: /home/runner/.local/share/pnpm/store/v11
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
@@ -118,7 +142,10 @@ jobs:
|
||||
node-24-consumers:
|
||||
needs: node-24
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: dsh-enterprise-ubuntu-latest-32core-test
|
||||
runs-on: >-
|
||||
${{ vars.DSH_CI_FAILOVER == 'selfhosted'
|
||||
&& fromJSON('["self-hosted", "linux", "x64", "vm-backup"]')
|
||||
|| 'dsh-enterprise-ubuntu-latest-32core-test' }}
|
||||
name: node 24 / snapshots and artifacts
|
||||
env:
|
||||
DSH_ESLINT_CACHE: '1'
|
||||
@@ -126,7 +153,8 @@ jobs:
|
||||
DSH_GATE_CONCURRENCY: '8'
|
||||
DSH_NODE_COMPAT_SKIP_TYPECHECK: '1'
|
||||
DSH_PUBLINT_CONCURRENCY: '8'
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '32'
|
||||
# Failover halves snapshot concurrency for the shared 64-core VM.
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER == 'selfhosted' && '16' || '32' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
@@ -140,7 +168,9 @@ jobs:
|
||||
- name: Restore built tree
|
||||
run: tar -xzf "$RUNNER_TEMP/node-24-built-tree.tar.gz"
|
||||
|
||||
# Skipped under failover — see the coverage lane's identical rationale.
|
||||
- uses: actions/cache/restore@v4
|
||||
if: vars.DSH_CI_FAILOVER != 'selfhosted'
|
||||
with:
|
||||
path: /home/runner/.local/share/pnpm/store/v11
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
|
||||
Reference in New Issue
Block a user