Merge pull request #2246 from deepseek-harness/worktree/python-pypi-release-prep
Prepare Python SDK public PyPI publication
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 01081f1e0b8027420fedbc599f99c67e67a4639b
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: fadfc8cdcb0651665b965126ab0b25ed4218a533
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 4fe9ea3c4073e249a8c961634be9d329b35a3f4e
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: d7a4b91a48175edcc624e6af44efa3e4c7676554
|
||||
|
||||
@@ -40,15 +40,15 @@ The deploy root is [`python/sdk-runtime/package.json`](../../../../python/sdk-ru
|
||||
|
||||
### Build pipeline and artifacts
|
||||
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local dependency tree and rejecting any remaining manifest gap → replace every staged dependency symlink with its target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source, so the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory; macOS uses its target prebuild and emits the required `-spawn-helper` beside the executable. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry.
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local dependency tree and rejecting any remaining manifest gap → replace every staged dependency symlink with its target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source; CI rebuilds that addon inside the matching manylinux 2.28 container before packaging, and the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory. macOS uses its target prebuild and emits the required `-spawn-helper` beside the executable. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry.
|
||||
|
||||
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml), triggered explicitly only — `workflow_dispatch`, or the `build-exe` label on a pull request; native builds on the three platforms linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64, with `~/.pkg-cache` cached; macOS ad-hoc signing is handled by pkg. Each leg drives a mock SSE model through the SDK with the default config and a custom `cordis.yml`, drives the exe directly over NDJSON JSON-RPC, verifies the JSONL and final response, and installs release-shaped wheels into a clean venv without `runtime_bin`; Linux additionally inspects GLIBC requirements and runs in a manylinux 2.28 container. A full three-target run retains four artifacts, each containing one release file: the platform-independent SDK wheel and three native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts only `python-vX.Y.Z` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and three native runtime wheels, then a single serialized job checks and publishes all four to the project PyPI registry. Windows is a non-goal.
|
||||
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml), triggered explicitly by `workflow_dispatch`, the `build-exe` label on a pull request, or the [public publication workflow](../process/2026-08-11-python-publication-workflow.md); native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64, with `~/.pkg-cache` cached, and pkg handles macOS ad-hoc signing. Each leg drives a mock SSE model through the SDK with the default config and a custom `cordis.yml`, drives the exe directly over NDJSON JSON-RPC, verifies the JSONL and final response, and installs release-shaped wheels into a clean venv without `runtime_bin`; Linux additionally inspects both the executable and native addon's GLIBC requirements and runs in a manylinux 2.28 container, while macOS verifies that the executable's deployment target fits the wheel tag. A full three-target run retains four artifacts, each containing one release file: the platform-independent SDK wheel and three native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v<repository-version>` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and three native runtime wheels, then a single serialized job checks and publishes all four to the project PyPI registry. Windows is a non-goal.
|
||||
|
||||
### Python SDK distribution: two carriers, exe for production, node for development
|
||||
|
||||
The Python SDK lives at [`python/`](../../../../python/README.md): `python/sdk` (the client) + `python/sdk-runtime` (the runtime carrier package). The runtime package's data directory holds the checked-in default `runtime/cordis.yml`, the build-injected platform exe and optional helper, and the build-injected `runtime/node/` closure tree. `resolve_bundled_launch_args()` automatic resolution **finds the exe only**; the node carrier is enabled only by an explicit `DSH_RUNTIME_MODE=node` (running `runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`, requiring a system node ≥22.19), positioned as the development-verification channel for members of this repo, and does not enter wheel distributions.
|
||||
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative stable `X.Y.Z` from the repository root `package.json` and stages both packages at that version, with `deepseek-harness-sdk` depending exactly on `deepseek-harness-runtime-bin==X.Y.Z`. An optional `python-vX.Y.Z` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe, and the macOS wheel also contains its architecture-matched helper. Runtime wheels use one of `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, or `py3-none-macosx_11_0_arm64`; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra helpers, and unsupported platforms.
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative `X.Y.Z` or prerelease version from the repository root `package.json`, converts prereleases to their PEP 440 spelling, and stages both packages at that wheel version, with `deepseek-harness-sdk` depending exactly on the matching `deepseek-harness-runtime-bin`. An optional `python-v<repository-version>` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. Staging also carries the repository license into both wheels and the third-party notices into the bundled runtime wheel. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe, and the macOS wheel also contains its architecture-matched helper. Runtime wheels use one of `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, or the conservative `py3-none-macosx_14_0_arm64` tag for the Node 24 executable's macOS 13.5 deployment target; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra helpers, and unsupported platforms.
|
||||
|
||||
The exe's "must be explicitly configured" hard semantic is unchanged; the zero-config experience is restored by the wrapper: when the caller gave no `cordis`, named no explicit runtime, and the environment has no `DSH_CORDIS_CONFIG`, the client explicitly injects the checked-in default `cordis.yml` (agent-core + preloaded llm-deepseek + JSONL persistence + bash-local + the `dsh-jsonrpc` serving entry, with `!!js` environment-variable fallbacks) via `DSH_CORDIS_CONFIG`.
|
||||
|
||||
|
||||
@@ -40,15 +40,15 @@ exe 的 VFS 内是**构建产物形态的真实包树**（各包的 `lib/` + 真
|
||||
|
||||
### 构建管线与产物
|
||||
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts)：运行时闭包校验 → `pnpm run build` →（清空后）`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包，同时省略其包内依赖树，并拒绝剩余的 manifest 缺口 → 将暂存依赖中的每个符号链接替换为目标文件内容，删除包管理器的 `.bin` 链接，并在仍有任何符号链接时失败 → 注入 pkg 配置（`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`；`assets` 使用全量 glob，因为动态 `import()` 对 pkg 静态分析不可见，必须显式打入全部内容）→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg-<platform>-<arch>` 写入 `dist-exe/`，并拷回运行时目录。Linux 安装会从源码构建 `pty.node`，而 `--legacy` 部署会省略该副作用目录，因此构建器会把它从根安装目录复制到暂存闭包；macOS 使用对应目标的预构建产物，并在可执行文件旁生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入，只保留对应平台的 wheel 包。四个部署标志都有实测依据：未启用 `inject-workspace-packages` 时必须使用 `--legacy`；`hoisted` 为 pkg 提供稳定的单实例布局，再由显式物化步骤消除符号链接；关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包；`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码，使 legacy deploy 不会从注册表解析这些未发布名称。
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts)：运行时闭包校验 → `pnpm run build` →（清空后）`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包，同时省略其包内依赖树，并拒绝剩余的 manifest 缺口 → 将暂存依赖中的每个符号链接替换为目标文件内容，删除包管理器的 `.bin` 链接，并在仍有任何符号链接时失败 → 注入 pkg 配置（`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`；`assets` 使用全量 glob，因为动态 `import()` 对 pkg 静态分析不可见，必须显式打入全部内容）→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg-<platform>-<arch>` 写入 `dist-exe/`，并拷回运行时目录。Linux 安装会从源码构建 `pty.node`；CI 会在打包前进入匹配架构的 manylinux 2.28 容器重新构建该 addon，而 `--legacy` 部署会省略这一副作用目录，因此构建器会把它从根安装目录复制到暂存闭包。macOS 使用对应目标的预构建产物，并在可执行文件旁生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入，只保留对应平台的 wheel 包。四个部署标志都有实测依据：未启用 `inject-workspace-packages` 时必须使用 `--legacy`；`hoisted` 为 pkg 提供稳定的单实例布局，再由显式物化步骤消除符号链接；关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包；`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码，使 legacy deploy 不会从注册表解析这些未发布名称。
|
||||
|
||||
CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml)，且只允许显式触发：手动派发 `workflow_dispatch`，或给 PR（Pull Request）添加 `build-exe` 标签。linux-x64、linux-arm64（`ubuntu-24.04-arm`）和 macos-arm64 三个平台分别进行原生构建，并缓存 `~/.pkg-cache`；macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用 mock SSE（Server-Sent Events）模型，分别通过默认配置和自定义 `cordis.yml` 驱动 SDK，再通过 NDJSON JSON-RPC 直接驱动 exe，校验 JSONL 与最终响应；最后把发布形态的 wheel 包安装到干净的 venv 中，并在不传 `runtime_bin` 的情况下运行。Linux 还会检查 GLIBC 依赖，并在 manylinux 2.28 容器中运行。完整构建三个目标时保留 4 个产物，每个产物只含一个发布文件：平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包；手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-vX.Y.Z` 标签流水线，构建一个 SDK wheel 包和 3 个原生运行时 wheel 包，再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。
|
||||
CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml)，由手动派发 `workflow_dispatch`、PR（Pull Request）的 `build-exe` 标签或[公开发布工作流](../process/2026-08-11-python-publication-workflow.md)显式触发。linux-x64、linux-arm64（`ubuntu-24.04-arm`）和 macos-arm64 三个平台分别进行原生构建，并缓存 `~/.pkg-cache`；macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用 mock SSE（Server-Sent Events）模型，分别通过默认配置和自定义 `cordis.yml` 驱动 SDK，再通过 NDJSON JSON-RPC 直接驱动 exe，校验 JSONL 与最终响应；最后把发布形态的 wheel 包安装到干净的 venv 中，并在不传 `runtime_bin` 的情况下运行。Linux 还会检查可执行文件和原生 addon 各自的 GLIBC 依赖，并在 manylinux 2.28 容器中运行；macOS 则验证可执行文件的部署目标符合 wheel 包标签。完整构建三个目标时保留 4 个产物，每个产物只含一个发布文件：平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包；手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v<repository-version>` 标签流水线，构建一个 SDK wheel 包和 3 个原生运行时 wheel 包，再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。
|
||||
|
||||
### Python SDK 分发：双载体，exe 用于生产，`node` 用于开发
|
||||
|
||||
Python SDK 位于 [`python/`](../../../../python/README.md)：`python/sdk` 是客户端，`python/sdk-runtime` 是运行时载体包。运行时包的数据目录包含检入的默认 `runtime/cordis.yml`、构建注入的平台 exe 与可选 helper，以及构建注入的 `runtime/node/` 闭包树。`resolve_bundled_launch_args()` 的自动解析**只查找 exe**；`node` 载体仅在显式设置 `DSH_RUNTIME_MODE=node` 时启用（运行 `runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`，需要系统 Node ≥22.19），定位为本仓库成员的开发验证通道，不随 wheel 包分发。
|
||||
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) 从仓库根目录的 `package.json` 读取权威的稳定版本 `X.Y.Z`，以该版本暂存两个包，并让 `deepseek-harness-sdk` 精确依赖 `deepseek-harness-runtime-bin==X.Y.Z`。可选的 `python-vX.Y.Z` 发布标签只是一项一致性断言，与仓库版本不同时会被拒绝；源码 `pyproject.toml` 中的开发占位版本从不决定发布版本。SDK 是 `py3-none-any` wheel 包；每个只提供 wheel 包的运行时包都包含一个 exe，macOS wheel 包还包含与其架构匹配的 helper。运行时 wheel 包使用 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 或 `py3-none-macosx_11_0_arm64` 三种标签之一；Hatch 钩子拒绝 sdist、通用标签、混合平台载荷、helper 缺失或多余，以及不支持的平台。
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) 从仓库根目录的 `package.json` 读取权威的 `X.Y.Z` 或预发布版本，把预发布版本转换为 PEP 440 写法，并以该 wheel 包版本暂存两个包，让 `deepseek-harness-sdk` 精确依赖匹配版本的 `deepseek-harness-runtime-bin`。可选的 `python-v<repository-version>` 发布标签只是一项一致性断言，与仓库版本不同时会被拒绝；源码 `pyproject.toml` 中的开发占位版本从不决定发布版本。暂存过程还会把仓库许可证放入两个 wheel 包，并把第三方声明放入内置运行时 wheel 包。SDK 是 `py3-none-any` wheel 包；每个只提供 wheel 包的运行时包都包含一个 exe，macOS wheel 包还包含与其架构匹配的 helper。运行时 wheel 包使用 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64`，或针对 Node 24 可执行文件 macOS 13.5 部署目标而保守选择的 `py3-none-macosx_14_0_arm64` 标签；Hatch 钩子拒绝 sdist、通用标签、混合平台载荷、helper 缺失或多余，以及不支持的平台。
|
||||
|
||||
exe「必须显式配置」的硬语义不变；零配置体验由包装层恢复：调用方没有提供 `cordis`、没有显式指定运行时，且环境中没有 `DSH_CORDIS_CONFIG` 时，客户端将检入的默认 `cordis.yml`（`agent-core` + 预载的 `llm-deepseek` + JSONL 持久化 + `bash-local` + `dsh-jsonrpc` 对外服务条目，并通过 `!!js` 使用环境变量兜底）显式注入 `DSH_CORDIS_CONFIG`。
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-11-python-publication-workflow.md
|
||||
2026-08-11-python-publication-workflow.md: 870db08e1d59ad7840fa9acf822915f83ecbd31b
|
||||
2026-08-11-python-publication-workflow.zh.md: 0b2b4a71b909a510bc5a7f52132dbb0ba2bf3e67
|
||||
@@ -0,0 +1,51 @@
|
||||
# Agent Note: Python public publication workflow
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-11-python-publication-workflow.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The Python SDK comprises one platform-independent client wheel and three native runtime wheels that must carry one version and become installable as a set. Public PyPI uploads expose package metadata and files immediately, cannot replace an uploaded filename, and create a temporarily unusable SDK if its exact runtime dependency has not arrived. The private repository needs to exercise the complete native build and validation sequence without publishing any artifact externally.
|
||||
|
||||
## Decision
|
||||
|
||||
The `Release (Python)` GitHub workflow exposes credential-free validation to pull requests labeled `python-release-dry-run` and to manual runs with `publish=false`. Both paths call the native wheel builder for all three platforms, install the Linux release set on Python 3.10 and 3.14, download the four resulting artifacts, verify their exact filenames and package metadata, enforce PyPI's default per-file size limit, record SHA-256 hashes, and retain one aggregate release candidate. These jobs have only repository read permission and no registry credential or OIDC permission, and pull request events cannot enter either publication job.
|
||||
|
||||
A run with `publish=true` must use the `python-v<repository-version>` tag in the private automation repository, match that repository's `github.repository` to its repository-scoped `PYPI_PUBLISHER_REPOSITORY` variable, find `PUBLIC_PYPI_RELEASE_ENABLED=true`, and receive approval from the `pypi-runtime` and `pypi` GitHub environments for runtime and SDK publication, respectively. The read-only public mirror supplies the package metadata URLs but does not run release Actions. Only the two publication jobs receive `id-token: write`; PyPI Trusted Publishing exchanges the private repository identity for short-lived project credentials, so the repository stores no PyPI token.
|
||||
|
||||
Publication consumes the aggregate artifact produced and checked in the same workflow run. Each publication job verifies the retained `SHA256SUMS` before selecting its upload set. A runtime job uploads all three platform wheels before a dependent job uploads the SDK wheel because PyPI uploads are not atomic and the SDK pins the runtime distribution at the exact same version. Neither job checks out source or rebuilds a wheel. Separating them lets GitHub's failed-job retry resume an SDK failure without attempting to replace immutable runtime files.
|
||||
|
||||
Both publication actions disable public attestations. The action still uses Trusted Publishing for authentication, while omitting provenance that would disclose the private publisher repository instead of the public source mirror.
|
||||
|
||||
Repository versions may be stable or use the supported prerelease spellings. Tags retain the repository spelling, while wheel filenames, metadata, dependency pins, and artifact lookup use the normalized PEP 440 spelling.
|
||||
|
||||
The runtime package's `platforms.json` is the source of truth for native wheel tags and executable names. The repository release builder and the isolated Hatch build hook validate and load that file independently. GitHub Actions and GitLab CI call one repository-owned macOS deployment-target check for both the runtime executable and its required spawn helper, so every Mach-O file in the wheel must fit the declared platform tag.
|
||||
|
||||
Both Python build-system requirements pin Hatchling 1.30.1. The next available Hatchling release emits Core Metadata 2.5, which the pinned Twine 6.2.0 validator rejects; keeping the builder exact makes local, GitHub, and GitLab output agree until the validation toolchain supports that metadata version.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**TestPyPI rehearsal.** TestPyPI is a public index, so uploading there would expose package names, metadata, and wheel contents before the repository opens. The credential-free aggregate artifact and the existing private GitLab package registry cover validation and upload-protocol rehearsal without that disclosure.
|
||||
|
||||
**A long-lived PyPI API token.** A stored token gives unrelated workflow steps a reusable secret and needs manual rotation. Trusted Publishing limits the credential to the registered repository, workflow, and environment and mints it only for each protected publication job.
|
||||
|
||||
**Building again inside the publication job.** A second build can differ from the candidate that passed native smoke tests. Publication downloads the same retained bytes and checks no source out.
|
||||
|
||||
**Uploading the SDK before its runtime carriers.** The SDK would become visible while its exact dependency remained unavailable if a later upload failed. Runtime-first ordering leaves partial failures without an installable client that points at missing files.
|
||||
|
||||
**Publishing from the public mirror.** The public mirror is a read-only source projection and does not run release Actions. Binding the PyPI publisher to it would leave no workload capable of presenting the registered OIDC identity.
|
||||
|
||||
**Publishing public attestations.** The default action behavior makes the Trusted Publisher repository identity publicly verifiable. That provenance identifies the private automation repository rather than the package's public source mirror, so the publication jobs disable it.
|
||||
|
||||
## Consequences
|
||||
|
||||
The complete release candidate and the public release both run from the private automation repository. Selecting `publish=true` fails before the protected publication jobs unless the publisher-repository variable, release switch, and tag identify an intentional public release. Mirroring code does not copy those private repository settings, so the read-only public mirror cannot satisfy the authorization checks.
|
||||
|
||||
The private automation repository owner and name, workflow filename, and each job's environment (`pypi-runtime` for runtime and `pypi` for SDK) are part of the Trusted Publisher identity. A source-repository transfer, workflow rename, or environment rename requires updating the affected PyPI publishers and the publisher-repository variable when the repository identity changes. Changing the read-only public mirror changes package metadata URLs instead, not the publishing identity.
|
||||
|
||||
PyPI publication remains non-atomic across the two distribution projects. Runtime-first ordering narrows the visible failure mode, while separate publication jobs and checksum verification let a failed SDK upload resume with the exact checked bytes; an uploaded filename is never replaced.
|
||||
|
||||
Disabling public attestations gives up public cryptographic provenance for the upload identity. Trusted Publishing still authenticates each upload, and the retained aggregate artifact keeps the checked wheel hashes inside the private release workflow.
|
||||
|
||||
Upgrading Hatchling now requires validating the emitted Core Metadata version with the release pipeline's pinned Twine version before changing both package build requirements together.
|
||||
@@ -0,0 +1,51 @@
|
||||
# Agent Note: Python 公开发布工作流
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-11-python-publication-workflow.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
Python SDK 由一个平台无关的客户端 wheel 包和三个原生运行时 wheel 包组成，它们必须使用同一版本，并作为一组可安装。public PyPI 上传会立即公开包元数据和文件，无法替换已上传的同名文件；如果精确版本的运行时依赖尚未到达，还会产生暂时不可用的 SDK。私有仓库需要在不向外发布任何产物的情况下，执行完整的原生构建与验证流程。
|
||||
|
||||
## 决策
|
||||
|
||||
GitHub 的 `Release (Python)` 工作流为带有 `python-release-dry-run` 标签的拉取请求和设置 `publish=false` 的手动运行提供无凭据验证。两条路径都会为全部三个平台调用原生 wheel 包构建器，在 Python 3.10 和 3.14 上安装 Linux 发行集合，下载所得四份产物，验证其精确文件名和包元数据，执行 PyPI 默认单文件大小限制，记录 SHA-256 哈希，并保留一份汇总候选发行版。这些作业只有仓库读取权限，没有注册表凭据或 OIDC 权限，拉取请求事件无法进入任何发布作业。
|
||||
|
||||
设置 `publish=true` 时，运行必须在私有自动化仓库使用 `python-v<repository-version>` 标签，将该仓库的 `github.repository` 与其仓库级 `PYPI_PUBLISHER_REPOSITORY` 变量匹配，找到 `PUBLIC_PYPI_RELEASE_ENABLED=true`，并分别获得 GitHub `pypi-runtime` 和 `pypi` 环境对运行时与 SDK 发布的批准。只读公开镜像提供包元数据 URL，但不运行发布 Actions。只有两个发布作业获得 `id-token: write`；PyPI Trusted Publishing 会把私有仓库身份换成短期项目凭据，因此仓库不保存 PyPI token。
|
||||
|
||||
发布过程使用同一次工作流运行中生成并检查过的汇总产物。每个发布作业都会在选择上传文件前验证保留的 `SHA256SUMS`。一个运行时作业先上传全部三个平台 wheel 包，再由依赖它的作业上传 SDK wheel 包，因为 PyPI 上传不是原子操作，而 SDK 会把运行时分发包固定到完全相同的版本。两个作业都不会检出源码，也不会重新构建 wheel 包。将它们拆开后，GitHub 的失败作业重试可以在 SDK 上传失败时继续执行，而不会尝试替换不可变的运行时文件。
|
||||
|
||||
两个发布 action 都会禁用公开 attestation。action 仍使用 Trusted Publishing 进行身份认证，同时不上传会披露私有发布仓库而非公开源码镜像的 provenance。
|
||||
|
||||
仓库版本可以是稳定版，也可以使用受支持的预发布写法。标签保留仓库写法，wheel 包文件名、元数据、依赖版本固定和产物查找则使用规范化的 PEP 440 写法。
|
||||
|
||||
运行时包的 `platforms.json` 是原生 wheel 包标签和可执行文件名的事实来源。仓库发行构建器与隔离 Hatch 构建钩子会分别校验并加载该文件。GitHub Actions 与 GitLab CI 对运行时可执行文件及其必需的 spawn helper 调用同一个仓库自有的 macOS 部署目标检查，因此 wheel 包中的每个 Mach-O 文件都必须符合声明的平台标签。
|
||||
|
||||
两个 Python 构建系统依赖都固定使用 Hatchling 1.30.1。下一个可用的 Hatchling 版本会生成 Core Metadata 2.5，而固定使用的 Twine 6.2.0 校验器会拒绝该版本；精确固定构建器后，本地、GitHub 与 GitLab 的输出会保持一致，直到校验工具链支持该元数据版本。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**使用 TestPyPI 演练。** TestPyPI 是公开索引，上传会在仓库开放前暴露包名、元数据和 wheel 包内容。无凭据的汇总产物与既有私有 GitLab 包注册表可以覆盖验证和上传协议演练，而不会造成这种披露。
|
||||
|
||||
**使用长期 PyPI API token。** 保存的 token 会让无关工作流步骤接触可复用的密钥，并需要人工轮换。Trusted Publishing 把凭据限制到已登记的仓库、工作流和环境，并且只为每个受保护的发布作业生成凭据。
|
||||
|
||||
**在发布作业中重新构建。** 第二次构建可能与通过原生冒烟测试的候选产物不同。发布过程下载并使用同一批已保留文件，且不检出任何源码。
|
||||
|
||||
**先上传 SDK，再上传运行时载体。** 如果后续上传失败，SDK 会先公开，而其精确依赖仍不可用。运行时优先的顺序使部分失败不会产生指向缺失文件的可安装客户端。
|
||||
|
||||
**从公开镜像发布。** 公开镜像是只读源码投影，不运行发布 Actions。将 PyPI Publisher 绑定到该镜像后，没有工作负载能够提供已登记的 OIDC 身份。
|
||||
|
||||
**发布公开 attestation。** action 默认行为会让 Trusted Publisher 仓库身份可公开验证。该 provenance 标识私有自动化仓库而非包的公开源码镜像，因此发布作业将其禁用。
|
||||
|
||||
## 后果
|
||||
|
||||
完整候选发行版与公开发布都从私有自动化仓库运行。选择 `publish=true` 后，只有发布仓库变量、发布开关和标签都能标识一次有意的公开发布，工作流才会进入受保护的发布作业，否则会提前失败。镜像代码不会复制这些私有仓库设置，因此只读公开镜像无法满足授权检查。
|
||||
|
||||
私有自动化仓库 owner 和仓库名、工作流文件名以及每个作业的环境（运行时使用 `pypi-runtime`，SDK 使用 `pypi`）都是 Trusted Publisher 身份的一部分。源码仓库转移、工作流改名或环境改名后，必须更新受影响的 PyPI Publisher；仓库身份变化时还必须更新发布仓库变量。只读公开镜像发生变化时，需要修改的是包元数据 URL，而不是发布身份。
|
||||
|
||||
两个分发项目之间的 PyPI 发布仍然不是原子操作。运行时优先的顺序会缩小可见的失败状态；独立的发布作业和校验和验证则让失败的 SDK 上传能够从经过检查的精确文件继续执行，并且绝不替换已上传的同名文件。
|
||||
|
||||
禁用公开 attestation 会放弃上传身份的公开密码学 provenance。Trusted Publishing 仍会认证每次上传，而保留的汇总产物会在私有发布工作流内部保存经过检查的 wheel 包哈希。
|
||||
|
||||
升级 Hatchling 时，必须先使用发布流水线固定的 Twine 版本验证其生成的 Core Metadata 版本，再同时修改两个包的构建依赖。
|
||||
Reference in New Issue
Block a user