Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(packaging): materialize runtime dependencies

Browse Source
This commit is contained in:
Yichen Jiang
2026-08-10 22:47:18 +08:00
parent 75f4434787
commit a7e8805340
6 changed files with 181 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 29aa14db2b3b64cefb7960ee8f316682ba1ac736
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: f10d72b394246e9e57033a5382ec977bdc6f3ab8
2026-07-10-single-file-executable-sdk-runtime-distribution.md: c2b6d9ff1825915e39738bf8f782c302ecfc1d0d
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: d7758a77083e07b1d2cac99ae2be3f15e6edd2dc

2
.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md
View File

@@ -40,7 +40,7 @@ The deploy root is [`python/sdk-runtime/package.json`](../../../../python/sdk-ru
### Build pipeline and artifacts
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local symlink tree and rejecting any remaining manifest gap → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source, so the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory; macOS uses its target prebuild and emits the required `-spawn-helper` beside the executable. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted yields a zero-symlink package payload (most stable for the pkg VFS, physically guaranteeing a single cordis instance); disabling automatic peer installation keeps unpublished package names from triggering registry resolution; link-workspace-packages points the closure at workspace/vendor sources.
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore any direct workspace package that legacy deploy hoisted back under the source manifest's `node_modules`, omitting its package-local dependency tree and rejecting any remaining manifest gap → replace every staged dependency symlink with its target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → stage the target `node-pty` addon → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` land in `dist-exe/` and are copied back into the runtime directory. Linux installs build `pty.node` from source, so the builder copies it from the root install into the staged closure because legacy deploy omits that side-effect directory; macOS uses its target prebuild and emits the required `-spawn-helper` beside the executable. CI treats these products as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry.
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml), triggered explicitly only — `workflow_dispatch`, or the `build-exe` label on a pull request; native builds on the three platforms linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64, with `~/.pkg-cache` cached; macOS ad-hoc signing is handled by pkg. Each leg drives a mock SSE model through the SDK with the default config and a custom `cordis.yml`, drives the exe directly over NDJSON JSON-RPC, verifies the JSONL and final response, and installs release-shaped wheels into a clean venv without `runtime_bin`; Linux additionally inspects GLIBC requirements and runs in a manylinux 2.28 container. A full three-target run retains four artifacts, each containing one release file: the platform-independent SDK wheel and three native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts only `python-vX.Y.Z` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and three native runtime wheels, then a single serialized job checks and publishes all four to the project PyPI registry. Windows is a non-goal.

2
.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md
View File

@@ -40,7 +40,7 @@ exe 的 VFS 内是**构建产物形态的真实包树**（各包的 `lib/` + 真
### 构建管线与产物
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts)：运行时闭包校验 → `pnpm run build` →（清空后）`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包，同时省略其包内符号链接树，并拒绝剩余的 manifest 缺口 → 注入 pkg 配置（`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`；`assets` 使用全量 glob，因为动态 `import()` 对 pkg 静态分析不可见，必须显式打入全部内容）→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg-<platform>-<arch>` 写入 `dist-exe/`，并拷回运行时目录。Linux 安装会从源码构建 `pty.node`，而 `--legacy` 部署会省略该副作用目录，因此构建器会把它从根安装目录复制到暂存闭包；macOS 使用对应目标的预构建产物，并在可执行文件旁生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入，只保留对应平台的 wheel 包。四个部署标志都有实测依据：未启用 `inject-workspace-packages` 时必须使用 `--legacy`；`hoisted` 产出无符号链接的包载荷（对 pkg VFS 最稳定，并从物理上保证只有一个 Cordis 实例）；关闭对等依赖自动安装可避免未发布包名触发注册表解析；`link-workspace-packages` 让闭包指向工作区/vendor 源码。
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts)：运行时闭包校验 → `pnpm run build` →（清空后）`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复被 legacy deploy 提升回源 manifest 的 `node_modules` 下的任何直接工作区包，同时省略其包内依赖树，并拒绝剩余的 manifest 缺口 → 将暂存依赖中的每个符号链接替换为目标文件内容，删除包管理器的 `.bin` 链接，并在仍有任何符号链接时失败 → 注入 pkg 配置（`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/packaged-bin.js`；`assets` 使用全量 glob，因为动态 `import()` 对 pkg 静态分析不可见，必须显式打入全部内容）→ 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg-<platform>-<arch>` 写入 `dist-exe/`，并拷回运行时目录。Linux 安装会从源码构建 `pty.node`，而 `--legacy` 部署会省略该副作用目录，因此构建器会把它从根安装目录复制到暂存闭包；macOS 使用对应目标的预构建产物，并在可执行文件旁生成所需的 `-spawn-helper`。CI 将这些产物作为测试中间输入，只保留对应平台的 wheel 包。四个部署标志都有实测依据：未启用 `inject-workspace-packages` 时必须使用 `--legacy`；`hoisted` 为 pkg 提供稳定的单实例布局，再由显式物化步骤消除符号链接；关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包；`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码，使 legacy deploy 不会从注册表解析这些未发布名称。
CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml)，且只允许显式触发：手动派发 `workflow_dispatch`，或给 PR（Pull Request）添加 `build-exe` 标签。linux-x64、linux-arm64（`ubuntu-24.04-arm`）和 macos-arm64 三个平台分别进行原生构建，并缓存 `~/.pkg-cache`；macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用 mock SSE（Server-Sent Events）模型，分别通过默认配置和自定义 `cordis.yml` 驱动 SDK，再通过 NDJSON JSON-RPC 直接驱动 exe，校验 JSONL 与最终响应；最后把发布形态的 wheel 包安装到干净的 venv 中，并在不传 `runtime_bin` 的情况下运行。Linux 还会检查 GLIBC 依赖，并在 manylinux 2.28 容器中运行。完整构建三个目标时保留 4 个产物，每个产物只含一个发布文件：平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包；手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-vX.Y.Z` 标签流水线，构建一个 SDK wheel 包和 3 个原生运行时 wheel 包，再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。