fix(repository-plugin): make GitHub source preparation self-contained

This commit is contained in:
Tianyi Cui
2026-08-08 17:46:56 +08:00
parent da2179dd2b
commit b91b1fdefe
20 changed files with 454 additions and 64 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-static-repository-plugin-format.md
2026-07-30-static-repository-plugin-format.md: c9d755b925a6ea05eed71e75803397d2672df9f4
2026-07-30-static-repository-plugin-format.zh.md: 361de64d2e98b9fb4ac42963e4ae48e77fbc7016
2026-07-30-static-repository-plugin-format.md: 4823495dabee2101713ac72c8d0cee5bcc6b38d1
2026-07-30-static-repository-plugin-format.zh.md: e5817ca982a71a609c86c5835b803b12f1317e1a

View File

@@ -14,7 +14,7 @@ The [package-manager-native repository cache](2026-07-30-package-manager-native-
`@deepseek-ai/dsh-repository-plugin` owns a restricted `.dsh-plugin` package format with two contribution kinds only: skill roots and one common `.mcp.json`. Its package metadata uses `package.json#dsh.skills` for relative skill-root paths and `package.json#dsh.mcpServers` for the relative MCP document path. At least one is required. Each path may leave `.dsh-plugin` to reuse repository content but must remain beneath the directory containing that `.dsh-plugin`; a nested selectable Plugin therefore owns the adjacent subtree above its package without gaining access to unrelated host paths.
The `.dsh-plugin` package declares `dsh-plugin-prepare` as its ordinary package-manager `prepare` script. The helper validates metadata and source types, strictly parses `.mcp.json`, copies static assets into `dsh-plugin-assets`, and writes `dsh-plugin.mjs`. The `.mjs` extension avoids imposing `type: module` on repository-authored package metadata. The generated module is a fixed import-free template containing only a normalized manifest, an `inject` list derived from it (`loader`, plus `skills` and/or `tools` per the declared capabilities, so the wrapper fiber gates on the services its children need), and delegation to the `dsh-repository-plugin` Loader builtin. Preparation never discovers, transpiles, bundles, or preserves a custom repository entry point.
The `.dsh-plugin` package declares exact `scripts.prepack: "dsh-plugin-prepare"` metadata without depending on a DSH npm package. During Git installation, the standalone runtime temporarily supplies that command from its own build on the isolated lifecycle `PATH`; `prepack` runs after dependency installation and before pnpm packs a selected subdirectory, including a Plugin nested inside another package-manager workspace. The helper validates metadata and source types, strictly parses `.mcp.json`, copies static assets into `dsh-plugin-assets`, and writes `dsh-plugin.mjs`; the source loader revalidates the installed package's exact lifecycle metadata before importing that wrapper. The `.mjs` extension avoids imposing `type: module` on repository-authored package metadata. The generated module is a fixed import-free template containing only a normalized manifest, an `inject` list derived from it (`loader`, plus `skills` and/or `tools` per the declared capabilities, so the wrapper fiber gates on the services its children need), and delegation to the `dsh-repository-plugin` Loader builtin. Preparation never discovers, transpiles, bundles, or preserves a custom repository entry point. The host-owned command rationale is in the [Git source preparation repair](../bug-fix/2026-08-08-host-owned-git-repository-plugin-preparation.md).
Loading the DSH package registers that builtin as an effect. A generated wrapper mounts the builtin as its child with `import.meta.url`, so all contributions belong to the wrapper fiber and disappear on Loader removal or rollback. The builtin revalidates the prepared manifest and path containment before reading assets. It composes the existing implementations rather than registering skills or MCP tools itself.
@@ -46,4 +46,4 @@ Unknown MCP fields reject. This intentionally excludes OAuth, `auth` objects, `C
## Testing
Focused tests prepare skills and MCP metadata, prove the emitted wrapper contains no imports, reject Work IQ-style OAuth fields, map Expo-style HTTP and DataJunction-style stdio plus environment values, and exercise missing variables. A real Loader test mounts a generated wrapper through the registered builtin, reads its skill through `ctx.skills`, removes the Loader entry, and observes provider cleanup. The keyless headless example loads a checked-in prepared wrapper through its real `cordis.yml` and snapshots the repository skill's logged model catalog row.
Focused tests prepare skills and MCP metadata, prove the emitted wrapper contains no imports, reject Work IQ-style OAuth fields, map Expo-style HTTP and DataJunction-style stdio plus environment values, and exercise missing variables. A real Loader test mounts a generated wrapper through the registered builtin, reads its skill through `ctx.skills`, removes the Loader entry, and observes provider cleanup. The CI built-entry acceptance invokes `dsh run` with a GitHub source pinned to the pull request head, lets bundled pnpm fetch and prepare a private dependency-free fixture, then observes the copied skill in the real model request and the prepared wrapper in the immutable cache.

View File

@@ -14,7 +14,7 @@
`@deepseek-ai/dsh-repository-plugin` 负责一个受限的 `.dsh-plugin` package 格式且只允许两类贡献skill 根和一个通用 `.mcp.json`。Package metadata 使用 `package.json#dsh.skills` 声明相对 skill 根路径,使用 `package.json#dsh.mcpServers` 声明相对 MCP 文档路径;两者至少需要一个。路径可以离开 `.dsh-plugin` 以复用仓库内容,但必须留在包含该 `.dsh-plugin` 的目录之下;因此,一个嵌套且可选择的 Plugin 可以拥有其 package 上方相邻的子树,却不能访问无关宿主路径。
`.dsh-plugin` package 把 `dsh-plugin-prepare` 声明为普通 package-manager `prepare` 脚本。Helper 会校验 metadata 与源码类型,严格解析 `.mcp.json`,把静态资源复制到 `dsh-plugin-assets`,并写入 `dsh-plugin.mjs``.mjs` 扩展名避免强迫仓库作者在 package metadata 中设置 `type: module`。生成模块来自固定、无 import 的模板,只包含规范化 manifest、由 manifest 派生的 `inject` 列表(`loader`,加上按声明能力加入的 `skills``tools`,使包装 fiber 在其子插件所需服务上门控),以及对 `dsh-repository-plugin` Loader builtin 的委托。准备阶段永远不会发现、转译、打包或保留自定义仓库入口。
`.dsh-plugin` 包声明精确的 `scripts.prepack: "dsh-plugin-prepare"` 元数据,且不依赖 DSH NPM 包。在 Git 安装期间,独立运行时会从自身构建产物中临时提供该命令,并将其放入隔离的生命周期 `PATH``prepack` 会在依赖安装后、pnpm 打包选定子目录前运行,即使插件嵌套在另一个包管理器工作区内也不例外。该辅助程序会校验元数据与源码类型,严格解析 `.mcp.json`,把静态资源复制到 `dsh-plugin-assets`,并写入 `dsh-plugin.mjs`;源码 loader 会在导入该包装层前重新校验已安装包的精确生命周期元数据`.mjs` 扩展名避免强迫仓库作者在包元数据中设置 `type: module`。生成模块来自固定、无 import 的模板,只包含规范化 manifest、由 manifest 派生的 `inject` 列表(`loader`,加上按声明能力加入的 `skills``tools`,使包装 fiber 在其子插件所需服务上门控),以及对 `dsh-repository-plugin` Loader builtin 的委托。准备阶段永远不会发现、转译、打包或保留自定义仓库入口。宿主自有命令的设计依据见[Git 源准备修复](../bug-fix/2026-08-08-host-owned-git-repository-plugin-preparation.md)。
加载 DSH package 会以 effect 方式注册该 builtin。生成的包装模块使用 `import.meta.url` 把 builtin 挂载为自己的子级,因此所有贡献都归属于包装 fiber并在 Loader 移除或回滚时消失。Builtin 会在读取资源前重新校验已准备 manifest 与路径包含关系。它只组合现有实现,而不自行注册 skills 或 MCP 工具。
@@ -46,4 +46,4 @@
## 测试
聚焦测试会准备 skills 与 MCP metadata证明生成包装模块不含 import拒绝 Work IQ 风格的 OAuth 字段,映射 Expo 风格 HTTP 与 DataJunction 风格 stdio 及环境变量,并覆盖缺失变量。真实 Loader 测试通过已注册 builtin 挂载生成包装模块,经 `ctx.skills` 读取其 skill移除 Loader 条目并观察提供方清理。Keyless headless 示例通过真实 `cordis.yml` 加载一份签入的已准备包装模块,并快照 repository skill 写入日志的模型目录行
聚焦测试会准备 skills 与 MCP metadata证明生成包装模块不含 import拒绝 Work IQ 风格的 OAuth 字段,映射 Expo 风格 HTTP 与 DataJunction 风格 stdio 及环境变量,并覆盖缺失变量。真实 Loader 测试通过已注册 builtin 挂载生成包装模块,经 `ctx.skills` 读取其 skill移除 Loader 条目并观察提供方清理。CI 的构建入口验收会用锁定到 PRPull Requesthead 的 GitHub 源调用 `dsh run`,让随附 pnpm 获取并准备一个私有且不含依赖的 fixture测试前置数据然后在真实模型请求中观察已复制的 skill并在不可变缓存中观察已准备的包装模块

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-08-host-owned-git-repository-plugin-preparation.md
2026-08-08-host-owned-git-repository-plugin-preparation.md: 45e84f9c8a89bb9d1eb7e4521634d16789dea2f0
2026-08-08-host-owned-git-repository-plugin-preparation.zh.md: ba88d5bb351427118d55c357209b496b39c2eb98

View File

@@ -0,0 +1,42 @@
# Agent Note: Host-owned preparation makes GitHub repository Plugins installable
Status: implemented
English | [中文](2026-08-08-host-owned-git-repository-plugin-preparation.zh.md)
## Problem
The repository Plugin authoring contract depended on `scripts.prepare: "dsh-plugin-prepare"` and told source repositories to add `@deepseek-ai/dsh-repository-plugin` as a development dependency. That package is private and not published to npm, so an otherwise valid external GitHub repository could not obtain the helper in a clean install.
The lifecycle choice also failed for a selectable `.dsh-plugin` inside a pnpm workspace. pnpm prepares a Git-hosted package by running the repository's preferred package manager before packing the selected subdirectory. A nested `pnpm install` joins the containing workspace and need not execute the unlisted `.dsh-plugin` package's `prepare` script. The install could therefore succeed and publish a cache generation containing only the source package metadata; real DSH startup failed later because `dsh-plugin.mjs` did not exist.
The checked-in headless fixture did not catch either defect because it mounted an already prepared wrapper. It proved runtime composition, not GitHub acquisition or package preparation.
## Decision
The fixed authoring format now requires exact `scripts.prepack: "dsh-plugin-prepare"` metadata and no DSH dependency. pnpm's Git-hosted package preparation invokes `prepack` explicitly after its dependency-install step and before packlist selects the `.dsh-plugin` subtree, so the helper can still copy sibling repository assets such as `../skills` into the package.
`@deepseek-ai/dsh-repository-plugin` materializes short-lived POSIX and Windows command wrappers that invoke its own built `dsh-plugin-prepare` entry. `RepositoryCache` accepts caller-owned executable directories, resolves them absolutely, and prepends them to the credential-scrubbed lifecycle `PATH` passed to bundled pnpm. The command directory exists only for the installation transaction and is removed on success or failure. The repository remains trusted package-manager input: DSH supplies one command, but other lifecycle scripts and dependencies still execute under the existing trust contract.
The Node 24 consumer lane passes an exact source derived from the pull request head repository and SHA. Its built-entry acceptance launches the real `apps/cli/lib/bin.js run` command with a one-run patch selecting a `private: true`, dependency-free GitHub fixture. It requires the run to reach the mock LLM, finds the repository skill description in the actual model request, and verifies the generated wrapper and copied skill under the immutable DSH cache. The test fails if CI omits the exact source instead of silently skipping.
## Alternatives considered
**Publish the prepare helper to npm.** Rejected because the source package would acquire a release/version dependency solely to call code already owned by the running DSH installation, and the existing helper is intentionally private.
**Keep `prepare` and only inject the command.** Rejected because command availability does not make a nested package's `prepare` lifecycle run when the Git repository's package manager treats it as part of another workspace.
**Prepare after RepositoryCache installs the selected package.** Rejected because pnpm's packed subdirectory no longer contains sibling source assets referenced by paths such as `../skills`; preparation must happen before packlist.
**Clone GitHub repositories in DSH and bypass pnpm's Git fetcher.** Rejected because it would duplicate ref resolution, subdirectory selection, dependency installation, packlist behavior, and cache integrity already owned by the pinned package manager.
## Consequences
- A repository author can commit the fixed `.dsh-plugin/package.json` and source assets to GitHub without publishing either the Plugin or its preparation helper to npm.
- `prepack`, not `prepare`, is part of the pre-release authoring format. Invalid lifecycle metadata fails during source preparation or installed-package validation instead of producing an ambiguous partial format.
- Exact source strings still identify immutable cache generations; a changed ref or source configuration selects another generation.
- This repair does not expand the contribution surface: prepared repository Plugins still contribute only declared skills and common MCP definitions, while arbitrary package lifecycle code remains trusted installation code rather than a model-facing Cordis Plugin API.
## Testing
`packages/ui/app-boot/tests/repository-cache.spec.ts` runs a local Git subpath through bundled pnpm with an injected command directory and proves that visible environment survives while credential-shaped variables are scrubbed. `packages/cordis/repository-plugin/tests/repository-plugin.spec.ts` pins the exact `prepack` metadata and temporary command cleanup. `apps/cli/tests/github-repository-plugin.built.e2e.ts` is the product acceptance: fresh DSH home, exact live GitHub source, actual built `dsh run`, real headless composition, mock LLM request observation, and prepared cache inspection.

View File

@@ -0,0 +1,42 @@
# Agent Note: 宿主自有的准备机制使 GitHub repository 插件可安装
状态:已实现
[English](2026-08-08-host-owned-git-repository-plugin-preparation.md) | 中文
## 问题
repository 插件的创作契约依赖 `scripts.prepare: "dsh-plugin-prepare"`,并要求源码仓库将 `@deepseek-ai/dsh-repository-plugin` 添加为开发依赖。该包是私有包,且未发布到 NPM因此即使外部 GitHub 仓库符合其他要求,也无法在全新安装中取得该辅助程序。
这种生命周期选择也无法支持 pnpm 工作区内可选的 `.dsh-plugin`。pnpm 会先运行 Git 托管仓库首选的包管理器,再打包选定的子目录,从而准备 Git 托管包。嵌套执行的 `pnpm install` 会加入外层工作区,而不一定执行未列入其中的 `.dsh-plugin` 包的 `prepare` 脚本。因此,安装可能成功并发布一个仅包含源包元数据的缓存 generation随后真实 DSH 启动因 `dsh-plugin.mjs` 不存在而失败。
签入仓库的 headless fixture测试前置数据没有捕获任一缺陷因为它挂载的是已准备好的包装层。它证明的是运行时组合而不是 GitHub 获取或包准备。
## 决策
修复后的创作格式要求元数据中精确包含 `scripts.prepack: "dsh-plugin-prepare"`,且不包含 DSH 依赖。pnpm 针对 Git 托管包的准备流程会在依赖安装步骤之后、打包清单选择 `.dsh-plugin` 子树之前显式调用 `prepack`,因此辅助程序仍可将 `../skills` 等同仓库的相邻资源复制进包内。
`@deepseek-ai/dsh-repository-plugin` 会生成临时的 POSIX 和 Windows 命令包装脚本,用于调用其自有的已构建 `dsh-plugin-prepare` 入口。`RepositoryCache` 接受由调用方持有的可执行文件目录,将它们解析为绝对路径,再前置到传给随附 pnpm、已清除凭据的包生命周期 `PATH`。该命令目录仅存在于安装事务期间无论成功还是失败都会被移除。仓库仍是受信任的包管理器输入DSH 仅提供这一条命令,其他生命周期脚本和依赖仍会按既有信任契约执行。
Node 24 消费方 CI 任务会传入从 PRPull Requesthead 仓库和 SHA 派生的精确源。其构建入口验收会启动真实的 `apps/cli/lib/bin.js run` 命令,并通过一个仅作用于当次运行的 patch 选择 `private: true`、不含依赖的 GitHub fixture。验收要求该次运行到达 mock LLM大语言模型在实际模型请求中找到 repository skill 描述,并验证不可变 DSH 缓存中的生成包装层和已复制 skill。如果 CI 遗漏精确源,测试会失败,而不是静默跳过。
## 考虑过的替代方案
**把准备辅助程序发布到 NPM。** 拒绝,因为源包会仅为了调用当前运行的 DSH 安装本就拥有的代码,而增加一个需要发布和管理版本的依赖;现有辅助程序又有意保持私有。
**保留 `prepare`,只注入命令。** 拒绝,因为当 Git 仓库的包管理器把嵌套包当作另一工作区的一部分时,即使命令可用,也不会使嵌套包的 `prepare` 生命周期得以运行。
**在 RepositoryCache 安装选定包后再准备。** 拒绝,因为 pnpm 打包后的子目录不再包含 `../skills` 等路径所引用的同仓库相邻资源;准备必须在生成打包清单前完成。
**在 DSH 中克隆 GitHub 仓库,并绕过 pnpm 的 Git 获取器。** 拒绝,因为这会重复实现已由锁定版本的包管理器负责的 ref 解析、子目录选择、依赖安装、打包清单行为和缓存完整性。
## 后果
- 仓库作者可以把修复后的 `.dsh-plugin/package.json` 和源资源提交到 GitHub而无需把插件或其准备辅助程序发布到 NPM。
- 预发布创作格式使用 `prepack` 而不是 `prepare`。无效的生命周期元数据会在源码准备或已安装包校验阶段导致失败,而不会留下状态不明的半成品格式。
- 精确源字符串仍标识不可变缓存 generation改变 ref 或源配置会选择另一个 generation。
- 本次修复不扩大贡献范围:已准备的 repository 插件仍只贡献已声明的 skills 和通用 MCP 定义,而任意包生命周期代码仍是受信任的安装代码,不是面向模型的 Cordis 插件 API。
## 测试
`packages/ui/app-boot/tests/repository-cache.spec.ts` 会用注入的命令目录通过随附 pnpm 运行本地 Git 子路径,并证明可见环境变量得以保留,而名称符合凭据模式的变量会被清除。`packages/cordis/repository-plugin/tests/repository-plugin.spec.ts` 锁定精确的 `prepack` 元数据和临时命令清理行为。`apps/cli/tests/github-repository-plugin.built.e2e.ts` 是产品验收测试:全新的 DSH 主目录、精确的真实 GitHub 源、实际构建产物的 `dsh run`、真实 headless 组合、mock LLM 请求观测,以及对已准备缓存的检查。