Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(cordis): add repository package cache

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-30 03:45:49 +08:00
parent f941ba1b5f
commit fd4d369907
14 changed files with 482 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.agents/notes/implemented/architecture/2026-07-30-package-manager-native-repository-cache.i18n.yaml Normal file
View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-package-manager-native-repository-cache.md
2026-07-30-package-manager-native-repository-cache.md: f8a6706065a936ca4a9abf2a50d266a60f09b252
2026-07-30-package-manager-native-repository-cache.zh.md: b1fea3d655f8d7aeb466744dc27bbf4ba69993ec

47
.agents/notes/implemented/architecture/2026-07-30-package-manager-native-repository-cache.md Normal file
View File

@@ -0,0 +1,47 @@
# Agent Note: Package-manager-native repository cache
Status: implemented
English | [中文](2026-07-30-package-manager-native-repository-cache.zh.md)
## Problem
A standalone Harness app cannot rely on a developer-owned SDK project to declare and install repository dependencies. Loading a configured GitHub repository therefore needs a persistent fetch, preparation, and cache boundary, but implementing Git transport, hosted-source syntax, package preparation, and a content store inside DSH would duplicate a package manager. Requiring a separately installed package manager would make a config-only feature depend on host setup.
The cache also needs an update identity. A mutable branch name cannot both remain permanently cached and reflect later commits without an independent refresh protocol.
## Decision
Vendored `@cordisjs/plugin-loader/repository` exports `RepositoryCache`, a generic Node-only package helper with no DSH plugin-format knowledge. Keeping it on a subpath prevents browser consumers of the Loader's main entry from traversing Node filesystem and child-process imports. The caller supplies a package-manager-native source specifier and a cache root. DSH-specific callers own accepted source syntax, path selection, and the cache-root location; the [SDK project dependency workflow](../../proposed/feature/2026-07-17-sdk-follow-up-capabilities.md#external-cordis-plugin-installation) remains a separate path owned by the developer project's selected package manager.
The Loader carries an exact runtime dependency on `pnpm@11.7.0` and invokes that package's JavaScript entry with the current Node executable. It never discovers a global executable or delegates through Corepack. Each cache miss creates an isolated project with one dependency named `repository`; pnpm owns Git/GitHub resolution, fetching, its content-addressed store, dependency installation, and lifecycle scripts in the repository's dependency graph.
The isolated workspace sets `dangerouslyAllowAllBuilds: true`. A configured repository and its dependency graph are trusted executable code: lifecycle scripts may run before DSH reads any declared assets. The child receives ordinary host process state needed by Git and pnpm, but ambient credential-shaped (`KEY`, `PASSWORD`, `SECRET`, `TOKEN`) variables are removed. No OAuth, token forwarding, or private-repository authentication contract is added.
The SHA-256 of the exact specifier names the cache entry. Concurrent same-process requests share one task. Installation occurs in a sibling temporary directory; only a successful install with a package directory and marker is atomically renamed into the final key. Failed staging is removed, and a competing process's already-published valid entry wins. A later process validates the marker and package directory before returning the stable `node_modules/repository` path.
An identical specifier permanently reuses its published entry. The caller changes the ref or another part of the specifier to request a new generation; the cache does not poll remotes, reinterpret mutable refs, expire entries, or garbage-collect old generations.
## Alternatives considered
**Implement GitHub download, archive extraction, preparation, and caching directly.** Rejected under the [dependency policy](../process/2026-07-26-dependencies-over-hand-rolling.md): pnpm already owns hosted Git syntax, Git execution, lifecycle policy, and a shared content store. A second resolver would add more code while still needing package semantics.
**Require `pnpm` on `PATH` or invoke Corepack.** Rejected because changing one app config must be sufficient on every supported installation. Pinning and shipping the CLI also makes the preparation policy reviewable and independent of the host's package-manager version.
**Resolve a branch or tag again on every startup.** Rejected because it turns startup into a network refresh, changes code without a config diff, and makes rollback depend on remote state. Explicit ref changes preserve auditability even when a user deliberately chooses a mutable ref.
**Disable repository lifecycle scripts.** Rejected because common plugin repositories need a declarative `prepare` step to validate and package their plugin subdirectory. The trust boundary is explicit configuration of executable source, not an incomplete illusion that only static files can run.
**Introduce a Cordis repository service.** Rejected because cache lookup has no runtime contribution registry or provider variation. A small helper lets the later host own Cordis lifecycle and HMR without adding a service seam prematurely.
## Consequences
- Standalone apps carry pnpm's approximately 18.6 MB unpacked runtime instead of requiring a global tool or owning a Git/package implementation.
- A repository author may use ordinary package preparation, and a malicious configured repository or dependency can execute code with the scrubbed child environment and the user's filesystem authority.
- Exact specifiers make startup deterministic after the first successful install; changing cached code requires a config/ref change.
- Failed installs leave no published cache entry and may be retried. Published corruption fails loud instead of silently reinstalling under the same identity.
- Cache generations consume disk until a future explicit cache-management policy removes them.
## Testing
`packages/ui/app-boot/tests/repository-cache.spec.ts` covers same-process single-flight, cross-instance cache reuse, exact-specifier separation, failed-stage cleanup and retry, and boundary validation. Its real local-Git case invokes the bundled pnpm, runs the fixture repository's `prepare` script, and reads the prepared file from the installed cache entry without network access.

47
.agents/notes/implemented/architecture/2026-07-30-package-manager-native-repository-cache.zh.md Normal file
View File

@@ -0,0 +1,47 @@
# Agent Note: 包管理器原生仓库缓存
Status: implemented
[English](2026-07-30-package-manager-native-repository-cache.md) | 中文
## 问题
独立运行的 Harness 应用不能依赖开发者自有的 SDK 工程来声明并安装仓库依赖。因此，加载配置中的 GitHub 仓库需要一道持久的获取、准备与缓存边界；但如果在 DSH 内实现 Git 传输、托管来源语法、包（package）准备流程和内容存储，就会重复实现包管理器。若要求用户另行安装包管理器，则只需修改配置即可使用的功能还会依赖宿主环境的额外配置。
缓存还需要明确更新标识。若没有独立的刷新协议，可变分支名无法既永久缓存，又反映后续 commit。
## 决策
vendor 中的 `@cordisjs/plugin-loader/repository` 导出 `RepositoryCache`：一个不包含 DSH 插件格式知识、仅限 Node 使用的通用包辅助工具。把它保留在子路径上，可以避免 Loader 主入口的浏览器消费方在解析依赖时遍历到 Node 文件系统和子进程 import。调用方提供包管理器原生的来源 specifier 和缓存根目录。DSH 专属调用方负责规定可接受的来源语法、路径选择与缓存根目录位置；[SDK 工程依赖工作流](../../proposed/feature/2026-07-17-sdk-follow-up-capabilities.md#external-cordis-plugin-installation)仍是另一条路径，由开发者工程选定的包管理器负责。
Loader 将 `pnpm@11.7.0` 作为固定版本的运行时依赖，并使用当前 Node 可执行文件调用该包的 JavaScript 入口。它绝不探测全局可执行文件，也不经 Corepack 调用。每次缓存未命中都会创建一个隔离工程，其中只有一个名为 `repository` 的依赖；Git 与 GitHub 来源的解析和获取、pnpm 自身的内容寻址 store、依赖安装，以及仓库依赖图中的生命周期脚本均由 pnpm 负责。
隔离工作区设置 `dangerouslyAllowAllBuilds: true`。用户配置的仓库及其依赖图都属于受信任的可执行代码：DSH 读取任何已声明资产之前，生命周期脚本就可能运行。子进程会收到 Git 与 pnpm 所需的常规宿主进程状态，但会移除环境中名称形似凭据（`KEY`、`PASSWORD`、`SECRET`、`TOKEN`）的变量。该机制不新增 OAuth、token 转发或私有仓库认证契约。
缓存项以精确 specifier 的 SHA-256 命名。同一进程内针对相同 specifier 的并发请求共享一项任务。安装在同级临时目录中进行；只有安装成功且存在包目录和标记时，系统才会把暂存目录原子重命名为最终键对应的目录。失败的暂存目录会被删除；如果另一进程已发布有效项，则以该项为准。后续进程会先校验标记与包目录，再返回稳定的 `node_modules/repository` 路径。
相同的 specifier 会永久复用已发布项。调用方通过修改 ref 或 specifier 的其他部分来请求新的缓存代次；缓存不会轮询远端、重新解释可变 ref、让条目过期，也不会垃圾回收旧代次。
## 曾考虑的替代方案
**直接实现 GitHub 下载、归档解压、准备与缓存。** 根据[依赖政策](../process/2026-07-26-dependencies-over-hand-rolling.md)不予采纳：pnpm 已负责托管 Git 语法、Git 执行、生命周期政策和共享内容存储。第二套解析器会增加更多代码，却仍需实现包语义。
**要求 `pnpm` 位于 `PATH` 上，或调用 Corepack。** 不予采纳：在每种受支持的安装形态中，只修改一份应用配置就必须足以启用该功能。固定并随应用分发 CLI（命令行界面）还能使准备政策可供评审，并与宿主的包管理器版本无关。
**每次启动都重新解析分支或 tag。** 不予采纳：这会把启动变成网络刷新，在配置 diff 未变化时更改代码，并让回滚依赖远端状态。即使用户有意选择可变 ref，显式修改 ref 仍能保持可审计性。
**禁用仓库生命周期脚本。** 不予采纳：常见插件仓库需要声明式 `prepare` 步骤来校验并打包插件子目录。信任边界是显式配置可执行来源，而不是营造一种不完整的假象，仿佛只有静态文件能够运行。
**引入 Cordis 仓库服务。** 不予采纳：缓存查找没有运行时贡献注册表，也不存在提供方变体。小型 helper 让后续宿主负责 Cordis 生命周期与 HMR（热模块替换），无需过早新增服务 seam。
## 后果
- 独立应用随附 pnpm 约 18.6 MB 的解压后运行时，不要求全局工具，也无需自行实现 Git 与包处理。
- 仓库作者可以使用常规包准备流程；恶意的已配置仓库或依赖可以在经过上述清理的子进程环境中，以用户的文件系统权限执行代码。
- 精确 specifier 使首次安装成功后的启动具有确定性；更改缓存代码必须修改配置或 ref。
- 安装失败不会留下已发布缓存项，可以再次重试。已发布缓存损坏时会明确报错，而不会在同一标识下静默重装。
- 缓存代次会持续占用磁盘，直到未来有明确的缓存管理政策将其移除。
## 测试
`packages/ui/app-boot/tests/repository-cache.spec.ts` 覆盖同进程 single-flight、跨实例缓存复用、精确 specifier 隔离、失败暂存清理与重试，以及边界校验。其真实本地 Git 用例会调用随附的 pnpm，运行 fixture（测试前置数据）仓库的 `prepare` 脚本，并在不访问网络的情况下，从已安装缓存项中读取准备后的文件。