docs: adopt dependencies-over-hand-rolling policy from NIH audit
A repo-wide Not Invented Here audit (ten parallel domain surveys covering every package group, scripts/, native/, vendor/ edges, python/, test infrastructure, and CI) asked of each hand-rolled surface whether a maintained external package or Node builtin deletes it with a net win. Policy: new implemented process note records that a dependency which genuinely deletes owned code is a preferred simplification (bar: net deletion, health, boundary fit, settled seams stay); root AGENTS.md carries the one-line rule and dsh-find-simplifications now surveys for hand-rolled-where-a-dependency-exists candidates. Findings, all bilingual from birth: - proposed/simplification: eventsource-parser for llm-deepseek SSE, node:timers/promises for three hand-rolled sleeps, turndown (or minimal 'entities') for tool-web HTML->markdown, gate-script consolidation onto mdast/parseArgs/globSync - proposed/testing: execa + parseArgs + loadEnvFile + vi.waitFor for hand-rolled test subprocess plumbing - proposed/process: pnpm/action-setup for symmetric CI caching - proposed/feature: evaluate landstrip before building a Windows sandbox launcher - rejected/simplification: ~30 swap verdicts recorded (vscode-jsonrpc, p-retry, Ajv, write-file-atomic, msw, hono, better-sqlite3, wireit, landstrip-for-linux, YAML consolidation, ...) so the survey is not re-litigated from scratch Also drops the stale prompt/ entry from the AGENTS.md layout map (workspace instructions live in packages/context/workspace-context).
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-evaluate-landstrip-for-windows-sandbox-rung.md: 047449f4915c973e86cdb9f05f6dc51535133534
|
||||
2026-07-26-evaluate-landstrip-for-windows-sandbox-rung.zh.md: 379d57e1e0006bf8f567d0b750ca0bb641ca6b49
|
||||
@@ -0,0 +1,34 @@
|
||||
# Agent Note: Evaluate landstrip before building a Windows sandbox launcher
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-evaluate-landstrip-for-windows-sandbox-rung.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The [sandbox decision](../../implemented/feature/2026-07-06-sandbox.md) leaves `PLATFORM_CHAINS.win32` empty and plans to fill it with "a confinement runner from the AppContainer/restricted-token family, shipped from its own repository on the `node-addon-landlock-run` template" — an estimated ~1,500-line new repo (the landlock-run subtree is ~1,460 lines of C/TS/scripts/tests plus docs and CI) authored and maintained in-house.
|
||||
|
||||
Since that note was written, a maintained third-party runner has appeared: `@landstrip/landstrip` (npm, actively developed, Rust core with prebuilt per-platform `optionalDependencies`) covers Landlock + seccomp on Linux, Seatbelt on macOS, and AppContainer/restricted-user on Windows, with JSON/YAML policy input and a trap-fd denial-reporting channel. It is exec-wrapped like bwrap, so it fits the chain's `confine(argv)` shape without touching the Linux/macOS rungs.
|
||||
|
||||
## Proposal
|
||||
|
||||
When the Windows sandbox phase is picked up, evaluate wrapping landstrip's Windows backend as the `win32` chain runner before authoring an in-house AppContainer launcher repository. The evaluation must answer:
|
||||
|
||||
- **Probe synthesis.** landstrip has no `--probe`; the chain's functional-probe contract would have to be synthesized from a trap run.
|
||||
- **Dialect mapping.** Denial and runner-failure stderr dialects, and fail-closed exit-code classification, need explicit mapping into the chain's vocabulary.
|
||||
- **License.** The binaries are LGPL-2.1-or-later; distribution review is required before it enters the shipped closure.
|
||||
- **Provenance.** The in-house launcher's value is byte-pinned native-CI provenance over a ~300-line reviewable C file; landstrip is a single-maintainer Rust binary set. For the *existing Linux rung* that trade is already settled — do not swap it ([sandbox note](../../implemented/feature/2026-07-06-sandbox.md) and the launcher's own migration away from a Rust dependency). For a rung we have not built, weighing third-party maintenance against a second in-house native repo is a genuinely open question.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Build the in-house AppContainer launcher as planned.** Still the default if the evaluation fails on license, provenance, or probe fit; the cost is owning a second native security launcher repo indefinitely.
|
||||
- **Swap the Linux Landlock rung to landstrip too.** Rejected outright: sandbox correctness is a security invariant, the current launcher's reviewability and provenance chain were chosen deliberately, and it already migrated away from a Rust dependency for exactly this reason.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Before any Windows-rung implementation starts, an evaluation records the probe, dialect, license, and provenance answers, and the go/no-go is added to the sandbox note's deferred-phases plan.
|
||||
|
||||
## Risks
|
||||
|
||||
- Single-maintainer supply chain in a security-critical position — the reason this is an evaluation gate, not an adoption decision.
|
||||
- The package is young; its API and packaging may churn before the Windows phase starts, so re-verify against the live registry then.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Agent Note: 在构建 Windows 沙箱启动器之前先评估 landstrip
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-evaluate-landstrip-for-windows-sandbox-rung.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
[沙箱决策](../../implemented/feature/2026-07-06-sandbox.md)将 `PLATFORM_CHAINS.win32` 留空,并计划用「AppContainer/受限令牌(restricted-token)家族的一个约束运行器,按 `node-addon-landlock-run` 模板从其独立仓库发布」来填充——一个估计约 1,500 行、需要自研编写并维护的新仓库(landlock-run 子树约为 1,460 行 C/TS/脚本/测试,外加文档与 CI)。
|
||||
|
||||
自那份决策记录写成以来,出现了一个持续维护的第三方运行器:`@landstrip/landstrip`(npm 包,活跃开发中,Rust 内核,附带按平台预构建的 `optionalDependencies`)覆盖 Linux 上的 Landlock + seccomp、macOS 上的 Seatbelt,以及 Windows 上的 AppContainer/受限用户,支持 JSON/YAML 策略输入和基于 trap-fd 的拒绝上报通道。它与 bwrap 一样采用 exec 包装方式,因此无需触碰 Linux/macOS 梯级即可契合链的 `confine(argv)` 形态。
|
||||
|
||||
## 提案
|
||||
|
||||
当 Windows 沙箱阶段启动时,在动手编写自研 AppContainer 启动器仓库之前,先评估将 landstrip 的 Windows 后端包装为 `win32` 链运行器。评估必须回答:
|
||||
|
||||
- **探测合成。** landstrip 没有 `--probe`;链所要求的功能探测契约必须从一次 trap 运行中合成出来。
|
||||
- **方言映射。** 拒绝与运行器失败两类 stderr 方言,以及失败即关闭(fail-closed)的退出码分类,都需要显式映射到链的词汇中。
|
||||
- **许可证。** 其二进制文件采用 LGPL-2.1-or-later 许可;在进入随产品发布的依赖闭包之前需要先做分发审查。
|
||||
- **溯源。** 自研启动器的价值在于对一个约 300 行、可审阅的 C 文件施以字节级锁定的原生 CI 溯源;而 landstrip 是单一维护者手中的一组 Rust 二进制文件。对*既有的 Linux 梯级*而言,这笔权衡早有定论——不要替换它(见[沙箱 Note](../../implemented/feature/2026-07-06-sandbox.md)以及该启动器自身摆脱 Rust 依赖的迁移)。而对一个我们尚未构建的梯级,在第三方维护与第二个自研原生仓库之间如何取舍,是一个真正悬而未决的问题。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **按原计划构建自研 AppContainer 启动器。** 若评估在许可证、溯源或探测契合度上不通过,这仍是默认选项;代价是要无限期持有第二个原生安全启动器仓库。
|
||||
- **把 Linux Landlock 梯级也换成 landstrip。** 直接否决:沙箱正确性是安全不变量,当前启动器的可审阅性与溯源链是刻意选择的结果,而且它正是出于这一原因才迁移摆脱了 Rust 依赖。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- 在任何 Windows 梯级实现开始之前,先有一份评估记录下探测、方言、许可证与溯源问题的答案,并把「做/不做」(go/no-go)的结论加入沙箱 Note 的延后阶段计划。
|
||||
|
||||
## 风险
|
||||
|
||||
- 处于安全关键位置的单一维护者供应链——这正是本提案定为一道评估门禁、而非采用决定的原因。
|
||||
- 该包尚且年轻;在 Windows 阶段启动之前其 API 与打包方式可能反复变动,届时需对照线上注册表重新核验。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md: 63e3f45ab2340ee2b732da286117e25be45bed08
|
||||
2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md: 2348e07d58f7f0ed39a1759cc30133c8e15dbc4a
|
||||
@@ -0,0 +1,31 @@
|
||||
# Agent Note: Use pnpm/action-setup for symmetric CI pnpm caching
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Five workflows repeat a hand-rolled three-step pnpm setup — `corepack enable`, `pnpm store path --silent >> $GITHUB_OUTPUT`, then `actions/cache@v4` keyed on `pnpm-lock.yaml`: `e2e.yml`, `docs-pages.yml`, `pi-ai-provider-e2e.yml`, `build-exe-for-python-sdk.yml`, and the node-compat, serial-linux, and benchmark jobs of `ci.yml` (~40–60 YAML lines total). The maintained equivalent — `pnpm/action-setup@v4` (reads `packageManager` from package.json) plus `actions/setup-node` with `cache: pnpm` — is already proven in-repo in `landlock-run.yml`, and also insulates against corepack's removal from newer Node distributions.
|
||||
|
||||
## Proposal
|
||||
|
||||
Convert the symmetric-cache workflows to `pnpm/action-setup@v4` + `setup-node` `cache: pnpm`. Explicitly do NOT convert:
|
||||
|
||||
- the three enterprise-runner PR jobs in `ci.yml` — they deliberately use `actions/cache/restore` only, keeping cache compression/upload off the paid latency-critical path, an asymmetry `setup-node`'s cache cannot express;
|
||||
- the Windows job, which deliberately skips the store cache.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the hand-rolled steps.** They work, but they are five drifting copies of setup boilerplate, and the corepack dependency is a known future break.
|
||||
- **Convert everything including the enterprise jobs.** Rejected: the restore-only asymmetry is a documented latency decision in `ci.yml`'s comments; erasing it to unify tooling inverts the priority.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- The five symmetric workflows set up pnpm via the actions; one cold run per lane repopulates the new cache-key format, after which cache hit rates match the old steps.
|
||||
- The enterprise-runner PR jobs and the Windows job are untouched.
|
||||
|
||||
## Risks
|
||||
|
||||
- Cache-key format changes once (one cold run per lane).
|
||||
- A third-party action in more workflows; it is already trusted in-repo (`landlock-run.yml`) and is the pnpm team's official action.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Agent Note: 用 pnpm/action-setup 实现对称的 CI pnpm 缓存
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
五个工作流重复着同一套手写(hand-rolled)的三步 pnpm 设置——`corepack enable`、`pnpm store path --silent >> $GITHUB_OUTPUT`、再加以 `pnpm-lock.yaml` 为缓存键的 `actions/cache@v4`:`e2e.yml`、`docs-pages.yml`、`pi-ai-provider-e2e.yml`、`build-exe-for-python-sdk.yml`,以及 `ci.yml` 的 node-compat、serial-linux 与 benchmark 作业(合计约 40–60 行 YAML)。与之等价、由官方维护的做法——`pnpm/action-setup@v4`(从 package.json 读取 `packageManager`)加带 `cache: pnpm` 的 `actions/setup-node`——已在仓库内的 `landlock-run.yml` 中得到验证,同时还能隔绝 corepack 被从较新 Node 发行版中移除的影响。
|
||||
|
||||
## 提案
|
||||
|
||||
将各对称缓存工作流改为 `pnpm/action-setup@v4` + `setup-node` `cache: pnpm`。以下明确不做转换:
|
||||
|
||||
- `ci.yml` 中运行在企业 runner 上的三个 PR(Pull Request)作业——它们刻意只用 `actions/cache/restore`,把缓存压缩/上传挡在付费且延迟敏感的关键路径之外,这种不对称是 `setup-node` 的缓存无法表达的;
|
||||
- Windows 作业,它刻意跳过 store 缓存。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **保留手写步骤。** 它们能用,但那是五份会各自漂移的设置样板副本,而且对 corepack 的依赖是已知的未来失效点。
|
||||
- **连企业作业在内全部转换。** 否决:只恢复不上传(restore-only)的不对称是 `ci.yml` 注释中有记录的延迟决策;为统一工具而抹掉它,属于颠倒优先级。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- 五个对称工作流经由上述 action 完成 pnpm 设置;每条泳道各跑一次冷运行以重建新的缓存键格式,此后缓存命中率与旧步骤持平。
|
||||
- 企业 runner 上的 PR 作业与 Windows 作业保持原样不动。
|
||||
|
||||
## 风险
|
||||
|
||||
- 缓存键格式变更一次(每条泳道各一次冷运行)。
|
||||
- 更多工作流引入一个第三方 action;它已在仓库内获得信任(`landlock-run.yml`),且是 pnpm 团队的官方 action。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-builtin-timer-promises-for-hand-rolled-sleeps.md: 036e2f2906ca99aaab30a2164649f9c750b4ad21
|
||||
2026-07-26-builtin-timer-promises-for-hand-rolled-sleeps.zh.md: 15a6f0dd412d142647df2722335a454a028cb798
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: Use node:timers/promises for hand-rolled cancellable sleeps
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-builtin-timer-promises-for-hand-rolled-sleeps.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Three packages hand-roll promise-wrapped timers that the `node:timers/promises` builtin already provides, while other packages (`dsh-llm-mock-server` `pause()`, `dsh-lsp-local`, `dsh-acp-snapshot`) already use the builtin — so the hand-rolled copies are also a consistency gap:
|
||||
|
||||
- `packages/llm/llm-retry/src/index.ts` `cancellableDelay()` (~14 lines): `new Promise` + `setTimeout` + manual abort-listener add/remove, resolving `true` on elapse and `false` on abort, consumed once for the backoff wait.
|
||||
- `packages/workflow/workflow-workerthread/src/host.ts` `sleep()` (~7 lines): promise-wrapped unref'd `setTimeout` used as the dispose-grace bound.
|
||||
- `packages/pty/pty-local/src/session.ts` `delay()` (~4 lines): bare promise-wrapped `setTimeout` used in polling/teardown waits.
|
||||
|
||||
## Proposal
|
||||
|
||||
Replace both with `import { setTimeout } from 'node:timers/promises'`:
|
||||
|
||||
- llm-retry: `try { await setTimeout(delayMs, undefined, { signal }); /* retry */ } catch { /* abort → fail */ }` — with a signal, the promise rejects only with the abort error, and a pre-aborted signal rejects immediately; behavior is identical, including timer clearing on abort. The empty `catch` names the abort rejection per the repo's empty-catch rule.
|
||||
- workflow-workerthread: `setTimeout(ms, undefined, { ref: false })` — exact semantics including not holding the event loop open.
|
||||
- pty-local: `import { setTimeout as delay } from 'node:timers/promises'` — identical signature, call sites unchanged.
|
||||
|
||||
No dedicated tests pin the helpers themselves; the packages' behavior suites keep passing.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **`p-timeout`/`p-defer` style packages.** Rejected: the builtin covers both call sites exactly; an external package for a one-line await is negative-net.
|
||||
- **Leave them.** Rejected only weakly — the cost is small, but the repo already uses the builtin idiom elsewhere, and two hand-rolled variants of a builtin invite a third.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Neither package defines a promise-wrapped `setTimeout` helper; both import from `node:timers/promises`.
|
||||
- `llm-retry` and `workflow-workerthread` test suites pass unchanged (behavioral parity).
|
||||
|
||||
## Risks
|
||||
|
||||
Essentially none: no model-visible output, no platform concerns, no new dependency. The llm-retry rewrite changes a boolean-returning helper into try/catch control flow — a local readability judgment the implementing PR makes.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: 用 node:timers/promises 替代手写的可取消休眠
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-builtin-timer-promises-for-hand-rolled-sleeps.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
三个包(package)手写了 promise 包装的定时器,而 `node:timers/promises` 内置模块早已提供同等能力;其他包(`dsh-llm-mock-server` 的 `pause()`、`dsh-lsp-local`、`dsh-acp-snapshot`)已经在使用该内置模块,因此这些手写副本同时也是一处一致性缺口:
|
||||
|
||||
- `packages/llm/llm-retry/src/index.ts` 的 `cancellableDelay()`(约 14 行):`new Promise` + `setTimeout` + 手动添加/移除 abort 监听器,计时走完时 resolve 为 `true`、被中止时 resolve 为 `false`,仅在退避等待处消费一次。
|
||||
- `packages/workflow/workflow-workerthread/src/host.ts` 的 `sleep()`(约 7 行):promise 包装、已 unref 的 `setTimeout`,用作 dispose(资源释放)宽限的时间上界。
|
||||
- `packages/pty/pty-local/src/session.ts` 的 `delay()`(约 4 行):朴素的 promise 包装 `setTimeout`,用于轮询与拆除等待。
|
||||
|
||||
## 提案
|
||||
|
||||
用 `import { setTimeout } from 'node:timers/promises'` 替换上述实现:
|
||||
|
||||
- llm-retry:`try { await setTimeout(delayMs, undefined, { signal }); /* retry */ } catch { /* abort → fail */ }`。传入 signal 后,该 promise 只会以 abort 错误拒绝,已提前中止的 signal 则立即拒绝;行为完全一致,包括中止时清除定时器。按仓库的空 catch 规则,这个空 `catch` 注明其吞下的是 abort 拒绝。
|
||||
- workflow-workerthread:`setTimeout(ms, undefined, { ref: false })`,语义完全等价,包括不会让事件循环保持存活。
|
||||
- pty-local:`import { setTimeout as delay } from 'node:timers/promises'`,签名完全相同,调用点无需改动。
|
||||
|
||||
没有专属测试固定这些辅助函数本身;各包的行为测试套件继续通过。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **`p-timeout`/`p-defer` 一类的包。** 不予采纳:内置模块恰好精确覆盖这些调用点;为一行 await 引入外部包是负收益。
|
||||
- **维持现状。** 不予采纳,但理由较弱:成本确实很小,但仓库其他地方已经在用这一内置惯用法,而同一内置能力存在两个手写变体,就会招来第三个。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- 上述包不再各自定义 promise 包装的 `setTimeout` 辅助函数,而是都从 `node:timers/promises` 导入。
|
||||
- `llm-retry` 与 `workflow-workerthread` 的测试套件原样通过(行为等价)。
|
||||
|
||||
## 风险
|
||||
|
||||
基本没有风险:不涉及模型可见的输出,没有平台顾虑,也不新增依赖。llm-retry 的改写把一个返回布尔值的辅助函数变成 try/catch 控制流,这是一项局部可读性判断,由实施 PR(Pull Request)裁量。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-consolidate-gate-scripts-on-existing-deps.md: 2b6c2f80b4fc3d3bf818b6789b5f40bb7a61b654
|
||||
2026-07-26-consolidate-gate-scripts-on-existing-deps.zh.md: b20a5bd9ba1661321721c0c9d62de8dc63ec645b
|
||||
@@ -0,0 +1,38 @@
|
||||
# Agent Note: Consolidate gate scripts on already-present deps and builtins
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-consolidate-gate-scripts-on-existing-deps.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The `scripts/` gates mostly use the right tools (`node:fs` `globSync` in 15+ gates, mdast/micromark in the markdown gates), but a handful of stragglers hand-roll what a sibling gate already does with an existing dependency or builtin:
|
||||
|
||||
- **Duplicated fence scanners.** `scripts/md-fences.ts` (~55 lines, consumed by `doc-typecheck.ts`) and `extractEquivBlocks` in `scripts/verify-type-equiv.ts` (~39 lines) are two copies of the same regex line-scanner for fenced code blocks, while `scripts/verify-mermaid.ts` already extracts fences by visiting mdast `code` nodes via the shared `scripts/markdown.ts` helpers — and `markdownProseLines` in `markdown.ts` itself parses to mdast but then hand-tracks fence state with a second regex. The regex scanners only recognize backtick fences at column 0, so they silently disagree with the mdast-based gates on tilde and indented fences.
|
||||
- **Hand-rolled argv parsing.** `parseOptions` in `scripts/publint-all.ts` and its near-identical copy in `scripts/verify-built-package-invariants.mjs` (~26 lines) step argv indexes manually, while sibling scripts (`verify-runtime-closure.ts`, `build-exe-for-python-sdk.ts`, `packages/sdk/scripts/src/args.ts`) already use the `node:util` `parseArgs` builtin.
|
||||
- **Hand-rolled directory walks.** Five sites re-derive nested `readdirSync` walks that `globSync` covers: `verify-runtime-closure.ts` (packages + vendor manifests), `dev-web.ts` `discoverPluginDirs`, `verify-package-paths.ts` `realPackageNames`, `verify-client-domain-graph.ts` `listSources`, and `publint-all.ts` `addPath` (~55–65 lines total). `scripts/package-invariants.ts` shows the one-line `globSync` template.
|
||||
|
||||
No new dependency is needed anywhere; every replacement is an existing devDep or a Node builtin.
|
||||
|
||||
## Proposal
|
||||
|
||||
- Extract a shared ~10–15-line mdast fence helper (visiting `code` nodes for `lang`, `meta`, `value`, `position.start.line`) into `scripts/markdown.ts`; rewrite `doc-typecheck.ts` and `verify-type-equiv.ts` onto it; delete `md-fences.ts` and the duplicated scanner; drop the redundant fence regex in `markdownProseLines`.
|
||||
- Replace both `parseOptions` copies with `parseArgs`.
|
||||
- Replace the five straggler walks with `globSync`. Keep the walks in `check-workspace-constraints.ts` and `clean.ts`: they need dirent-level detail to diagnose malformed trees, which glob-by-pattern cannot report.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **A new glob/walking dependency (`tinyglobby`, `fdir`).** Rejected: the builtin already won repo-wide; these are stragglers, not a gap.
|
||||
- **`p-map` for `publint-all.ts`'s ~19-line ordered worker pool.** Deliberately left out: one new devDep for one small deletion is at the edge of the [dependency policy](../../implemented/process/2026-07-26-dependencies-over-hand-rolling.md) bar, and the pool's requirements (bounded workers, deterministic order, env override) are documented in the [parallel-gates note](../../implemented/process/2026-07-06-parallel-pre-push-gates.md). Fold it in only if `p-map` earns a second consumer.
|
||||
- **Leaving the fence scanners.** Rejected: two drifting copies of a parser beside a third correct implementation is exactly the duplication the shared `markdown.ts` helper exists to prevent, and the column-0-backtick-only limitation is a latent inconsistency between sibling gates.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `md-fences.ts` is gone; `doc-typecheck` and `verify-type-equiv` extract fences through `scripts/markdown.ts`; `pnpm run doc-sync` passes with unchanged results on the current tree (any delta traces to a fence shape the regex scanners mishandled).
|
||||
- Both CLIs parse via `parseArgs`; unknown options still fail loud.
|
||||
- The five walk sites use `globSync`; the gates they feed pass unchanged.
|
||||
|
||||
## Risks
|
||||
|
||||
- Behavioral deltas on pathological markdown: mdast honors tilde/indented fences the regex scanners ignored, so `doc-typecheck`'s opt-out ratio could shift if any stray fence shape exists in the docs tree; verify by running `doc-sync` before/after.
|
||||
- `parseArgs` keeps the last value of a duplicated option instead of erroring and consumes a `--`-prefixed next token as a value; both are dev-tool edge cases the tests don't pin.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Agent Note: 把门禁脚本统一到已有依赖与内置模块上
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-consolidate-gate-scripts-on-existing-deps.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`scripts/` 下的门禁大多已经在用正确的工具(15 个以上的门禁使用 `node:fs` 的 `globSync`,markdown 门禁使用 mdast/micromark),但少数几个掉队的脚本仍在手写同类门禁早已用既有依赖或内置模块完成的事情:
|
||||
|
||||
- **重复的围栏扫描器。**`scripts/md-fences.ts`(约 55 行,由 `doc-typecheck.ts` 消费)和 `scripts/verify-type-equiv.ts` 中的 `extractEquivBlocks`(约 39 行)是同一个围栏代码块正则行扫描器的两份拷贝,而 `scripts/verify-mermaid.ts` 已经通过共享的 `scripts/markdown.ts` 辅助函数访问 mdast `code` 节点来提取代码围栏;`markdown.ts` 自己的 `markdownProseLines` 也是先解析成 mdast,再用第二个正则手工跟踪围栏状态。这两个正则扫描器只识别第 0 列的反引号围栏,因此在波浪线围栏和缩进围栏上与基于 mdast 的门禁悄悄不一致。
|
||||
- **手写的 argv 解析。**`scripts/publint-all.ts` 中的 `parseOptions` 和 `scripts/verify-built-package-invariants.mjs` 中与之几乎相同的拷贝(约 26 行)手工推进 argv 下标,而同类脚本(`verify-runtime-closure.ts`、`build-exe-for-python-sdk.ts`、`packages/sdk/scripts/src/args.ts`)已经在使用 `node:util` 的内置 `parseArgs`。
|
||||
- **手写的目录遍历。**五处代码各自重写了 `globSync` 已覆盖的嵌套 `readdirSync` 遍历:`verify-runtime-closure.ts` 对 packages 与 vendor manifest(元数据清单)的扫描、`dev-web.ts` 的 `discoverPluginDirs`、`verify-package-paths.ts` 的 `realPackageNames`、`verify-client-domain-graph.ts` 的 `listSources`,以及 `publint-all.ts` 的 `addPath`(合计约 55–65 行)。`scripts/package-invariants.ts` 展示了一行式的 `globSync` 模板。
|
||||
|
||||
所有替换都不需要引入新依赖;每一处替换用的都是既有的 devDependency 或 Node 内置模块。
|
||||
|
||||
## 提案
|
||||
|
||||
- 在 `scripts/markdown.ts` 中提取一个约 10–15 行的共享 mdast 围栏辅助函数(访问 `code` 节点,读取 `lang`、`meta`、`value`、`position.start.line`);把 `doc-typecheck.ts` 和 `verify-type-equiv.ts` 改写到它上面;删除 `md-fences.ts` 和重复的扫描器;去掉 `markdownProseLines` 中冗余的围栏正则。
|
||||
- 用 `parseArgs` 替换两份 `parseOptions` 拷贝。
|
||||
- 用 `globSync` 替换那五处掉队的目录遍历。保留 `check-workspace-constraints.ts` 和 `clean.ts` 中的遍历:它们需要 dirent 级别的细节来诊断结构异常的目录树,按模式匹配的 glob 报告不了这些信息。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **新的 glob/目录遍历依赖(`tinyglobby`、`fdir`)。**不予采纳:内置模块已在全仓库范围内胜出;这几处只是掉队者,不是能力缺口。
|
||||
- **用 `p-map` 替换 `publint-all.ts` 中约 19 行的有序 worker 池。**刻意未纳入:为一次小删除引入一个新 devDependency,正处在[依赖策略](../../implemented/process/2026-07-26-dependencies-over-hand-rolling.md)门槛的边缘,而且该池的需求(worker 数量有界、确定性顺序、环境变量覆盖)已记录在[并行 pre-push 门禁决策记录](../../implemented/process/2026-07-06-parallel-pre-push-gates.md)中。仅当 `p-map` 赢得第二个消费方时再顺带纳入。
|
||||
- **保留这两个围栏扫描器。**不予采纳:在第三个正确实现旁边放着两份逐渐漂移的解析器拷贝,正是共享的 `markdown.ts` 辅助函数要防止的那种重复;「只认第 0 列反引号」的限制也是同类门禁之间的潜在不一致。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- `md-fences.ts` 已删除;`doc-typecheck` 与 `verify-type-equiv` 通过 `scripts/markdown.ts` 提取代码围栏;`pnpm run doc-sync` 在当前代码树上通过且结果不变(如有差异,必须能追溯到正则扫描器处理有误的某种围栏形态)。
|
||||
- 两个 CLI 都改用 `parseArgs` 解析;未知选项仍然大声失败。
|
||||
- 五处遍历代码改用 `globSync`;它们供给的门禁保持原样通过。
|
||||
|
||||
## 风险
|
||||
|
||||
- 病态 markdown 上的行为差异:mdast 会承认正则扫描器忽略的波浪线围栏和缩进围栏,因此如果文档树中存在任何零散的此类围栏形态,`doc-typecheck` 的 opt-out 比例可能变化;应在改动前后分别运行 `doc-sync` 加以验证。
|
||||
- `parseArgs` 对重复出现的选项保留最后一个值而不报错,还会把下一个以 `--` 开头的 token 当作值消费;这两种情况都是测试未固定的开发工具边缘用例。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-eventsource-parser-for-deepseek-sse.md: 8a93b7f6c7aa0d428f25e87c44e1d29e884ecc81
|
||||
2026-07-26-eventsource-parser-for-deepseek-sse.zh.md: b16109d9458f487c7e463cf02e6b2d22fbbde015
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: Replace the hand-rolled SSE parser in llm-deepseek with eventsource-parser
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-eventsource-parser-for-deepseek-sse.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`packages/llm/llm-deepseek/src/sse.ts` hand-implements Server-Sent Events parsing: a streaming `TextDecoder`, event-block splitting on `\r?\n\r?\n`, `data:` payload extraction and joining, comment/field skipping, the `[DONE]` sentinel, a `STREAM_CLOSED` error on EOF without it, and a flush of a final unterminated event block. The file is ~67 lines with ~108 lines of dedicated tests (`tests/sse.spec.ts`) re-proving SSE spec behavior — UTF-8 split across chunks, CRLF handling, multi-`data:` joining, no-space-after-colon — that a maintained parser already guarantees. Its only consumer is `adapter.ts` (`yield* translate(parseSse(response.body))`).
|
||||
|
||||
This is exactly the surface `eventsource-parser` owns: the de-facto standard SSE parser (it underlies the Vercel AI SDK and the MCP SDK), zero-dependency, actively maintained, and already present in this repo's lockfile transitively via `@modelcontextprotocol/sdk` — so adopting it directly adds no new supply-chain surface in practice.
|
||||
|
||||
## Proposal
|
||||
|
||||
Replace `sse.ts` with `EventSourceParserStream` from `eventsource-parser/stream`: `response.body.pipeThrough(new TextDecoderStream()).pipeThrough(new EventSourceParserStream())`, keeping only the DeepSeek protocol shim (~10–25 lines): yield each event's `data`, terminate on `[DONE]`, and throw `LlmError('STREAM_CLOSED')` when the stream ends without the sentinel. All required builtins (`TextDecoderStream`, `pipeThrough`, async-iterable `ReadableStream`) exist at the Node ^22.19 engine floor. Delete the spec-conformance tests; keep the `[DONE]`/`STREAM_CLOSED`/EOF contract tests. Add `eventsource-parser` to `llm-deepseek`'s dependencies (its second runtime dep after schemastery). Update the [twin-adapters note](../../implemented/architecture/2026-06-13-twin-llm-adapters.md) and the `dsh-llm` JSDoc that brand this adapter "hand-rolled fetch + SSE parsing" in the same PR.
|
||||
|
||||
The library also strips a leading BOM (the hand-rolled parser would fail to match `data:` after one) and offers `maxBufferSize` hardening the current parser lacks.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the hand-rolled parser.** Defensible under the [twin-adapters decision](../../implemented/architecture/2026-06-13-twin-llm-adapters.md): the adapter is deliberately the hand-rolled design-verification twin of the pi-ai adapter. But the note's load-bearing distinction is owning the fetch/translate internals versus delegating to a full provider SDK; a ~700-byte SSE micro-parser is transport plumbing, not the design under verification. Whether that reading stands is the twin-note owner's call — this proposal explicitly needs their sign-off.
|
||||
- **`createParser({onEvent})` callback API instead of the stream.** Works fed by a manual `TextDecoder` loop, but the `pipeThrough` composition deletes more of the hand-rolled code.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `sse.ts`'s parsing internals are gone; the remaining shim only encodes the DeepSeek `[DONE]`/`STREAM_CLOSED` protocol.
|
||||
- `llm-deepseek` unit tests and the real-API e2e suite pass; keyless snapshots are unchanged (parsing is transport-internal and payload extraction is equivalent).
|
||||
- The twin-adapters note and `dsh-llm` JSDoc no longer claim hand-rolled SSE parsing.
|
||||
|
||||
## Risks
|
||||
|
||||
- One deliberate robustness deviation is lost: the hand-rolled parser flushes a final event block that lacks its terminating blank line, and `tests/sse.spec.ts` pins that a trailing `data: [DONE]` without `\n\n` still yields DONE. eventsource-parser is spec-strict and only dispatches on the blank line, so that shape becomes `STREAM_CLOSED`. Real providers and `dsh-llm-mock-server` always terminate events properly, so the pinned behavior is a robustness nicety, not an observed provider shape — drop the test, or keep a tiny buffer-tail check if the deviation is judged load-bearing.
|
||||
- Dilutes the documented "hand-rolled" identity of the twin adapter; mitigated by updating the note in the same change rather than leaving the claim stale.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: 用 eventsource-parser 替换 llm-deepseek 中手写的 SSE 解析器
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-eventsource-parser-for-deepseek-sse.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`packages/llm/llm-deepseek/src/sse.ts` 手写实现了 SSE(Server-Sent Events)解析:一个流式 `TextDecoder`、按 `\r?\n\r?\n` 切分事件块、提取并拼接 `data:` 载荷、跳过注释与其他字段、`[DONE]` 哨兵、在未见哨兵即 EOF 时抛出 `STREAM_CLOSED` 错误,以及对最后一个未终结事件块的 flush。该文件约 67 行,另有约 108 行专属测试(`tests/sse.spec.ts`)重复验证 SSE 规范行为——UTF-8 字符被切分到多个分片、CRLF 处理、多条 `data:` 拼接、冒号后无空格——而这些行为,持续维护的解析器早已有保证。它唯一的消费方是 `adapter.ts`(`yield* translate(parseSse(response.body))`)。
|
||||
|
||||
这恰好是 `eventsource-parser` 负责的接口面:事实标准的 SSE 解析器(Vercel AI SDK 和 MCP SDK 都构建在它之上),零依赖,持续维护,并且已通过 `@modelcontextprotocol/sdk` 作为传递依赖出现在本仓库的 lockfile 中——因此直接采用它实际上不增加新的供应链接触面。
|
||||
|
||||
## 提案
|
||||
|
||||
用 `eventsource-parser/stream` 的 `EventSourceParserStream` 替换 `sse.ts`:`response.body.pipeThrough(new TextDecoderStream()).pipeThrough(new EventSourceParserStream())`,只保留 DeepSeek 协议垫层(约 10–25 行):逐个产出事件的 `data`,遇到 `[DONE]` 终止,流在未见哨兵时结束则抛出 `LlmError('STREAM_CLOSED')`。所需的全部内置能力(`TextDecoderStream`、`pipeThrough`、可异步迭代的 `ReadableStream`)在 Node ^22.19 引擎下限即已存在。删除规范符合性测试;保留 `[DONE]`/`STREAM_CLOSED`/EOF 契约测试。将 `eventsource-parser` 加入 `llm-deepseek` 的依赖(这是它继 schemastery 之后的第二个运行时依赖)。在同一个 PR(Pull Request)中更新[孪生适配器 Agent Note(agent 决策记录)](../../implemented/architecture/2026-06-13-twin-llm-adapters.md)以及 `dsh-llm` 中把该适配器标为「手写 fetch + SSE 解析」的 JSDoc。
|
||||
|
||||
该库还会剥离开头的 BOM(手写解析器在 BOM 之后会无法匹配 `data:`),并提供当前解析器缺少的 `maxBufferSize` 加固能力。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **保留手写解析器。** 依据[孪生适配器决策](../../implemented/architecture/2026-06-13-twin-llm-adapters.md),这一选择有辩护余地:该适配器有意作为 pi-ai 适配器的手写设计验证孪生体。但那份 Agent Note 起支撑作用的区分在于「自行持有 fetch/translate 内部实现」与「委托给完整的提供方 SDK」;一个约 700 字节的 SSE 微型解析器属于传输层管道,不是被验证的设计本身。这一解读是否成立由孪生 Agent Note 的所有者裁定——本提案明确需要其签署确认。
|
||||
- **改用 `createParser({onEvent})` 回调 API 而非流。** 配合手动的 `TextDecoder` 循环可以工作,但 `pipeThrough` 组合方式能删除更多手写代码。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- `sse.ts` 的解析内部实现消失;剩下的垫层只编码 DeepSeek 的 `[DONE]`/`STREAM_CLOSED` 协议。
|
||||
- `llm-deepseek` 单元测试与真实 API 的 e2e 套件通过;无密钥快照不变(解析属于传输层内部,载荷提取等价)。
|
||||
- 孪生适配器 Agent Note 与 `dsh-llm` 的 JSDoc 不再声称手写 SSE 解析。
|
||||
|
||||
## 风险
|
||||
|
||||
- 会失去一处有意为之的健壮性偏离:手写解析器会 flush 缺少终结空行的最后一个事件块,`tests/sse.spec.ts` 固定了「末尾的 `data: [DONE]` 即使没有 `\n\n` 也仍产出 DONE」这一行为。eventsource-parser 严格遵循规范,只在空行处分发事件,因此这种形态会变成 `STREAM_CLOSED`。真实提供方和 `dsh-llm-mock-server` 总是正确终结事件,所以被固定的行为只是健壮性上的锦上添花,并非实际观测到的提供方形态:可以删除该测试;若判定该偏离确有支撑作用,也可以保留一个小型的缓冲区尾部检查。
|
||||
- 稀释了孪生适配器有文档记录的「手写」身份;缓解方式是在同一次变更中更新那份 Agent Note,而不是让声明陈旧下去。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-turndown-for-tool-web-html-markdown.md: 7f25e51bf6e6fc9313a880abee737bca80a472af
|
||||
2026-07-26-turndown-for-tool-web-html-markdown.zh.md: 3a59b08e13fd392e4f34ac543f32f5b4648f3c1c
|
||||
@@ -0,0 +1,32 @@
|
||||
# Agent Note: Replace tool-web's regex HTML-to-markdown converter with turndown
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-turndown-for-tool-web-html-markdown.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`packages/web/tool-web/src/html.ts` (~86 lines, ~40 lines of dedicated tests) converts fetched HTML to markdown with regexes: strip script/style/noscript/comments, convert `<a>`/`<h1-6>`/`<li>`, decode numeric entities plus a 12-entry named-entity table, collapse whitespace. The module's own JSDoc says "A richer converter can replace it without changing the seam or tool schema", and the README's Known Limitations documents it as "a minimal regex converter, not an HTML parser — tables, images, and nested formatting are lost." The [web capability seam note](../../implemented/architecture/2026-06-24-web-capability-seam.md) assigns HTML→markdown to this package as presentation, so the swap point is exactly here. The converter's output is model-visible on every fetched HTML page; no keyless snapshot currently exercises `web_fetch`, so no expected outputs pin it.
|
||||
|
||||
## Proposal
|
||||
|
||||
Replace `htmlToMarkdown` with `turndown` (`new TurndownService().turndown(html)`), optionally with `turndown-plugin-gfm` for tables. The consumer switch in `fetch.ts` and the status-header/truncation-footer formatting stay. Wrap the call in try/catch falling back to the raw text path: the regex version could never throw; turndown on pathological HTML could. Delete `html.ts` and its conversion tests; keep tests for the fallback and the surrounding formatting. Update the README's Known Limitations to drop the regex-converter caveat.
|
||||
|
||||
If the "deliberately minimal fallback" stance is preferred instead, a minimal variant still deletes the worst part: replace the entity-decoding third of the file (~30 lines: `decodeEntities`, `NAMED_ENTITIES`, `safeFromCodePoint`) with the zero-dependency `entities` package (already in the lockfile transitively), erasing the documented "about a dozen entities" limitation at near-zero risk.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **`@mozilla/readability` + a DOM.** Solves a different problem (content extraction, not conversion) and drags a heavier DOM dependency; the seam only asks for markdown rendering of whatever the fetch returned.
|
||||
- **Keep the regex converter.** It was an explicit v1 placeholder per its own JSDoc; keeping it means model-visible quality (tables, images, nested formatting) stays lost for the cost of maintaining bespoke entity tables.
|
||||
- **The minimal `entities`-only variant.** Kept in the proposal as the fallback position; it deletes less but avoids the dependency-weight question entirely.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `web_fetch` renders tables/nested formatting via turndown (or, minimal variant: decodes all named entities), with the README limitation updated.
|
||||
- Unit tests cover the fallback path; `pnpm run test` passes for the package.
|
||||
- A keyless snapshot exercising `web_fetch` markdown rendering is added per testing policy (the missing snapshot coverage is part of the change, and it pins the new output).
|
||||
|
||||
## Risks
|
||||
|
||||
- Model-visible output changes on every fetched HTML page — transcript drift is acceptable pre-release, and nothing currently pins the old output.
|
||||
- Dependency weight: turndown's one dependency (`@mixmark-io/domino`) is a ~200 KB DOM that would enter the single-file-executable closure if tool-web ships in it ([single-exe note](../../implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md)); the minimal `entities` variant avoids this if closure size is the deciding factor.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Agent Note: 用 turndown 替换 tool-web 的正则 HTML 转 markdown 转换器
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-turndown-for-tool-web-html-markdown.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`packages/web/tool-web/src/html.ts`(约 86 行,另有约 40 行专属测试)用正则表达式把抓取到的 HTML 转成 markdown:剥离 script、style、noscript 标签与注释,转换 `<a>`/`<h1-6>`/`<li>`,解码数字实体外加一张 12 项的命名实体表,并折叠空白。该模块自身的 JSDoc 写明「A richer converter can replace it without changing the seam or tool schema」,README 的 Known Limitations 章节也把它记载为「a minimal regex converter, not an HTML parser — tables, images, and nested formatting are lost」。[web 能力 seam 决策记录](../../implemented/architecture/2026-06-24-web-capability-seam.md)把 HTML 转 markdown 作为呈现职责划归本包(package),因此替换点恰好就在这里。每个抓取到的 HTML 页面上,该转换器的输出都对模型可见;当前没有任何无密钥快照执行到 `web_fetch`,因此没有预期输出固定它的行为。
|
||||
|
||||
## 提案
|
||||
|
||||
用 `turndown` 替换 `htmlToMarkdown`(`new TurndownService().turndown(html)`),可选择配合 `turndown-plugin-gfm` 支持表格。`fetch.ts` 中的消费方分支与状态头、截断页脚的格式化保持不变。把调用包在 try/catch 中,失败时回退到原始文本路径:正则版本从不可能抛异常,而 turndown 处理病态 HTML 时可能抛出。删除 `html.ts` 及其转换测试;保留回退路径与外围格式化的测试。更新 README 的 Known Limitations 章节,移除正则转换器的警示说明。
|
||||
|
||||
如果更倾向于「刻意保持最小回退实现」的立场,最小变体仍能删掉最糟的部分:用零依赖的 `entities` 包(已通过传递依赖存在于 lockfile 中)替换文件中占三分之一的实体解码部分(约 30 行:`decodeEntities`、`NAMED_ENTITIES`、`safeFromCodePoint`),以近乎为零的风险抹掉文档记载的「about a dozen entities」限制。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **`@mozilla/readability` 加一个 DOM。** 它解决的是另一个问题(内容提取,而非格式转换),还会拖入更重的 DOM 依赖;这个 seam 只要求把抓取返回的内容渲染成 markdown。
|
||||
- **保留正则转换器。** 按其自身 JSDoc 的说法,它本来就是明确的 v1 占位实现;保留它意味着模型可见的质量(表格、图片、嵌套格式)继续缺失,代价还是维护一套自制实体表。
|
||||
- **仅引入 `entities` 的最小变体。** 已作为退守方案保留在提案中;它删得更少,但完全避开了依赖体积问题。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- `web_fetch` 经由 turndown 渲染表格与嵌套格式(或在最小变体下:解码全部命名实体),README 中的限制说明同步更新。
|
||||
- 单元测试覆盖回退路径;该包的 `pnpm run test` 通过。
|
||||
- 按测试政策补充一个执行 `web_fetch` markdown 渲染的无密钥快照(缺失的快照覆盖是本变更的一部分,它同时固定新输出)。
|
||||
|
||||
## 风险
|
||||
|
||||
- 模型可见的输出在每个抓取到的 HTML 页面上都会变化:预发布阶段的 transcript(文本记录)漂移可以接受,且当前没有任何东西固定旧输出。
|
||||
- 依赖体积:turndown 的唯一依赖(`@mixmark-io/domino`)是一个约 200 KB 的 DOM 实现,若 tool-web 进入单文件可执行文件,它会一并进入闭包([single-exe 决策记录](../../implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md));若闭包体积是决定因素,最小的 `entities` 变体可以避开这一点。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-26-execa-for-test-subprocess-plumbing.md: 3b2ba9062a72dfe03c9e9a84fa13fe23da39302a
|
||||
2026-07-26-execa-for-test-subprocess-plumbing.zh.md: 61e12233fc49ca7788882ca409d6f67f030d2475
|
||||
@@ -0,0 +1,41 @@
|
||||
# Agent Note: Adopt execa for hand-rolled test subprocess plumbing
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-26-execa-for-test-subprocess-plumbing.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Roughly ten e2e/smoke files re-derive the same spawn-collect-timeout choreography by hand: `let stdout = ''` accumulation with `setEncoding` and `data` handlers, a `setTimeout` → `kill('SIGKILL')` deadline, and `once('exit')`/`once('error')` settlement, each with small variations. The sites: the inner spawn block of `runLoaderSmoke` (`packages/support/loader-smoke/src/index.ts`), `runBuiltBin` in `apps/cli/tests/built-bin.e2e.ts` and `packages/examples/cli-demo/tests/built-bin.e2e.ts`, `runBinExpectingExit` in `packages/examples/acp-demo/tests/built-bin.e2e.ts`, the built-lib e2e helpers in `lsp-local` and `code-runtime-worker`, the outer collector of `examples/tui-agent/tests/pty-harness.ts`, `examples/jsonrpc-agent/tests/keyless-smoke.e2e.ts`, and partially `apps/web/tests/smoke-real.e2e.ts` and `session-checkpoint-policy/tests/crash-recovery.e2e.ts`. Net deletable: ~100–150 lines of test infrastructure.
|
||||
|
||||
Two related test-infra hand-rolls compound the case:
|
||||
|
||||
- `packages/support/llm-mock-server/src/cli.ts` hand-tokenizes 18 `--flag value` options (~45–60 lines of loop and value-extraction helpers) where the `node:util` `parseArgs` builtin is already the repo idiom (`cli-demo`, `acp-demo`, `verify-runtime-closure.ts`, `packages/sdk/scripts`).
|
||||
- `apps/web/tests/smoke-real.e2e.ts` and `apps/web/tests/scaffold.ts` carry two verbatim copies of a regex `.env` parser (~20 lines) where the `process.loadEnvFile` builtin has exactly the required no-override semantics — and the vitest e2e/snapshot/web configs already load root `.env` with it before these files run, making the copies arguably dead.
|
||||
- The snapshot harness hand-rolls three poll-until-deadline loops (`waitForPersistedTurnStart`/`waitForPersistedTurnEnd`/`waitForWorkspaceFile` in `packages/support/acp-snapshot/src/harness.ts`, ~55 lines) plus `waitForFile` in `crash-recovery.e2e.ts`, where `vi.waitFor`/`expect.poll` cover the shape — vitest is already a runtime dependency of `dsh-acp-snapshot`, so this adds nothing.
|
||||
|
||||
## Proposal
|
||||
|
||||
- Add `execa` as a root devDependency and rewrite the spawn-collect-timeout sites onto `await execa(cmd, args, { cwd, env, timeout, killSignal: 'SIGKILL', reject: false })`, whose result reports `{ stdout, stderr, exitCode, signal, timedOut }` as independent fields — matching the repo's own defensive-patterns rule to report orthogonal subprocess outcomes independently. Keep the genuinely custom parts custom: cli-demo's interrupt-on-marker mid-stream logic, jsonrpc's line-predicate protocol driving, and crash-recovery's SIGKILL-at-failpoint choreography.
|
||||
- Swap `llm-mock-server`'s CLI tokenizer for `parseArgs` (numeric coercion, bounds, and cross-option constraints stay manual; pinned error-message texts update with the tests).
|
||||
- Delete both `loadRootEnv` copies in favor of `process.loadEnvFile` in a try/catch, or remove them outright if the vitest-config loading already covers them.
|
||||
- Replace the four poll loops with `vi.waitFor`/`expect.poll`, passing explicit `{ interval, timeout }` and throwing descriptive errors from the callback.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **`tinyexec` instead of execa.** Already in `node_modules` transitively via vitest, smaller API — but no kill-escalation, no rich error output embedding, and being transitive is not a contract; if the lighter package is preferred the swap shape is identical.
|
||||
- **A repo-local shared spawn helper (no new dep).** Viable and cheaper on supply chain, but it keeps the maintenance of deadline/kill/settlement logic in-repo when a battle-tested package owns exactly this; contrary to the [dependency policy](../../implemented/process/2026-07-26-dependencies-over-hand-rolling.md), it also has to re-earn Windows behavior (taskkill, exit codes) that execa already carries.
|
||||
- **`get-port`, `wait-on`, `tempy`, `tree-kill`.** Rejected individually: the repo's single port probe is break-even, the file waits are dominated by `vi.waitFor`, temp-dir handling already uses `mkdtemp` + `rm {recursive}` builtins everywhere, and acp-snapshot's `close()` is drain-ordering logic, not tree traversal.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- The listed sites spawn through execa (or the chosen equivalent); the hand-rolled collect/timeout blocks and the two `/* v8 ignore */` un-inducible OS-error branches in `loader-smoke` are gone.
|
||||
- `llm-mock-server` CLI parses via `parseArgs`; its cli spec passes with updated message expectations.
|
||||
- No hand-rolled `.env` parser remains under `apps/web/tests`.
|
||||
- The affected e2e and snapshot suites pass on both POSIX and Windows CI lanes.
|
||||
|
||||
## Risks
|
||||
|
||||
- `loader-smoke` is a `src/` file under the per-file-100% coverage gate; the swap actually simplifies its coverage story (removes un-inducible branches) but the new call shape needs coverage.
|
||||
- Each rewritten e2e must be re-run on both platforms; subtle differences in kill escalation or stdin-close semantics (`input: ''` for loader-smoke's stdin-close contract) are the risk to verify per site.
|
||||
- execa is a new root devDependency (currently absent from the lockfile entirely); it is one of the most-depended-on packages on npm and actively maintained, so health is not a concern, but the exe/runtime closure is unaffected either way (tests only).
|
||||
@@ -0,0 +1,41 @@
|
||||
# Agent Note: 采用 execa 替换手写的测试子进程管道代码
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-26-execa-for-test-subprocess-plumbing.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
大约十个 e2e/冒烟测试文件各自手工重写同一套「spawn、收集输出、超时终止」编排:用 `setEncoding` 加 `data` 处理器做 `let stdout = ''` 式累积,用 `setTimeout` → `kill('SIGKILL')` 设定超时截止,再以 `once('exit')`/`once('error')` 结算结果,各处只有细微差别。这些位置是:`runLoaderSmoke` 的内层 spawn 代码块(`packages/support/loader-smoke/src/index.ts`)、`apps/cli/tests/built-bin.e2e.ts` 与 `packages/examples/cli-demo/tests/built-bin.e2e.ts` 中的 `runBuiltBin`、`packages/examples/acp-demo/tests/built-bin.e2e.ts` 中的 `runBinExpectingExit`、`lsp-local` 与 `code-runtime-worker` 中基于构建产物的 e2e 辅助函数、`examples/tui-agent/tests/pty-harness.ts` 的外层收集器、`examples/jsonrpc-agent/tests/keyless-smoke.e2e.ts`,以及部分涉及的 `apps/web/tests/smoke-real.e2e.ts` 和 `session-checkpoint-policy/tests/crash-recovery.e2e.ts`。净可删除量:约 100–150 行测试基础设施代码。
|
||||
|
||||
另有两处相关的测试基础设施手写代码进一步强化了替换的理由:
|
||||
|
||||
- `packages/support/llm-mock-server/src/cli.ts` 手工逐个切分 18 个 `--flag value` 选项(约 45–60 行的循环与取值辅助函数),而 `node:util` 内置的 `parseArgs` 早已是本仓库的惯用写法(`cli-demo`、`acp-demo`、`verify-runtime-closure.ts`、`packages/sdk/scripts`)。
|
||||
- `apps/web/tests/smoke-real.e2e.ts` 与 `apps/web/tests/scaffold.ts` 携带两份逐字相同的正则 `.env` 解析器拷贝(约 20 行),而内置的 `process.loadEnvFile` 恰好具备所需的「不覆盖已有值」语义;并且 vitest 的 e2e/snapshot/web 配置在这些文件运行之前就已用它加载了根 `.env`,这两份拷贝几乎可以视为死代码。
|
||||
- 快照 harness 手写了三个「轮询直到截止时间」的循环(`packages/support/acp-snapshot/src/harness.ts` 中的 `waitForPersistedTurnStart`/`waitForPersistedTurnEnd`/`waitForWorkspaceFile`,约 55 行),外加 `crash-recovery.e2e.ts` 中的 `waitForFile`,而 `vi.waitFor`/`expect.poll` 正好覆盖这种形态;vitest 本来就是 `dsh-acp-snapshot` 的运行时依赖,因此这不新增任何东西。
|
||||
|
||||
## 提案
|
||||
|
||||
- 将 `execa` 添加为根 devDependency,把上述 spawn、收集、超时的代码位置改写到 `await execa(cmd, args, { cwd, env, timeout, killSignal: 'SIGKILL', reject: false })` 上:其结果以相互独立的字段报告 `{ stdout, stderr, exitCode, signal, timedOut }`,与本仓库防御模式中「正交的子进程结果各自独立上报」的规则一致。真正定制的部分继续保持定制:cli-demo 在流中遇到标记即中断的逻辑、jsonrpc 基于行谓词的协议驱动,以及 crash-recovery 在故障点发送 SIGKILL 的编排。
|
||||
- 把 `llm-mock-server` 的 CLI 切分器换成 `parseArgs`(数值转换、边界检查与跨选项约束仍手工实现;被固定的错误消息文本随测试一并更新)。
|
||||
- 删除两份 `loadRootEnv` 拷贝,改用包在 try/catch 中的 `process.loadEnvFile`;如果 vitest 配置的加载已经覆盖了它们,则直接整体移除。
|
||||
- 用 `vi.waitFor`/`expect.poll` 替换那四个轮询循环,显式传入 `{ interval, timeout }`,并在回调中抛出带描述信息的错误。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **用 `tinyexec` 代替 execa。**它已经作为 vitest 的传递依赖存在于 `node_modules` 中,API 也更小;但它没有终止信号逐级升级,不会把丰富的输出嵌入错误对象,而且传递依赖并不构成契约。如果最终更倾向这个更轻的包,替换的形态完全相同。
|
||||
- **仓库内共享的 spawn 辅助函数(不引入新依赖)。**可行,供应链成本也更低,但当一个久经实战的包恰好负责这件事时,它把截止时限、终止与结算逻辑的维护留在了仓库内;这与[依赖策略](../../implemented/process/2026-07-26-dependencies-over-hand-rolling.md)背道而驰,它还得重新踩坑换来 execa 已经自带的 Windows 行为(taskkill、退出码)。
|
||||
- **`get-port`、`wait-on`、`tempy`、`tree-kill`。**逐一不予采纳:仓库仅有的一处端口探测替换后收支相抵;文件等待场景已由 `vi.waitFor` 更优地覆盖;临时目录处理在各处已经使用内置的 `mkdtemp` + `rm {recursive}`;acp-snapshot 的 `close()` 是排空顺序逻辑,不是进程树遍历。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- 所列位置全部通过 execa(或最终选定的等价包)spawn 子进程;手写的收集/超时代码块,连同 `loader-smoke` 中两个标注 `/* v8 ignore */`、无法人为诱发的 OS 错误分支,全部移除。
|
||||
- `llm-mock-server` 的 CLI 经由 `parseArgs` 解析;其 cli 测试文件在更新消息期望后通过。
|
||||
- `apps/web/tests` 下不再存在手写的 `.env` 解析器。
|
||||
- 受影响的 e2e 与快照测试套件在 POSIX 与 Windows 两条 CI 车道上均通过。
|
||||
|
||||
## 风险
|
||||
|
||||
- `loader-smoke` 是逐文件 100% 覆盖率门禁下的 `src/` 文件;这次替换实际上简化了它的覆盖率问题(移除了无法人为诱发的分支),但新的调用形态需要补齐覆盖。
|
||||
- 每个改写后的 e2e 都必须在两个平台上重新运行;终止信号升级或 stdin 关闭语义上的细微差异(loader-smoke 的 stdin 关闭契约对应 `input: ''`)是需要逐处核验的风险。
|
||||
- execa 是新增的根 devDependency(当前完全不存在于 lockfile 中);它是 npm 上被依赖最多的包之一且维护活跃,健康度不是顾虑;至于 exe/运行时闭包,无论选哪个包都不受影响(仅测试使用)。
|
||||
Reference in New Issue
Block a user