Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows
Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
This commit is contained in:
Tianyi Cui
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"AGENTS.md": 1590,
|
||||
"docs/AGENTS.md": 1315,
|
||||
"docs/architecture.md": 1890,
|
||||
"docs/architecture.md": 1630,
|
||||
"docs/cordis-primer.md": 550,
|
||||
"docs/defensive-patterns.md": 550,
|
||||
"docs/testing.md": 800,
|
||||
"examples/AGENTS.md": 610,
|
||||
|
||||
@@ -556,7 +556,7 @@ function renderEvents(events: EventEntry[]): string {
|
||||
'',
|
||||
'The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary this repo owns, grouped by scope. The **inherited tier** at the end is the cordis-core + loader/hmr/timer event surface a plugin also sees — pinned vendor source, summarized tersely.',
|
||||
'',
|
||||
'Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out; all listeners run), **serial** (awaited in registration order until one returns a bail value — anything other than `null`, `false`, or `undefined`).',
|
||||
'Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../cordis-primer.md#cordis-waterfall-semantics)), **parallel** (awaited fan-out; all listeners run), **serial** (awaited in registration order until one returns a bail value — anything other than `null`, `false`, or `undefined`).',
|
||||
'',
|
||||
]
|
||||
const scopes = [...new Set(events.map(e => e.scope))].sort()
|
||||
|
||||
785
scripts/gen-doc-graphs.ts
Normal file
785
scripts/gen-doc-graphs.ts
Normal file
@@ -0,0 +1,785 @@
|
||||
/**
|
||||
* Generate (and verify) the relationship-diagram docs.
|
||||
*
|
||||
* This is the relationship layer above the existing catalogs:
|
||||
* - module-graph.md answers "which packages depend on which packages?"
|
||||
* - cordis-catalog/ answers "which events and services exist?"
|
||||
* - tool-catalog/ answers "which tools does the model see?"
|
||||
* - generated relationship diagrams answer "how do those pieces fit together?"
|
||||
*
|
||||
* Generated pages discover the enumerable facts from source. Hybrid pages use
|
||||
* discovered inventory plus small manifests for policy that source cannot infer
|
||||
* (for example, whether a package is an implementation or consumer in a seam).
|
||||
* Curated pages are still emitted here so the graph docs are one regenerated unit,
|
||||
* but their diagrams intentionally explain flow and ownership rather than
|
||||
* pretending to enumerate every source edge.
|
||||
*
|
||||
* `tsx scripts/gen-doc-graphs.ts` -> write generated diagram docs
|
||||
* `tsx scripts/gen-doc-graphs.ts --check` -> exit 1 if any file is stale
|
||||
*/
|
||||
|
||||
import { existsSync, globSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { dirname, relative, resolve } from 'node:path'
|
||||
import ts from 'typescript'
|
||||
import { collectEvents, collectServices } from './gen-cordis-catalog.ts'
|
||||
|
||||
const root = resolve(import.meta.dirname, '..')
|
||||
const SCOPE = '@deepseek-ai/dsh-'
|
||||
|
||||
interface PkgJson {
|
||||
name: string
|
||||
peerDependencies?: Record<string, string>
|
||||
}
|
||||
|
||||
interface Pkg {
|
||||
short: string
|
||||
name: string
|
||||
group: string
|
||||
rel: string
|
||||
deps: string[]
|
||||
}
|
||||
|
||||
interface GraphDoc {
|
||||
rel: string
|
||||
content: string
|
||||
}
|
||||
|
||||
interface ServiceRole {
|
||||
key: string
|
||||
pkg: string
|
||||
title: string
|
||||
mode: 'core' | 'seam' | 'bundle'
|
||||
implementations?: string[]
|
||||
consumers?: string[]
|
||||
companions?: string[]
|
||||
note: string
|
||||
}
|
||||
|
||||
interface ExamplePlugin {
|
||||
id: string
|
||||
name: string
|
||||
}
|
||||
|
||||
interface EventRelation {
|
||||
dispatchers: Map<string, Set<string>>
|
||||
listeners: Set<string>
|
||||
}
|
||||
|
||||
const GROUP_ORDER = [
|
||||
'util',
|
||||
'llm',
|
||||
'core',
|
||||
'bash',
|
||||
'fs',
|
||||
'compact',
|
||||
'subagent',
|
||||
'web',
|
||||
'todo',
|
||||
'hooks',
|
||||
'session-persistence',
|
||||
'support',
|
||||
'ui',
|
||||
]
|
||||
|
||||
const SERVICE_ROLES: ServiceRole[] = [
|
||||
{
|
||||
key: 'llm',
|
||||
pkg: 'llm',
|
||||
title: 'LLM adapter registry',
|
||||
mode: 'seam',
|
||||
implementations: ['llm-deepseek', 'llm-pi-ai', 'llm-replay'],
|
||||
consumers: ['agent-loop', 'compact-basic'],
|
||||
note: 'Adapters register provider implementations; the loop and compaction call the provider-neutral stream service.',
|
||||
},
|
||||
{
|
||||
key: 'sessions',
|
||||
pkg: 'session',
|
||||
title: 'In-memory session store',
|
||||
mode: 'core',
|
||||
consumers: ['agent-loop', 'agent', 'session-persistence', 'subagent-inprocess', 'invariants'],
|
||||
note: 'Owns append-only Session instances and emits the durable session event feed.',
|
||||
},
|
||||
{
|
||||
key: 'sessionPersistence',
|
||||
pkg: 'session-persistence',
|
||||
title: 'Durable session persistence seam',
|
||||
mode: 'seam',
|
||||
implementations: ['session-persistence-jsonl', 'session-persistence-sqlite'],
|
||||
consumers: ['agent-loop', 'acp'],
|
||||
note: 'Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time.',
|
||||
},
|
||||
{
|
||||
key: 'systemPrompt',
|
||||
pkg: 'system-prompt',
|
||||
title: 'System prompt assembly registry',
|
||||
mode: 'core',
|
||||
consumers: ['agent-loop', 'tools', 'tool-fs', 'tool-web'],
|
||||
note: 'Collects prompt sections and model-facing tool schemas for each step.',
|
||||
},
|
||||
{
|
||||
key: 'tools',
|
||||
pkg: 'tools',
|
||||
title: 'Tool registry and execution waterfall',
|
||||
mode: 'core',
|
||||
consumers: ['agent-loop', 'tool-bash', 'tool-fs', 'tool-subagent', 'tool-todo', 'tool-web', 'acp'],
|
||||
note: 'Registers tool definitions, exposes schemas to the prompt, and routes calls through tools/pre-execute and tools/post-execute.',
|
||||
},
|
||||
{
|
||||
key: 'agents',
|
||||
pkg: 'agent',
|
||||
title: 'Agent registry',
|
||||
mode: 'core',
|
||||
consumers: ['agent-loop', 'acp', 'subagent-inprocess', 'stdio-agent', 'invariants'],
|
||||
note: 'Owns live Agent handles and the create/resume factory seam.',
|
||||
},
|
||||
{
|
||||
key: 'agentLoop',
|
||||
pkg: 'agent-loop',
|
||||
title: 'Concrete loop driver',
|
||||
mode: 'bundle',
|
||||
consumers: ['agent-core'],
|
||||
note: 'The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package.',
|
||||
},
|
||||
{
|
||||
key: 'bash',
|
||||
pkg: 'bash',
|
||||
title: 'Bash executor seam',
|
||||
mode: 'seam',
|
||||
implementations: ['bash-local'],
|
||||
consumers: ['tool-bash', 'hooks-claude', 'hooks-codex'],
|
||||
note: 'The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors can replace bash-local.',
|
||||
},
|
||||
{
|
||||
key: 'fs',
|
||||
pkg: 'fs',
|
||||
title: 'Filesystem provider seam',
|
||||
mode: 'seam',
|
||||
implementations: ['fs-local'],
|
||||
consumers: ['tool-fs'],
|
||||
companions: ['fs-policy'],
|
||||
note: 'tool-fs executes read/write/edit through ctx.fs; fs-policy contributes observed-state checks through the fs/* event gate.',
|
||||
},
|
||||
{
|
||||
key: 'compact',
|
||||
pkg: 'compact',
|
||||
title: 'Compaction seam',
|
||||
mode: 'seam',
|
||||
implementations: ['compact-basic'],
|
||||
consumers: ['compact-basic'],
|
||||
note: 'The basic backend currently consumes the pre-step event directly; a model-facing compact tool remains deferred.',
|
||||
},
|
||||
{
|
||||
key: 'subagents',
|
||||
pkg: 'subagent',
|
||||
title: 'Subagent provider registry',
|
||||
mode: 'seam',
|
||||
implementations: ['subagent-spawn', 'subagent-fork', 'subagent-acp', 'subagent-mock'],
|
||||
consumers: ['tool-subagent'],
|
||||
note: 'Providers implement transports; tool-subagent exposes one configured provider as a model-facing tool name.',
|
||||
},
|
||||
{
|
||||
key: 'web',
|
||||
pkg: 'web',
|
||||
title: 'Web access provider registry',
|
||||
mode: 'seam',
|
||||
implementations: ['web-search-exa', 'web-search-perplexity', 'web-search-deepseek', 'web-fetch-local'],
|
||||
consumers: ['tool-web'],
|
||||
note: 'Search and fetch providers register into one ctx.web seam; tool-web owns the stable model-facing names.',
|
||||
},
|
||||
{
|
||||
key: 'workflows',
|
||||
pkg: 'workflow',
|
||||
title: 'Workflow script engine',
|
||||
mode: 'seam',
|
||||
implementations: ['workflow-vm'],
|
||||
consumers: ['tool-workflow'],
|
||||
note: 'One engine per context (bash shape, no named-provider registry); the vm engine fans agent() calls out through ctx.subagents.',
|
||||
},
|
||||
]
|
||||
|
||||
const DYNAMIC_EVENT_DISPATCHERS: Array<{ event: string; pkg: string; method: string }> = [
|
||||
// Subagent lifecycle events intentionally bypass ctx.emit and call
|
||||
// ctx.events.dispatch directly so one throwing listener cannot starve later
|
||||
// listeners or strand an already-started child run.
|
||||
{ event: 'subagent/start', pkg: 'subagent', method: 'events.dispatch' },
|
||||
{ event: 'subagent/end', pkg: 'subagent', method: 'events.dispatch' },
|
||||
]
|
||||
|
||||
function generatedHeader(title: string): string[] {
|
||||
return [
|
||||
'<!-- Generated by scripts/gen-doc-graphs.ts - do not edit by hand.',
|
||||
' Run `pnpm run gen-doc-graphs` to regenerate. -->',
|
||||
'',
|
||||
`# ${title}`,
|
||||
'',
|
||||
]
|
||||
}
|
||||
|
||||
function maintenanceFooter(source: string): string[] {
|
||||
return [`Maintenance mode: ${source}.`, '']
|
||||
}
|
||||
|
||||
function graphIndexLink(rel: string): string {
|
||||
return relative('docs', rel).replaceAll('\\', '/')
|
||||
}
|
||||
|
||||
function linkFromDoc(docRel: string, targetRel: string): string {
|
||||
return relative(dirname(docRel), targetRel).replaceAll('\\', '/')
|
||||
}
|
||||
|
||||
function collectPackages(): Pkg[] {
|
||||
const pkgs: Pkg[] = []
|
||||
for (const rel of globSync('packages/*/*/package.json', { cwd: root }).sort()) {
|
||||
const json = JSON.parse(readFileSync(resolve(root, rel), 'utf8')) as PkgJson
|
||||
if (!json.name.startsWith(SCOPE)) continue
|
||||
const [, group, leaf] = rel.split('/')
|
||||
if (group === undefined || leaf === undefined) throw new Error(`gen-doc-graphs: unexpected package path ${rel}`)
|
||||
const deps = Object.keys(json.peerDependencies ?? {})
|
||||
.filter(dep => dep.startsWith(SCOPE))
|
||||
.map(dep => dep.slice(SCOPE.length))
|
||||
.sort()
|
||||
pkgs.push({
|
||||
short: json.name.slice(SCOPE.length),
|
||||
name: json.name,
|
||||
group,
|
||||
rel: dirname(rel),
|
||||
deps,
|
||||
})
|
||||
}
|
||||
return topoSort(pkgs)
|
||||
}
|
||||
|
||||
function topoSort(pkgs: Pkg[]): Pkg[] {
|
||||
const remaining = new Map(pkgs.map(p => [p.short, p]))
|
||||
const placed = new Set<string>()
|
||||
const out: Pkg[] = []
|
||||
while (remaining.size > 0) {
|
||||
const ready = [...remaining.values()]
|
||||
.filter(pkg => pkg.deps.every(dep => placed.has(dep)))
|
||||
.sort(comparePackages)
|
||||
if (ready.length === 0) throw new Error(`gen-doc-graphs: dependency cycle among ${[...remaining.keys()].join(', ')}`)
|
||||
for (const pkg of ready) {
|
||||
out.push(pkg)
|
||||
placed.add(pkg.short)
|
||||
remaining.delete(pkg.short)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function comparePackages(a: Pkg, b: Pkg): number {
|
||||
const groupA = GROUP_ORDER.indexOf(a.group)
|
||||
const groupB = GROUP_ORDER.indexOf(b.group)
|
||||
const normA = groupA === -1 ? Number.MAX_SAFE_INTEGER : groupA
|
||||
const normB = groupB === -1 ? Number.MAX_SAFE_INTEGER : groupB
|
||||
return normA - normB || a.group.localeCompare(b.group) || a.short.localeCompare(b.short)
|
||||
}
|
||||
|
||||
function nodeId(prefix: string, value: string): string {
|
||||
return `${prefix}_${value.replace(/[^a-zA-Z0-9_]/g, '_')}`
|
||||
}
|
||||
|
||||
function escLabel(value: string): string {
|
||||
return value.replace(/"/g, '\\"')
|
||||
}
|
||||
|
||||
function mermaidCode(value: string): string {
|
||||
return `<code>${value.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>')}</code>`
|
||||
}
|
||||
|
||||
function repoLink(path: string, label: string, up = '..'): string {
|
||||
return `[${label}](${up}/${path})`
|
||||
}
|
||||
|
||||
function sourceLink(source: string, up = '..'): string {
|
||||
return repoLink(source.split(':')[0] ?? source, `\`${source}\``, up)
|
||||
}
|
||||
|
||||
function pkgLink(pkg: Pkg | undefined, fallback: string, up = '..'): string {
|
||||
return pkg ? repoLink(pkg.rel, `\`${pkg.short}\``, up) : `\`${fallback}\``
|
||||
}
|
||||
|
||||
function pkgList(names: string[] | undefined, pkgsByShort: Map<string, Pkg>): string {
|
||||
if (!names || names.length === 0) return '-'
|
||||
return names.map(name => pkgLink(pkgsByShort.get(name), name)).join(', ')
|
||||
}
|
||||
|
||||
function tableCell(value: string): string {
|
||||
return value.replace(/\|/g, '\\|').replace(/\n/g, '<br>')
|
||||
}
|
||||
|
||||
function assertServiceRolesComplete(): void {
|
||||
const discovered = new Set(collectServices().map(service => service.key))
|
||||
const classified = new Set(SERVICE_ROLES.map(role => role.key))
|
||||
const missing = [...discovered].filter(key => !classified.has(key)).sort()
|
||||
const stale = [...classified].filter(key => !discovered.has(key)).sort()
|
||||
if (missing.length || stale.length) {
|
||||
throw new Error([
|
||||
missing.length ? `missing service role classification: ${missing.join(', ')}` : '',
|
||||
stale.length ? `stale service role classification: ${stale.join(', ')}` : '',
|
||||
].filter(Boolean).join('; '))
|
||||
}
|
||||
}
|
||||
|
||||
function renderCapabilitySeams(pkgs: Pkg[]): string {
|
||||
assertServiceRolesComplete()
|
||||
const pkgsByShort = new Map(pkgs.map(pkg => [pkg.short, pkg]))
|
||||
const maintenance = 'hybrid: services are discovered from Cordis declarations; interface/implementation/consumer roles are classified in `scripts/gen-doc-graphs.ts` with a completeness guard'
|
||||
const nodes = new Map<string, string>()
|
||||
const edges = new Set<string>()
|
||||
const companionEdges = new Set<string>()
|
||||
const addNode = (id: string, label: string): void => {
|
||||
if (!nodes.has(id)) nodes.set(id, ` ${id}["${escLabel(label)}"]`)
|
||||
}
|
||||
const addEdge = (from: string, to: string): void => { edges.add(` ${from} --> ${to}`) }
|
||||
const lines = generatedHeader('Capability Seams And Core Services')
|
||||
lines.push(
|
||||
'A service can be a core spine service, a swappable capability seam, or a bundle/composition point. The graph shows the package that owns the service declaration, known implementation packages, and packages that consume the service directly.',
|
||||
'',
|
||||
'```mermaid',
|
||||
'flowchart LR',
|
||||
)
|
||||
for (const role of SERVICE_ROLES) {
|
||||
const svc = nodeId('svc', role.key)
|
||||
const owner = nodeId('pkg', role.pkg)
|
||||
addNode(owner, role.pkg)
|
||||
addNode(svc, `ctx.${role.key}<br/>${role.title}`)
|
||||
addEdge(owner, svc)
|
||||
for (const impl of role.implementations ?? []) {
|
||||
addNode(nodeId('pkg', impl), impl)
|
||||
addEdge(nodeId('pkg', impl), svc)
|
||||
}
|
||||
for (const consumer of role.consumers ?? []) {
|
||||
addNode(nodeId('pkg', consumer), consumer)
|
||||
addEdge(svc, nodeId('pkg', consumer))
|
||||
}
|
||||
for (const companion of role.companions ?? []) {
|
||||
addNode(nodeId('pkg', companion), companion)
|
||||
companionEdges.add(` ${svc} -. event gate .-> ${nodeId('pkg', companion)}`)
|
||||
}
|
||||
}
|
||||
lines.push(...nodes.values(), ...[...edges].sort(), ...[...companionEdges].sort())
|
||||
lines.push('```', '', '| ctx key | Role | Owner | Implementations | Direct consumers | Companion plugins | Note |', '| --- | --- | --- | --- | --- | --- | --- |')
|
||||
for (const role of SERVICE_ROLES) {
|
||||
lines.push(`| \`ctx.${role.key}\` | \`${role.mode}\` | ${pkgLink(pkgsByShort.get(role.pkg), role.pkg)} | ${pkgList(role.implementations, pkgsByShort)} | ${pkgList(role.consumers, pkgsByShort)} | ${pkgList(role.companions, pkgsByShort)} | ${tableCell(role.note)} |`)
|
||||
}
|
||||
lines.push('', ...maintenanceFooter(maintenance))
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
function parseExampleCordis(rel: string): ExamplePlugin[] {
|
||||
const text = readFileSync(resolve(root, rel), 'utf8')
|
||||
const plugins: ExamplePlugin[] = []
|
||||
let current: { id: string; name?: string } | null = null
|
||||
const flush = (): void => {
|
||||
if (current?.name) plugins.push({ id: current.id, name: current.name })
|
||||
}
|
||||
for (const line of text.split('\n')) {
|
||||
const id = /^-\s+id:\s+(.+?)\s*$/.exec(line)
|
||||
if (id?.[1] !== undefined) {
|
||||
flush()
|
||||
current = { id: stripYamlScalar(id[1]) }
|
||||
continue
|
||||
}
|
||||
const name = /^\s+name:\s+(.+?)\s*$/.exec(line)
|
||||
if (name?.[1] !== undefined && current) current.name = stripYamlScalar(name[1])
|
||||
}
|
||||
flush()
|
||||
return plugins
|
||||
}
|
||||
|
||||
function stripYamlScalar(value: string): string {
|
||||
return value.trim().replace(/^['"]|['"]$/g, '')
|
||||
}
|
||||
|
||||
const APP_EXAMPLES = [
|
||||
{
|
||||
id: 'echo',
|
||||
rel: 'examples/echo-agent/composition.md',
|
||||
title: 'Echo Agent App Composition',
|
||||
label: 'examples/echo-agent',
|
||||
config: 'examples/echo-agent/cordis.yml',
|
||||
summary: 'The echo demo swaps in a local mock LLM and teaching echo tool, then loads the stdio app package for the shared spine and terminal front door.',
|
||||
},
|
||||
{
|
||||
id: 'coding',
|
||||
rel: 'examples/coding-agent/composition.md',
|
||||
title: 'Coding Agent App Composition',
|
||||
label: 'examples/coding-agent',
|
||||
config: 'examples/coding-agent/cordis.yml',
|
||||
summary: 'The coding REPL demo adds the real DeepSeek adapter, filesystem tools, todo_write, compaction, and both subagent transports on top of the stdio app package.',
|
||||
},
|
||||
{
|
||||
id: 'acp',
|
||||
rel: 'examples/acp-agent/composition.md',
|
||||
title: 'ACP Agent App Composition',
|
||||
label: 'examples/acp-agent',
|
||||
config: 'examples/acp-agent/cordis.yml',
|
||||
summary: 'The ACP demo exposes the same agent spine over JSON-RPC stdio, with no stdout logger and no pre-created agent; clients create sessions through the ACP bridge.',
|
||||
},
|
||||
]
|
||||
|
||||
type AppExample = typeof APP_EXAMPLES[number]
|
||||
|
||||
function renderAppExpansion(lines: string[], appNode: string, pluginName: string): void {
|
||||
const agentCore = nodeId('bundle', 'agent_core')
|
||||
const jsonl = nodeId('bundle', 'jsonl')
|
||||
lines.push(` ${appNode} --> ${agentCore}["@deepseek-ai/dsh-agent-core"]`)
|
||||
lines.push(` ${appNode} --> ${jsonl}["@deepseek-ai/dsh-session-persistence-jsonl"]`)
|
||||
if (pluginName === '@deepseek-ai/dsh-stdio-agent') {
|
||||
lines.push(` ${appNode} --> ${nodeId('frontdoor', 'stdio')}["readline UI<br/>console logger<br/>pre-created main agent"]`)
|
||||
} else if (pluginName === '@deepseek-ai/dsh-acp-agent') {
|
||||
lines.push(` ${appNode} --> ${nodeId('frontdoor', 'acp')}["@deepseek-ai/dsh-acp<br/>JSON-RPC stdio bridge<br/>sessions created by client"]`)
|
||||
}
|
||||
lines.push(
|
||||
` ${agentCore} --> ${nodeId('spine', 'llm')}["ctx.llm"]`,
|
||||
` ${agentCore} --> ${nodeId('spine', 'sessions')}["ctx.sessions"]`,
|
||||
` ${agentCore} --> ${nodeId('spine', 'tools')}["ctx.tools + tool-bash"]`,
|
||||
` ${agentCore} --> ${nodeId('spine', 'loop')}["ctx.agents + ctx.agentLoop"]`,
|
||||
)
|
||||
}
|
||||
|
||||
function renderAppComposition(example: AppExample): string {
|
||||
const plugins = parseExampleCordis(example.config)
|
||||
const maintenance = 'hybrid: the leaf plugin list is parsed from its `cordis.yml`; app package expansion is curated from package source'
|
||||
const lines = generatedHeader(example.title)
|
||||
lines.push(
|
||||
example.summary,
|
||||
'',
|
||||
'```mermaid',
|
||||
'flowchart LR',
|
||||
` cfg["${escLabel(example.label)}<br/>cordis.yml"]`,
|
||||
)
|
||||
for (const plugin of plugins) {
|
||||
const pluginNode = nodeId(`plugin_${example.id}`, plugin.id)
|
||||
lines.push(` ${pluginNode}["${escLabel(plugin.id)}<br/>${escLabel(plugin.name)}"]`)
|
||||
lines.push(` cfg --> ${pluginNode}`)
|
||||
if (plugin.name === '@deepseek-ai/dsh-stdio-agent' || plugin.name === '@deepseek-ai/dsh-acp-agent') {
|
||||
renderAppExpansion(lines, pluginNode, plugin.name)
|
||||
}
|
||||
}
|
||||
lines.push(
|
||||
'```',
|
||||
'',
|
||||
'| Plugin id | Package / module |',
|
||||
'| --- | --- |',
|
||||
...plugins.map(plugin => `| \`${plugin.id}\` | \`${plugin.name}\` |`),
|
||||
'',
|
||||
`Source config: [\`${example.config}\`](${linkFromDoc(example.rel, example.config)}).`,
|
||||
)
|
||||
lines.push('', ...maintenanceFooter(maintenance))
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
function collectEventRelations(): Map<string, EventRelation> {
|
||||
const out = new Map<string, EventRelation>()
|
||||
const ensure = (event: string): EventRelation => {
|
||||
const existing = out.get(event)
|
||||
if (existing) return existing
|
||||
const next = { dispatchers: new Map<string, Set<string>>(), listeners: new Set<string>() }
|
||||
out.set(event, next)
|
||||
return next
|
||||
}
|
||||
for (const rel of globSync('packages/*/*/src/**/*.ts', { cwd: root }).sort()) {
|
||||
const [, , leaf] = rel.split('/')
|
||||
if (leaf === undefined) continue
|
||||
const text = readFileSync(resolve(root, rel), 'utf8')
|
||||
const sf = ts.createSourceFile(rel, text, ts.ScriptTarget.Latest, true)
|
||||
const visit = (node: ts.Node): void => {
|
||||
if (ts.isCallExpression(node) && ts.isPropertyAccessExpression(node.expression)) {
|
||||
const method = node.expression.name.text
|
||||
if (!isCordisContextReceiver(node.expression, sf)) {
|
||||
ts.forEachChild(node, visit)
|
||||
return
|
||||
}
|
||||
if (method === 'on') {
|
||||
const event = eventArg(node.arguments, method)
|
||||
if (event) ensure(event).listeners.add(leaf)
|
||||
} else if (method === 'emit' || method === 'parallel' || method === 'serial' || method === 'waterfall') {
|
||||
const event = eventArg(node.arguments, method)
|
||||
if (event) {
|
||||
const relation = ensure(event)
|
||||
const methods = relation.dispatchers.get(leaf) ?? new Set<string>()
|
||||
methods.add(method)
|
||||
relation.dispatchers.set(leaf, methods)
|
||||
}
|
||||
}
|
||||
}
|
||||
ts.forEachChild(node, visit)
|
||||
}
|
||||
visit(sf)
|
||||
}
|
||||
for (const entry of DYNAMIC_EVENT_DISPATCHERS) {
|
||||
const relation = ensure(entry.event)
|
||||
const methods = relation.dispatchers.get(entry.pkg) ?? new Set<string>()
|
||||
methods.add(entry.method)
|
||||
relation.dispatchers.set(entry.pkg, methods)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function isCordisContextReceiver(expr: ts.PropertyAccessExpression, sf: ts.SourceFile): boolean {
|
||||
const target = expr.expression.getText(sf)
|
||||
return target === 'ctx' || target === 'this.ctx'
|
||||
}
|
||||
|
||||
function eventArg(args: ts.NodeArray<ts.Expression>, method: string): string | undefined {
|
||||
if (method === 'waterfall') {
|
||||
const arg = args.find(ts.isStringLiteralLike)
|
||||
return arg?.text
|
||||
}
|
||||
const first = args[0]
|
||||
return first && ts.isStringLiteralLike(first) ? first.text : undefined
|
||||
}
|
||||
|
||||
function relationPackages(map: Map<string, Set<string>>, pkgsByShort: Map<string, Pkg>): string {
|
||||
if (map.size === 0) return '-'
|
||||
return [...map.entries()]
|
||||
.sort(([a], [b]) => a.localeCompare(b))
|
||||
.map(([pkg, methods]) => `${pkgLink(pkgsByShort.get(pkg), pkg)} (${[...methods].sort().map(m => `\`${m}\``).join(', ')})`)
|
||||
.join(', ')
|
||||
}
|
||||
|
||||
function listenerPackages(listeners: Set<string>, pkgsByShort: Map<string, Pkg>): string {
|
||||
if (listeners.size === 0) return '-'
|
||||
return [...listeners].sort().map(pkg => pkgLink(pkgsByShort.get(pkg), pkg)).join(', ')
|
||||
}
|
||||
|
||||
function renderEventRelations(pkgs: Pkg[]): string {
|
||||
const events = collectEvents()
|
||||
const relations = collectEventRelations()
|
||||
const pkgsByShort = new Map(pkgs.map(pkg => [pkg.short, pkg]))
|
||||
const maintenance = 'hybrid generated: Cordis event declarations and most producer/listener edges are AST-scanned; dynamic dispatch sites are classified in `scripts/gen-doc-graphs.ts`'
|
||||
const lines = generatedHeader('Event Producer And Consumer Matrix')
|
||||
lines.push(
|
||||
'This matrix shows which packages dispatch each harness-owned event and which packages listen to it. It is intentionally a table rather than one large graph: events are many-to-many, and dense relation data is easier to review in rows. Dynamic dispatch overrides cover sites that deliberately bypass `ctx.emit`, such as subagent lifecycle containment.',
|
||||
'',
|
||||
'| Event | Mode | Declared in | Dispatchers | Listeners |',
|
||||
'| --- | --- | --- | --- | --- |',
|
||||
)
|
||||
for (const event of [...events].sort((a, b) => a.name.localeCompare(b.name))) {
|
||||
const relation = relations.get(event.name) ?? { dispatchers: new Map<string, Set<string>>(), listeners: new Set<string>() }
|
||||
lines.push(`| \`${event.name}\` | \`${event.mode}\` | ${sourceLink(event.source)} | ${relationPackages(relation.dispatchers, pkgsByShort)} | ${listenerPackages(relation.listeners, pkgsByShort)} |`)
|
||||
}
|
||||
const declared = new Set(events.map(event => event.name))
|
||||
const extra = [...relations.keys()].filter(event => !declared.has(event)).sort()
|
||||
if (extra.length > 0) {
|
||||
lines.push('', '## Non-harness or undeclared event strings seen in package source', '', '| Event string | Dispatchers | Listeners |', '| --- | --- | --- |')
|
||||
for (const event of extra) {
|
||||
const relation = relations.get(event)
|
||||
if (!relation) continue
|
||||
lines.push(`| \`${event}\` | ${relationPackages(relation.dispatchers, pkgsByShort)} | ${listenerPackages(relation.listeners, pkgsByShort)} |`)
|
||||
}
|
||||
}
|
||||
lines.push('', ...maintenanceFooter(maintenance))
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
function renderLifecycle(): string {
|
||||
const maintenance = 'curated Mermaid sequence; exact event signatures live in the generated Cordis catalog'
|
||||
return [
|
||||
...generatedHeader('Agent Turn And Step Lifecycle'),
|
||||
'This sequence is the visual companion to [architecture.md](architecture.md#loop-lifecycle-session--turn--step). It keeps durable replay facts on `session/event` and live control/status on `agent/*`.',
|
||||
'',
|
||||
'```mermaid',
|
||||
'sequenceDiagram',
|
||||
' participant User',
|
||||
' participant Agent',
|
||||
' participant Driver',
|
||||
' participant Hooks as hook listeners',
|
||||
' participant Prompt as ctx.systemPrompt',
|
||||
' participant LLM as ctx.llm',
|
||||
' participant Tools as ctx.tools',
|
||||
' participant Session',
|
||||
' participant Persistence',
|
||||
' participant SDK as UI or SDK listener',
|
||||
' User->>Agent: send(content)',
|
||||
` Agent-->>SDK: ${mermaidCode('agent/queued')}`,
|
||||
' Agent->>Driver: queued work wakes driver',
|
||||
` Driver-->>SDK: ${mermaidCode('agent/status')} running`,
|
||||
` Driver->>Session: ${mermaidCode('turn/start')}`,
|
||||
` Driver->>Hooks: ${mermaidCode('agent/prompt-submit')} waterfall`,
|
||||
' Hooks-->>Driver: allow, block, or add context',
|
||||
` Driver->>Session: ${mermaidCode('user/message')} or rejected ${mermaidCode('turn/end')}`,
|
||||
` Driver->>Prompt: ${mermaidCode('system-prompt/assemble')} waterfall`,
|
||||
` Driver-->>Driver: ${mermaidCode('agent/pre-step')} serial checkpoint`,
|
||||
` Driver->>Session: ${mermaidCode('step/start')}`,
|
||||
` Driver->>LLM: ${mermaidCode('agent/request')} waterfall, then ${mermaidCode('llm/stream')} waterfall`,
|
||||
' LLM-->>Driver: StreamChunk*',
|
||||
` Driver->>Session: ${mermaidCode('assistant/chunk')}*`,
|
||||
` Session-->>SDK: ${mermaidCode('session/event')} ${mermaidCode('assistant/chunk')}*`,
|
||||
` Driver->>Hooks: ${mermaidCode('agent/step-result')} waterfall`,
|
||||
` Driver->>Session: ${mermaidCode('assistant/message')}`,
|
||||
` Driver->>Session: ${mermaidCode('tool/call')}`,
|
||||
' Driver->>Tools: execute through pre and post waterfalls',
|
||||
' Tools-->>Session: tool-owned events when applicable',
|
||||
` Driver->>Session: ${mermaidCode('tool/result')} and ${mermaidCode('step/end')}`,
|
||||
` Driver->>Hooks: ${mermaidCode('agent/turn-continuation')} waterfall`,
|
||||
` Driver->>Session: ${mermaidCode('turn/end')}`,
|
||||
` Driver->>Persistence: ${mermaidCode('session/flush')} parallel checkpoint`,
|
||||
` Driver-->>SDK: ${mermaidCode('agent/status')} idle`,
|
||||
'```',
|
||||
'',
|
||||
'SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.',
|
||||
'',
|
||||
...maintenanceFooter(maintenance),
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
function renderToolPipeline(): string {
|
||||
const maintenance = 'curated Mermaid flow; exact tool schemas and event signatures live in generated catalogs'
|
||||
return [
|
||||
...generatedHeader('Tool Execution Pipeline'),
|
||||
'This graph shows where policy, hooks, sandboxing, filesystem guards, result rewriting, and UI rendering fit without changing the loop. The key extension points are the `tools/pre-execute` and `tools/post-execute` waterfalls.',
|
||||
'',
|
||||
'```mermaid',
|
||||
'flowchart TD',
|
||||
' model["Assistant message contains tool-call block"]',
|
||||
` toolCall["Session event: ${mermaidCode('tool/call')}<br/>logged before execution"]`,
|
||||
' presentCall["UI pending card<br/>presentCall(args)"]',
|
||||
` pre["${mermaidCode('tools/pre-execute')} waterfall<br/>hooks, permission, sandbox"]`,
|
||||
' denied["deny or ask<br/>tool body skipped"]',
|
||||
' toolBody["Registered tool execute() body"]',
|
||||
` fsGate["${mermaidCode('fs/write-intent')} or ${mermaidCode('fs/edit-intent')}<br/>tool-fs mutations only"]`,
|
||||
` owned["Tool-owned session events<br/>${mermaidCode('todo/write')}, ${mermaidCode('fs/observed')}, ${mermaidCode('hook/invoked')}, ${mermaidCode('hook/result')}"]`,
|
||||
` post["${mermaidCode('tools/post-execute')} waterfall<br/>accept, block, replace, add context"]`,
|
||||
' context["Buffered additionalContext<br/>context/message after all tool results"]',
|
||||
` toolResult["Session event: ${mermaidCode('tool/result')}<br/>single model-facing outcome"]`,
|
||||
' presentResult["UI completed card<br/>presentResult(args, result)"]',
|
||||
' model --> toolCall',
|
||||
' toolCall --> presentCall',
|
||||
' toolCall --> pre',
|
||||
' pre -->|allow| toolBody',
|
||||
' pre -->|deny or ask| denied',
|
||||
' denied --> post',
|
||||
' toolBody --> fsGate',
|
||||
' fsGate --> toolBody',
|
||||
' toolBody --> owned',
|
||||
' toolBody --> post',
|
||||
' post --> context',
|
||||
' post --> toolResult',
|
||||
' toolResult --> presentResult',
|
||||
'```',
|
||||
'',
|
||||
'Filesystem read-before-edit checks live below `tool-fs` on the `fs/*` event gate, while hook bridges and future permission prompts live on the generic tool waterfalls. That split lets the same hooks observe bash, fs, web, todo, and subagent calls without coupling those tools to one policy service.',
|
||||
'',
|
||||
...maintenanceFooter(maintenance),
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
function renderSnapshotReplay(): string {
|
||||
const maintenance = 'curated Mermaid sequence based on the snapshot test harness'
|
||||
return [
|
||||
...generatedHeader('ACP Snapshot Replay'),
|
||||
'This graph explains what a snapshot scenario proves: recorded real-model session logs are replayed keylessly, ACP stdout is normalized and diffed, and scenario workspaces preserve tool side effects that the UI stream alone cannot prove.',
|
||||
'',
|
||||
'```mermaid',
|
||||
'sequenceDiagram',
|
||||
' participant Recorder as Real API recording',
|
||||
' participant Fixture as snapshot fixture',
|
||||
' participant Workspace',
|
||||
' participant Replay as llm-replay adapter',
|
||||
' participant ACP as acp-agent subprocess',
|
||||
' participant Golden as stdout golden',
|
||||
' Recorder->>Fixture: session.jsonl + workspace inputs',
|
||||
' Fixture->>Workspace: seed files and hook configs',
|
||||
' Fixture->>Replay: recorded StreamChunk script',
|
||||
` Replay->>ACP: deterministic ${mermaidCode('llm/stream')} chunks`,
|
||||
' ACP->>Workspace: bash, fs, and hook side effects',
|
||||
' ACP->>Golden: normalized sessionUpdate stream',
|
||||
' Golden-->>ACP: diff must be empty',
|
||||
'```',
|
||||
'',
|
||||
'The fs and hook snapshot matrix is valuable because it proves world state, hook decisions, and failed tool-card rendering, not just that replay returns text.',
|
||||
'',
|
||||
...maintenanceFooter(maintenance),
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
function renderDocs(): GraphDoc[] {
|
||||
const pkgs = collectPackages()
|
||||
const docs: GraphDoc[] = [
|
||||
{ rel: 'docs/capability-seams.md', content: renderCapabilitySeams(pkgs) },
|
||||
...APP_EXAMPLES.map(example => ({ rel: example.rel, content: renderAppComposition(example) })),
|
||||
{ rel: 'docs/event-producer-consumer.md', content: renderEventRelations(pkgs) },
|
||||
{ rel: 'docs/agent-lifecycle.md', content: renderLifecycle() },
|
||||
{ rel: 'docs/tool-execution-pipeline.md', content: renderToolPipeline() },
|
||||
{ rel: 'packages/ui/acp/snapshot-replay.md', content: renderSnapshotReplay() },
|
||||
]
|
||||
docs.unshift({ rel: 'docs/graph-atlas.md', content: renderIndex(docs) })
|
||||
return docs
|
||||
}
|
||||
|
||||
function renderIndex(docs: GraphDoc[]): string {
|
||||
const labels: Record<string, string> = {
|
||||
'docs/capability-seams.md': 'capability seams and core services',
|
||||
'examples/echo-agent/composition.md': 'echo-agent app composition',
|
||||
'examples/coding-agent/composition.md': 'coding-agent app composition',
|
||||
'examples/acp-agent/composition.md': 'acp-agent app composition',
|
||||
'docs/event-producer-consumer.md': 'event producer/consumer matrix',
|
||||
'docs/agent-lifecycle.md': 'agent turn and step lifecycle',
|
||||
'docs/tool-execution-pipeline.md': 'tool execution pipeline',
|
||||
'packages/ui/acp/snapshot-replay.md': 'ACP snapshot replay',
|
||||
}
|
||||
const modes: Record<string, string> = {
|
||||
'docs/capability-seams.md': 'hybrid generated',
|
||||
'examples/echo-agent/composition.md': 'hybrid generated',
|
||||
'examples/coding-agent/composition.md': 'hybrid generated',
|
||||
'examples/acp-agent/composition.md': 'hybrid generated',
|
||||
'docs/event-producer-consumer.md': 'hybrid generated',
|
||||
'docs/agent-lifecycle.md': 'curated',
|
||||
'docs/tool-execution-pipeline.md': 'curated',
|
||||
'packages/ui/acp/snapshot-replay.md': 'curated',
|
||||
}
|
||||
const rows = [
|
||||
'| [module dependency graph](module-graph.md) | `generated` |',
|
||||
'| [tool schema catalog and package map](tool-catalog/tools.md) | `generated` |',
|
||||
...docs.map((doc) => {
|
||||
const link = graphIndexLink(doc.rel)
|
||||
return `| [${labels[doc.rel] ?? link}](${link}) | \`${modes[doc.rel] ?? 'generated'}\` |`
|
||||
}),
|
||||
]
|
||||
const maintenance = 'mixed: each linked page declares generated, hybrid, or curated mode'
|
||||
return [
|
||||
...generatedHeader('Documentation Graph Index'),
|
||||
'These diagrams are the relationship layer above the generated catalogs. Use them to navigate package topology, capability seams, event flow, model-facing tools, app composition, and runtime lifecycle paths. Exact signatures and type shapes still live in the generated [events](cordis-catalog/events.md) / [services](cordis-catalog/services.md) catalogs, [tool-catalog/](tool-catalog/tools.md), and [core-data-structures/](core-data-structures/core.md).',
|
||||
'',
|
||||
'The process decision behind this index is recorded in [the documentation graph RFC](rfc/implemented/process/2026-07-03-documentation-graph-atlas.md).',
|
||||
'',
|
||||
'| Graph | Mode |',
|
||||
'| --- | --- |',
|
||||
...rows,
|
||||
'',
|
||||
'Regenerate with `pnpm run gen-doc-graphs`; verify freshness with `pnpm run verify-doc-graphs`.',
|
||||
'',
|
||||
...maintenanceFooter(maintenance),
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
function main(): void {
|
||||
const docs = renderDocs()
|
||||
if (process.argv.includes('--check')) {
|
||||
const stale: string[] = []
|
||||
for (const doc of docs) {
|
||||
const abs = resolve(root, doc.rel)
|
||||
const committed = existsSync(abs) ? readFileSync(abs, 'utf8') : null
|
||||
if (committed !== doc.content) stale.push(doc.rel)
|
||||
}
|
||||
if (stale.length === 0) {
|
||||
console.log(`gen-doc-graphs: ${docs.length} graph doc(s) are up to date.`)
|
||||
return
|
||||
}
|
||||
console.error(`gen-doc-graphs: stale graph doc(s): ${stale.join(', ')}. Run \`pnpm run gen-doc-graphs\` and commit the result.`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
for (const doc of docs) {
|
||||
mkdirSync(dirname(resolve(root, doc.rel)), { recursive: true })
|
||||
writeFileSync(resolve(root, doc.rel), doc.content)
|
||||
}
|
||||
console.log(`gen-doc-graphs: wrote ${docs.length} graph doc(s).`)
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) {
|
||||
main()
|
||||
}
|
||||
@@ -6,7 +6,8 @@
|
||||
* these as `workspace:^` plus test-only extras, which would add noise). This
|
||||
* script reads every `packages/* /* /package.json`, keeps only the
|
||||
* `@deepseek-ai/dsh-*` peer edges (dropping the `cordis` peer), and renders a
|
||||
* GitHub-viewable Mermaid graph plus a dependency table.
|
||||
* GitHub-viewable Mermaid graph grouped by `packages/<group>/` plus a
|
||||
* dependency table.
|
||||
*
|
||||
* The file is fully generated — never hand-edit it. Output is deterministic
|
||||
* (packages and edges sorted) so a regenerate-and-diff freshness check is
|
||||
@@ -17,8 +18,8 @@
|
||||
* is stale (CI / pre-push gate)
|
||||
*/
|
||||
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import { globSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
|
||||
const root = resolve(import.meta.dirname, '..')
|
||||
const OUT = 'docs/module-graph.md'
|
||||
@@ -27,10 +28,30 @@ const SCOPE = '@deepseek-ai/dsh-'
|
||||
interface Pkg {
|
||||
/** Short name, `@deepseek-ai/dsh-` prefix stripped (e.g. `agent-loop`). */
|
||||
short: string
|
||||
/** Package group from `packages/<group>/<pkg>`. */
|
||||
group: string
|
||||
/** Repo-relative package directory. */
|
||||
rel: string
|
||||
/** Short names of this package's in-repo peer dependencies, sorted. */
|
||||
deps: string[]
|
||||
}
|
||||
|
||||
const GROUP_ORDER = [
|
||||
'util',
|
||||
'llm',
|
||||
'core',
|
||||
'bash',
|
||||
'fs',
|
||||
'compact',
|
||||
'subagent',
|
||||
'web',
|
||||
'todo',
|
||||
'hooks',
|
||||
'session-persistence',
|
||||
'support',
|
||||
'ui',
|
||||
]
|
||||
|
||||
/** Read every workspace package and its `@deepseek-ai/dsh-*` peer edges. */
|
||||
function collect(): Pkg[] {
|
||||
const pkgs: Pkg[] = []
|
||||
@@ -44,7 +65,9 @@ function collect(): Pkg[] {
|
||||
.filter(d => d.startsWith(SCOPE))
|
||||
.map(d => d.slice(SCOPE.length))
|
||||
.sort()
|
||||
pkgs.push({ short: json.name.slice(SCOPE.length), deps })
|
||||
const [, group, leaf] = rel.split('/')
|
||||
if (group === undefined || leaf === undefined) throw new Error(`gen-module-graph: unexpected package path ${rel}`)
|
||||
pkgs.push({ short: json.name.slice(SCOPE.length), group, rel: dirname(rel), deps })
|
||||
}
|
||||
return topoSort(pkgs)
|
||||
}
|
||||
@@ -63,7 +86,7 @@ function topoSort(pkgs: Pkg[]): Pkg[] {
|
||||
while (remaining.size > 0) {
|
||||
const ready = [...remaining.values()]
|
||||
.filter(p => p.deps.every(d => placed.has(d)))
|
||||
.sort((a, b) => a.short.localeCompare(b.short))
|
||||
.sort(comparePackages)
|
||||
if (ready.length === 0) throw new Error(`gen-module-graph: dependency cycle among ${[...remaining.keys()].join(', ')}`)
|
||||
for (const p of ready) {
|
||||
out.push(p)
|
||||
@@ -74,28 +97,71 @@ function topoSort(pkgs: Pkg[]): Pkg[] {
|
||||
return out
|
||||
}
|
||||
|
||||
function comparePackages(a: Pkg, b: Pkg): number {
|
||||
const groupA = GROUP_ORDER.indexOf(a.group)
|
||||
const groupB = GROUP_ORDER.indexOf(b.group)
|
||||
const normA = groupA === -1 ? Number.MAX_SAFE_INTEGER : groupA
|
||||
const normB = groupB === -1 ? Number.MAX_SAFE_INTEGER : groupB
|
||||
return normA - normB || a.group.localeCompare(b.group) || a.short.localeCompare(b.short)
|
||||
}
|
||||
|
||||
function nodeId(prefix: string, value: string): string {
|
||||
return `${prefix}_${value.replace(/[^a-zA-Z0-9_]/g, '_')}`
|
||||
}
|
||||
|
||||
function escLabel(value: string): string {
|
||||
return value.replace(/"/g, '\\"')
|
||||
}
|
||||
|
||||
function packageLink(pkg: Pkg): string {
|
||||
return `[\`${pkg.short}\`](../${pkg.rel})`
|
||||
}
|
||||
|
||||
/** Render the full docs/module-graph.md content (pure, deterministic). */
|
||||
function render(pkgs: Pkg[]): string {
|
||||
const edges: string[] = []
|
||||
for (const p of pkgs) {
|
||||
for (const d of p.deps) edges.push(` ${p.short} --> ${d}`)
|
||||
for (const d of p.deps) edges.push(` ${nodeId('pkg', p.short)} --> ${nodeId('pkg', d)}`)
|
||||
}
|
||||
const rows = pkgs.map(p => `| \`${p.short}\` | ${p.deps.length ? p.deps.map(d => `\`${d}\``).join(', ') : '—'} |`)
|
||||
const byShort = new Map(pkgs.map(pkg => [pkg.short, pkg]))
|
||||
const groups = [...new Set(pkgs.map(pkg => pkg.group))].sort((a, b) => {
|
||||
const ia = GROUP_ORDER.indexOf(a)
|
||||
const ib = GROUP_ORDER.indexOf(b)
|
||||
const na = ia === -1 ? Number.MAX_SAFE_INTEGER : ia
|
||||
const nb = ib === -1 ? Number.MAX_SAFE_INTEGER : ib
|
||||
return na - nb || a.localeCompare(b)
|
||||
})
|
||||
const groupBlocks: string[] = []
|
||||
for (const group of groups) {
|
||||
groupBlocks.push(` subgraph ${nodeId('group', group)}["packages/${escLabel(group)}"]`)
|
||||
for (const pkg of pkgs.filter(p => p.group === group).sort((a, b) => a.short.localeCompare(b.short))) {
|
||||
groupBlocks.push(` ${nodeId('pkg', pkg.short)}["${escLabel(pkg.short)}"]`)
|
||||
}
|
||||
groupBlocks.push(' end')
|
||||
}
|
||||
const rows = pkgs.map((p) => {
|
||||
const deps = p.deps.length ? p.deps.map((d) => {
|
||||
const dep = byShort.get(d)
|
||||
return dep ? packageLink(dep) : `\`${d}\``
|
||||
}).join(', ') : '—'
|
||||
return `| ${packageLink(p)} | \`${p.group}\` | ${deps} |`
|
||||
})
|
||||
return [
|
||||
'<!-- Generated by scripts/gen-module-graph.ts — do not edit by hand.',
|
||||
' Run `pnpm run gen-module-graph` to regenerate. -->',
|
||||
'',
|
||||
'# Module dependency graph',
|
||||
'',
|
||||
'Inter-package dependencies among the `@deepseek-ai/dsh-*` harness packages, derived from each package\'s `peerDependencies` (the canonical runtime-dependency signal). An edge `a --> b` means package `a` depends on package `b`. Names have the `@deepseek-ai/dsh-` prefix stripped.',
|
||||
'Inter-package dependencies among the `@deepseek-ai/dsh-*` harness packages, derived from each package\'s `peerDependencies` (the canonical runtime-dependency signal) and grouped by the `packages/<group>/<pkg>` hierarchy. An edge `a --> b` means package `a` depends on package `b`. Names have the `@deepseek-ai/dsh-` prefix stripped.',
|
||||
'',
|
||||
'```mermaid',
|
||||
'graph TD',
|
||||
'flowchart TD',
|
||||
...groupBlocks,
|
||||
...edges,
|
||||
'```',
|
||||
'',
|
||||
'| Package | Depends on |',
|
||||
'| --- | --- |',
|
||||
'| Package | Group | Depends on |',
|
||||
'| --- | --- | --- |',
|
||||
...rows,
|
||||
'',
|
||||
].join('\n')
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
/**
|
||||
* Regenerate the RFC index tables in `docs/rfc/README.md` from the RFC tree
|
||||
* (see [rfc-index.ts](./rfc-index.ts) for the layout contract and rendering
|
||||
* rules). Rewrites ONLY the marker-delimited regions; the curated prose is
|
||||
* untouched. Freshness is asserted by `verify-rfc-classification.ts` (a
|
||||
* `doc-sync` member), so a stale committed index fails CI.
|
||||
* Regenerate `docs/rfc/INDEX.md` — the fully generated RFC index — from the
|
||||
* RFC tree (see [rfc-index.ts](./rfc-index.ts) for the layout contract and
|
||||
* rendering rules). The whole file is generated state; the curated prose lives
|
||||
* in `docs/rfc/README.md`. Freshness is asserted by
|
||||
* `verify-rfc-classification.ts` (a `doc-sync` member), so a stale committed
|
||||
* index fails CI.
|
||||
*
|
||||
* Run: `pnpm run gen-rfc-index`.
|
||||
*/
|
||||
|
||||
import { readFileSync, writeFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { rfcRoot, spliceReadme, walkRfcTree } from './rfc-index.ts'
|
||||
import { renderIndex, rfcRoot, walkRfcTree } from './rfc-index.ts'
|
||||
|
||||
const { rfcs, errors } = walkRfcTree()
|
||||
if (errors.length > 0) {
|
||||
@@ -19,12 +20,17 @@ if (errors.length > 0) {
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const readmePath = resolve(rfcRoot, 'README.md')
|
||||
const readme = readFileSync(readmePath, 'utf8')
|
||||
const next = spliceReadme(readme, rfcs)
|
||||
if (next === readme) {
|
||||
console.log(`gen-rfc-index: docs/rfc/README.md is up to date (${rfcs.length} RFCs).`)
|
||||
} else {
|
||||
writeFileSync(readmePath, next)
|
||||
console.log(`gen-rfc-index: docs/rfc/README.md regenerated (${rfcs.length} RFCs).`)
|
||||
const indexPath = resolve(rfcRoot, 'INDEX.md')
|
||||
const next = renderIndex(rfcs)
|
||||
let current: string | undefined
|
||||
try {
|
||||
current = readFileSync(indexPath, 'utf8')
|
||||
} catch {
|
||||
// Missing INDEX.md is the fresh-generation case, not an error: fall through and write it.
|
||||
}
|
||||
if (next === current) {
|
||||
console.log(`gen-rfc-index: docs/rfc/INDEX.md is up to date (${rfcs.length} RFCs).`)
|
||||
} else {
|
||||
writeFileSync(indexPath, next)
|
||||
console.log(`gen-rfc-index: docs/rfc/INDEX.md regenerated (${rfcs.length} RFCs).`)
|
||||
}
|
||||
|
||||
@@ -77,6 +77,12 @@ interface ToolPackage {
|
||||
dir: string
|
||||
/** Repo-relative source path linked from the catalog entry. */
|
||||
source: string
|
||||
/** Services or owning runtime surfaces the package requires at execution time. */
|
||||
requires: string[]
|
||||
/** Session events or other visible state the tools write or affect. */
|
||||
writes: string[]
|
||||
/** Additional model-visible names shipped by example/app config. */
|
||||
shippedNames?: string[]
|
||||
/** Plug the injected seams + the tool plugin onto a context that already
|
||||
* carries `systemPrompt` + `tools`. */
|
||||
mount: (ctx: Context) => Promise<void>
|
||||
@@ -100,15 +106,21 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
pkg: '@deepseek-ai/dsh-tool-bash',
|
||||
dir: 'tool-bash',
|
||||
source: 'packages/bash/tool-bash/src/index.ts',
|
||||
requires: ['ctx.tools', 'ctx.bash'],
|
||||
writes: ['tool/call', 'tool/result', 'context/message via agent.inject() for background completion notices'],
|
||||
async mount(ctx) {
|
||||
await ctx.plugin(LocalBashExecutor)
|
||||
await ctx.plugin(ToolBash)
|
||||
},
|
||||
note:
|
||||
'The bash/bash_output/bash_kill tools are model-facing consumers of the bash executor seam.',
|
||||
},
|
||||
{
|
||||
pkg: '@deepseek-ai/dsh-tool-fs',
|
||||
dir: 'tool-fs',
|
||||
source: 'packages/fs/tool-fs/src/index.ts',
|
||||
requires: ['ctx.tools', 'ctx.fs', 'ctx.systemPrompt'],
|
||||
writes: ['tool/call', 'fs/write-intent or fs/edit-intent for mutations', 'fs/observed after successful file operations', 'tool/result'],
|
||||
async mount(ctx) {
|
||||
// The tool injects `fs`; boot the local backend to satisfy it. The schemas
|
||||
// do not depend on the policy plugin (an event gate that changes behavior,
|
||||
@@ -123,6 +135,9 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
pkg: '@deepseek-ai/dsh-tool-subagent',
|
||||
dir: 'tool-subagent',
|
||||
source: 'packages/subagent/tool-subagent/src/index.ts',
|
||||
requires: ['ctx.tools', 'ctx.subagents'],
|
||||
writes: ['tool/call', 'tool/result', 'child session events through the chosen provider'],
|
||||
shippedNames: ['subagent', 'subagent_fork'],
|
||||
async mount(ctx) {
|
||||
await ctx.plugin(SubagentService)
|
||||
// Register a scripted provider under the name the tool delegates to.
|
||||
@@ -136,14 +151,20 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
pkg: '@deepseek-ai/dsh-tool-todo',
|
||||
dir: 'tool-todo',
|
||||
source: 'packages/todo/tool-todo/src/index.ts',
|
||||
requires: ['ctx.tools', 'owning Agent session'],
|
||||
writes: ['tool/call', 'todo/write', 'tool/result'],
|
||||
async mount(ctx) {
|
||||
await ctx.plugin(ToolTodo)
|
||||
},
|
||||
note:
|
||||
'todo_write is session-owned state; UIs render the latest todo/write event as a checklist or ACP plan.',
|
||||
},
|
||||
{
|
||||
pkg: '@deepseek-ai/dsh-tool-workflow',
|
||||
dir: 'tool-workflow',
|
||||
source: 'packages/workflow/tool-workflow/src/index.ts',
|
||||
requires: ['ctx.tools', 'ctx.workflows', 'ctx.systemPrompt', 'a calling Agent (exec.agent parents the script children)'],
|
||||
writes: ['tool/call', 'tool/result'],
|
||||
async mount(ctx) {
|
||||
// The tool injects `workflows`; boot the vm engine over a scripted
|
||||
// subagent provider to satisfy it. The schema does not depend on which
|
||||
@@ -158,6 +179,8 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
pkg: '@deepseek-ai/dsh-tool-web',
|
||||
dir: 'tool-web',
|
||||
source: 'packages/web/tool-web/src/index.ts',
|
||||
requires: ['ctx.tools', 'ctx.web', 'ctx.systemPrompt'],
|
||||
writes: ['tool/call', 'tool/result'],
|
||||
async mount(ctx) {
|
||||
// The tools inject `web`; boot the seam plus one search and one fetch
|
||||
// provider so both `web_search` and `web_fetch` register. The schemas do
|
||||
@@ -168,6 +191,8 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
await ctx.plugin(WebFetchLocal)
|
||||
await ctx.plugin(ToolWeb)
|
||||
},
|
||||
note:
|
||||
'web_search and web_fetch keep provider selection behind ctx.web so model-visible schemas stay stable across backend swaps.',
|
||||
},
|
||||
]
|
||||
|
||||
@@ -175,6 +200,9 @@ const TOOL_PACKAGES: ToolPackage[] = [
|
||||
interface CatalogPackage {
|
||||
pkg: string
|
||||
source: string
|
||||
requires: string[]
|
||||
writes: string[]
|
||||
shippedNames?: string[]
|
||||
schemas: ToolSchema[]
|
||||
/** A deployment note (see {@link ToolPackage.note}), rendered after the tools. */
|
||||
note?: string
|
||||
@@ -224,7 +252,15 @@ export async function collectToolCatalog(packages: ToolPackage[] = TOOL_PACKAGES
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await entry.mount(ctx)
|
||||
const schemas = ctx.tools.schemas().sort((a, b) => a.name.localeCompare(b.name))
|
||||
catalog.push({ pkg: entry.pkg, source: entry.source, schemas, ...entry.note !== undefined ? { note: entry.note } : {} })
|
||||
catalog.push({
|
||||
pkg: entry.pkg,
|
||||
source: entry.source,
|
||||
requires: entry.requires,
|
||||
writes: entry.writes,
|
||||
schemas,
|
||||
...entry.shippedNames !== undefined ? { shippedNames: entry.shippedNames } : {},
|
||||
...entry.note !== undefined ? { note: entry.note } : {},
|
||||
})
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
@@ -241,6 +277,14 @@ function renderTool(schema: ToolSchema, source: string): string[] {
|
||||
return out
|
||||
}
|
||||
|
||||
function codeList(values: string[] | undefined): string {
|
||||
return values?.length ? values.map(value => `\`${value}\``).join(', ') : '-'
|
||||
}
|
||||
|
||||
function tableCell(value: string | undefined): string {
|
||||
return value ? value.replace(/\|/g, '\\|').replace(/\n/g, '<br>') : '-'
|
||||
}
|
||||
|
||||
/** Render the full catalog (pure, deterministic given the manifest-ordered input). */
|
||||
export function render(catalog: ToolCatalog): string {
|
||||
const lines: string[] = [
|
||||
@@ -255,6 +299,14 @@ export function render(catalog: ToolCatalog): string {
|
||||
'',
|
||||
'Scope: shipped product tools under `packages/*/tool-*`, each booted with its DEFAULT config. The registered tool NAME can be a load-time config (e.g. `tool-subagent`\'s `toolName`), so a deployment may surface a package under a different or additional name — a per-package note records those shipped aliases where they exist. The `examples/` demo tools (e.g. `echo`) are excluded, matching the cordis catalog\'s packages-only scope.',
|
||||
'',
|
||||
'## Tool Package Map',
|
||||
'',
|
||||
'This table connects model-visible tool names to the plugin package and service seams behind them. Exact JSON Schemas follow in the package sections below.',
|
||||
'',
|
||||
'| Tool package | Model-visible names | Requires | Writes / affects | Shipped aliases | Deployment note |',
|
||||
'| --- | --- | --- | --- | --- | --- |',
|
||||
...catalog.map(entry => `| \`${entry.pkg}\` | ${codeList(entry.schemas.map(schema => schema.name))} | ${codeList(entry.requires)} | ${codeList(entry.writes)} | ${codeList(entry.shippedNames)} | ${tableCell(entry.note)} |`),
|
||||
'',
|
||||
]
|
||||
for (const entry of catalog) {
|
||||
lines.push(`## \`${entry.pkg}\``, '')
|
||||
|
||||
@@ -9,11 +9,11 @@
|
||||
* folder IS the label, and both sets are CLOSED — extending either means
|
||||
* amending this module AND the README's Classification prose.
|
||||
*
|
||||
* The README's per-lifecycle tables are GENERATED between marker comments
|
||||
* (`<!-- gen-rfc-index:begin {lifecycle} -->` … `end`): section headings and
|
||||
* rows are derived from each RFC's path (lifecycle/class), H1 (title, with an
|
||||
* optional `RFC: ` prefix stripped), and filename date, sorted by date then
|
||||
* filename. Prose outside the markers is curated by hand and never touched.
|
||||
* The index (`docs/rfc/INDEX.md`) is GENERATED in full: per-lifecycle sections
|
||||
* whose rows are derived from each RFC's path (lifecycle/class), H1 (title,
|
||||
* with an optional `RFC: ` prefix stripped), and filename date, sorted by date
|
||||
* then filename. The curated prose lives in README.md, which carries no index
|
||||
* rows at all.
|
||||
*/
|
||||
|
||||
import { readFileSync, readdirSync } from 'node:fs'
|
||||
@@ -101,14 +101,8 @@ export function walkRfcTree(): { rfcs: Rfc[]; errors: string[] } {
|
||||
return { rfcs, errors }
|
||||
}
|
||||
|
||||
/** The begin/end marker lines that delimit one lifecycle's generated region. */
|
||||
const markers = (lifecycle: string): { begin: string; end: string } => ({
|
||||
begin: `<!-- gen-rfc-index:begin ${lifecycle} -->`,
|
||||
end: `<!-- gen-rfc-index:end ${lifecycle} -->`,
|
||||
})
|
||||
|
||||
/**
|
||||
* Render one lifecycle's generated region body: a `### {Class}` heading plus a
|
||||
* Render one lifecycle's section body: a `### {Class}` heading plus a
|
||||
* `| Title | First proposed |` table for every non-empty class, in CLASSES
|
||||
* order, rows sorted by date then filename.
|
||||
*/
|
||||
@@ -126,48 +120,21 @@ function renderLifecycle(rfcs: Rfc[], lifecycle: string): string {
|
||||
}
|
||||
|
||||
/**
|
||||
* Splice freshly rendered regions into the README text. Throws when a marker
|
||||
* pair is missing, duplicated, or out of order, when a region does not sit
|
||||
* under its own `## {Lifecycle}` heading, or when an index-shaped table row
|
||||
* (a `| [title](lifecycle/…)` line) appears OUTSIDE the generated regions —
|
||||
* the markers are part of the curated prose, the heading above each region is
|
||||
* the one its lifecycle names, and index rows live only inside the regions
|
||||
* (prose links to RFCs remain fine anywhere).
|
||||
* Render the complete `docs/rfc/INDEX.md` content: a generated-file banner
|
||||
* followed by one `## {Lifecycle}` section per lifecycle in canonical order.
|
||||
* The whole file is generated state — there is no curated region to preserve.
|
||||
*/
|
||||
export function spliceReadme(readme: string, rfcs: Rfc[]): string {
|
||||
let out = readme
|
||||
const regions: Array<{ from: number; to: number }> = []
|
||||
export function renderIndex(rfcs: Rfc[]): string {
|
||||
const parts = [
|
||||
'# RFC index',
|
||||
'',
|
||||
'Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; `verify-rfc-classification` fails when this file is stale. The curated front door — layout, classification, when to write one, and the in-file format — is [README.md](README.md).',
|
||||
]
|
||||
for (const lifecycle of LIFECYCLES) {
|
||||
const { begin, end } = markers(lifecycle)
|
||||
const beginAt = out.indexOf(begin)
|
||||
const endAt = out.indexOf(end)
|
||||
if (beginAt === -1 || endAt === -1 || endAt < beginAt) {
|
||||
throw new Error(`README.md is missing the ${JSON.stringify(begin)} … ${JSON.stringify(end)} marker pair`)
|
||||
}
|
||||
if (out.indexOf(begin, beginAt + 1) !== -1 || out.indexOf(end, endAt + 1) !== -1) {
|
||||
throw new Error(`README.md has a duplicated ${lifecycle} index marker`)
|
||||
}
|
||||
// The region must sit directly under its own lifecycle heading: the last
|
||||
// H2 above the begin marker is `## {Heading(lifecycle)}`, or the heading
|
||||
// itself has drifted while the generated table stayed put.
|
||||
const before = out.slice(0, beginAt)
|
||||
const lastH2 = [...before.matchAll(/^##\s+(.+?)\s*$/gm)].at(-1)?.[1]
|
||||
if (lastH2 !== heading(lifecycle)) {
|
||||
throw new Error(`README.md: the ${lifecycle} index region is not under a "## ${heading(lifecycle)}" heading (found "## ${lastH2 ?? '<none>'}")`)
|
||||
}
|
||||
out = `${out.slice(0, beginAt + begin.length)}\n${renderLifecycle(rfcs, lifecycle)}\n${out.slice(endAt)}`
|
||||
regions.push({ from: out.indexOf(begin), to: out.indexOf(markers(lifecycle).end) + markers(lifecycle).end.length })
|
||||
parts.push('', `## ${heading(lifecycle)}`, '', renderLifecycle(rfcs, lifecycle))
|
||||
}
|
||||
// Index rows are generated state: a table row linking into a lifecycle
|
||||
// folder anywhere OUTSIDE the regions is a hand-added index entry the
|
||||
// generator would never reconcile.
|
||||
let offset = 0
|
||||
for (const line of out.split('\n')) {
|
||||
const inRegion = regions.some(r => offset >= r.from && offset < r.to)
|
||||
if (!inRegion && /^\|\s*\[[^\]]+\]\((?:proposed|implemented|rejected)\//.test(line)) {
|
||||
throw new Error(`README.md: index-shaped row outside the generated regions: ${JSON.stringify(line.slice(0, 80))}`)
|
||||
}
|
||||
offset += line.length + 1
|
||||
}
|
||||
return out
|
||||
return `${parts.join('\n')}\n`
|
||||
}
|
||||
|
||||
/** Matches an index-shaped table row (a `| [title](lifecycle/…) |` line) — generated state that must not appear in curated prose. */
|
||||
export const INDEX_ROW = /^\|\s*\[[^\]]+\]\((?:proposed|implemented|rejected)\//
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
* Doc-sync gate: verify that doc references written in TypeScript COMMENTS
|
||||
* resolve to a file that exists. Source comments cite docs by root-relative
|
||||
* prose path — `see docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md`,
|
||||
* `docs/architecture.md § plugin checklist`. `verify-md-links` parses Markdown
|
||||
* `docs/architecture.md § Where New Behavior Goes`. `verify-md-links` parses Markdown
|
||||
* link AST and never sees these, so a doc rename or move could silently orphan
|
||||
* a `.ts` comment that points at it. The RFC classification reorg
|
||||
* ([the classification RFC](../docs/rfc/implemented/process/2026-06-20-rfc-classification.md))
|
||||
@@ -12,7 +12,7 @@
|
||||
* Detection is a token scan, NOT an AST walk: doc refs live in free prose inside
|
||||
* comments, not in a structured form. We match `docs/<path>.md` tokens and
|
||||
* REQUIRE the `.md` extension, so extensionless prose (`docs/postmortem/0001`,
|
||||
* `docs/architecture.md § plugin checklist` — the section suffix is outside the
|
||||
* `docs/architecture.md § Where New Behavior Goes` — the section suffix is outside the
|
||||
* token) is left alone rather than misread as a path. Each token is resolved
|
||||
* ROOT-RELATIVE (the way the comments are written) and must exist on disk. This
|
||||
* is checker, not fixer: it reports and never rewrites.
|
||||
@@ -43,7 +43,7 @@ const isExcluded = (p: string): boolean =>
|
||||
* Match a `docs/….md` reference token. The `.md` extension is required so a
|
||||
* bare `docs/postmortem/0001` (no extension) does not register as a path. The
|
||||
* character class stops at whitespace, backticks, parens, and the section sign,
|
||||
* so trailing prose (`… .md § plugin checklist`) is not swallowed into the path.
|
||||
* so trailing prose (`… .md § Where New Behavior Goes`) is not swallowed into the path.
|
||||
*/
|
||||
const DOC_REF = /\bdocs\/[A-Za-z0-9._/-]+\.md/g
|
||||
|
||||
|
||||
108
scripts/verify-mermaid.ts
Normal file
108
scripts/verify-mermaid.ts
Normal file
@@ -0,0 +1,108 @@
|
||||
/**
|
||||
* Doc-sync gate: verify every fenced ```mermaid block parses with Mermaid's
|
||||
* own parser. Markdown link/type/code gates can say a diagram block exists and
|
||||
* is linked, but only Mermaid can catch syntax errors that GitHub would fail to
|
||||
* render.
|
||||
*
|
||||
* Scope matches the Markdown link gate so any Mermaid diagram in repo-authored
|
||||
* docs is checked: README.md, README.zh.md, docs/** /*.md,
|
||||
* packages/* /*.md, packages/* /* /*.md, examples/** /*.md, AGENTS.md,
|
||||
* packages/AGENTS.md, and .agents/skills/** /*.md.
|
||||
*
|
||||
* Run: `tsx scripts/verify-mermaid.ts`.
|
||||
*/
|
||||
|
||||
import { readFileSync, realpathSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { glob } from 'node:fs/promises'
|
||||
import { fromMarkdown } from 'mdast-util-from-markdown'
|
||||
import { gfmFromMarkdown } from 'mdast-util-gfm'
|
||||
import { gfm } from 'micromark-extension-gfm'
|
||||
import { JSDOM } from 'jsdom'
|
||||
import type { Nodes } from 'mdast'
|
||||
|
||||
const root = resolve(import.meta.dirname, '..')
|
||||
|
||||
const PATTERNS = [
|
||||
'README.md',
|
||||
'README.zh.md',
|
||||
'docs/**/*.md',
|
||||
'packages/*/*.md',
|
||||
'packages/*/*/*.md',
|
||||
'examples/**/*.md',
|
||||
'AGENTS.md',
|
||||
'packages/AGENTS.md',
|
||||
'.agents/skills/**/*.md',
|
||||
]
|
||||
|
||||
interface Block {
|
||||
file: string
|
||||
line: number
|
||||
source: string
|
||||
}
|
||||
|
||||
interface Violation {
|
||||
file: string
|
||||
line: number
|
||||
message: string
|
||||
}
|
||||
|
||||
function extractMermaidBlocks(file: string): Block[] {
|
||||
const source = readFileSync(resolve(root, file), 'utf8')
|
||||
const tree = fromMarkdown(source, { extensions: [gfm()], mdastExtensions: [gfmFromMarkdown()] })
|
||||
const out: Block[] = []
|
||||
const visit = (node: Nodes): void => {
|
||||
if (node.type === 'code' && node.lang === 'mermaid') {
|
||||
out.push({ file, line: node.position?.start.line ?? 0, source: node.value })
|
||||
}
|
||||
if ('children' in node) {
|
||||
for (const child of node.children) visit(child)
|
||||
}
|
||||
}
|
||||
visit(tree)
|
||||
return out
|
||||
}
|
||||
|
||||
function formatError(error: unknown): string {
|
||||
if (error instanceof Error) return error.message.replace(/\s+/g, ' ').trim()
|
||||
return String(error).replace(/\s+/g, ' ').trim()
|
||||
}
|
||||
|
||||
const blocks: Block[] = []
|
||||
const seen = new Set<string>()
|
||||
let checkedFiles = 0
|
||||
for (const pattern of PATTERNS) {
|
||||
for await (const match of glob(pattern, { cwd: root })) {
|
||||
const real = realpathSync(resolve(root, match))
|
||||
if (seen.has(real)) continue
|
||||
seen.add(real)
|
||||
checkedFiles++
|
||||
blocks.push(...extractMermaidBlocks(match))
|
||||
}
|
||||
}
|
||||
|
||||
const violations: Violation[] = []
|
||||
const { window } = new JSDOM('')
|
||||
Object.defineProperty(globalThis, 'window', { value: window })
|
||||
Object.defineProperty(globalThis, 'document', { value: window.document })
|
||||
Object.defineProperty(globalThis, 'navigator', { value: window.navigator })
|
||||
const mermaid = (await import('mermaid')).default
|
||||
mermaid.initialize({ startOnLoad: false })
|
||||
for (const block of blocks) {
|
||||
try {
|
||||
await mermaid.parse(block.source, { suppressErrors: false })
|
||||
} catch (error: unknown) {
|
||||
violations.push({ file: block.file, line: block.line, message: formatError(error) })
|
||||
}
|
||||
}
|
||||
|
||||
if (violations.length === 0) {
|
||||
console.log(`verify-mermaid: ${blocks.length} mermaid block(s) parsed across ${checkedFiles} file(s).`)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
console.error('verify-mermaid: Mermaid syntax errors found:')
|
||||
for (const violation of violations) {
|
||||
console.error(` ${violation.file}:${violation.line} ${violation.message}`)
|
||||
}
|
||||
process.exit(1)
|
||||
@@ -1,13 +1,13 @@
|
||||
/**
|
||||
* Doc-sync gate: enforce the RFC classification scheme
|
||||
* ([the classification RFC](../docs/rfc/implemented/process/2026-06-20-rfc-classification.md))
|
||||
* and the freshness of the generated index tables
|
||||
* and the freshness of the generated index
|
||||
* ([the index-generation RFC](../docs/rfc/implemented/process/2026-07-04-generate-rfc-index-tables.md)).
|
||||
* Every RFC is filed at `docs/rfc/{lifecycle}/{class}/yyyy-mm-dd-topic.md`; the
|
||||
* folder IS the label. This gate is the machine source of truth for the closed
|
||||
* class set and keeps the README index honest.
|
||||
* class set and keeps the generated index honest.
|
||||
*
|
||||
* Two checks (both against [rfc-index.ts](./rfc-index.ts), the shared walker
|
||||
* Three checks (all against [rfc-index.ts](./rfc-index.ts), the shared walker
|
||||
* and renderer):
|
||||
*
|
||||
* 1. STRUCTURE — every `.md` under a lifecycle folder lives in a class folder
|
||||
@@ -17,31 +17,38 @@
|
||||
* file at an unexpected depth fails. This is what makes the set CLOSED: a
|
||||
* new class folder can't appear without amending CLASSES (and the README's
|
||||
* Classification section, per the RFC).
|
||||
*
|
||||
* 2. FRESHNESS — the marker-delimited index regions in `docs/rfc/README.md`
|
||||
* byte-match a fresh render from the tree, so every RFC is listed exactly
|
||||
* once, under the heading matching its path, with its H1 title and filename
|
||||
* date. The fix for a stale index is `pnpm run gen-rfc-index`, never a hand
|
||||
* edit. This is checker, not fixer: it reports and never rewrites.
|
||||
* 2. FRESHNESS — the committed `docs/rfc/INDEX.md` byte-matches a fresh render
|
||||
* from the tree, so every RFC is listed exactly once, under the heading
|
||||
* matching its path, with its H1 title and filename date. The fix for a
|
||||
* stale index is `pnpm run gen-rfc-index`, never a hand edit. This is
|
||||
* checker, not fixer: it reports and never rewrites.
|
||||
* 3. NO STRAY ROWS — `docs/rfc/README.md` (the curated front door) carries no
|
||||
* index-shaped table rows; the list lives only in the generated INDEX.md.
|
||||
*
|
||||
* Run: `tsx scripts/verify-rfc-classification.ts`.
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { rfcRoot, spliceReadme, walkRfcTree } from './rfc-index.ts'
|
||||
import { INDEX_ROW, renderIndex, rfcRoot, walkRfcTree } from './rfc-index.ts'
|
||||
|
||||
const { rfcs, errors } = walkRfcTree()
|
||||
|
||||
const readmePath = resolve(rfcRoot, 'README.md')
|
||||
const readme = readFileSync(readmePath, 'utf8')
|
||||
if (errors.length === 0) {
|
||||
let index: string | undefined
|
||||
try {
|
||||
if (spliceReadme(readme, rfcs) !== readme) {
|
||||
errors.push('index: docs/rfc/README.md is stale — run `pnpm run gen-rfc-index` and commit the result')
|
||||
index = readFileSync(resolve(rfcRoot, 'INDEX.md'), 'utf8')
|
||||
} catch {
|
||||
// A missing INDEX.md is reported below as staleness, exactly like a drifted one.
|
||||
}
|
||||
if (renderIndex(rfcs) !== index) {
|
||||
errors.push('index: docs/rfc/INDEX.md is stale or missing — run `pnpm run gen-rfc-index` and commit the result')
|
||||
}
|
||||
const readme = readFileSync(resolve(rfcRoot, 'README.md'), 'utf8')
|
||||
for (const line of readme.split('\n')) {
|
||||
if (INDEX_ROW.test(line)) {
|
||||
errors.push(`readme: index-shaped row in the curated README (the list lives in INDEX.md): ${JSON.stringify(line.slice(0, 80))}`)
|
||||
}
|
||||
} catch (error) {
|
||||
errors.push(`index: ${error instanceof Error ? error.message : String(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
119
scripts/verify-rfc-format.ts
Normal file
119
scripts/verify-rfc-format.ts
Normal file
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* Doc-sync gate: enforce the RFC in-file format
|
||||
* ([README.md § The file format](../docs/rfc/README.md), the contract; rationale in
|
||||
* [the uniform-format RFC](../docs/rfc/implemented/process/2026-07-05-uniform-rfc-format.md)).
|
||||
* The classification gate owns WHERE a file sits and how it is named; this gate
|
||||
* owns what is INSIDE: the header block, the per-lifecycle body skeleton, and
|
||||
* the Alternatives-considered mandate.
|
||||
*
|
||||
* Per English RFC (`.zh.md` counterparts are the pairing gate's concern):
|
||||
*
|
||||
* 1. HEADER — line 1 is `# RFC: <title>`, line 2 blank, line 3 the one
|
||||
* `Status:` line in the file, line 4 blank. The status is the dateless enum
|
||||
* matching the lifecycle folder: `Status: proposed`, `Status: implemented`,
|
||||
* or `Status: rejected — <reason>`.
|
||||
* 2. SKELETON — the first `##` section is `## Problem`; the lifecycle's
|
||||
* required sections are present under their canonical names (`proposed/`:
|
||||
* Proposal, Acceptance criteria, Risks; `implemented/`: Decision,
|
||||
* Consequences; `rejected/`: Proposal); `implemented/` must not carry the
|
||||
* proposal-era headings (Proposal, Plan, Migration plan, Acceptance
|
||||
* criteria) that the docs standard's slop checklist outlaws there.
|
||||
* 3. ALTERNATIVES — `## Alternatives considered` is present, or the file is a
|
||||
* pre-format RFC (dated before the format landed) carrying the exact
|
||||
* grandfather comment instead. Carrying both, or grandfathering a
|
||||
* post-format RFC, fails.
|
||||
* 4. DEBT MARKER — the retired legacy-format debt comment may not reappear.
|
||||
*
|
||||
* Checker, not fixer: it reports and never rewrites.
|
||||
* Run: `tsx scripts/verify-rfc-format.ts`.
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { rfcRoot, walkRfcTree } from './rfc-index.ts'
|
||||
|
||||
/** The date the format contract landed; the grandfather comment is valid only before it. */
|
||||
const FORMAT_ADOPTED = '2026-07-05'
|
||||
|
||||
/** The exact comment a pre-format RFC carries in place of `## Alternatives considered`. */
|
||||
const GRANDFATHER = '<!-- rfc-format: alternatives-not-recorded (pre-format RFC) -->'
|
||||
|
||||
/** The retired debt marker that flagged pre-format bodies; banned so it cannot creep back. */
|
||||
const LEGACY_MARKER = 'XXX: legacy ADR/RFC body format'
|
||||
|
||||
/** Status-line grammar per lifecycle folder. */
|
||||
const STATUS: Record<string, RegExp> = {
|
||||
proposed: /^Status: proposed$/,
|
||||
implemented: /^Status: implemented$/,
|
||||
rejected: /^Status: rejected — .+$/,
|
||||
}
|
||||
|
||||
/** Required `##` headings per lifecycle, beyond the universal `## Problem` opener. */
|
||||
const REQUIRED: Record<string, string[]> = {
|
||||
proposed: ['## Proposal', '## Acceptance criteria', '## Risks'],
|
||||
implemented: ['## Decision', '## Consequences'],
|
||||
rejected: ['## Proposal'],
|
||||
}
|
||||
|
||||
/** Headings banned in `implemented/` — proposal-era spec-speak per the slop checklist. */
|
||||
const BANNED_IMPLEMENTED = /^## (?:Proposal\b|Plan\b|Migration plan\b|Acceptance criteria\b)/i
|
||||
|
||||
const { rfcs, errors } = walkRfcTree()
|
||||
|
||||
for (const rfc of rfcs) {
|
||||
const fail = (msg: string): void => {
|
||||
errors.push(`format: ${rfc.rel} — ${msg}`)
|
||||
}
|
||||
const lines = readFileSync(resolve(rfcRoot, rfc.rel), 'utf8').split('\n')
|
||||
// Content scans ignore fenced code blocks: an RFC may legitimately QUOTE a
|
||||
// status line, a banned heading, or the grandfather comment inside a fence
|
||||
// (the README's own format section does), and only real prose counts.
|
||||
let inFence = false
|
||||
const prose = lines.filter((l) => {
|
||||
if (l.startsWith('```')) {
|
||||
inFence = !inFence
|
||||
return false
|
||||
}
|
||||
return !inFence
|
||||
})
|
||||
|
||||
if (!/^# RFC: \S/.test(lines[0] ?? '')) fail('line 1 must be `# RFC: <title>`')
|
||||
if (lines[1] !== '') fail('line 2 must be blank')
|
||||
const status = STATUS[rfc.lifecycle]
|
||||
if (status !== undefined && !status.test(lines[2] ?? '')) {
|
||||
fail(`line 3 must match the ${rfc.lifecycle} status grammar (${String(status)})`)
|
||||
}
|
||||
if (lines[3] !== '') fail('line 4 must be blank')
|
||||
const statusLines = prose.filter(l => l.startsWith('Status:') && l !== lines[2])
|
||||
if (statusLines.length > 0 || prose.filter(l => l === lines[2]).length > 1) {
|
||||
fail('the line-3 `Status:` line must be the only one in the file')
|
||||
}
|
||||
|
||||
const h2s = prose.filter(l => l.startsWith('## ')).map(l => l.trimEnd())
|
||||
if (h2s[0] !== '## Problem') fail(`the first section must be \`## Problem\` (got ${JSON.stringify(h2s[0] ?? '<none>')})`)
|
||||
for (const required of REQUIRED[rfc.lifecycle] ?? []) {
|
||||
if (!h2s.includes(required)) fail(`missing the required \`${required}\` section`)
|
||||
}
|
||||
if (rfc.lifecycle === 'implemented') {
|
||||
for (const h2 of h2s.filter(h => BANNED_IMPLEMENTED.test(h))) {
|
||||
fail(`\`${h2}\` is a proposal-era heading; an implemented RFC states what is (fold it into Decision/Consequences/Testing)`)
|
||||
}
|
||||
}
|
||||
|
||||
const hasSection = h2s.includes('## Alternatives considered')
|
||||
const hasGrandfather = prose.includes(GRANDFATHER)
|
||||
if (hasSection && hasGrandfather) fail('carries both `## Alternatives considered` and the grandfather comment — drop the comment')
|
||||
if (!hasSection && !hasGrandfather) fail('missing `## Alternatives considered` (a pre-format RFC whose alternatives are not reconstructible carries the grandfather comment instead — see docs/rfc/README.md § The file format)')
|
||||
if (hasGrandfather && rfc.date >= FORMAT_ADOPTED) fail(`the grandfather comment is only valid for RFCs dated before ${FORMAT_ADOPTED}`)
|
||||
|
||||
if (prose.some(l => l.includes(LEGACY_MARKER))) fail('carries the retired legacy-format debt marker')
|
||||
}
|
||||
|
||||
if (errors.length === 0) {
|
||||
console.log(`verify-rfc-format: ${rfcs.length} RFC(s) checked, all conform to docs/rfc/README.md § The file format.`)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
console.error('verify-rfc-format: violations found:')
|
||||
for (const e of errors) console.error(` ${e}`)
|
||||
process.exit(1)
|
||||
Reference in New Issue
Block a user