From ab57d1ea817502982eb59fd2a8aa93deb99bd046 Mon Sep 17 00:00:00 2001 From: Coder Date: Thu, 27 Aug 2026 11:47:43 +0700 Subject: [PATCH] feat: sync 12 new skills into catalog (total 31 skills), update manifest and README --- README.md | 58 +- manifest.json | 128 ++- skills/api-designer/SKILL.md | 219 +++++ .../api-designer/references/error-handling.md | 541 +++++++++++ skills/api-designer/references/openapi.md | 824 ++++++++++++++++ skills/api-designer/references/pagination.md | 494 ++++++++++ .../api-designer/references/rest-patterns.md | 335 +++++++ skills/api-designer/references/versioning.md | 391 ++++++++ skills/brainstorming/SKILL.md | 250 +++++ skills/brainstorming/scripts/start-server.sh | 209 +++++ skills/brainstorming/scripts/stop-server.sh | 120 +++ .../spec-document-reviewer-prompt.md | 49 + skills/brainstorming/visual-companion.md | 299 ++++++ skills/codebase-design/DEEPENING.md | 37 + skills/codebase-design/DESIGN-IT-TWICE.md | 44 + skills/codebase-design/SKILL.md | 114 +++ skills/codebase-design/agents/openai.yaml | 3 + skills/drawio-skill/SKILL.md | 354 +++++++ .../drawio-skill/data/SHAPE-INDEX-NOTICE.md | 17 + skills/drawio-skill/data/lobe-icons.json | 878 ++++++++++++++++++ skills/drawio-skill/data/shape-index.json.gz | Bin 0 -> 436148 bytes skills/drawio-skill/references/autolayout.md | 177 ++++ skills/drawio-skill/references/derasterize.md | 54 ++ .../drawio-skill/references/diagram-types.md | 205 ++++ skills/drawio-skill/references/live-infra.md | 71 ++ .../references/mermaid-authoring.md | 58 ++ skills/drawio-skill/references/pr-bot.md | 73 ++ skills/drawio-skill/references/shapes.md | 151 +++ .../references/style-extraction.md | 255 +++++ .../drawio-skill/references/style-presets.md | 115 +++ skills/drawio-skill/references/toolbox.md | 107 +++ .../references/troubleshooting.md | 63 ++ skills/drawio-skill/references/tubemap.md | 67 ++ .../drawio-skill/references/xml-authoring.md | 208 +++++ skills/drawio-skill/scripts/aiicons.py | 205 ++++ skills/drawio-skill/scripts/autolayout.py | 410 ++++++++ skills/drawio-skill/scripts/buildup.py | 331 +++++++ skills/drawio-skill/scripts/c4.py | 163 ++++ skills/drawio-skill/scripts/ciimports.py | 182 ++++ skills/drawio-skill/scripts/composeimports.py | 118 +++ skills/drawio-skill/scripts/compress.py | 257 +++++ skills/drawio-skill/scripts/dockerimports.py | 148 +++ skills/drawio-skill/scripts/drawio2mermaid.py | 153 +++ skills/drawio-skill/scripts/drawio2pptx.py | 112 +++ skills/drawio-skill/scripts/drawiodiff.py | 164 ++++ skills/drawio-skill/scripts/drawiohtml.py | 230 +++++ skills/drawio-skill/scripts/edgeports.py | 211 +++++ .../drawio-skill/scripts/encode_drawio_url.py | 58 ++ skills/drawio-skill/scripts/explain.py | 156 ++++ skills/drawio-skill/scripts/goimports.py | 144 +++ skills/drawio-skill/scripts/heatmap.py | 226 +++++ skills/drawio-skill/scripts/jsimports.py | 164 ++++ skills/drawio-skill/scripts/k8simports.py | 242 +++++ skills/drawio-skill/scripts/openapiimports.py | 165 ++++ skills/drawio-skill/scripts/prdiff.py | 191 ++++ skills/drawio-skill/scripts/pyclasses.py | 158 ++++ skills/drawio-skill/scripts/pyimports.py | 155 ++++ skills/drawio-skill/scripts/raster2drawio.py | 196 ++++ skills/drawio-skill/scripts/relabel.py | 102 ++ skills/drawio-skill/scripts/repair_png.py | 37 + skills/drawio-skill/scripts/restyle.py | 161 ++++ skills/drawio-skill/scripts/runbook.py | 241 +++++ skills/drawio-skill/scripts/rustimports.py | 205 ++++ skills/drawio-skill/scripts/seqlayout.py | 222 +++++ skills/drawio-skill/scripts/shapesearch.py | 162 ++++ skills/drawio-skill/scripts/sqlerd.py | 167 ++++ skills/drawio-skill/scripts/svgflow.py | 85 ++ skills/drawio-skill/scripts/tfimports.py | 262 ++++++ skills/drawio-skill/scripts/tfstate.py | 132 +++ skills/drawio-skill/scripts/timelapse.py | 257 +++++ skills/drawio-skill/scripts/tubemap.py | 181 ++++ skills/drawio-skill/scripts/validate.py | 340 +++++++ .../styles/built-in/colorblind-safe.json | 49 + .../styles/built-in/corporate.json | 49 + skills/drawio-skill/styles/built-in/dark.json | 52 ++ .../drawio-skill/styles/built-in/default.json | 49 + .../styles/built-in/handdrawn.json | 49 + skills/drawio-skill/styles/schema.json | 264 ++++++ skills/frontend-design/LICENSE.txt | 177 ++++ skills/frontend-design/SKILL.md | 55 ++ skills/ln-24-architecture-auditor/SKILL.md | 127 +++ .../SKILL.md | 106 +++ .../SKILL.md | 103 ++ .../SKILL.md | 105 +++ .../SKILL.md | 103 ++ .../SKILL.md | 103 ++ .../SKILL.md | 105 +++ 87 files changed, 15489 insertions(+), 68 deletions(-) create mode 100644 skills/api-designer/SKILL.md create mode 100644 skills/api-designer/references/error-handling.md create mode 100644 skills/api-designer/references/openapi.md create mode 100644 skills/api-designer/references/pagination.md create mode 100644 skills/api-designer/references/rest-patterns.md create mode 100644 skills/api-designer/references/versioning.md create mode 100644 skills/brainstorming/SKILL.md create mode 100644 skills/brainstorming/scripts/start-server.sh create mode 100644 skills/brainstorming/scripts/stop-server.sh create mode 100644 skills/brainstorming/spec-document-reviewer-prompt.md create mode 100644 skills/brainstorming/visual-companion.md create mode 100644 skills/codebase-design/DEEPENING.md create mode 100644 skills/codebase-design/DESIGN-IT-TWICE.md create mode 100644 skills/codebase-design/SKILL.md create mode 100644 skills/codebase-design/agents/openai.yaml create mode 100644 skills/drawio-skill/SKILL.md create mode 100644 skills/drawio-skill/data/SHAPE-INDEX-NOTICE.md create mode 100644 skills/drawio-skill/data/lobe-icons.json create mode 100644 skills/drawio-skill/data/shape-index.json.gz create mode 100644 skills/drawio-skill/references/autolayout.md create mode 100644 skills/drawio-skill/references/derasterize.md create mode 100644 skills/drawio-skill/references/diagram-types.md create mode 100644 skills/drawio-skill/references/live-infra.md create mode 100644 skills/drawio-skill/references/mermaid-authoring.md create mode 100644 skills/drawio-skill/references/pr-bot.md create mode 100644 skills/drawio-skill/references/shapes.md create mode 100644 skills/drawio-skill/references/style-extraction.md create mode 100644 skills/drawio-skill/references/style-presets.md create mode 100644 skills/drawio-skill/references/toolbox.md create mode 100644 skills/drawio-skill/references/troubleshooting.md create mode 100644 skills/drawio-skill/references/tubemap.md create mode 100644 skills/drawio-skill/references/xml-authoring.md create mode 100644 skills/drawio-skill/scripts/aiicons.py create mode 100644 skills/drawio-skill/scripts/autolayout.py create mode 100644 skills/drawio-skill/scripts/buildup.py create mode 100644 skills/drawio-skill/scripts/c4.py create mode 100644 skills/drawio-skill/scripts/ciimports.py create mode 100644 skills/drawio-skill/scripts/composeimports.py create mode 100644 skills/drawio-skill/scripts/compress.py create mode 100644 skills/drawio-skill/scripts/dockerimports.py create mode 100644 skills/drawio-skill/scripts/drawio2mermaid.py create mode 100644 skills/drawio-skill/scripts/drawio2pptx.py create mode 100644 skills/drawio-skill/scripts/drawiodiff.py create mode 100644 skills/drawio-skill/scripts/drawiohtml.py create mode 100644 skills/drawio-skill/scripts/edgeports.py create mode 100644 skills/drawio-skill/scripts/encode_drawio_url.py create mode 100644 skills/drawio-skill/scripts/explain.py create mode 100644 skills/drawio-skill/scripts/goimports.py create mode 100644 skills/drawio-skill/scripts/heatmap.py create mode 100644 skills/drawio-skill/scripts/jsimports.py create mode 100644 skills/drawio-skill/scripts/k8simports.py create mode 100644 skills/drawio-skill/scripts/openapiimports.py create mode 100644 skills/drawio-skill/scripts/prdiff.py create mode 100644 skills/drawio-skill/scripts/pyclasses.py create mode 100644 skills/drawio-skill/scripts/pyimports.py create mode 100644 skills/drawio-skill/scripts/raster2drawio.py create mode 100644 skills/drawio-skill/scripts/relabel.py create mode 100644 skills/drawio-skill/scripts/repair_png.py create mode 100644 skills/drawio-skill/scripts/restyle.py create mode 100644 skills/drawio-skill/scripts/runbook.py create mode 100644 skills/drawio-skill/scripts/rustimports.py create mode 100644 skills/drawio-skill/scripts/seqlayout.py create mode 100644 skills/drawio-skill/scripts/shapesearch.py create mode 100644 skills/drawio-skill/scripts/sqlerd.py create mode 100644 skills/drawio-skill/scripts/svgflow.py create mode 100644 skills/drawio-skill/scripts/tfimports.py create mode 100644 skills/drawio-skill/scripts/tfstate.py create mode 100644 skills/drawio-skill/scripts/timelapse.py create mode 100644 skills/drawio-skill/scripts/tubemap.py create mode 100644 skills/drawio-skill/scripts/validate.py create mode 100644 skills/drawio-skill/styles/built-in/colorblind-safe.json create mode 100644 skills/drawio-skill/styles/built-in/corporate.json create mode 100644 skills/drawio-skill/styles/built-in/dark.json create mode 100644 skills/drawio-skill/styles/built-in/default.json create mode 100644 skills/drawio-skill/styles/built-in/handdrawn.json create mode 100644 skills/drawio-skill/styles/schema.json create mode 100644 skills/frontend-design/LICENSE.txt create mode 100644 skills/frontend-design/SKILL.md create mode 100644 skills/ln-24-architecture-auditor/SKILL.md create mode 100644 skills/ln-71-system-design-baseline-builder/SKILL.md create mode 100644 skills/ln-72-current-architecture-documenter/SKILL.md create mode 100644 skills/ln-73-system-design-proposal-builder/SKILL.md create mode 100644 skills/ln-74-architecture-decision-recorder/SKILL.md create mode 100644 skills/ln-75-architecture-diagram-builder/SKILL.md create mode 100644 skills/ln-76-architecture-migration-planner/SKILL.md diff --git a/README.md b/README.md index 7618b02..2286e4f 100644 --- a/README.md +++ b/README.md @@ -3,31 +3,43 @@ Общий каталог скиллов для DeepSeek Harness (`~/.dsh/skills`). Каждый скилл — папка `skills//SKILL.md` (+ вложенные ресурсы/скрипты), по схеме провайдера `dsh-skill-filesystem`. -## Состав +## Состав (31 скиллов) -См. [manifest.json](manifest.json) — программный каталог. Краткий список: +См. [manifest.json](manifest.json) — программный каталог метаданных. Список скиллов: | Скилл | Описание | |---|---| -| `code-review-and-quality` | Code review & software quality assurance | -| `conventional-commits-and-pr` | Conventional commits & PR documentation | -| `db-migration-safety` | Safe DB migrations & SQL optimization | -| `documentation-and-adrs` | ADRs & technical docs | -| `e2e-playwright-expert` | Playwright E2E testing | -| `github-actions-workflow-architect` | GitHub Actions CI/CD design | -| `grill-me` | Architecture/plan stress-testing | -| `lab-imagegen` | Home-lab ComfyUI image generation | -| `lab-pdf-ocr` | Home-lab Docling PDF OCR | -| `lab-speech-to-text` | Home-lab Whishper speech-to-text | -| `research` | Deep research outline workflow | -| `research-add-fields` | Add fields to research outline | -| `research-add-items` | Add research items | -| `research-deep` | Deep research via subagents | -| `research-report` | Summarize research to report | -| `security-audit` | Security audit & threat modeling | -| `systematic-debugging` | Scientific debugging method | -| `test-driven-development` | TDD red-green-refactor | -| `web-performance-audit` | Core Web Vitals optimization | +| `api-designer` | Use when designing REST or GraphQL APIs, creating OpenAPI specifications, or ... | +| `brainstorming` | "You MUST use this before any creative work - creating features, building com... | +| `code-review-and-quality` | Use when conducting comprehensive code reviews on pull requests, diffs, or ne... | +| `codebase-design` | Shared vocabulary for designing deep modules. Use when the user wants to desi... | +| `conventional-commits-and-pr` | Use when generating commit messages, structuring pull request descriptions, o... | +| `db-migration-safety` | Use when designing database schemas, writing schema migrations, optimizing sl... | +| `documentation-and-adrs` | Use when documenting architectural decisions, drafting ADRs (Architecture Dec... | +| `drawio-skill` | Use when the user requests diagrams, flowcharts, architecture diagrams, ER di... | +| `e2e-playwright-expert` | Use when writing, maintaining, or debugging end-to-end (E2E) browser tests wi... | +| `frontend-design` | Guidance for distinctive, intentional visual design when building new UI or r... | +| `github-actions-workflow-architect` | Use when designing, optimizing, or securing GitHub Actions CI/CD workflows, c... | +| `grill-me` | Use when the user wants to stress-test an architecture plan, interrogate a de... | +| `lab-imagegen` | Generate images using the user's home-lab ComfyUI server (SDXL-turbo on 2x AM... | +| `lab-pdf-ocr` | Extract text from PDF documents using the user's home-lab Docling OCR server ... | +| `lab-speech-to-text` | Transcribe speech from audio files (wav, mp3, ogg, m4a) using the user's home... | +| `ln-24-architecture-auditor` | "Audits implemented architecture fitness, boundaries, contracts, dependencies... | +| `ln-71-system-design-baseline-builder` | "Creates a project baseline of architecture drivers and constraints. Use befo... | +| `ln-72-current-architecture-documenter` | "Documents implemented current-state architecture from repository evidence. U... | +| `ln-73-system-design-proposal-builder` | "Creates a decision-complete target system design from requirements and const... | +| `ln-74-architecture-decision-recorder` | "Records one architecture decision with context, alternatives, tradeoffs, and... | +| `ln-75-architecture-diagram-builder` | "Creates evidence-backed current or target architecture diagrams when the dia... | +| `ln-76-architecture-migration-planner` | "Plans a reversible architecture migration with compatibility, data movement,... | +| `research` | Conduct preliminary research on a topic and generate research outline. For ac... | +| `research-add-fields` | Add field definitions to existing research outline. | +| `research-add-items` | Add items (research objects) to existing research outline. | +| `research-deep` | Read research outline, launch independent agent for each item for deep resear... | +| `research-report` | Summarize deep research results into markdown report, cover all fields, skip ... | +| `security-audit` | Use when performing security audits, vulnerability scanning, threat modeling,... | +| `systematic-debugging` | Use when debugging complex bugs, unexpected test failures, race conditions, o... | +| `test-driven-development` | Use when building new features, refactoring existing modules, or fixing bugs ... | +| `web-performance-audit` | Use when analyzing web application performance, Core Web Vitals (LCP, INP, CL... | ## Установка @@ -39,7 +51,7 @@ .\install.ps1 ``` -Клонирует/обновляет репозиторий в `$env:DSH_HOME\skills\dsh-skills` и (опционально) ставит symlink-переходник — либо копирует `.agents/skills`-подкаталог, что прописано в скрипте. +Синхронизирует директорию `skills/` напрямую в `$env:DSH_HOME\skills` (по умолчанию `~/.dsh/skills`). ### Bash/macOS/Linux @@ -51,4 +63,4 @@ ## Проверка -`validate.py` — проверяет frontmatter каждого `SKILL.md` (обязательные `name`, `description`; kebab-case) по правилам `dsh-skill-filesystem`. \ No newline at end of file +`validate.py` — проверяет frontmatter каждого `SKILL.md` (обязательные `name`, `description`; kebab-case) по правилам `dsh-skill-filesystem`. diff --git a/manifest.json b/manifest.json index 87ef182..f302162 100644 --- a/manifest.json +++ b/manifest.json @@ -1,124 +1,162 @@ -{ - "source": "git.byte-mate.ru/Coder/dsh-skills", - "install": "clone/symlink into ~/.dsh/skills", - "count": 19, +{ + "version": "1.0.0", + "updatedAt": "2026-08-27T04:46:59.502Z", + "count": 31, "skills": [ + { + "name": "api-designer", + "description": "Use when designing REST or GraphQL APIs, creating OpenAPI specifications, or planning API architecture. Invoke for resource modeling, versioning strategies, pagination patterns, error handling standards.", + "path": "skills/api-designer/SKILL.md" + }, + { + "name": "brainstorming", + "description": "\"You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation.\"", + "path": "skills/brainstorming/SKILL.md" + }, { "name": "code-review-and-quality", "description": "Use when conducting comprehensive code reviews on pull requests, diffs, or newly implemented features to ensure architectural cleanliness, maintainability, performance, and best engineering practices.", - "resources": [] + "path": "skills/code-review-and-quality/SKILL.md" + }, + { + "name": "codebase-design", + "description": "Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary.", + "path": "skills/codebase-design/SKILL.md" }, { "name": "conventional-commits-and-pr", "description": "Use when generating commit messages, structuring pull request descriptions, or maintaining changelogs following the Conventional Commits specification.", - "resources": [] + "path": "skills/conventional-commits-and-pr/SKILL.md" }, { "name": "db-migration-safety", "description": "Use when designing database schemas, writing schema migrations, optimizing slow SQL queries, or planning zero-downtime database changes.", - "resources": [] + "path": "skills/db-migration-safety/SKILL.md" }, { "name": "documentation-and-adrs", "description": "Use when documenting architectural decisions, drafting ADRs (Architecture Decision Records), or writing technical specifications and system documentation.", - "resources": [] + "path": "skills/documentation-and-adrs/SKILL.md" + }, + { + "name": "drawio-skill", + "description": "Use when the user requests diagrams, flowcharts, architecture diagrams, ER diagrams, UML / sequence / class diagrams, SysML / MBSE diagrams (block definition, internal block, requirement, parametric), BPMN business process diagrams, swimlane / cross-functional flowcharts, network topology, cloud architecture from Terraform or Kubernetes manifests, ML/DL model figures (Transformer/CNN/LSTM), mind maps, or any visualization. Also use proactively when explaining systems with 3+ components, complex data flows, or relationships that benefit from visual representation. Best suited when the diagram needs custom styling, rich shape vocabulary, swimlanes, or exportable images (PNG/SVG/PDF/JPG). Generates .drawio XML and exports locally via the native draw.io desktop CLI.", + "path": "skills/drawio-skill/SKILL.md" }, { "name": "e2e-playwright-expert", "description": "Use when writing, maintaining, or debugging end-to-end (E2E) browser tests with Playwright. Ensures rock-solid locators, auto-waiting, network mocking, and flakiness prevention.", - "resources": [] + "path": "skills/e2e-playwright-expert/SKILL.md" + }, + { + "name": "frontend-design", + "description": "Guidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.", + "path": "skills/frontend-design/SKILL.md" }, { "name": "github-actions-workflow-architect", "description": "Use when designing, optimizing, or securing GitHub Actions CI/CD workflows, caching dependencies, configuring matrix builds, and managing secrets.", - "resources": [] + "path": "skills/github-actions-workflow-architect/SKILL.md" }, { "name": "grill-me", "description": "Use when the user wants to stress-test an architecture plan, interrogate a design, or explicitly says \"grill me\". Relentlessly stress-tests technical plans before implementation starts.", - "resources": [] + "path": "skills/grill-me/SKILL.md" }, { "name": "lab-imagegen", "description": "Generate images using the user's home-lab ComfyUI server (SDXL-turbo on 2x AMD Radeon Pro VII). Use when the user asks to generate, render, or create an image or picture.", - "resources": [] + "path": "skills/lab-imagegen/SKILL.md" }, { "name": "lab-pdf-ocr", "description": "Extract text from PDF documents using the user's home-lab Docling OCR server (GPU-backed). Use when the user wants to recognize text, OCR a PDF, or convert a scanned or structured PDF to markdown text.", - "resources": [] + "path": "skills/lab-pdf-ocr/SKILL.md" }, { "name": "lab-speech-to-text", "description": "Transcribe speech from audio files (wav, mp3, ogg, m4a) using the user's home-lab Whishper speech-to-text server (GPU-backed). Use when the user wants to convert audio/voice to text, transcribe a recording, or subtitle.", - "resources": [] + "path": "skills/lab-speech-to-text/SKILL.md" + }, + { + "name": "ln-24-architecture-auditor", + "description": "\"Audits implemented architecture fitness, boundaries, contracts, dependencies, and configuration ownership. Use for system structure; not for current-state documentation, diagrams, or plan review.\"", + "path": "skills/ln-24-architecture-auditor/SKILL.md" + }, + { + "name": "ln-71-system-design-baseline-builder", + "description": "\"Creates a project baseline of architecture drivers and constraints. Use before design or planning; not for target design, plan review, implementation, or architecture audit.\"", + "path": "skills/ln-71-system-design-baseline-builder/SKILL.md" + }, + { + "name": "ln-72-current-architecture-documenter", + "description": "\"Documents implemented current-state architecture from repository evidence. Use for onboarding or migration baselines; not for target design, audit verdicts, or code changes.\"", + "path": "skills/ln-72-current-architecture-documenter/SKILL.md" + }, + { + "name": "ln-73-system-design-proposal-builder", + "description": "\"Creates a decision-complete target system design from requirements and constraints. Use before implementation planning; not for requirements baselines, reviews, audits, or code changes.\"", + "path": "skills/ln-73-system-design-proposal-builder/SKILL.md" + }, + { + "name": "ln-74-architecture-decision-recorder", + "description": "\"Records one architecture decision with context, alternatives, tradeoffs, and consequences. Use for a significant choice; not for broad design, audit, or implementation.\"", + "path": "skills/ln-74-architecture-decision-recorder/SKILL.md" + }, + { + "name": "ln-75-architecture-diagram-builder", + "description": "\"Creates evidence-backed current or target architecture diagrams when the diagram is the primary deliverable. Not for UI design, architecture audit, or invented structure.\"", + "path": "skills/ln-75-architecture-diagram-builder/SKILL.md" + }, + { + "name": "ln-76-architecture-migration-planner", + "description": "\"Plans a reversible architecture migration with compatibility, data movement, rollout, and rollback. Use for current-to-target transitions; not execution, generic planning, or delivery review.\"", + "path": "skills/ln-76-architecture-migration-planner/SKILL.md" }, { "name": "research", "description": "Conduct preliminary research on a topic and generate research outline. For academic research, benchmark research, technology selection, etc.", - "userInvocable": "true", - "resources": [ - "agents/web-search-agent.md", - "agents/web-search-modules/academic-papers.md", - "agents/web-search-modules/chinese-tech.md", - "agents/web-search-modules/general-web.md", - "agents/web-search-modules/github-debug.md", - "agents/web-search-modules/stackoverflow.md", - "LICENSE" - ] + "path": "skills/research/SKILL.md" }, { "name": "research-add-fields", "description": "Add field definitions to existing research outline.", - "userInvocable": "true", - "resources": [ - "LICENSE" - ] + "path": "skills/research-add-fields/SKILL.md" }, { "name": "research-add-items", "description": "Add items (research objects) to existing research outline.", - "userInvocable": "true", - "resources": [ - "LICENSE" - ] + "path": "skills/research-add-items/SKILL.md" }, { "name": "research-deep", "description": "Read research outline, launch independent agent for each item for deep research. Disable task output.", - "userInvocable": "true", - "resources": [ - "LICENSE", - "validate_json.py" - ] + "path": "skills/research-deep/SKILL.md" }, { "name": "research-report", "description": "Summarize deep research results into markdown report, cover all fields, skip uncertain values.", - "userInvocable": "true", - "resources": [ - "LICENSE" - ] + "path": "skills/research-report/SKILL.md" }, { "name": "security-audit", "description": "Use when performing security audits, vulnerability scanning, threat modeling, or reviewing code for OWASP Top 10 vulnerabilities, injection flaws, and authentication/authorization issues.", - "resources": [] + "path": "skills/security-audit/SKILL.md" }, { "name": "systematic-debugging", "description": "Use when debugging complex bugs, unexpected test failures, race conditions, or production anomalies. Enforces a rigorous scientific debugging method over trial-and-error edits.", - "resources": [] + "path": "skills/systematic-debugging/SKILL.md" }, { "name": "test-driven-development", "description": "Use when building new features, refactoring existing modules, or fixing bugs using Test-Driven Development (TDD: Red-Green-Refactor cycle).", - "resources": [] + "path": "skills/test-driven-development/SKILL.md" }, { "name": "web-performance-audit", "description": "Use when analyzing web application performance, Core Web Vitals (LCP, INP, CLS), JavaScript bundle size optimization, rendering bottlenecks, and memory leaks.", - "resources": [] + "path": "skills/web-performance-audit/SKILL.md" } ] } diff --git a/skills/api-designer/SKILL.md b/skills/api-designer/SKILL.md new file mode 100644 index 0000000..9d68465 --- /dev/null +++ b/skills/api-designer/SKILL.md @@ -0,0 +1,219 @@ +--- +name: api-designer +description: Use when designing REST or GraphQL APIs, creating OpenAPI specifications, or planning API architecture. Invoke for resource modeling, versioning strategies, pagination patterns, error handling standards. +license: MIT +metadata: + author: https://github.com/Jeffallan + version: "1.1.0" + domain: api-architecture + triggers: API design, REST API, OpenAPI, API specification, API architecture, resource modeling, API versioning, GraphQL schema, API documentation + role: architect + scope: design + output-format: specification + related-skills: graphql-architect, fastapi-expert, nestjs-expert, spring-boot-engineer, security-reviewer +--- + +# API Designer + +Senior API architect specializing in REST and GraphQL APIs with comprehensive OpenAPI 3.1 specifications. + +## Core Workflow + +1. **Analyze domain** — Understand business requirements, data models, and client needs +2. **Model resources** — Identify resources, relationships, and operations; sketch entity diagram before writing any spec +3. **Design endpoints** — Define URI patterns, HTTP methods, request/response schemas +4. **Specify contract** — Create OpenAPI 3.1 spec; validate before proceeding: `npx @redocly/cli lint openapi.yaml` +5. **Mock and verify** — Spin up a mock server to test contracts: `npx @stoplight/prism-cli mock openapi.yaml` +6. **Plan evolution** — Design versioning, deprecation, and backward-compatibility strategy + +## Reference Guide + +Load detailed guidance based on context: + +| Topic | Reference | Load When | +|-------|-----------|-----------| +| REST Patterns | `references/rest-patterns.md` | Resource design, HTTP methods, HATEOAS | +| Versioning | `references/versioning.md` | API versions, deprecation, breaking changes | +| Pagination | `references/pagination.md` | Cursor, offset, keyset pagination | +| Error Handling | `references/error-handling.md` | Error responses, RFC 7807, status codes | +| OpenAPI | `references/openapi.md` | OpenAPI 3.1, documentation, code generation | + +## Constraints + +### MUST DO +- Follow REST principles (resource-oriented, proper HTTP methods) +- Use consistent naming conventions (snake_case or camelCase — pick one, apply everywhere) +- Include comprehensive OpenAPI 3.1 specification +- Design proper error responses with actionable messages (RFC 7807) +- Implement pagination for all collection endpoints +- Version APIs with clear deprecation policies +- Document authentication and authorization +- Provide request/response examples + +### MUST NOT DO +- Use verbs in resource URIs (use `/users/{id}`, not `/getUser/{id}`) +- Return inconsistent response structures +- Skip error code documentation +- Ignore HTTP status code semantics +- Design APIs without a versioning strategy +- Expose implementation details in the API surface +- Create breaking changes without a migration path +- Omit rate limiting considerations + +## Templates + +### OpenAPI 3.1 Resource Endpoint (copy-paste starter) + +```yaml +openapi: "3.1.0" +info: + title: Example API + version: "1.1.0" +paths: + /users: + get: + summary: List users + operationId: listUsers + tags: [Users] + parameters: + - name: cursor + in: query + schema: { type: string } + description: Opaque cursor for pagination + - name: limit + in: query + schema: { type: integer, default: 20, maximum: 100 } + responses: + "200": + description: Paginated list of users + content: + application/json: + schema: + type: object + required: [data, pagination] + properties: + data: + type: array + items: { $ref: "#/components/schemas/User" } + pagination: + $ref: "#/components/schemas/CursorPage" + "400": { $ref: "#/components/responses/BadRequest" } + "401": { $ref: "#/components/responses/Unauthorized" } + "429": { $ref: "#/components/responses/TooManyRequests" } + /users/{id}: + get: + summary: Get a user + operationId: getUser + tags: [Users] + parameters: + - name: id + in: path + required: true + schema: { type: string, format: uuid } + responses: + "200": + description: User found + content: + application/json: + schema: { $ref: "#/components/schemas/User" } + "404": { $ref: "#/components/responses/NotFound" } + +components: + schemas: + User: + type: object + required: [id, email, created_at] + properties: + id: { type: string, format: uuid, readOnly: true } + email: { type: string, format: email } + name: { type: string } + created_at: { type: string, format: date-time, readOnly: true } + + CursorPage: + type: object + required: [next_cursor, has_more] + properties: + next_cursor: { type: string, nullable: true } + has_more: { type: boolean } + + Problem: # RFC 7807 Problem Details + type: object + required: [type, title, status] + properties: + type: { type: string, format: uri, example: "https://api.example.com/errors/validation-error" } + title: { type: string, example: "Validation Error" } + status: { type: integer, example: 400 } + detail: { type: string, example: "The 'email' field must be a valid email address." } + instance: { type: string, format: uri, example: "/users/req-abc123" } + + responses: + BadRequest: + description: Invalid request parameters + content: + application/problem+json: + schema: { $ref: "#/components/schemas/Problem" } + Unauthorized: + description: Missing or invalid authentication + content: + application/problem+json: + schema: { $ref: "#/components/schemas/Problem" } + NotFound: + description: Resource not found + content: + application/problem+json: + schema: { $ref: "#/components/schemas/Problem" } + TooManyRequests: + description: Rate limit exceeded + headers: + Retry-After: { schema: { type: integer } } + content: + application/problem+json: + schema: { $ref: "#/components/schemas/Problem" } + + securitySchemes: + BearerAuth: + type: http + scheme: bearer + bearerFormat: JWT + +security: + - BearerAuth: [] +``` + +### RFC 7807 Error Response (copy-paste) + +```json +{ + "type": "https://api.example.com/errors/validation-error", + "title": "Validation Error", + "status": 422, + "detail": "The 'email' field must be a valid email address.", + "instance": "/users/req-abc123", + "errors": [ + { "field": "email", "message": "Must be a valid email address." } + ] +} +``` + +- Always use `Content-Type: application/problem+json` for error responses. +- `type` must be a stable, documented URI — never a generic string. +- `detail` must be human-readable and actionable. +- Extend with `errors[]` for field-level validation failures. + +## Output Checklist + +When delivering an API design, provide: +1. Resource model and relationships (diagram or table) +2. Endpoint specifications with URIs and HTTP methods +3. OpenAPI 3.1 specification (YAML) +4. Authentication and authorization flows +5. Error response catalog (all 4xx/5xx with `type` URIs) +6. Pagination and filtering patterns +7. Versioning and deprecation strategy +8. Validation result: `npx @redocly/cli lint openapi.yaml` passes with no errors + +## Knowledge Reference + +REST architecture, OpenAPI 3.1, GraphQL, HTTP semantics, JSON:API, HATEOAS, OAuth 2.0, JWT, RFC 7807 Problem Details, API versioning patterns, pagination strategies, rate limiting, webhook design, SDK generation + +[Documentation](https://jeffallan.github.io/claude-skills/skills/api-architecture/api-designer/) diff --git a/skills/api-designer/references/error-handling.md b/skills/api-designer/references/error-handling.md new file mode 100644 index 0000000..cfc1a01 --- /dev/null +++ b/skills/api-designer/references/error-handling.md @@ -0,0 +1,541 @@ +# API Error Handling + +## Error Response Design + +Consistent, informative error responses are critical for API usability. + +## Standard Error Format + +### Basic Error Response + +```json +{ + "error": { + "code": "RESOURCE_NOT_FOUND", + "message": "User with ID 123 not found", + "details": null + } +} +``` + +### RFC 7807 Problem Details + +Standardized error format (application/problem+json): + +```http +HTTP/1.1 404 Not Found +Content-Type: application/problem+json + +{ + "type": "https://api.example.com/errors/resource-not-found", + "title": "Resource Not Found", + "status": 404, + "detail": "User with ID 123 does not exist", + "instance": "/users/123" +} +``` + +**Fields:** +- `type` - URI reference identifying error type +- `title` - Short, human-readable summary +- `status` - HTTP status code +- `detail` - Human-readable explanation specific to this occurrence +- `instance` - URI reference for this specific occurrence + +### Extended Error Response + +```json +{ + "error": { + "code": "VALIDATION_ERROR", + "message": "Request validation failed", + "details": [ + { + "field": "email", + "code": "INVALID_FORMAT", + "message": "Email must be a valid email address" + }, + { + "field": "age", + "code": "OUT_OF_RANGE", + "message": "Age must be between 18 and 120" + } + ], + "request_id": "req_123456", + "timestamp": "2024-01-15T10:30:00Z", + "documentation_url": "https://api.example.com/docs/errors#validation-error" + } +} +``` + +## Error Categories + +### 1. Validation Errors (400 Bad Request) + +Client sent invalid data. + +```http +POST /users +Content-Type: application/json + +{ + "name": "", + "email": "invalid-email", + "age": 15 +} + +Response: 400 Bad Request +{ + "error": { + "code": "VALIDATION_ERROR", + "message": "Request validation failed", + "details": [ + { + "field": "name", + "code": "REQUIRED", + "message": "Name is required" + }, + { + "field": "email", + "code": "INVALID_FORMAT", + "message": "Email must be a valid email address" + }, + { + "field": "age", + "code": "OUT_OF_RANGE", + "message": "Age must be at least 18", + "constraints": { + "min": 18, + "max": 120 + } + } + ] + } +} +``` + +### 2. Authentication Errors (401 Unauthorized) + +Missing or invalid authentication credentials. + +```http +GET /users/123 +Authorization: Bearer invalid_token + +Response: 401 Unauthorized +WWW-Authenticate: Bearer realm="api", error="invalid_token" + +{ + "error": { + "code": "INVALID_TOKEN", + "message": "The access token is invalid or has expired", + "details": { + "reason": "token_expired", + "expired_at": "2024-01-15T10:00:00Z" + } + } +} +``` + +**Common auth error codes:** +- `MISSING_TOKEN` - No auth token provided +- `INVALID_TOKEN` - Token is malformed or invalid +- `EXPIRED_TOKEN` - Token has expired +- `REVOKED_TOKEN` - Token has been revoked + +### 3. Authorization Errors (403 Forbidden) + +Authenticated but not authorized to perform action. + +```http +DELETE /users/123 +Authorization: Bearer valid_token + +Response: 403 Forbidden +{ + "error": { + "code": "INSUFFICIENT_PERMISSIONS", + "message": "You do not have permission to delete this user", + "details": { + "required_permission": "users:delete", + "your_permissions": ["users:read", "users:update"] + } + } +} +``` + +### 4. Not Found Errors (404 Not Found) + +Resource doesn't exist. + +```http +GET /users/99999 + +Response: 404 Not Found +{ + "error": { + "code": "RESOURCE_NOT_FOUND", + "message": "User with ID 99999 not found", + "details": { + "resource_type": "User", + "resource_id": "99999" + } + } +} +``` + +### 5. Conflict Errors (409 Conflict) + +Request conflicts with current state. + +```http +POST /users +Content-Type: application/json + +{ + "email": "existing@example.com", + "name": "John Doe" +} + +Response: 409 Conflict +{ + "error": { + "code": "RESOURCE_ALREADY_EXISTS", + "message": "User with email 'existing@example.com' already exists", + "details": { + "field": "email", + "value": "existing@example.com", + "existing_resource": "/users/123" + } + } +} +``` + +### 6. Rate Limiting (429 Too Many Requests) + +Client exceeded rate limit. + +```http +GET /users + +Response: 429 Too Many Requests +Retry-After: 60 +X-RateLimit-Limit: 100 +X-RateLimit-Remaining: 0 +X-RateLimit-Reset: 1705320000 + +{ + "error": { + "code": "RATE_LIMIT_EXCEEDED", + "message": "You have exceeded the rate limit", + "details": { + "limit": 100, + "window": "1 hour", + "retry_after": 60, + "reset_at": "2024-01-15T11:00:00Z" + } + } +} +``` + +### 7. Server Errors (500 Internal Server Error) + +Unexpected server error. + +```http +GET /users/123 + +Response: 500 Internal Server Error +{ + "error": { + "code": "INTERNAL_SERVER_ERROR", + "message": "An unexpected error occurred. Please try again later.", + "request_id": "req_123456", + "timestamp": "2024-01-15T10:30:00Z" + } +} +``` + +**Never expose:** +- Stack traces +- Database errors +- Internal paths +- Sensitive configuration + +### 8. Service Unavailable (503 Service Unavailable) + +Service temporarily unavailable. + +```http +GET /users + +Response: 503 Service Unavailable +Retry-After: 300 + +{ + "error": { + "code": "SERVICE_UNAVAILABLE", + "message": "Service is temporarily unavailable due to maintenance", + "details": { + "retry_after": 300, + "maintenance_end": "2024-01-15T12:00:00Z" + } + } +} +``` + +## Error Code Catalog + +Define standard error codes for your API: + +```json +{ + "VALIDATION_ERROR": { + "status": 400, + "description": "Request validation failed", + "subcodes": { + "REQUIRED": "Required field is missing", + "INVALID_FORMAT": "Field has invalid format", + "OUT_OF_RANGE": "Value is out of allowed range", + "INVALID_ENUM": "Value is not in allowed set" + } + }, + "AUTHENTICATION_ERROR": { + "status": 401, + "description": "Authentication failed", + "subcodes": { + "MISSING_TOKEN": "No authentication token provided", + "INVALID_TOKEN": "Token is invalid", + "EXPIRED_TOKEN": "Token has expired" + } + }, + "AUTHORIZATION_ERROR": { + "status": 403, + "description": "Insufficient permissions", + "subcodes": { + "INSUFFICIENT_PERMISSIONS": "Missing required permission", + "RESOURCE_FORBIDDEN": "Access to resource is forbidden" + } + }, + "RESOURCE_NOT_FOUND": { + "status": 404, + "description": "Resource not found" + }, + "CONFLICT_ERROR": { + "status": 409, + "description": "Request conflicts with current state", + "subcodes": { + "RESOURCE_ALREADY_EXISTS": "Resource already exists", + "CONCURRENT_MODIFICATION": "Resource was modified by another request" + } + }, + "RATE_LIMIT_EXCEEDED": { + "status": 429, + "description": "Rate limit exceeded" + }, + "INTERNAL_SERVER_ERROR": { + "status": 500, + "description": "Internal server error" + } +} +``` + +## Validation Error Details + +### Field-Level Validation + +```json +{ + "error": { + "code": "VALIDATION_ERROR", + "message": "Request validation failed", + "details": [ + { + "field": "credit_card.number", + "code": "INVALID_FORMAT", + "message": "Credit card number must be 16 digits", + "value_provided": "1234", + "constraints": { + "pattern": "^[0-9]{16}$" + } + }, + { + "field": "items[0].quantity", + "code": "OUT_OF_RANGE", + "message": "Quantity must be at least 1", + "value_provided": 0, + "constraints": { + "min": 1, + "max": 1000 + } + } + ] + } +} +``` + +### Cross-Field Validation + +```json +{ + "error": { + "code": "VALIDATION_ERROR", + "message": "Request validation failed", + "details": [ + { + "fields": ["start_date", "end_date"], + "code": "INVALID_RANGE", + "message": "End date must be after start date", + "values_provided": { + "start_date": "2024-01-20", + "end_date": "2024-01-15" + } + } + ] + } +} +``` + +## Request ID Tracking + +Always include request ID for debugging: + +```http +Response Headers: +X-Request-ID: req_abc123 + +Response Body: +{ + "error": { + "code": "INTERNAL_SERVER_ERROR", + "message": "An unexpected error occurred", + "request_id": "req_abc123" + } +} +``` + +Clients can reference request ID in support tickets. + +## Error Documentation + +Document all possible errors for each endpoint: + +```yaml +/users/{id}: + get: + responses: + '200': + description: Success + '401': + description: Authentication failed + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + examples: + missing_token: + value: + error: + code: MISSING_TOKEN + message: No authentication token provided + invalid_token: + value: + error: + code: INVALID_TOKEN + message: Token is invalid or expired + '404': + description: User not found + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + examples: + not_found: + value: + error: + code: RESOURCE_NOT_FOUND + message: User with ID 123 not found +``` + +## Retry Guidance + +Help clients understand if they should retry: + +```json +{ + "error": { + "code": "SERVICE_UNAVAILABLE", + "message": "Service temporarily unavailable", + "retry": { + "retryable": true, + "retry_after": 60, + "max_retries": 3, + "backoff": "exponential" + } + } +} +``` + +### Retryable Errors + +- 408 Request Timeout +- 429 Too Many Requests (with Retry-After) +- 500 Internal Server Error (sometimes) +- 502 Bad Gateway +- 503 Service Unavailable +- 504 Gateway Timeout + +### Non-Retryable Errors + +- 400 Bad Request +- 401 Unauthorized +- 403 Forbidden +- 404 Not Found +- 409 Conflict +- 422 Unprocessable Entity + +## Multi-Language Support + +Support error messages in multiple languages: + +```http +GET /users/invalid +Accept-Language: es + +Response: 404 Not Found +Content-Language: es +{ + "error": { + "code": "RESOURCE_NOT_FOUND", + "message": "Usuario con ID 'invalid' no encontrado" + } +} +``` + +Always include `code` so clients can implement their own translations. + +## Best Practices + +1. **Use standard HTTP status codes** - Don't return 200 for errors +2. **Include machine-readable codes** - Error codes for client logic +3. **Provide human-readable messages** - Clear explanations +4. **Be specific but safe** - Don't expose sensitive information +5. **Include request ID** - For tracking and debugging +6. **Document all errors** - Every possible error for each endpoint +7. **Be consistent** - Same format across all endpoints +8. **Help clients retry** - Indicate if error is retryable +9. **Validate early** - Return validation errors immediately +10. **Log errors server-side** - Track errors for monitoring + +## Anti-Patterns + +Avoid these mistakes: + +- **Generic error messages** - "Error occurred" without details +- **Exposing stack traces** - Security risk +- **Inconsistent error format** - Different structure per endpoint +- **Missing error codes** - Only human-readable messages +- **Wrong status codes** - Returning 200 with error in body +- **No request ID** - Makes debugging impossible +- **Undocumented errors** - Clients don't know what to expect +- **Too much information** - Exposing internal implementation diff --git a/skills/api-designer/references/openapi.md b/skills/api-designer/references/openapi.md new file mode 100644 index 0000000..8448e6a --- /dev/null +++ b/skills/api-designer/references/openapi.md @@ -0,0 +1,824 @@ +# OpenAPI 3.1 Specification + +## What is OpenAPI? + +OpenAPI (formerly Swagger) is a standard for describing REST APIs. It enables: +- Interactive documentation +- Code generation (SDKs, clients, servers) +- API testing tools +- Contract validation +- Mock servers + +## Basic Structure + +### Minimal OpenAPI 3.1 Spec + +```yaml +openapi: 3.1.0 +info: + title: My API + version: 1.0.0 + description: A sample API + contact: + name: API Support + email: support@example.com + url: https://example.com/support + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0.html + +servers: + - url: https://api.example.com/v1 + description: Production server + - url: https://staging-api.example.com/v1 + description: Staging server + - url: http://localhost:3000/v1 + description: Local development + +paths: + /users: + get: + summary: List users + description: Retrieve a paginated list of users + operationId: listUsers + tags: + - Users + responses: + '200': + description: Successful response + content: + application/json: + schema: + type: object + properties: + data: + type: array + items: + $ref: '#/components/schemas/User' + +components: + schemas: + User: + type: object + required: + - id + - email + properties: + id: + type: integer + format: int64 + example: 123 + email: + type: string + format: email + example: john@example.com + name: + type: string + example: John Doe +``` + +## Info Object + +Metadata about the API: + +```yaml +info: + title: Users API + version: 1.0.0 + description: | + # Users API + + This API manages user accounts and profiles. + + ## Features + - User CRUD operations + - Authentication with JWT + - Role-based authorization + + termsOfService: https://example.com/terms + + contact: + name: API Support Team + email: api-support@example.com + url: https://example.com/support + + license: + name: MIT + url: https://opensource.org/licenses/MIT + + x-api-id: users-api-v1 + x-audience: external +``` + +## Servers + +Define API base URLs: + +```yaml +servers: + - url: https://api.example.com/v1 + description: Production + variables: + version: + default: v1 + enum: + - v1 + - v2 + + - url: https://{environment}.example.com/v1 + description: Dynamic environment + variables: + environment: + default: api + enum: + - api + - staging + - dev +``` + +## Paths and Operations + +### Complete Endpoint Example + +```yaml +paths: + /users: + get: + summary: List users + description: Retrieve a paginated list of users with optional filtering + operationId: listUsers + tags: + - Users + + parameters: + - name: offset + in: query + description: Number of items to skip + required: false + schema: + type: integer + minimum: 0 + default: 0 + + - name: limit + in: query + description: Maximum number of items to return + required: false + schema: + type: integer + minimum: 1 + maximum: 100 + default: 20 + + - name: status + in: query + description: Filter by user status + required: false + schema: + type: string + enum: + - active + - inactive + - suspended + + security: + - bearerAuth: [] + + responses: + '200': + description: Successful response + content: + application/json: + schema: + $ref: '#/components/schemas/UserListResponse' + examples: + success: + $ref: '#/components/examples/UserListSuccess' + + '401': + $ref: '#/components/responses/Unauthorized' + + '429': + $ref: '#/components/responses/RateLimitExceeded' + + post: + summary: Create user + description: Create a new user account + operationId: createUser + tags: + - Users + + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/CreateUserRequest' + examples: + basic: + $ref: '#/components/examples/CreateUserBasic' + + responses: + '201': + description: User created successfully + headers: + Location: + description: URL of the created user + schema: + type: string + format: uri + content: + application/json: + schema: + $ref: '#/components/schemas/User' + + '400': + $ref: '#/components/responses/ValidationError' + + '409': + description: User already exists + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + + /users/{userId}: + parameters: + - name: userId + in: path + description: User ID + required: true + schema: + type: integer + format: int64 + + get: + summary: Get user + description: Retrieve a specific user by ID + operationId: getUser + tags: + - Users + + responses: + '200': + description: Successful response + content: + application/json: + schema: + $ref: '#/components/schemas/User' + + '404': + $ref: '#/components/responses/NotFound' + + put: + summary: Update user + description: Replace user data + operationId: updateUser + tags: + - Users + + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateUserRequest' + + responses: + '200': + description: User updated successfully + content: + application/json: + schema: + $ref: '#/components/schemas/User' + + '404': + $ref: '#/components/responses/NotFound' + + delete: + summary: Delete user + description: Delete a user account + operationId: deleteUser + tags: + - Users + + responses: + '204': + description: User deleted successfully + + '404': + $ref: '#/components/responses/NotFound' +``` + +## Components + +Reusable components for your API spec. + +### Schemas + +```yaml +components: + schemas: + User: + type: object + required: + - id + - email + - name + properties: + id: + type: integer + format: int64 + readOnly: true + example: 123 + email: + type: string + format: email + example: john@example.com + name: + type: string + minLength: 1 + maxLength: 100 + example: John Doe + status: + type: string + enum: + - active + - inactive + - suspended + default: active + created_at: + type: string + format: date-time + readOnly: true + example: "2024-01-15T10:30:00Z" + metadata: + type: object + additionalProperties: + type: string + + CreateUserRequest: + type: object + required: + - email + - name + properties: + email: + type: string + format: email + name: + type: string + minLength: 1 + maxLength: 100 + metadata: + type: object + additionalProperties: + type: string + + UserListResponse: + type: object + properties: + data: + type: array + items: + $ref: '#/components/schemas/User' + pagination: + $ref: '#/components/schemas/Pagination' + + Pagination: + type: object + properties: + offset: + type: integer + minimum: 0 + limit: + type: integer + minimum: 1 + total: + type: integer + minimum: 0 + has_more: + type: boolean + + Error: + type: object + required: + - error + properties: + error: + type: object + required: + - code + - message + properties: + code: + type: string + example: RESOURCE_NOT_FOUND + message: + type: string + example: User with ID 123 not found + details: + type: object + request_id: + type: string + example: req_abc123 +``` + +### Security Schemes + +```yaml +components: + securitySchemes: + bearerAuth: + type: http + scheme: bearer + bearerFormat: JWT + description: JWT access token + + apiKey: + type: apiKey + in: header + name: X-API-Key + description: API key for authentication + + oauth2: + type: oauth2 + flows: + authorizationCode: + authorizationUrl: https://auth.example.com/oauth/authorize + tokenUrl: https://auth.example.com/oauth/token + scopes: + users:read: Read user data + users:write: Create and update users + users:delete: Delete users +``` + +Apply security globally or per-operation: + +```yaml +# Global security +security: + - bearerAuth: [] + +# Or per-operation +paths: + /users: + get: + security: + - bearerAuth: [] + - apiKey: [] # Alternative auth method +``` + +### Responses + +Reusable response definitions: + +```yaml +components: + responses: + NotFound: + description: Resource not found + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + example: + error: + code: RESOURCE_NOT_FOUND + message: The requested resource was not found + + Unauthorized: + description: Authentication required + headers: + WWW-Authenticate: + schema: + type: string + description: Authentication method + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + + ValidationError: + description: Validation failed + content: + application/json: + schema: + $ref: '#/components/schemas/Error' + example: + error: + code: VALIDATION_ERROR + message: Request validation failed + details: + - field: email + code: INVALID_FORMAT + message: Email must be a valid email address + + RateLimitExceeded: + description: Rate limit exceeded + headers: + X-RateLimit-Limit: + schema: + type: integer + description: Request limit per hour + X-RateLimit-Remaining: + schema: + type: integer + description: Remaining requests + X-RateLimit-Reset: + schema: + type: integer + format: int64 + description: Time when limit resets (Unix timestamp) + Retry-After: + schema: + type: integer + description: Seconds to wait before retry + content: + application/json: + schema: + $ref: '#/components/schemas/Error' +``` + +### Examples + +```yaml +components: + examples: + UserListSuccess: + summary: Successful user list response + value: + data: + - id: 1 + email: john@example.com + name: John Doe + status: active + created_at: "2024-01-15T10:30:00Z" + - id: 2 + email: jane@example.com + name: Jane Smith + status: active + created_at: "2024-01-16T14:20:00Z" + pagination: + offset: 0 + limit: 20 + total: 150 + has_more: true + + CreateUserBasic: + summary: Create user with minimal fields + value: + email: newuser@example.com + name: New User +``` + +## Data Types + +### Primitive Types + +```yaml +# String +type: string +example: "Hello World" + +# String with format +type: string +format: email +example: "user@example.com" + +# Integer +type: integer +format: int64 +example: 123 + +# Number (float) +type: number +format: double +example: 99.99 + +# Boolean +type: boolean +example: true + +# Date-time +type: string +format: date-time +example: "2024-01-15T10:30:00Z" + +# Date +type: string +format: date +example: "2024-01-15" + +# UUID +type: string +format: uuid +example: "550e8400-e29b-41d4-a716-446655440000" + +# URI +type: string +format: uri +example: "https://example.com/users/123" +``` + +### Arrays + +```yaml +type: array +items: + type: string +minItems: 1 +maxItems: 10 +uniqueItems: true +example: ["tag1", "tag2", "tag3"] + +# Array of objects +type: array +items: + $ref: '#/components/schemas/User' +``` + +### Objects + +```yaml +type: object +required: + - name + - email +properties: + name: + type: string + email: + type: string + format: email + age: + type: integer + minimum: 0 + maximum: 120 + +# Additional properties +additionalProperties: false # Strict - no extra properties +additionalProperties: true # Allow any extra properties +additionalProperties: # Extra properties must be strings + type: string +``` + +### Enums + +```yaml +type: string +enum: + - active + - inactive + - suspended +default: active +``` + +### OneOf / AnyOf / AllOf + +```yaml +# OneOf - exactly one schema matches +oneOf: + - $ref: '#/components/schemas/CreditCard' + - $ref: '#/components/schemas/BankAccount' + +# AnyOf - one or more schemas match +anyOf: + - $ref: '#/components/schemas/User' + - $ref: '#/components/schemas/Organization' + +# AllOf - all schemas must match (inheritance) +allOf: + - $ref: '#/components/schemas/BaseUser' + - type: object + properties: + admin_level: + type: integer +``` + +## Validation + +### String Validation + +```yaml +type: string +minLength: 1 +maxLength: 100 +pattern: "^[a-zA-Z0-9_-]+$" +format: email +``` + +### Number Validation + +```yaml +type: integer +minimum: 0 +maximum: 100 +exclusiveMinimum: true # > 0 instead of >= 0 +multipleOf: 5 +``` + +### Array Validation + +```yaml +type: array +minItems: 1 +maxItems: 10 +uniqueItems: true +``` + +## Tags + +Organize endpoints into logical groups: + +```yaml +tags: + - name: Users + description: User management operations + - name: Orders + description: Order management + - name: Products + description: Product catalog + +paths: + /users: + get: + tags: + - Users +``` + +## Documentation + +### Markdown Support + +```yaml +description: | + # User Management + + This endpoint allows you to manage users. + + ## Features + - Create users + - Update profiles + - Delete accounts + + ## Authentication + Requires JWT bearer token. + + ## Example + ```json + { + "name": "John Doe", + "email": "john@example.com" + } + ``` +``` + +## Code Generation + +Generate SDKs from OpenAPI spec: + +```bash +# Generate TypeScript client +openapi-generator-cli generate \ + -i openapi.yaml \ + -g typescript-axios \ + -o ./client + +# Generate Python client +openapi-generator-cli generate \ + -i openapi.yaml \ + -g python \ + -o ./python-client + +# Generate server stub +openapi-generator-cli generate \ + -i openapi.yaml \ + -g nodejs-express-server \ + -o ./server +``` + +## Validation Tools + +Validate OpenAPI spec: + +```bash +# Using Swagger CLI +swagger-cli validate openapi.yaml + +# Using Spectral (advanced linting) +spectral lint openapi.yaml +``` + +## Best Practices + +1. **Use components** - Reuse schemas, responses, parameters +2. **Add examples** - Include realistic examples for all schemas +3. **Document thoroughly** - Every endpoint, parameter, response +4. **Version your spec** - Track changes to the specification +5. **Validate regularly** - Use tools to catch errors +6. **Use $ref** - Reference components instead of duplicating +7. **Include error responses** - Document all possible errors +8. **Add operationId** - Unique ID for each operation (for code gen) +9. **Tag endpoints** - Organize into logical groups +10. **Provide security schemes** - Document authentication clearly diff --git a/skills/api-designer/references/pagination.md b/skills/api-designer/references/pagination.md new file mode 100644 index 0000000..9874b50 --- /dev/null +++ b/skills/api-designer/references/pagination.md @@ -0,0 +1,494 @@ +# Pagination Patterns + +## Why Paginate? + +Large collections can't be returned all at once due to: +- Performance (slow queries, large payloads) +- Memory constraints (server and client) +- Network timeouts +- Poor user experience + +Always paginate collection endpoints. + +## Pagination Strategies + +### 1. Offset-Based Pagination + +Most common and intuitive. Uses `offset` (skip) and `limit` (page size). + +**Request:** +```http +GET /users?offset=20&limit=10 +``` + +**Response:** +```json +{ + "data": [ + {"id": 21, "name": "User 21"}, + {"id": 22, "name": "User 22"} + ], + "pagination": { + "offset": 20, + "limit": 10, + "total": 150, + "has_more": true + }, + "links": { + "first": "/users?offset=0&limit=10", + "prev": "/users?offset=10&limit=10", + "next": "/users?offset=30&limit=10", + "last": "/users?offset=140&limit=10" + } +} +``` + +**Advantages:** +- Simple to implement +- Easy to understand +- Random access (jump to any page) +- Shows total count + +**Disadvantages:** +- Performance degrades with large offsets (database scans many rows) +- Inconsistent results if data changes during pagination +- Inefficient for real-time data +- Database must count total rows (expensive) + +**Use when:** +- Small to medium datasets +- Data doesn't change frequently +- Need random page access +- Need total count + +### 2. Page-Based Pagination + +Simplified offset pagination using page numbers. + +**Request:** +```http +GET /users?page=3&per_page=10 +``` + +**Response:** +```json +{ + "data": [...], + "pagination": { + "page": 3, + "per_page": 10, + "total_pages": 15, + "total_count": 150 + }, + "links": { + "first": "/users?page=1&per_page=10", + "prev": "/users?page=2&per_page=10", + "next": "/users?page=4&per_page=10", + "last": "/users?page=15&per_page=10" + } +} +``` + +**Calculation:** +- `offset = (page - 1) * per_page` +- `total_pages = ceil(total_count / per_page)` + +**Same pros/cons as offset-based, but:** +- More intuitive for users (page 1, page 2) +- Common in web applications + +### 3. Cursor-Based Pagination + +Uses an opaque cursor (pointer) to the next set of results. + +**Request:** +```http +GET /users?limit=10 +GET /users?cursor=eyJpZCI6MTIzfQ&limit=10 +``` + +**Response:** +```json +{ + "data": [ + {"id": 21, "name": "User 21"}, + {"id": 22, "name": "User 22"} + ], + "pagination": { + "next_cursor": "eyJpZCI6MzB9", + "prev_cursor": "eyJpZCI6MjB9", + "has_more": true + }, + "links": { + "next": "/users?cursor=eyJpZCI6MzB9&limit=10", + "prev": "/users?cursor=eyJpZCI6MjB9&limit=10" + } +} +``` + +**Cursor structure (base64 encoded):** +```json +{"id": 30, "sort": "created_at"} +``` + +**Implementation:** +```sql +-- First page +SELECT * FROM users ORDER BY created_at DESC LIMIT 10; + +-- Next page (cursor points to last item) +SELECT * FROM users +WHERE created_at < '2024-01-15T10:30:00Z' +ORDER BY created_at DESC +LIMIT 10; +``` + +**Advantages:** +- Consistent results (no skipped/duplicate items) +- Efficient for large datasets +- Works well with real-time data +- No expensive COUNT query +- Better database performance + +**Disadvantages:** +- No random access (can't jump to page 10) +- No total count +- More complex to implement +- Cursor is opaque (users can't modify it) + +**Use when:** +- Large datasets +- Data changes frequently +- Infinite scroll UI +- Real-time feeds +- Performance is critical + +### 4. Keyset Pagination + +Similar to cursor but uses actual field values instead of opaque cursor. + +**Request:** +```http +GET /users?after_id=20&limit=10 +GET /users?after_created_at=2024-01-15T10:30:00Z&limit=10 +``` + +**Response:** +```json +{ + "data": [ + {"id": 21, "name": "User 21", "created_at": "2024-01-15T11:00:00Z"}, + {"id": 22, "name": "User 22", "created_at": "2024-01-15T11:30:00Z"} + ], + "pagination": { + "after_id": 30, + "limit": 10, + "has_more": true + }, + "links": { + "next": "/users?after_id=30&limit=10" + } +} +``` + +**Implementation:** +```sql +SELECT * FROM users +WHERE id > 20 +ORDER BY id ASC +LIMIT 10; +``` + +**Advantages:** +- Very efficient (uses index) +- Transparent cursor (human readable) +- Consistent results +- Simple implementation + +**Disadvantages:** +- Requires indexed column +- No random access +- Sorting limited to cursor field +- Complex for multi-field sorting + +**Use when:** +- Simple ordering (by ID, timestamp) +- Need efficient pagination +- Want transparent cursor +- Have proper indexes + +### 5. Seek Pagination (Time-Based) + +Specialized keyset pagination for time-series data. + +**Request:** +```http +GET /events?since=2024-01-15T10:00:00Z&until=2024-01-15T11:00:00Z&limit=100 +``` + +**Response:** +```json +{ + "data": [...], + "pagination": { + "since": "2024-01-15T10:00:00Z", + "until": "2024-01-15T11:00:00Z", + "limit": 100, + "has_more": true + }, + "links": { + "next": "/events?since=2024-01-15T11:00:00Z&until=2024-01-15T12:00:00Z&limit=100" + } +} +``` + +**Use for:** +- Time-series data +- Logs and events +- Activity streams +- Analytics data + +## Default Limits + +Always set reasonable defaults and maximum limits: + +```json +{ + "default_limit": 20, + "max_limit": 100, + "min_limit": 1 +} +``` + +**Validation:** +```http +GET /users?limit=1000 + +Response: 400 Bad Request +{ + "error": { + "code": "INVALID_LIMIT", + "message": "Limit must be between 1 and 100. Default is 20." + } +} +``` + +## Response Format + +### Standard Pagination Object + +```json +{ + "data": [...], + "pagination": { + "limit": 10, + "offset": 20, + "total": 150, + "has_more": true, + "has_previous": true + } +} +``` + +### Link Header (RFC 5988) + +```http +Link: ; rel="first", + ; rel="prev", + ; rel="next", + ; rel="last" +``` + +**Used by:** GitHub API + +### Embedded Links + +```json +{ + "data": [...], + "_links": { + "self": { "href": "/users?offset=20&limit=10" }, + "first": { "href": "/users?offset=0&limit=10" }, + "prev": { "href": "/users?offset=10&limit=10" }, + "next": { "href": "/users?offset=30&limit=10" }, + "last": { "href": "/users?offset=140&limit=10" } + } +} +``` + +## Sorting with Pagination + +Always support sorting when paginating: + +```http +GET /users?sort=created_at&order=desc&limit=10 +GET /users?sort=-created_at&limit=10 # Descending +GET /users?sort=last_name,first_name&limit=10 # Multi-field +``` + +**For cursor pagination, cursor must include sort fields:** +```json +{ + "cursor": { + "id": 123, + "created_at": "2024-01-15T10:30:00Z", + "sort_fields": ["created_at", "id"] + } +} +``` + +## Filtering with Pagination + +Combine filtering with pagination: + +```http +GET /users?status=active&role=admin&offset=0&limit=10 +``` + +**Important:** Apply filters before pagination: +1. Filter records +2. Count filtered results +3. Apply pagination +4. Return paginated subset + +## Total Count + +### Include Total Count + +```json +{ + "data": [...], + "pagination": { + "total": 1523, + "limit": 10, + "offset": 20 + } +} +``` + +**Pros:** +- Clients know total results +- Can calculate total pages +- Better UX (show "Page 3 of 153") + +**Cons:** +- COUNT query is expensive +- Slows down response +- Inaccurate for large/changing datasets + +### Omit Total Count + +```json +{ + "data": [...], + "pagination": { + "has_more": true, + "limit": 10 + } +} +``` + +**Use when:** +- Large datasets (COUNT is too slow) +- Real-time data (count changes constantly) +- Cursor pagination +- Infinite scroll UI + +### Optional Total Count + +Let client request total count: + +```http +GET /users?limit=10&include_total=true +``` + +## Edge Cases + +### Empty Results + +```json +{ + "data": [], + "pagination": { + "offset": 0, + "limit": 10, + "total": 0, + "has_more": false + } +} +``` + +### Last Page + +```json +{ + "data": [{"id": 150, "name": "Last User"}], + "pagination": { + "offset": 140, + "limit": 10, + "total": 150, + "has_more": false + }, + "links": { + "first": "/users?offset=0&limit=10", + "prev": "/users?offset=130&limit=10", + "next": null + } +} +``` + +### Out of Range + +```http +GET /users?offset=10000&limit=10 + +Response: 200 OK (empty results) +{ + "data": [], + "pagination": { + "offset": 10000, + "limit": 10, + "total": 150, + "has_more": false + } +} +``` + +Or return 404 for pages that don't exist: +```http +GET /users?page=1000&per_page=10 + +Response: 404 Not Found +{ + "error": { + "code": "PAGE_NOT_FOUND", + "message": "Page 1000 does not exist. Total pages: 15" + } +} +``` + +## Best Practices + +1. **Always paginate collections** - Never return unbounded lists +2. **Set reasonable defaults** - Default limit of 20-50 items +3. **Enforce maximum limits** - Prevent excessive loads (max 100-1000) +4. **Include has_more flag** - Tell clients if more results exist +5. **Provide navigation links** - Make it easy to get next/prev pages +6. **Document pagination** - Explain cursor format, limits, defaults +7. **Be consistent** - Use same pagination pattern across all endpoints +8. **Consider performance** - Choose strategy based on data size/type +9. **Support sorting** - Let clients control result order +10. **Handle edge cases** - Empty results, last page, invalid cursors + +## Comparison Matrix + +| Feature | Offset | Page | Cursor | Keyset | +|---------|--------|------|--------|--------| +| Performance | Poor for large offsets | Poor | Excellent | Excellent | +| Random access | Yes | Yes | No | No | +| Total count | Yes | Yes | No | Optional | +| Consistency | Poor | Poor | Excellent | Excellent | +| Complexity | Simple | Simple | Medium | Medium | +| Real-time data | Poor | Poor | Excellent | Excellent | +| Database load | High | High | Low | Low | +| Use case | Small datasets | Web UIs | Feeds/streams | Large datasets | diff --git a/skills/api-designer/references/rest-patterns.md b/skills/api-designer/references/rest-patterns.md new file mode 100644 index 0000000..6f493d3 --- /dev/null +++ b/skills/api-designer/references/rest-patterns.md @@ -0,0 +1,335 @@ +# REST Design Patterns + +## Resource-Oriented Architecture + +REST APIs are built around resources, not actions. Resources are the nouns of your API. + +### Resource Identification + +**Good Resource URIs:** +``` +GET /users # Collection +GET /users/{id} # Individual resource +GET /users/{id}/orders # Nested collection +POST /users # Create resource +PUT /users/{id} # Replace resource +PATCH /users/{id} # Update resource +DELETE /users/{id} # Delete resource +``` + +**Bad Resource URIs:** +``` +POST /getUser # Verb in URI +POST /createUser # Verb in URI +GET /user?action=delete # Action as query param +``` + +### Resource Naming Conventions + +- Use plural nouns for collections: `/users`, `/orders`, `/products` +- Use lowercase and hyphens for readability: `/shipping-addresses` +- Avoid deep nesting (max 2-3 levels): `/users/{id}/orders/{orderId}` +- Use query parameters for filtering: `/users?status=active&role=admin` + +## HTTP Method Semantics + +### Safe and Idempotent Methods + +| Method | Safe | Idempotent | Use Case | +|--------|------|------------|----------| +| GET | Yes | Yes | Retrieve resource(s) | +| POST | No | No | Create resource, non-idempotent operations | +| PUT | No | Yes | Replace entire resource | +| PATCH | No | No | Partial update | +| DELETE | No | Yes | Remove resource | +| HEAD | Yes | Yes | Get metadata only | +| OPTIONS | Yes | Yes | Get allowed methods | + +### Method Usage + +**GET - Retrieve Resources** +```http +GET /users/123 +Accept: application/json + +Response: 200 OK +{ + "id": 123, + "name": "John Doe", + "email": "john@example.com", + "created_at": "2024-01-15T10:30:00Z" +} +``` + +**POST - Create Resources** +```http +POST /users +Content-Type: application/json + +{ + "name": "Jane Smith", + "email": "jane@example.com" +} + +Response: 201 Created +Location: /users/124 +{ + "id": 124, + "name": "Jane Smith", + "email": "jane@example.com", + "created_at": "2024-01-16T14:20:00Z" +} +``` + +**PUT - Replace Resource** +```http +PUT /users/123 +Content-Type: application/json + +{ + "name": "John Doe Updated", + "email": "john.new@example.com" +} + +Response: 200 OK +{ + "id": 123, + "name": "John Doe Updated", + "email": "john.new@example.com", + "updated_at": "2024-01-17T09:15:00Z" +} +``` + +**PATCH - Partial Update** +```http +PATCH /users/123 +Content-Type: application/json + +{ + "email": "john.updated@example.com" +} + +Response: 200 OK +{ + "id": 123, + "name": "John Doe", + "email": "john.updated@example.com", + "updated_at": "2024-01-17T10:00:00Z" +} +``` + +**DELETE - Remove Resource** +```http +DELETE /users/123 + +Response: 204 No Content +``` + +## HTTP Status Codes + +### Success Codes (2xx) + +- **200 OK** - Request succeeded (GET, PUT, PATCH) +- **201 Created** - Resource created (POST), include Location header +- **202 Accepted** - Request accepted for async processing +- **204 No Content** - Success with no response body (DELETE) + +### Redirection (3xx) + +- **301 Moved Permanently** - Resource permanently moved +- **302 Found** - Temporary redirect +- **304 Not Modified** - Cached version is still valid + +### Client Errors (4xx) + +- **400 Bad Request** - Invalid request syntax or validation error +- **401 Unauthorized** - Authentication required or failed +- **403 Forbidden** - Authenticated but not authorized +- **404 Not Found** - Resource doesn't exist +- **405 Method Not Allowed** - HTTP method not supported for resource +- **409 Conflict** - Request conflicts with current state (e.g., duplicate) +- **422 Unprocessable Entity** - Valid syntax but semantic errors +- **429 Too Many Requests** - Rate limit exceeded + +### Server Errors (5xx) + +- **500 Internal Server Error** - Unexpected server error +- **502 Bad Gateway** - Invalid response from upstream server +- **503 Service Unavailable** - Server temporarily unavailable +- **504 Gateway Timeout** - Upstream server timeout + +## HATEOAS (Hypermedia) + +### Hypermedia-Driven APIs + +Include links to related resources and available actions: + +```json +{ + "id": 123, + "name": "John Doe", + "email": "john@example.com", + "_links": { + "self": { "href": "/users/123" }, + "orders": { "href": "/users/123/orders" }, + "update": { "href": "/users/123", "method": "PATCH" }, + "delete": { "href": "/users/123", "method": "DELETE" } + } +} +``` + +### HAL (Hypertext Application Language) + +```json +{ + "id": 123, + "name": "John Doe", + "_links": { + "self": { "href": "/users/123" } + }, + "_embedded": { + "orders": [ + { + "id": 456, + "total": 99.99, + "_links": { + "self": { "href": "/orders/456" } + } + } + ] + } +} +``` + +## Content Negotiation + +### Accept Headers + +```http +GET /users/123 +Accept: application/json + +GET /users/123 +Accept: application/xml + +GET /users/123 +Accept: application/hal+json +``` + +### Response Content-Type + +```http +Content-Type: application/json; charset=utf-8 +Content-Type: application/problem+json +Content-Type: application/hal+json +``` + +## Idempotency + +### Idempotent Operations + +**PUT - Always idempotent:** +Multiple identical PUT requests produce the same result as a single request. + +**DELETE - Idempotent:** +First DELETE returns 204, subsequent DELETEs return 404 (same end state). + +**POST - Not idempotent by default:** +Use `Idempotency-Key` header for idempotent POST: + +```http +POST /payments +Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000 +Content-Type: application/json + +{ + "amount": 100.00, + "currency": "USD" +} +``` + +Server stores idempotency key and returns same response for duplicate requests. + +## Cache Control + +### Cache Headers + +```http +Cache-Control: public, max-age=3600 +Cache-Control: private, no-cache +Cache-Control: no-store +ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4" +Last-Modified: Wed, 15 Jan 2024 10:30:00 GMT +``` + +### Conditional Requests + +```http +GET /users/123 +If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4" + +Response: 304 Not Modified +``` + +```http +PUT /users/123 +If-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4" +Content-Type: application/json + +{ + "name": "Updated Name" +} + +Response: 412 Precondition Failed (if ETag doesn't match) +``` + +## URI Patterns + +### Consistent URI Structure + +``` +/{version}/{resource} +/{version}/{resource}/{id} +/{version}/{resource}/{id}/{sub-resource} +/{version}/{resource}/{id}/{sub-resource}/{sub-id} +``` + +### Query Parameters + +**Filtering:** +``` +GET /users?status=active&role=admin +GET /products?category=electronics&price_min=100&price_max=500 +``` + +**Sorting:** +``` +GET /users?sort=created_at +GET /users?sort=-created_at # Descending +GET /users?sort=name,created_at # Multiple fields +``` + +**Field Selection:** +``` +GET /users?fields=id,name,email +GET /users?exclude=password,social_security_number +``` + +**Search:** +``` +GET /users?q=john +GET /products?search=laptop +``` + +## Best Practices + +1. **Use nouns, not verbs** - Resources are nouns, methods are verbs +2. **Plural collections** - Use `/users` not `/user` +3. **Consistent naming** - Choose snake_case or camelCase and stick to it +4. **Proper status codes** - Use appropriate HTTP status codes +5. **Include metadata** - Pagination, filtering, sorting info in responses +6. **Version your API** - Plan for evolution from day one +7. **Document everything** - OpenAPI specs, examples, error codes +8. **Security by default** - HTTPS, authentication, rate limiting +9. **Support filtering** - Enable clients to get exactly what they need +10. **Implement HATEOAS** - Make APIs self-documenting and discoverable diff --git a/skills/api-designer/references/versioning.md b/skills/api-designer/references/versioning.md new file mode 100644 index 0000000..c4840b1 --- /dev/null +++ b/skills/api-designer/references/versioning.md @@ -0,0 +1,391 @@ +# API Versioning Strategies + +## Why Version APIs? + +API versioning allows you to evolve your API while maintaining backward compatibility for existing clients. Breaking changes require a new version. + +### Breaking Changes + +Changes that require a new version: +- Removing or renaming fields +- Changing field types (string to integer) +- Adding required fields to requests +- Changing response structure +- Removing endpoints +- Changing HTTP status codes for same scenario +- Changing authentication mechanisms + +### Non-Breaking Changes + +Safe changes that don't require a new version: +- Adding new endpoints +- Adding optional request fields +- Adding new fields to responses (clients should ignore unknown fields) +- Fixing bugs +- Performance improvements +- Adding new HTTP methods to existing resources + +## Versioning Strategies + +### 1. URI Versioning + +Most common and visible approach. Version is part of the URL path. + +```http +GET /v1/users/123 +GET /v2/users/123 +``` + +**Advantages:** +- Clear and visible in URLs +- Easy to understand and implement +- Simple routing and caching +- Can run multiple versions simultaneously + +**Disadvantages:** +- Violates REST principle (same resource, different URIs) +- Requires updating client code to change version +- Can lead to URI proliferation + +**Implementation:** +``` +/v1/users +/v1/products +/v2/users # New version with breaking changes +/v2/products +``` + +### 2. Header Versioning + +Version specified in HTTP headers (Accept header or custom header). + +**Accept Header:** +```http +GET /users/123 +Accept: application/vnd.myapi.v1+json + +GET /users/123 +Accept: application/vnd.myapi.v2+json +``` + +**Custom Header:** +```http +GET /users/123 +API-Version: 1 + +GET /users/123 +API-Version: 2 +``` + +**Advantages:** +- URIs remain stable +- More RESTful (same resource, same URI) +- Separates versioning from resource identification + +**Disadvantages:** +- Less visible (harder to debug) +- More complex routing +- Difficult to test in browser +- Cache complexity + +### 3. Query Parameter Versioning + +Version specified as query parameter. + +```http +GET /users/123?version=1 +GET /users/123?version=2 + +# or +GET /users/123?api-version=1 +GET /users/123?api-version=2 +``` + +**Advantages:** +- Simple to implement +- Easy to test +- Visible in URLs + +**Disadvantages:** +- Pollutes query string +- Not semantic (version not a filter) +- Can interfere with other query params + +### 4. Content Negotiation + +Client specifies desired version through content negotiation. + +```http +GET /users/123 +Accept: application/vnd.myapi+json; version=1 + +GET /users/123 +Accept: application/vnd.myapi+json; version=2 +``` + +**Advantages:** +- Very RESTful +- Flexible content type negotiation +- Stable URIs + +**Disadvantages:** +- Complex implementation +- Less intuitive for developers +- Harder to test + +## Recommended Approach + +**URI versioning is recommended for most APIs** because: +- It's the most explicit and discoverable +- Easy to understand and debug +- Simple to implement and maintain +- Clear separation between versions + +``` +/v1/users +/v2/users +/v3/users +``` + +## Version Format + +### Major Versions Only + +Use simple major versions (v1, v2, v3) for public APIs: +``` +/v1/users +/v2/users +``` + +**Advantages:** +- Simple and clear +- Easy to communicate +- Forces thoughtful breaking changes + +### Date-Based Versions + +Some APIs use dates for versions: +``` +/2024-01-01/users +/2024-06-15/users +``` + +**Used by:** Stripe, GitHub API + +**Advantages:** +- Clear when version was released +- Easy to understand timeline +- No confusion about major/minor + +**Disadvantages:** +- Less intuitive for clients +- Harder to understand what changed + +## Version Lifecycle + +### 1. Introduction Phase + +New version is released alongside existing version: +``` +/v1/users # Still supported +/v2/users # New version available +``` + +Announce new version: +- Blog post explaining changes +- Migration guide +- Breaking changes list +- Timeline for v1 deprecation + +### 2. Deprecation Phase + +Mark old version as deprecated but keep it running: + +```http +GET /v1/users/123 + +Response: +Deprecation: true +Sunset: Wed, 15 Jan 2025 00:00:00 GMT +Link: ; rel="successor-version" + +{ + "id": 123, + "name": "John Doe" +} +``` + +**Deprecation Headers:** +- `Deprecation: true` - Indicates version is deprecated +- `Sunset: ` - When version will be removed (RFC 8594) +- `Link: ; rel="successor-version"` - Points to new version + +### 3. Sunset Phase + +Old version is shut down on announced date. + +Return 410 Gone for deprecated endpoints: +```http +GET /v1/users/123 + +Response: 410 Gone +{ + "error": { + "code": "VERSION_SUNSET", + "message": "API v1 was sunset on 2025-01-15. Please use v2.", + "documentation_url": "https://api.example.com/docs/migration-v1-to-v2" + } +} +``` + +## Deprecation Policy + +### Recommended Timeline + +1. **Announce deprecation** - At least 6 months before sunset +2. **Support period** - Run both versions for 6-12 months +3. **Sunset date** - Clear date communicated in advance +4. **Grace period** - 30 days of 410 Gone responses before complete shutdown + +### Communication Channels + +- API response headers +- Email to registered developers +- Blog posts and changelog +- Dashboard notifications +- Documentation updates +- Status page announcements + +## Migration Strategy + +### Provide Migration Guide + +```markdown +# Migrating from v1 to v2 + +## Breaking Changes + +### User Resource Changes + +**v1:** +```json +{ + "id": 123, + "name": "John Doe", + "email": "john@example.com" +} +``` + +**v2:** +```json +{ + "id": 123, + "first_name": "John", + "last_name": "Doe", + "email": "john@example.com" +} +``` + +**Migration:** +- Split `name` field into `first_name` and `last_name` +- Update client code to use new fields +``` + +### Offer Tools + +- Migration scripts +- SDK updates +- API diff viewer +- Compatibility layer (temporary) + +## Version Discovery + +### Root Endpoint + +```http +GET / + +Response: +{ + "versions": { + "v1": { + "status": "deprecated", + "sunset_date": "2025-01-15", + "documentation_url": "https://api.example.com/docs/v1" + }, + "v2": { + "status": "current", + "documentation_url": "https://api.example.com/docs/v2" + }, + "v3": { + "status": "beta", + "documentation_url": "https://api.example.com/docs/v3" + } + } +} +``` + +### Version Info Endpoint + +```http +GET /v2/version + +Response: +{ + "version": "v2", + "released": "2024-01-15", + "status": "stable", + "sunset_date": null +} +``` + +## OpenAPI Versioning + +### Separate Specs per Version + +``` +openapi-v1.yaml +openapi-v2.yaml +openapi-v3.yaml +``` + +Each spec is complete and independent. + +### Single Spec with Servers + +```yaml +openapi: 3.1.0 +info: + title: My API + version: 2.0.0 +servers: + - url: https://api.example.com/v1 + description: Version 1 (deprecated) + - url: https://api.example.com/v2 + description: Version 2 (current) +``` + +## Best Practices + +1. **Version from day one** - Start with /v1, not /api +2. **Major versions only** - Use v1, v2, v3 (not v1.1, v1.2) +3. **Long deprecation periods** - Give clients time to migrate (6-12 months) +4. **Clear communication** - Use headers, docs, emails +5. **Maintain old versions** - Support at least 2 versions simultaneously +6. **Document changes** - Provide detailed migration guides +7. **Use semantic versioning** - For internal/SDK versioning +8. **Never break without warning** - Always announce breaking changes +9. **Provide tools** - Migration scripts, updated SDKs +10. **Monitor usage** - Track which versions are being used + +## Anti-Patterns + +Avoid these mistakes: + +- **Breaking changes without version bump** - Breaks existing clients +- **Too many versions** - Maintenance nightmare (max 2-3 active versions) +- **Short deprecation periods** - Frustrates developers +- **No migration path** - Makes upgrades painful +- **Surprise sunsets** - Breaks production apps without warning +- **Inconsistent versioning** - Different strategies for different endpoints +- **Versioning individual endpoints** - Use consistent version across API diff --git a/skills/brainstorming/SKILL.md b/skills/brainstorming/SKILL.md new file mode 100644 index 0000000..b56a3b5 --- /dev/null +++ b/skills/brainstorming/SKILL.md @@ -0,0 +1,250 @@ +--- +name: brainstorming +description: "You MUST use this before any creative work - creating features, building components, adding functionality, or modifying behavior. Explores user intent, requirements and design before implementation." +--- + +# Brainstorming Ideas Into Designs + +Help turn ideas into fully formed designs and specs through natural collaborative dialogue. + +Start by classifying how much process the request needs, then work +through your path: understand the context, refine the idea, present a +design, and get your human partner's approval. + + +Do NOT invoke any implementation skill, write any code, scaffold any +project, or take any implementation action until you have told your +human partner what you intend and they have approved it. This applies +to EVERY task on EVERY path below — the ceremony scales with the task; +the approval gate never does. + + +## Three Paths + +Before your first question, classify the request and say the +classification out loud — "this looks bounded, so I'll present a short +design here rather than write a spec" — so your human partner can +override it: + +- **Spike** — a feasibility question ("can we...", "is it possible...", + "quick and dirty is fine") whose output is an answer, not code you + keep. Present the question and what you'll try in 2-3 sentences, get + a nod, then find out as cheaply as correctness allows. No design + doc, no spec file. Report findings as a recommendation; anything you + built stays labeled throwaway. +- **Bounded** — a well-scoped change to code that already exists in + this repo: a new flag, a small endpoint, a one-file fix. + Understanding the kind of app is not enough — bounded means the flow + you are changing is already here to read. If there is no existing + flow to change, the task is not bounded. Ask the clarifying + questions that matter, present a short design IN CHAT (a few + sentences to a few short paragraphs), and STOP. Implementation + starts only after your human partner says yes to that design — a + bounded task's approval is as hard a gate as an architectural + one. No spec file, no implementation plan document. +- **Architectural** — new projects, new subsystems, changes that + restructure how components fit together or alter interfaces others + depend on. Follow the full process: questions, approaches, sectioned + design, written spec, then the writing-plans skill. + +When in doubt between two paths, take the heavier one. The ratchet is +one-way: hidden complexity discovered mid-task upgrades the path — +stop, say so, and step up. Nothing downgrades mid-task. + +## Anti-Pattern: "Too Simple To Need Approval" + +Every path ends with your human partner approving your intent before +implementation. A todo list, a single-function utility, a config +change — the design may be two sentences in chat, but you MUST present +it and get approval. "Simple" tasks are where unexamined assumptions +cause the most wasted work. What scales with simplicity is the +artifact, never the approval. + +## Red Flags + +| Thought | Reality | +|---------|---------| +| "This is too simple to need a design" | Simple means a short design, not no design. Two sentences in chat, then approval. | +| "I'll call it bounded and skip the spec" | Reaching for a label to skip work IS the doubt — take the heavier path. | +| "It's bounded and the design is obvious — I'll start while they read it" | The gate is the approval, not the design's length. Present, then stop until you hear yes. | +| "I understand this kind of app, so it's bounded" | Bounded measures the repo, not your familiarity. A new project has no existing flow — it is architectural. | +| "The spike works, so I'll keep the code" | A spike's output is an answer. Keeping the code is a new request — classify it. | +| "It grew, but I'm almost done — no need to re-classify" | Hidden complexity upgrades the path mid-task. Stop and say so. | +| "They approved the spike, so the follow-up change is approved too" | Each task gets its own classification and its own approval. | + +## Checklist + +Classify first, announce the path, then create a task for each item on +your path and complete them in order. + +**Spike:** +1. **Explore project context** — enough to frame the probe +2. **Present question + probe plan** — 2-3 sentences +3. **Get approval** — a nod is enough +4. **Investigate** — as cheaply as correctness allows +5. **Report findings** — a recommendation; label anything built as throwaway + +**Bounded:** +1. **Explore project context** — check files, docs, recent commits +2. **Ask clarifying questions** — one at a time, the ones that matter +3. **Present short design in chat** — approach, files touched, testing +4. **Get approval** — STOP and wait for an explicit yes; presenting the design and starting in the same breath is skipping the gate +5. **Implement** — proceed with the normal development workflow (TDD applies); no plan document + +**Architectural:** +1. **Explore project context** — check files, docs, recent commits +2. **Offer the visual companion just-in-time** — NOT upfront. The first time a question would genuinely be clearer shown than described, offer it then (its own message); on approval its browser tab opens for you. If no visual question ever arises, never offer it. See the Visual Companion section below. +3. **Ask clarifying questions** — one at a time, understand purpose/constraints/success criteria +4. **Propose 2-3 approaches** — with trade-offs and your recommendation +5. **Present design** — in sections scaled to their complexity, get user approval after each section +6. **Write design doc** — save to `docs/superpowers/specs/YYYY-MM-DD--design.md` and commit +7. **Spec self-review** — quick inline check for placeholders, contradictions, ambiguity, scope (see below) +8. **User reviews written spec** — ask user to review the spec file before proceeding +9. **Transition to implementation** — invoke writing-plans skill to create implementation plan + +## Process Flow + +```dot +digraph brainstorming { + "Classify: spike / bounded / architectural" [shape=diamond]; + "Present question + probe (2-3 sentences)" [shape=box]; + "Ask clarifying questions (bounded)" [shape=box]; + "Present short design in chat" [shape=box]; + "Human approves?" [shape=diamond]; + "Investigate; report recommendation" [shape=doublecircle]; + "Implement via normal workflow (no plan doc)" [shape=doublecircle]; + "Explore project context" [shape=box]; + "Ask clarifying questions" [shape=box]; + "Propose 2-3 approaches" [shape=box]; + "Present design sections" [shape=box]; + "User approves design?" [shape=diamond]; + "Write design doc" [shape=box]; + "Spec self-review\n(fix inline)" [shape=box]; + "User reviews spec?" [shape=diamond]; + "Invoke writing-plans skill" [shape=doublecircle]; + "Hidden complexity? Upgrade path" [shape=box]; + + "Classify: spike / bounded / architectural" -> "Present question + probe (2-3 sentences)" [label="spike"]; + "Classify: spike / bounded / architectural" -> "Ask clarifying questions (bounded)" [label="bounded"]; + "Classify: spike / bounded / architectural" -> "Explore project context" [label="architectural"]; + "Present question + probe (2-3 sentences)" -> "Human approves?"; + "Ask clarifying questions (bounded)" -> "Present short design in chat"; + "Present short design in chat" -> "Human approves?"; + "Human approves?" -> "Investigate; report recommendation" [label="spike: yes"]; + "Human approves?" -> "Implement via normal workflow (no plan doc)" [label="bounded: yes"]; + "Hidden complexity? Upgrade path" -> "Classify: spike / bounded / architectural"; + "Explore project context" -> "Ask clarifying questions"; + "Ask clarifying questions" -> "Propose 2-3 approaches"; + "Propose 2-3 approaches" -> "Present design sections"; + "Present design sections" -> "User approves design?"; + "User approves design?" -> "Present design sections" [label="no, revise"]; + "User approves design?" -> "Write design doc" [label="yes"]; + "Write design doc" -> "Spec self-review\n(fix inline)"; + "Spec self-review\n(fix inline)" -> "User reviews spec?"; + "User reviews spec?" -> "Write design doc" [label="changes requested"]; + "User reviews spec?" -> "Invoke writing-plans skill" [label="approved"]; +} +``` + +**Terminal states are path-bound.** Architectural: the ONLY skill you +invoke after brainstorming is writing-plans — never frontend-design, +mcp-builder, or any other implementation skill. Bounded: after +approval, implementation proceeds directly through the normal +development workflow; no plan document. Spike: the terminal state is a +reported recommendation. + +## The Process + +The subsections below serve the bounded and architectural paths (a +spike stops at "present the probe, get a nod"). Sections from +**Exploring approaches** onward are architectural-path depth — for +bounded work, context plus a few questions plus a short in-chat design +is the whole process. + +**Understanding the idea:** + +- Check out the current project state first (files, docs, recent commits) +- Before asking detailed questions, assess scope: if the request describes multiple independent subsystems (e.g., "build a platform with chat, file storage, billing, and analytics"), flag this immediately. Don't spend questions refining details of a project that needs to be decomposed first. +- If the project is too large for a single spec, help the user decompose into sub-projects: what are the independent pieces, how do they relate, what order should they be built? Then brainstorm the first sub-project through the normal design flow. Each sub-project gets its own spec → plan → implementation cycle. +- For appropriately-scoped projects, ask questions one at a time to refine the idea +- Prefer multiple choice questions when possible, but open-ended is fine too +- Only one question per message - if a topic needs more exploration, break it into multiple questions +- Focus on understanding: purpose, constraints, success criteria + +**Exploring approaches:** + +- Propose 2-3 different approaches with trade-offs +- Present options conversationally with your recommendation and reasoning +- Lead with your recommended option and explain why +- YAGNI ruthlessly - remove unnecessary features from every approach and design + +**Presenting the design:** + +- Once you believe you understand what you're building, present the design +- Scale each section to its complexity: a few sentences if straightforward, up to 200-300 words if nuanced +- Ask after each section whether it looks right so far +- Cover: architecture, components, data flow, error handling, testing +- Be ready to go back and clarify if something doesn't make sense + +**Design for isolation and clarity:** + +- Break the system into smaller units that each have one clear purpose, communicate through well-defined interfaces, and can be understood and tested independently +- For each unit, you should be able to answer: what does it do, how do you use it, and what does it depend on? +- Can someone understand what a unit does without reading its internals? Can you change the internals without breaking consumers? If not, the boundaries need work. +- Smaller, well-bounded units are also easier for you to work with - you reason better about code you can hold in context at once, and your edits are more reliable when files are focused. When a file grows large, that's often a signal that it's doing too much. + +**Working in existing codebases:** + +- Explore the current structure before proposing changes. Follow existing patterns. +- Where existing code has problems that affect the work (e.g., a file that's grown too large, unclear boundaries, tangled responsibilities), include targeted improvements as part of the design - the way a good developer improves code they're working in. +- Don't propose unrelated refactoring. Stay focused on what serves the current goal. + +## After the Design (architectural path) + +**Documentation:** + +- Write the validated design (spec) to `docs/superpowers/specs/YYYY-MM-DD--design.md` + - (User preferences for spec location override this default) +- Use elements-of-style:writing-clearly-and-concisely skill if available +- Commit the design document to git + +**Spec Self-Review:** +After writing the spec document, look at it with fresh eyes: + +1. **Placeholder scan:** Any "TBD", "TODO", incomplete sections, or vague requirements? Fix them. +2. **Internal consistency:** Do any sections contradict each other? Does the architecture match the feature descriptions? +3. **Scope check:** Is this focused enough for a single implementation plan, or does it need decomposition? +4. **Ambiguity check:** Could any requirement be interpreted two different ways? If so, pick one and make it explicit. + +Fix any issues inline. No need to re-review — just fix and move on. + +**User Review Gate:** +After the spec review loop passes, ask the user to review the written spec before proceeding: + +> "Spec written and committed to ``. Please review it and let me know if you want to make any changes before we start writing out the implementation plan." + +Wait for the user's response. If they request changes, make them and re-run the spec review loop. Only proceed once the user approves. + +**Implementation:** + +- Invoke the writing-plans skill to create a detailed implementation plan +- Do NOT invoke any other skill. writing-plans is the next step. + +## Visual Companion + +A browser-based companion for showing mockups, diagrams, and visual options during brainstorming. Available as a tool — not a mode. Accepting the companion means it's available for questions that benefit from visual treatment; it does NOT mean every question goes through the browser. + +**Offering the companion (just-in-time):** Do NOT offer it upfront. Wait until a question would genuinely be clearer shown than told — a real mockup / layout / diagram question, not merely a UI *topic*. The first time that happens, offer it then, as its own message: +> "This next part might be easier if I show you — I can put together mockups, diagrams, and comparisons in a browser tab as we go. It's still new and can be token-intensive. Want me to? I'll open it for you." + +**This offer MUST be its own message.** Only the offer — no clarifying question, summary, or other content. Wait for the user's response. If they accept, start the server with `--open` so their browser opens to the first screen automatically. If they decline, continue text-only and don't offer again unless they raise it. + +**Per-question decision:** Even after the user accepts, decide FOR EACH QUESTION whether to use the browser or the terminal. The test: **would the user understand this better by seeing it than reading it?** + +- **Use the browser** for content that IS visual — mockups, wireframes, layout comparisons, architecture diagrams, side-by-side visual designs +- **Use the terminal** for content that is text — requirements questions, conceptual choices, tradeoff lists, A/B/C/D text options, scope decisions + +A question about a UI topic is not automatically a visual question. "What does personality mean in this context?" is a conceptual question — use the terminal. "Which wizard layout works better?" is a visual question — use the browser. + +If they agree to the companion, read the detailed guide before proceeding: +`skills/brainstorming/visual-companion.md` diff --git a/skills/brainstorming/scripts/start-server.sh b/skills/brainstorming/scripts/start-server.sh new file mode 100644 index 0000000..016a8e4 --- /dev/null +++ b/skills/brainstorming/scripts/start-server.sh @@ -0,0 +1,209 @@ +#!/usr/bin/env bash +# Start the brainstorm server and output connection info +# Usage: start-server.sh [--project-dir ] [--host ] [--url-host ] [--foreground] [--background] +# +# Starts server on a random high port, outputs JSON with URL. +# Each session gets its own directory to avoid conflicts. +# +# Options: +# --project-dir Store session files under /.superpowers/brainstorm/ +# instead of /tmp. Files persist after server stops. +# --host Host/interface to bind (default: 127.0.0.1). +# Use 0.0.0.0 in remote/containerized environments. +# --url-host Hostname shown in returned URL JSON. +# --idle-timeout-minutes Shut down after n minutes idle (default 240 = 4h). +# --open Auto-open the browser on the first screen (use only +# after the user approves the visual companion). +# --foreground Run server in the current terminal (no backgrounding). +# --background Force background mode (overrides Codex auto-foreground). + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" + +# Parse arguments +PROJECT_DIR="" +FOREGROUND="false" +FORCE_BACKGROUND="false" +BIND_HOST="127.0.0.1" +URL_HOST="" +IDLE_TIMEOUT_MINUTES="" +while [[ $# -gt 0 ]]; do + case "$1" in + --project-dir) + PROJECT_DIR="$2" + shift 2 + ;; + --host) + BIND_HOST="$2" + shift 2 + ;; + --url-host) + URL_HOST="$2" + shift 2 + ;; + --idle-timeout-minutes) + IDLE_TIMEOUT_MINUTES="$2" + shift 2 + ;; + --open) + export BRAINSTORM_OPEN=1 + shift + ;; + --foreground|--no-daemon) + FOREGROUND="true" + shift + ;; + --background|--daemon) + FORCE_BACKGROUND="true" + shift + ;; + *) + echo "{\"error\": \"Unknown argument: $1\"}" + exit 1 + ;; + esac +done + +if [[ -z "$URL_HOST" ]]; then + if [[ "$BIND_HOST" == "127.0.0.1" || "$BIND_HOST" == "localhost" ]]; then + URL_HOST="localhost" + else + URL_HOST="$BIND_HOST" + fi +fi + +if [[ -n "$IDLE_TIMEOUT_MINUTES" ]]; then + if ! [[ "$IDLE_TIMEOUT_MINUTES" =~ ^[0-9]+$ ]] || [[ "$IDLE_TIMEOUT_MINUTES" -lt 1 ]]; then + echo "{\"error\": \"--idle-timeout-minutes must be a positive integer\"}" + exit 1 + fi + export BRAINSTORM_IDLE_TIMEOUT_MS=$(( IDLE_TIMEOUT_MINUTES * 60 * 1000 )) +fi + +is_windows_like_shell() { + case "${OSTYPE:-}" in + msys*|cygwin*|mingw*) return 0 ;; + esac + if [[ -n "${MSYSTEM:-}" ]]; then + return 0 + fi + local uname_s + uname_s="$(uname -s 2>/dev/null || true)" + case "$uname_s" in + MSYS*|MINGW*|CYGWIN*) return 0 ;; + esac + return 1 +} + +# Some environments reap detached/background processes. Auto-foreground when detected. +if [[ -n "${CODEX_CI:-}" && "$FOREGROUND" != "true" && "$FORCE_BACKGROUND" != "true" ]]; then + FOREGROUND="true" +fi + +# Windows/Git Bash reaps nohup background processes. Auto-foreground when detected. +if [[ "$FOREGROUND" != "true" && "$FORCE_BACKGROUND" != "true" ]]; then + if is_windows_like_shell; then + FOREGROUND="true" + fi +fi + +# Session files (server.log, server-info, .last-token) embed the session key — +# keep everything this script and the server create owner-only. +umask 077 + +# Generate unique session directory +SESSION_ID="$$-$(date +%s)" + +if [[ -n "$PROJECT_DIR" ]]; then + SESSION_DIR="${PROJECT_DIR}/.superpowers/brainstorm/${SESSION_ID}" + # Persist the bound port and key per project so a restart reuses them and an + # already-open browser tab reconnects to the same URL with a valid cookie. + export BRAINSTORM_PORT_FILE="${PROJECT_DIR}/.superpowers/brainstorm/.last-port" + export BRAINSTORM_TOKEN_FILE="${PROJECT_DIR}/.superpowers/brainstorm/.last-token" +else + SESSION_DIR="/tmp/brainstorm-${SESSION_ID}" +fi + +STATE_DIR="${SESSION_DIR}/state" +PID_FILE="${STATE_DIR}/server.pid" +LOG_FILE="${STATE_DIR}/server.log" +SERVER_ID_FILE="${STATE_DIR}/server-instance-id" + +# Create fresh session directory with content and state peers +mkdir -p "${SESSION_DIR}/content" "$STATE_DIR" + +SERVER_ID="" +if [[ -r /dev/urandom ]]; then + SERVER_ID="$(od -An -N24 -tx1 /dev/urandom 2>/dev/null | tr -d ' \n' || true)" +fi +if ! [[ "$SERVER_ID" =~ ^[A-Za-z0-9_-]{32,64}$ ]]; then + SERVER_ID="$(printf '%08x%08x%08x%08x' "$$" "$(date +%s)" "${RANDOM:-0}" "${RANDOM:-0}")" +fi +printf '%s\n' "$SERVER_ID" > "$SERVER_ID_FILE" +chmod 600 "$SERVER_ID_FILE" 2>/dev/null || true + +# Kill any existing server +if [[ -f "$PID_FILE" ]]; then + old_pid=$(cat "$PID_FILE") + kill "$old_pid" 2>/dev/null + rm -f "$PID_FILE" +fi + +cd "$SCRIPT_DIR" || exit 1 + +# Resolve the harness PID (grandparent of this script). +# $PPID is the ephemeral shell the harness spawned to run us — it dies +# when this script exits. The harness itself is $PPID's parent. +OWNER_PID="$(ps -o ppid= -p "$PPID" 2>/dev/null | tr -d ' ')" +if [[ -z "$OWNER_PID" || "$OWNER_PID" == "1" ]]; then + OWNER_PID="$PPID" +fi + +# Windows/MSYS2: Node.js cannot see POSIX PIDs from the MSYS2 namespace. +# Passing a PID node cannot verify causes server to log owner-pid-invalid +# and self-terminate at the 60-second lifecycle check. Clear it so the +# watchdog is disabled and the idle timeout becomes the only shutdown trigger. +if is_windows_like_shell; then + OWNER_PID="" +fi + +# Foreground mode for environments that reap detached/background processes. +if [[ "$FOREGROUND" == "true" ]]; then + env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs "--brainstorm-server-id=$SERVER_ID" & + SERVER_PID=$! + echo "$SERVER_PID" > "$PID_FILE" + wait "$SERVER_PID" + exit $? +fi + +# Start server, capturing output to log file +# Use nohup to survive shell exit; disown to remove from job table +nohup env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs "--brainstorm-server-id=$SERVER_ID" > "$LOG_FILE" 2>&1 & +SERVER_PID=$! +disown "$SERVER_PID" 2>/dev/null +echo "$SERVER_PID" > "$PID_FILE" + +# Wait for server-started message (check log file) +for _ in {1..50}; do + if grep -q "server-started" "$LOG_FILE" 2>/dev/null; then + # Verify server is still alive after a short window (catches process reapers) + alive="true" + for _ in {1..20}; do + if ! kill -0 "$SERVER_PID" 2>/dev/null; then + alive="false" + break + fi + sleep 0.1 + done + if [[ "$alive" != "true" ]]; then + echo "{\"error\": \"Server started but was killed. Retry in a persistent terminal with: $SCRIPT_DIR/start-server.sh${PROJECT_DIR:+ --project-dir $PROJECT_DIR} --host $BIND_HOST --url-host $URL_HOST --foreground\"}" + exit 1 + fi + grep "server-started" "$LOG_FILE" | head -1 + exit 0 + fi + sleep 0.1 +done + +# Timeout - server didn't start +echo '{"error": "Server failed to start within 5 seconds"}' +exit 1 diff --git a/skills/brainstorming/scripts/stop-server.sh b/skills/brainstorming/scripts/stop-server.sh new file mode 100644 index 0000000..7cacfe9 --- /dev/null +++ b/skills/brainstorming/scripts/stop-server.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# Stop the brainstorm server and clean up +# Usage: stop-server.sh +# +# Kills the server process. Only deletes session directory if it's +# under /tmp (ephemeral). Persistent directories (.superpowers/) are +# kept so mockups can be reviewed later. + +SESSION_DIR="$1" + +if [[ -z "$SESSION_DIR" ]]; then + echo '{"error": "Usage: stop-server.sh "}' + exit 1 +fi + +STATE_DIR="${SESSION_DIR}/state" +PID_FILE="${STATE_DIR}/server.pid" +SERVER_ID_FILE="${STATE_DIR}/server-instance-id" + +mark_stopped() { + local reason="$1" + rm -f "${STATE_DIR}/server-info" + printf '{"reason":"%s","timestamp":%s}\n' "$reason" "$(date +%s)" > "${STATE_DIR}/server-stopped" +} + +read_expected_server_id() { + [[ -f "$SERVER_ID_FILE" ]] || return 1 + local id + id="$(tr -d '\r\n' < "$SERVER_ID_FILE" 2>/dev/null || true)" + [[ "$id" =~ ^[A-Za-z0-9_-]{32,64}$ ]] || return 1 + printf '%s\n' "$id" +} + +command_line_for_pid() { + local pid="$1" + if [[ -r "/proc/$pid/cmdline" ]]; then + tr '\0' '\n' < "/proc/$pid/cmdline" 2>/dev/null || true + return 0 + fi + ps -ww -p "$pid" -o command= 2>/dev/null || ps -f -p "$pid" 2>/dev/null | sed '1d' || true +} + +command_has_server_id() { + local pid="$1" + local expected="$2" + local expected_arg="--brainstorm-server-id=$expected" + if [[ -r "/proc/$pid/cmdline" ]]; then + local arg + while IFS= read -r -d '' arg || [[ -n "$arg" ]]; do + [[ "$arg" == "$expected_arg" ]] && return 0 + done < "/proc/$pid/cmdline" + return 1 + fi + local command_line + command_line="$(command_line_for_pid "$pid")" + [[ -n "$command_line" ]] || return 1 + case " $command_line " in + *" $expected_arg "*) return 0 ;; + *) return 1 ;; + esac +} + +# Confirm a PID has this session's per-start instance id, not just a familiar +# process name. Ambiguous or legacy metadata fails closed as stale_pid. +is_brainstorm_server() { + kill -0 "$1" 2>/dev/null || return 1 + local expected_id + expected_id="$(read_expected_server_id)" || return 1 + command_has_server_id "$1" "$expected_id" || return 1 + return 0 +} + +if [[ -f "$PID_FILE" ]]; then + pid=$(cat "$PID_FILE") + + # Refuse to signal a PID we can't prove is our server. A stale pid file may + # point at an unrelated process after a reboot/PID wraparound. + if ! is_brainstorm_server "$pid"; then + rm -f "$PID_FILE" "$SERVER_ID_FILE" + mark_stopped "stale_pid" + echo '{"status": "stale_pid"}' + exit 0 + fi + + # Try to stop gracefully, fallback to force if still alive + kill "$pid" 2>/dev/null || true + + # Wait for graceful shutdown (up to ~2s) + for _ in {1..20}; do + if ! kill -0 "$pid" 2>/dev/null; then + break + fi + sleep 0.1 + done + + # If still running, escalate to SIGKILL + if kill -0 "$pid" 2>/dev/null; then + kill -9 "$pid" 2>/dev/null || true + + # Give SIGKILL a moment to take effect + sleep 0.1 + fi + + if kill -0 "$pid" 2>/dev/null; then + echo '{"status": "failed", "error": "process still running"}' + exit 1 + fi + + rm -f "$PID_FILE" "$SERVER_ID_FILE" "${STATE_DIR}/server.log" + mark_stopped "stop-server.sh" + + # Only delete ephemeral /tmp directories + if [[ "$SESSION_DIR" == /tmp/* ]]; then + rm -rf "$SESSION_DIR" + fi + + echo '{"status": "stopped"}' +else + echo '{"status": "not_running"}' +fi diff --git a/skills/brainstorming/spec-document-reviewer-prompt.md b/skills/brainstorming/spec-document-reviewer-prompt.md new file mode 100644 index 0000000..6099312 --- /dev/null +++ b/skills/brainstorming/spec-document-reviewer-prompt.md @@ -0,0 +1,49 @@ +# Spec Document Reviewer Prompt Template + +Use this template when dispatching a spec document reviewer subagent. + +**Purpose:** Verify the spec is complete, consistent, and ready for implementation planning. + +**Dispatch after:** Spec document is written to docs/superpowers/specs/ + +``` +Subagent (general-purpose): + description: "Review spec document" + prompt: | + You are a spec document reviewer. Verify this spec is complete and ready for planning. + + **Spec to review:** [SPEC_FILE_PATH] + + ## What to Check + + | Category | What to Look For | + |----------|------------------| + | Completeness | TODOs, placeholders, "TBD", incomplete sections | + | Consistency | Internal contradictions, conflicting requirements | + | Clarity | Requirements ambiguous enough to cause someone to build the wrong thing | + | Scope | Focused enough for a single plan — not covering multiple independent subsystems | + | YAGNI | Unrequested features, over-engineering | + + ## Calibration + + **Only flag issues that would cause real problems during implementation planning.** + A missing section, a contradiction, or a requirement so ambiguous it could be + interpreted two different ways — those are issues. Minor wording improvements, + stylistic preferences, and "sections less detailed than others" are not. + + Approve unless there are serious gaps that would lead to a flawed plan. + + ## Output Format + + ## Spec Review + + **Status:** Approved | Issues Found + + **Issues (if any):** + - [Section X]: [specific issue] - [why it matters for planning] + + **Recommendations (advisory, do not block approval):** + - [suggestions for improvement] +``` + +**Reviewer returns:** Status, Issues (if any), Recommendations diff --git a/skills/brainstorming/visual-companion.md b/skills/brainstorming/visual-companion.md new file mode 100644 index 0000000..c145e64 --- /dev/null +++ b/skills/brainstorming/visual-companion.md @@ -0,0 +1,299 @@ +# Visual Companion Guide + +Browser-based visual brainstorming companion for showing mockups, diagrams, and options. + +## When to Use + +Decide per-question, not per-session. The test: **would the user understand this better by seeing it than reading it?** + +**Use the browser** when the content itself is visual: + +- **UI mockups** — wireframes, layouts, navigation structures, component designs +- **Architecture diagrams** — system components, data flow, relationship maps +- **Side-by-side visual comparisons** — comparing two layouts, two color schemes, two design directions +- **Design polish** — when the question is about look and feel, spacing, visual hierarchy +- **Spatial relationships** — state machines, flowcharts, entity relationships rendered as diagrams + +**Use the terminal** when the content is text or tabular: + +- **Requirements and scope questions** — "what does X mean?", "which features are in scope?" +- **Conceptual A/B/C choices** — picking between approaches described in words +- **Tradeoff lists** — pros/cons, comparison tables +- **Technical decisions** — API design, data modeling, architectural approach selection +- **Clarifying questions** — anything where the answer is words, not a visual preference + +A question *about* a UI topic is not automatically a visual question. "What kind of wizard do you want?" is conceptual — use the terminal. "Which of these wizard layouts feels right?" is visual — use the browser. + +## How It Works + +The server watches a directory for HTML files and serves the newest one to the browser. You write HTML content to `screen_dir`, the user sees it in their browser and can click to select options. Selections are recorded to `state_dir/events` that you read on your next turn. + +**Content fragments vs full documents:** If your HTML file starts with `/.superpowers/brainstorm/` for the session directory. + +**Note:** Pass the project root as `--project-dir` so mockups persist in `.superpowers/brainstorm/` and survive server restarts. Without it, files go to `/tmp` and get cleaned up. Remind the user to add `.superpowers/` to `.gitignore` if it's not already there. + +**Launching the server by platform:** + +**Claude Code:** +```bash +# Default mode works — the script backgrounds the server itself. +scripts/start-server.sh --project-dir /path/to/project --open +``` + +On Windows, the script auto-detects and switches to foreground mode (which blocks the tool call). Use `run_in_background: true` on the Bash tool call so the server survives across conversation turns, then read `$STATE_DIR/server-info` on the next turn to get the URL and port. + +**Codex:** +```bash +# Codex reaps background processes. The script auto-detects CODEX_CI and +# switches to foreground mode. Run it normally — no extra flags needed. +scripts/start-server.sh --project-dir /path/to/project --open +``` + +**Gemini CLI:** +```bash +# Use --foreground and set is_background: true on your shell tool call +# so the process survives across turns +scripts/start-server.sh --project-dir /path/to/project --open --foreground +``` + +**Copilot CLI:** +```bash +# Start it with Copilot CLI's non-blocking/background shell mechanism so the +# server survives across turns. Keep --foreground so the harness, not the +# script, owns backgrounding. The launcher is a .sh, so invoke it via bash +# (on Windows, call Git Bash's bash.exe from the PowerShell tool). +bash scripts/start-server.sh --project-dir /path/to/project --open --foreground +``` + +**Other environments:** The server must keep running in the background across conversation turns. If your environment reaps detached processes, use `--foreground` and launch the command with your platform's background execution mechanism. + +If the URL is unreachable from your browser (common in remote/containerized setups), bind a non-loopback host: + +```bash +scripts/start-server.sh \ + --project-dir /path/to/project \ + --host 0.0.0.0 \ + --url-host localhost +``` + +Use `--url-host` to control what hostname is printed in the returned URL JSON. + +## The Loop + +1. **Check server is alive**, then **write HTML** to a new file in `screen_dir`: + - **Required: confirm the server is alive before referring to the URL or pushing a screen.** Check that `$STATE_DIR/server-info` exists and `$STATE_DIR/server-stopped` does not. If it has shut down, restart it with `start-server.sh` using the **same `--project-dir`** — it reuses the same port, so the user's open tab reconnects on its own (it shows a "paused" overlay while the server is down) and you don't need to send a new URL. The server auto-exits after 4 hours idle (configurable with `--idle-timeout-minutes`). + - Use semantic filenames: `platform.html`, `visual-style.html`, `layout.html` + - **Never reuse filenames** — each screen gets a fresh file + - Use your file-creation tool — **never use cat/heredoc** (dumps noise into terminal) + - Server automatically serves the newest file + +2. **Tell user what to expect and end your turn:** + - Remind them of the URL (every step, not just first) + - Give a brief text summary of what's on screen (e.g., "Showing 3 layout options for the homepage") + - Ask them to respond in the terminal: "Take a look and let me know what you think. Click to select an option if you'd like." + +3. **On your next turn** — after the user responds in the terminal: + - Read `$STATE_DIR/events` if it exists — this contains the user's browser interactions (clicks, selections) as JSON lines + - Merge with the user's terminal text to get the full picture + - The terminal message is the primary feedback; `state_dir/events` provides structured interaction data + +4. **Iterate or advance** — if feedback changes current screen, write a new file (e.g., `layout-v2.html`). Only move to the next question when the current step is validated. + +5. **Unload when returning to terminal** — when the next step doesn't need the browser (e.g., a clarifying question, a tradeoff discussion), push a waiting screen to clear the stale content: + + ```html + +
+

Continuing in terminal...

+
+ ``` + + This prevents the user from staring at a resolved choice while the conversation has moved on. When the next visual question comes up, push a new content file as usual. + +6. Repeat until done. + +## Writing Content Fragments + +Write just the content that goes inside the page. The server wraps it in the frame template automatically (header, theme CSS, connection status, and all interactive infrastructure). + +**Minimal example:** + +```html +

Which layout works better?

+

Consider readability and visual hierarchy

+ +
+
+
A
+
+

Single Column

+

Clean, focused reading experience

+
+
+
+
B
+
+

Two Column

+

Sidebar navigation with main content

+
+
+
+``` + +That's it. No ``, no CSS, no `""" + + +def make_gif(pngs, out_path, fps, hold): + """Assemble PNG frame bytes into an animated GIF via Pillow. Skips with a + stderr warning (not fatal) if Pillow isn't installed.""" + try: + from PIL import Image + except ImportError: + sys.stderr.write("warning: Pillow not installed, skipping --gif (pip install Pillow)\n") + return + frames = [Image.open(io.BytesIO(p)).convert("RGB") for p in pngs] + duration = [int(1000 / fps)] * (len(frames) - 1) + [int(hold * 1000)] + frames[0].save(out_path, save_all=True, append_images=frames[1:], + duration=duration, loop=0) + sys.stderr.write(f"wrote {out_path} ({len(frames)} frames)\n") + + +def main(): + ap = argparse.ArgumentParser(description="Animate a .drawio building itself -> self-contained HTML player.") + ap.add_argument("file", help="input .drawio (uncompressed)") + ap.add_argument("-o", "--output", help="output .html (default: buildup.html alongside input)") + ap.add_argument("--gif", help="also assemble frames into an animated GIF (needs Pillow)") + ap.add_argument("--fps", type=float, default=2.0, help="GIF frames per second (default 2)") + ap.add_argument("--hold", type=float, default=1.5, help="seconds to hold the final GIF frame") + ap.add_argument("--keep-frames", action="store_true", help="also write the PNG frames next to the output") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + if not shutil.which("drawio"): + sys.exit("error: draw.io CLI not found on PATH (is the draw.io CLI installed?)") + + tree, cells = parse_page(args.file) + leaves, containers, edge_list = classify(cells) + if not leaves: + sys.exit(f"error: no revealable vertices found in {args.file}") + + order = reveal_order(leaves, [(s, t) for _, s, t in edge_list]) + node_step, edge_step = reveal_steps(order, edge_list) + width, height = bounding_box(cells) + labels = {c["id"]: label_of(c["el"]) for c in cells} + n_total = len(order) + + out = args.output or os.path.join( + os.path.dirname(os.path.abspath(args.file)) or ".", "buildup.html") + + frames = [] + with tempfile.TemporaryDirectory() as tmp: + for k in range(n_total): + revealed_nodes = set(order[:k + 1]) + revealed_edges = {eid for eid, _, _ in edge_list if edge_step[eid] <= k} + keep = {"0", "1"} | containers | revealed_nodes | revealed_edges + keep |= {c["id"] for c in cells if c["id"] not in keep and c["parent"] in keep} + + frame_tree = copy.deepcopy(tree) + model = frame_tree.getroot().find("diagram").find("mxGraphModel") + model.set("pageWidth", str(width)) + model.set("pageHeight", str(height)) + froot = model.find("root") + for child in list(froot): + if child.get("id") not in keep: + froot.remove(child) + + src = os.path.join(tmp, f"step{k:03d}.drawio") + frame_tree.write(src, encoding="utf-8", xml_declaration=False) + png_path = os.path.join(tmp, f"step{k:03d}.png") + r = subprocess.run(["drawio", "-x", "-f", "png", "--page-index", "1", + "--width", "2000", "-o", png_path, src], capture_output=True) + if r.returncode != 0 or not os.path.exists(png_path): + sys.stderr.write(f"warning: step {k + 1}/{n_total} export failed — skipped\n") + continue + with open(png_path, "rb") as f: + png = f.read() + label = labels.get(order[k], order[k]) + frames.append((png, label, k + 1, n_total)) + if args.keep_frames: + with open(f"{os.path.splitext(out)[0]}-frame{k + 1:03d}.png", "wb") as f: + f.write(png) + sys.stderr.write(f"[{k + 1}/{n_total}] revealed {label!r}\n") + + if not frames: + sys.exit("error: no frames exported (is the draw.io CLI installed?)") + + title = os.path.splitext(os.path.basename(args.file))[0] + " — build-up" + with open(out, "w", encoding="utf-8") as f: + f.write(build_html(frames, title)) + sys.stderr.write(f"wrote {out} ({len(frames)} frames)\n") + + if args.gif: + make_gif([f[0] for f in frames], args.gif, args.fps, args.hold) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/c4.py b/skills/drawio-skill/scripts/c4.py new file mode 100644 index 0000000..dab2bf1 --- /dev/null +++ b/skills/drawio-skill/scripts/c4.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python3 +"""C4 model diagrams: levels JSON -> one multi-page .drawio with drill-down. + +Generates a C4 architecture diagram set (System Context -> Containers -> +Components, as many levels as you define) in a single `.drawio` file: one +page per level, official draw.io C4 shapes and colors, Graphviz placement +per page (via autolayout), and **drill-down links** — an element with a +`"children"` key becomes clickable and jumps to that level's page in +draw.io / the diagrams.net viewer. + + python3 c4.py c4.json -o architecture.drawio + +Input JSON: + { + "title": "Internet Banking", + "levels": [ + { + "name": "System Context", + "elements": [ + {"id": "customer", "type": "person", "label": "Personal Customer", + "desc": "A customer of the bank"}, + {"id": "ibs", "type": "system", "label": "Internet Banking System", + "desc": "Lets customers manage accounts", "children": "Containers"}, + {"id": "email", "type": "external", "label": "E-mail System", + "desc": "Microsoft Exchange"} + ], + "relations": [ + {"from": "customer", "to": "ibs", "label": "Uses"}, + {"from": "ibs", "to": "email", "label": "Sends e-mail via"} + ] + }, + { + "name": "Containers", + "elements": [ + {"id": "spa", "type": "container", "label": "Single-Page App", + "tech": "React", "desc": "Banking UI in the browser"}, + {"id": "api", "type": "container", "label": "API Application", + "tech": "Java/Spring", "children": "Components"}, + {"id": "db", "type": "database", "label": "Database", + "tech": "PostgreSQL"} + ], + "relations": [ + {"from": "spa", "to": "api", "label": "JSON/HTTPS"}, + {"from": "api", "to": "db", "label": "JDBC"} + ] + } + ] + } + +Element types: person, system, external (greyed external system), container, +component, database. `tech` renders as the [Type: Tech] line, `desc` as the +description line — the standard C4 label. Element ids must be unique across +ALL levels (pages share one link namespace). Requires Graphviz `dot`. + +Usage: python3 c4.py [-o out.drawio] [--direction TB|LR] +""" +import argparse +import importlib.util +import json +import os +import re +import sys + +# Official draw.io C4 template styles (colors from c4model.com). +_BASE = "html=1;whiteSpace=wrap;fontSize=12;fontColor=#ffffff;align=center;" +STYLES = { + "person": ("shape=mxgraph.c4.person2;" + _BASE + + "fillColor=#083F75;strokeColor=#06315C;", 200, 180), + "system": ("rounded=1;arcSize=10;" + _BASE + + "fillColor=#1061B0;strokeColor=#0D5091;", 240, 120), + "external": ("rounded=1;arcSize=10;" + _BASE + + "fillColor=#8C8496;strokeColor=#736782;", 240, 120), + "container": ("rounded=1;arcSize=10;" + _BASE + + "fillColor=#23A2D9;strokeColor=#0E7DAD;", 240, 120), + "component": ("rounded=1;arcSize=10;" + _BASE + + "fillColor=#63BEF2;strokeColor=#2086C9;", 240, 120), + "database": ("shape=cylinder3;size=15;boundedLbl=1;" + _BASE + + "fillColor=#23A2D9;strokeColor=#0E7DAD;", 240, 120), +} +TYPE_WORD = {"person": "Person", "system": "Software System", + "external": "Software System", "container": "Container", + "component": "Component", "database": "Container"} +EDGE = ("endArrow=blockThin;endFill=1;endSize=10;html=1;fontSize=11;" + "fontColor=#404040;strokeColor=#828282;labelBackgroundColor=#ffffff;" + "rounded=0;") + + +def load_autolayout(): + path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "autolayout.py") + spec = importlib.util.spec_from_file_location("autolayout", path) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +def slug(name): + return re.sub(r"[^a-z0-9]+", "-", str(name).lower()).strip("-") or "page" + + +def c4_label(el): + """Standard C4 element label: Name / [Type: Tech] / description.""" + kind = TYPE_WORD.get(el.get("type", "system"), "Software System") + bracket = f"[{kind}: {el['tech']}]" if el.get("tech") else f"[{kind}]" + lines = [el.get("label", el["id"]), bracket] + if el.get("desc"): + lines.append(el["desc"]) + return "\n".join(lines) + + +def main(): + ap = argparse.ArgumentParser(description="C4 levels JSON -> multi-page draw.io.") + ap.add_argument("input", help="C4 JSON file") + ap.add_argument("-o", "--output", help="output .drawio path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + args = ap.parse_args() + with open(args.input, encoding="utf-8") as f: + spec = json.load(f) + levels = spec.get("levels") or [] + if not levels: + sys.exit("error: no levels in input") + al = load_autolayout() + + page_ids = {lv["name"]: slug(lv["name"]) for lv in levels} + seen = set() + pages = [] + for lv in levels: + nodes = [] + for el in lv.get("elements", []): + if el["id"] in seen: + sys.exit(f"error: duplicate element id {el['id']!r} " + "(ids must be unique across all levels)") + seen.add(el["id"]) + style, w, h = STYLES.get(el.get("type", "system"), STYLES["system"]) + node = {"id": el["id"], "label": c4_label(el), "style": style, + "width": w, "height": h} + child = el.get("children") + if child: + if child not in page_ids: + sys.exit(f"error: element {el['id']!r} drills down to " + f"unknown level {child!r}") + node["link"] = f"data:page/id,{page_ids[child]}" + nodes.append(node) + edges = [{"source": r["from"], "target": r["to"], + "label": r.get("label", ""), "style": EDGE} + for r in lv.get("relations", [])] + graph = {"direction": args.direction, "nodes": nodes, "edges": edges, + "ranksep": 0.9, "nodesep": 0.5} + height, pos, edge_pts = al.layout(al.build_dot(graph)) + pages.append(al.wrap_page(al.page_cells(graph, height, pos, edge_pts, color=False), + page_id=page_ids[lv["name"]], name=lv["name"])) + + xml = "\n" + "".join(pages) + "\n" + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(xml) + print(f"wrote {args.output} ({len(pages)} pages, {len(seen)} elements)", + file=sys.stderr) + else: + sys.stdout.write(xml) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/ciimports.py b/skills/drawio-skill/scripts/ciimports.py new file mode 100644 index 0000000..1c865ac --- /dev/null +++ b/skills/drawio-skill/scripts/ciimports.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +"""Extract a CI pipeline (GitHub Actions / GitLab CI) as autolayout graph JSON. + +GitHub Actions: every job becomes a node (label: name, runner, matrix size, +reusable-workflow target), `needs:` become edges, and each workflow gets a +trigger node (its `on:` events) feeding the jobs that have no `needs`. Given a +repo root, all of `.github/workflows/*.yml|yaml` are read and each workflow is +boxed in its own container. + +GitLab CI (`.gitlab-ci.yml`, auto-detected): jobs become nodes grouped by +stage; edges come from `needs:`, and jobs without `needs` inherit the stage +DAG (every job of the previous stage), matching GitLab's execution order. + + python3 ciimports.py . # repo root -> all workflows + python3 ciimports.py .github/workflows/ci.yml -o graph.json + python3 autolayout.py graph.json -o pipeline.drawio + +Requires PyYAML (pip install pyyaml). + +Usage: python3 ciimports.py [-o graph.json] + [--direction TB|LR] +""" +import argparse +import json +import os +import sys + +JOB_STYLE = "rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" +REUSE_STYLE = "rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" +TRIGGER_STYLE = "ellipse;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" + +GITLAB_RESERVED = {"stages", "variables", "workflow", "default", "include", "image", + "services", "before_script", "after_script", "cache", "pages"} + + +def find_workflows(path): + """Workflow files for a path: file(s) as-is, a repo root via .github/workflows.""" + if os.path.isfile(path): + return [path] + wfdir = os.path.join(path, ".github", "workflows") + files = sorted(os.path.join(wfdir, f) for f in os.listdir(wfdir) + if f.endswith((".yml", ".yaml"))) if os.path.isdir(wfdir) else [] + gitlab = os.path.join(path, ".gitlab-ci.yml") + if os.path.isfile(gitlab): + files.append(gitlab) + if not files: + sys.exit(f"error: no workflow files under {path}") + return files + + +def matrix_size(strategy): + n = 1 + matrix = (strategy or {}).get("matrix") or {} + if not isinstance(matrix, dict): + return 0 # dynamic (fromJSON) — unknown + for key, vals in matrix.items(): + if key not in ("include", "exclude") and isinstance(vals, list): + n *= len(vals) + n += len(matrix.get("include") or []) - len(matrix.get("exclude") or []) + return max(n, 1) + + +def parse_actions(spec, wf_id, wf_name, group): + """One GitHub Actions workflow -> (nodes, edges).""" + nodes, edges = [], [] + # YAML 1.1 quirk: bare `on:` parses as boolean True + on = spec.get("on", spec.get(True, {})) + events = sorted(on) if isinstance(on, dict) else \ + ([on] if isinstance(on, str) else sorted(on or [])) + trig_id = f"{wf_id}//trigger" + nodes.append({"id": trig_id, "label": "on: " + (", ".join(events) or "?"), + "style": TRIGGER_STYLE, "width": 160, "height": 50, "group": group}) + jobs = spec.get("jobs") or {} + for jid, job in jobs.items(): + job = job or {} + lines = [job.get("name") or jid] + if job.get("uses"): + lines.append("uses: " + os.path.basename(str(job["uses"]))) + style = REUSE_STYLE + else: + style = JOB_STYLE + runner = job.get("runs-on") + if runner: + lines.append(str(runner if isinstance(runner, str) else ", ".join(runner))) + n = matrix_size(job.get("strategy")) + if n > 1: + lines.append(f"matrix ×{n}") + elif n == 0: + lines.append("matrix (dynamic)") + nodes.append({"id": f"{wf_id}//{jid}", "label": "\n".join(lines), + "style": style, "width": 180, "height": 60, "group": group}) + needs = job.get("needs") or [] + needs = [needs] if isinstance(needs, str) else needs + for dep in needs: + if dep in jobs: + edges.append({"source": f"{wf_id}//{dep}", "target": f"{wf_id}//{jid}"}) + if not needs: + edges.append({"source": trig_id, "target": f"{wf_id}//{jid}"}) + return nodes, edges + + +def parse_gitlab(spec, wf_id, group_prefix): + """A .gitlab-ci.yml -> (nodes, edges); jobs grouped by stage.""" + stages = spec.get("stages") or ["build", "test", "deploy"] + jobs = {k: v for k, v in spec.items() + if isinstance(v, dict) and k not in GITLAB_RESERVED and not k.startswith(".") + and ("script" in v or "trigger" in v or "extends" in v or "stage" in v)} + nodes, edges = [], [] + by_stage = {} + for jid, job in jobs.items(): + stage = job.get("stage") or "test" + by_stage.setdefault(stage, []).append(jid) + nodes.append({"id": f"{wf_id}//{jid}", "label": jid, "style": JOB_STYLE, + "width": 160, "height": 50, + "group": f"{group_prefix}{stage}"}) + order = [s for s in stages if s in by_stage] + for jid, job in jobs.items(): + needs = [(n.get("job") if isinstance(n, dict) else n) for n in job.get("needs") or []] + needs = [n for n in needs if n in jobs] + if needs: + edges.extend({"source": f"{wf_id}//{n}", "target": f"{wf_id}//{jid}"} for n in needs) + else: # stage DAG: all jobs of the previous stage + stage = job.get("stage") or "test" + i = order.index(stage) if stage in order else 0 + if i > 0: + edges.extend({"source": f"{wf_id}//{p}", "target": f"{wf_id}//{jid}"} + for p in by_stage[order[i - 1]]) + return nodes, edges + + +def main(): + ap = argparse.ArgumentParser(description="CI pipeline -> autolayout graph JSON.") + ap.add_argument("paths", nargs="+", + help="repo root, or workflow file(s) (.github/workflows/*.yml, .gitlab-ci.yml)") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="LR", choices=["TB", "LR"]) + args = ap.parse_args() + + try: + import yaml + except ImportError: + sys.exit("error: PyYAML is required (pip install pyyaml)") + + files = [f for p in args.paths for f in find_workflows(p)] + nodes, edges = [], [] + for path in files: + with open(path, encoding="utf-8") as f: + try: + spec = yaml.safe_load(f) or {} + except yaml.YAMLError as e: + sys.stderr.write(f"warning: skipping {path}: {e}\n") + continue + wf_id = os.path.splitext(os.path.basename(path))[0] + if os.path.basename(path) == ".gitlab-ci.yml" or ( + "jobs" not in spec and "stages" in spec): + n, e = parse_gitlab(spec, wf_id, "stage: " if len(files) == 1 + else f"{wf_id} / stage: ") + elif spec.get("jobs"): + wf_name = spec.get("name") or wf_id + group = wf_name if len(files) > 1 else None + n, e = parse_actions(spec, wf_id, wf_name, group) + else: + sys.stderr.write(f"warning: {path} has no jobs — skipped\n") + continue + nodes.extend(n) + edges.extend(e) + if not nodes: + sys.exit("error: no CI jobs found") + + graph = {"direction": args.direction, "nodes": nodes, "edges": edges} + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + sys.stderr.write(f"{len(nodes)} nodes, {len(edges)} edges from {len(files)} file(s)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/composeimports.py b/skills/drawio-skill/scripts/composeimports.py new file mode 100644 index 0000000..0de3509 --- /dev/null +++ b/skills/drawio-skill/scripts/composeimports.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +"""Extract a docker-compose file's service graph as autolayout graph JSON. + +Services become rounded boxes (labeled name + image), named volumes become +cylinders, and edges come from real wiring: `depends_on` (list or mapping +form), `links`, `volumes_from`, and named-volume mounts (short "vol:/path" +and long {type: volume, source: ...} syntax). The output feeds autolayout.py: + + python3 composeimports.py docker-compose.yml -o graph.json + python3 autolayout.py graph.json -o stack.drawio + +Given a directory, the usual compose file names are tried +(compose.yaml/compose.yml/docker-compose.yml/docker-compose.yaml). +Requires PyYAML (pip install pyyaml). `--group` boxes services by their +first network. + +Usage: python3 composeimports.py [-o graph.json] + [--direction TB|LR] [--group] +""" +import argparse +import json +import os +import sys + +SERVICE_STYLE = "rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" +VOLUME_STYLE = ("shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;size=15;" + "fillColor=#f5f5f5;strokeColor=#666666;") + + +def find_compose(path): + if os.path.isfile(path): + return path + for name in ("compose.yaml", "compose.yml", "docker-compose.yml", "docker-compose.yaml"): + cand = os.path.join(path, name) + if os.path.isfile(cand): + return cand + sys.exit(f"error: no compose file found under {path}") + + +def volume_mounts(svc): + """Named volumes a service mounts (short and long syntax).""" + for v in svc.get("volumes") or []: + if isinstance(v, str): + src = v.split(":", 1)[0] + if src and not src.startswith((".", "/", "~", "$")): + yield src + elif isinstance(v, dict) and v.get("type", "volume") == "volume" and v.get("source"): + yield v["source"] + + +def dependencies(svc): + dep = svc.get("depends_on") or [] + deps = list(dep) if isinstance(dep, (list, dict)) else [] + for link in svc.get("links") or []: + deps.append(str(link).split(":", 1)[0]) + for vf in svc.get("volumes_from") or []: + deps.append(str(vf).split(":", 1)[0]) + return deps + + +def main(): + ap = argparse.ArgumentParser(description="docker-compose -> autolayout graph JSON.") + ap.add_argument("path", help="compose file, or directory containing one") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group services into containers by their first network") + args = ap.parse_args() + + try: + import yaml + except ImportError: + sys.exit("error: PyYAML is required (pip install pyyaml)") + path = find_compose(args.path) + with open(path, encoding="utf-8") as f: + spec = yaml.safe_load(f) or {} + services = spec.get("services") or {} + if not services: + sys.exit(f"error: no services in {path}") + declared_volumes = set(spec.get("volumes") or {}) + + nodes, edges = [], set() + for name, svc in services.items(): + svc = svc or {} + image = svc.get("image") or ("build: " + str((svc.get("build") or {}).get("context", ".") + if isinstance(svc.get("build"), dict) + else svc.get("build", "."))) + node = {"id": name, "label": f"{name}\n{image}", "style": SERVICE_STYLE, + "width": 160, "height": 60} + nets = svc.get("networks") + first_net = (sorted(nets)[0] if isinstance(nets, dict) else nets[0]) if nets else None + if args.group and first_net: + node["group"] = str(first_net) + nodes.append(node) + for dep in dependencies(svc): + if dep in services and dep != name: + edges.add((name, dep)) + for vol in volume_mounts(svc): + if vol in declared_volumes: + edges.add((name, f"vol:{vol}")) + for vol in sorted({t[4:] for _, t in edges if t.startswith("vol:")}): + nodes.append({"id": f"vol:{vol}", "label": vol, "style": VOLUME_STYLE, + "width": 120, "height": 70}) + + graph = {"direction": args.direction, "nodes": nodes, + "edges": [{"source": s, "target": t} for s, t in sorted(edges)]} + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + sys.stderr.write(f"{len(nodes)} nodes, {len(edges)} edges\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/compress.py b/skills/drawio-skill/scripts/compress.py new file mode 100644 index 0000000..ac4ce49 --- /dev/null +++ b/skills/drawio-skill/scripts/compress.py @@ -0,0 +1,257 @@ +#!/usr/bin/env python3 +"""Collapse a big .drawio into a boardroom-friendly executive summary. + +Detects clusters in a large diagram with a deterministic pure-Python label +propagation pass (no networkx), replaces each cluster with ONE labeled group +node, keeps aggregated inter-cluster edges, and emits a 2-page .drawio: page 1 +is the executive view (auto-laid-out via autolayout.py), page 2 is the +original full diagram, copied verbatim. Each executive node is wrapped in a +draw.io UserObject `data:page/id,...` drill-down link to page 2, so clicking +"Auth (5)" jumps straight into the full detail. + +Community detection is unsupervised — it finds however many clusters the +graph naturally has; `--clusters` is only a soft hint and may be ignored. +Clusters are named after the longest common leading token shared by their +members' labels (falling back to the highest-degree member's label), with the +member count appended, e.g. "Auth (5)". Rename them by hand afterward for a +more semantic label — label propagation does not know what your system does. + +Requires Graphviz `dot` on PATH (shells out to autolayout.py to place the +executive nodes). + + python3 compress.py big-system.drawio -o exec-view.drawio + +Usage: python3 compress.py [-o out.drawio] [--clusters N] +""" +import argparse +import copy +import json +import os +import re +import subprocess +import sys +import tempfile +import xml.etree.ElementTree as ET + +HERE = os.path.dirname(os.path.abspath(__file__)) + + +def parse(path): + """Return (nodes, edges) for a .drawio: nodes {id: (label, style)} for leaf + vertices, edges {(source_id, target_id)}. Cells are flattened across pages; + UserObject/object wrappers are unwrapped (id on the wrapper, cell inside). + Copied from drawiodiff.parse() — see SHARED CONVENTIONS.""" + try: + tree = ET.parse(path) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + cells, labels = [], {} + for page in pages: + model = page.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + if (page.text or "").strip(): + sys.stderr.write(f"warning: {path}: a page is compressed, skipped\n") + continue + for child in root: + if child.tag == "mxCell": + cells.append(child) + labels[child.get("id")] = child.get("value") or "" + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + cells.append(inner) + labels[child.get("id")] = child.get("label") or child.get("value") or "" + parents = {c.get("parent") for c in cells} # ids that have children + nodes, edges = {}, set() + for c in cells: + cid = c.get("id") + if c.get("edge") == "1": + s, t = c.get("source"), c.get("target") + if s and t: + edges.add((s, t)) + elif c.get("vertex") == "1" and cid not in parents: # leaf vertices only + if "edgeLabel" in (c.get("style") or ""): + continue + g = c.find("mxGeometry") + if g is not None and g.get("relative") == "1": # edge-label child + continue + nodes[cid] = (labels.get(cid, ""), c.get("style") or "") + return nodes, edges + + +def label_propagation(node_ids, edges, max_passes=20): + """Deterministic pure-Python label propagation for community detection. + + Edges are treated as undirected for clustering. Each pass computes every + node's new label synchronously from the PREVIOUS pass's labels (most + frequent label among neighbours, ties -> smallest label), then applies + them all at once — this keeps a thin bridge between two dense clusters + from cascading a merge within a single pass. Stops early once no label + changes, else after `max_passes`. Returns {node_id: community_label}. + """ + nodes = sorted(set(node_ids)) + neighbours = {n: set() for n in nodes} + for s, t in edges: + if s in neighbours and t in neighbours and s != t: + neighbours[s].add(t) + neighbours[t].add(s) + labels = {n: n for n in nodes} + for _ in range(max_passes): + new_labels = {} + for n in nodes: + if not neighbours[n]: + new_labels[n] = labels[n] + continue + counts = {} + for nb in neighbours[n]: + lbl = labels[nb] + counts[lbl] = counts.get(lbl, 0) + 1 + best = max(counts.values()) + new_labels[n] = min(lbl for lbl, c in counts.items() if c == best) + if new_labels == labels: + break + labels = new_labels + return labels + + +def compute_degree(node_ids, edges): + """Undirected degree per node (used as the naming tiebreak).""" + degree = {n: 0 for n in node_ids} + for s, t in edges: + if s in degree: + degree[s] += 1 + if t in degree: + degree[t] += 1 + return degree + + +def aggregate_edges(edges, community_of): + """Roll original edges up to inter-community edges: for every edge whose + endpoints fall in two different communities, count crossings by + (source_community, target_community) and dedupe into one entry per pair. + Same-community (internal) edges are dropped. Returns + {(src_community, tgt_community): crossing_count}.""" + counts = {} + for s, t in edges: + cs, ct = community_of.get(s), community_of.get(t) + if cs is None or ct is None or cs == ct: + continue + counts[(cs, ct)] = counts.get((cs, ct), 0) + 1 + return counts + + +def cluster_name(member_ids, node_labels, degree): + """Heuristic community name: the longest common leading token shared by + every member's label (split on whitespace), else the highest-degree + member's label. The member count is appended, e.g. "Auth (5)".""" + token_lists = [str(node_labels.get(m, m)).split() for m in member_ids] + common = [] + if token_lists and all(token_lists): + for tokens in zip(*token_lists): + if len(set(tokens)) == 1: + common.append(tokens[0]) + else: + break + if common: + base = " ".join(common) + else: + top = max(member_ids, key=lambda m: (degree.get(m, 0), m)) + base = node_labels.get(top) or top + return f"{base} ({len(member_ids)})" + + +def layout_exec_page(graph): + """Shell out to autolayout.py to place the executive nodes; return the + rendered ... page, renamed to a friendlier id/title.""" + with tempfile.TemporaryDirectory() as d: + gpath = os.path.join(d, "exec.json") + with open(gpath, "w", encoding="utf-8") as f: + json.dump(graph, f) + opath = os.path.join(d, "exec.drawio") + r = subprocess.run( + [sys.executable, os.path.join(HERE, "autolayout.py"), gpath, "-o", opath], + capture_output=True, text=True, + ) + if r.returncode != 0 or not os.path.exists(opath): + sys.exit(f"error: autolayout failed: {r.stderr.strip()}") + with open(opath, encoding="utf-8") as f: + xml = f.read() + m = re.search(r"()", xml, re.S) + if not m: + sys.exit("error: autolayout produced no page") + page = m.group(1).replace('id="autolayout"', 'id="exec-view"', 1) + page = page.replace('name="Page-1"', 'name="Executive View"', 1) + return page + "\n" + + +def copy_original_page(path, page2_id): + """Copy the source's first page verbatim (cells untouched) into a new + with id=page2_id, so exec-node drill-down links resolve to it.""" + try: + tree = ET.parse(path) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + page = copy.deepcopy(pages[0]) + if page.find("mxGraphModel/root") is None: + sys.exit(f"error: {path}: page is compressed (no ), cannot copy verbatim") + page.set("id", page2_id) + page.set("name", "Full Diagram") + return ET.tostring(page, encoding="unicode") + "\n" + + +def main(): + ap = argparse.ArgumentParser( + description="Collapse a big .drawio into an executive-summary view with drill-down.") + ap.add_argument("input", help="source .drawio") + ap.add_argument("-o", "--output", help="output .drawio path (default: stdout)") + ap.add_argument("--clusters", type=int, + help="soft hint for cluster count; label propagation picks the " + "count automatically and may ignore this") + args = ap.parse_args() + + if args.clusters: + sys.stderr.write("note: --clusters is a soft hint; label propagation " + "determines the actual cluster count automatically\n") + + nodes, edges = parse(args.input) + if not nodes: + sys.exit(f"error: no leaf vertices found in {args.input}") + + community_of = label_propagation(nodes.keys(), edges) + communities = {} + for nid in sorted(nodes): + communities.setdefault(community_of[nid], []).append(nid) + + degree = compute_degree(nodes.keys(), edges) + node_labels = {nid: label for nid, (label, _style) in nodes.items()} + names = {c: cluster_name(members, node_labels, degree) for c, members in communities.items()} + + crossings = aggregate_edges(edges, community_of) + + page2_id = "full-diagram" + exec_nodes = [{"id": f"c_{c}", "label": names[c], "link": f"data:page/id,{page2_id}"} + for c in communities] + exec_edges = [{"source": f"c_{s}", "target": f"c_{t}", "label": str(n) if n > 1 else ""} + for (s, t), n in sorted(crossings.items())] + exec_graph = {"direction": "TB", "nodes": exec_nodes, "edges": exec_edges} + + page1 = layout_exec_page(exec_graph) + page2 = copy_original_page(args.input, page2_id) + xml = "\n" + page1 + page2 + "\n" + + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(xml) + sys.stderr.write(f"wrote {args.output} ({len(nodes)} nodes -> " + f"{len(communities)} clusters)\n") + else: + sys.stdout.write(xml) + sys.stderr.write(f"{len(nodes)} nodes -> {len(communities)} clusters\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/dockerimports.py b/skills/drawio-skill/scripts/dockerimports.py new file mode 100644 index 0000000..9c74396 --- /dev/null +++ b/skills/drawio-skill/scripts/dockerimports.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""Draw the containers that are ACTUALLY running from `docker inspect` output. + +Where composeimports.py reads the *declared* stack (compose file), this reads +the *live* one: pipe `docker inspect` of the running containers and it maps the +real topology — every container, the user networks they are attached to, the +named volumes they mount, and the container->container edges recorded in +`links` / compose `depends_on` labels. The output feeds autolayout.py: + + docker inspect $(docker ps -q) | python3 dockerimports.py - -o graph.json + python3 autolayout.py graph.json -o running.drawio + +Input is the JSON array `docker inspect` prints (a file path, or `-` for +stdin). Containers become rounded boxes (name + image), user networks become +green ellipses, named volumes become cylinders — visually matching the compose +importer so declared and live diagrams read alike. `--group` boxes containers +by their compose project (falling back to their first user network). + +Usage: docker inspect $(docker ps -q) | python3 dockerimports.py - [-o graph.json] + [--direction TB|LR] [--group] +""" +import argparse +import json +import sys + +CONTAINER_STYLE = "rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" +NETWORK_STYLE = "ellipse;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" +VOLUME_STYLE = ("shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;size=15;" + "fillColor=#f5f5f5;strokeColor=#666666;") +# Docker's built-in networks are topology noise — a compose stack's own +# networks are what tell the architecture story. +BUILTIN_NETS = {"bridge", "host", "none", "ingress"} + + +def cname(obj): + """A container's short name (strip docker's leading slash).""" + return (obj.get("Name") or obj.get("Id", "")[:12]).lstrip("/") + + +def links_of(obj): + """Container names this one links to (HostConfig.Links + per-network Links).""" + out = set() + raw = list((obj.get("HostConfig") or {}).get("Links") or []) + for net in ((obj.get("NetworkSettings") or {}).get("Networks") or {}).values(): + raw.extend((net or {}).get("Links") or []) + for link in raw: + # "/db:/web/db" -> target container is the part before the first colon. + target = str(link).lstrip("/").split(":", 1)[0] + if target: + out.add(target) + return out + + +def depends_on(obj): + """Compose service names this container depends on (label form).""" + label = (obj.get("Config") or {}).get("Labels", {}).get("com.docker.compose.depends_on") + if not label: + return set() + # "db:service_healthy:false,cache:service_started:false" -> {db, cache} + return {part.split(":", 1)[0] for part in label.split(",") if part.strip()} + + +def main(): + ap = argparse.ArgumentParser(description="`docker inspect` output -> autolayout graph JSON.") + ap.add_argument("input", help="`docker inspect` JSON file, or - for stdin") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group containers by compose project (else first network)") + args = ap.parse_args() + + if args.input == "-": + text = sys.stdin.read() + else: + with open(args.input, encoding="utf-8") as f: + text = f.read() + try: + data = json.loads(text) + except json.JSONDecodeError as exc: + sys.exit(f"error: input is not valid JSON ({exc}) — feed `docker inspect ...`") + containers = data if isinstance(data, list) else [data] + containers = [c for c in containers if isinstance(c, dict) and c.get("Id")] + if not containers: + sys.exit("error: no containers found (feed `docker inspect $(docker ps -q)`)") + + names = {cname(c) for c in containers} + # compose service label -> container name, so depends_on (which names + # services) can resolve to the real container node. + svc_to_name = {} + for c in containers: + svc = (c.get("Config") or {}).get("Labels", {}).get("com.docker.compose.service") + if svc: + svc_to_name[svc] = cname(c) + + nodes, edges, nets, vols = [], set(), set(), set() + for c in containers: + name = cname(c) + image = (c.get("Config") or {}).get("Image") or "?" + labels = (c.get("Config") or {}).get("Labels", {}) or {} + node = {"id": name, "label": f"{name}\n{image}", "style": CONTAINER_STYLE, + "width": 160, "height": 60} + + attached = [n for n in ((c.get("NetworkSettings") or {}).get("Networks") or {}) + if n not in BUILTIN_NETS] + if args.group: + project = labels.get("com.docker.compose.project") + grp = project or (attached[0] if attached else None) + if grp: + node["group"] = str(grp) + nodes.append(node) + + for net in attached: + nets.add(net) + edges.add((name, f"net:{net}")) + for m in c.get("Mounts") or []: + if m.get("Type") == "volume" and m.get("Name"): + vols.add(m["Name"]) + edges.add((name, f"vol:{m['Name']}")) + for target in links_of(c): + if target in names and target != name: + edges.add((name, target)) + for dep in depends_on(c): + target = svc_to_name.get(dep, dep) + if target in names and target != name: + edges.add((name, target)) + + for net in sorted(nets): + nodes.append({"id": f"net:{net}", "label": net, "style": NETWORK_STYLE, + "width": 120, "height": 70}) + for vol in sorted(vols): + nodes.append({"id": f"vol:{vol}", "label": vol, "style": VOLUME_STYLE, + "width": 120, "height": 70}) + + graph = {"direction": args.direction, "nodes": nodes, + "edges": [{"source": s, "target": t} for s, t in sorted(edges)]} + out = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(out) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(out) + sys.stderr.write(f"{len(nodes)} nodes ({len(containers)} containers, " + f"{len(nets)} networks, {len(vols)} volumes), {len(edges)} edges\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/drawio2mermaid.py b/skills/drawio-skill/scripts/drawio2mermaid.py new file mode 100644 index 0000000..cd582af --- /dev/null +++ b/skills/drawio-skill/scripts/drawio2mermaid.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python3 +"""Convert a .drawio into Mermaid flowchart text (diagrams-as-code). + +The other reverse tool, `explain.py`, turns a diagram into prose; this turns it +into a **Mermaid `flowchart`** you can paste into a Markdown file and have +GitHub / GitLab / docs render natively — handy when you want the diagram to live +as maintainable text next to the code. Containers become `subgraph`s, edge +labels are kept, and a few shapes map to Mermaid node forms (cylinder → database +`[( )]`, rhombus → decision `{ }`, else `[ ]`). + + python3 drawio2mermaid.py architecture.drawio # Mermaid to stdout + python3 drawio2mermaid.py c4.drawio --fenced -o out.md # ```mermaid fenced + +Multi-page files emit one flowchart per page. This is a structural conversion — +styling, colours and vendor icons do not survive (Mermaid has no equivalent); +for a faithful, richly-styled diagram keep the `.drawio`. + +Usage: python3 drawio2mermaid.py [-o out] [--direction TD|LR] [--fenced] +""" +import argparse +import html +import re +import sys +import xml.etree.ElementTree as ET + + +def clean(text): + """Strip the HTML draw.io stores in labels; keep line breaks as
.""" + if not text: + return "" + text = re.sub(r"", "\n", text, flags=re.I) + text = re.sub(r"<[^>]+>", "", text) + text = html.unescape(text) + return re.sub(r"[ \t]+", " ", text).strip() + + +def esc(label): + """Mermaid-safe quoted label: escape quotes, newlines ->
.""" + label = label.replace('"', """).replace("\n", "
") + return label or " " + + +def node_form(safe_id, label, style): + """Mermaid node declaration, shape chosen from the draw.io style.""" + lbl = f'"{esc(label)}"' + if "shape=cylinder" in style or "shape=datastore" in style: + return f"{safe_id}[({lbl})]" # database + if "rhombus" in style: + return f"{safe_id}{{{lbl}}}" # decision + if "ellipse" in style or "shape=cloud" in style: + return f"{safe_id}(({lbl}))" # circle-ish + return f"{safe_id}[{lbl}]" # default box + + +def cells_of(page): + """(cell, id, label) for a page, unwrapping UserObject/object wrappers.""" + model = page.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + return None + out = [] + for child in root: + if child.tag == "mxCell": + out.append((child, child.get("id"), clean(child.get("value")))) + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + out.append((inner, child.get("id"), + clean(child.get("label") or child.get("value")))) + return out + + +def page_to_mermaid(page, direction): + cells = cells_of(page) + if cells is None: + return "%% (compressed page — skipped)" + label = {cid: lbl for _, cid, lbl in cells} + style = {cid: (c.get("style") or "") for c, cid, _ in cells} + parents = {c.get("parent") for c, _, _ in cells if c.get("parent")} + + verts = [(c, cid) for c, cid, _ in cells if c.get("vertex") == "1"] + containers = {cid for c, cid in verts if cid in parents} + leaves = [(c, cid) for c, cid in verts + if cid not in containers and "edgeLabel" not in style.get(cid, "")] + + sid = {cid: f"n{i}" for i, (_, cid) in enumerate(leaves)} # mermaid-safe ids + lines = [f"flowchart {direction}"] + + # Nodes, grouped into subgraphs by their container. + by_container = {} + for c, cid in leaves: + parent = c.get("parent") + key = parent if parent in containers else None + by_container.setdefault(key, []).append(cid) + + def emit_node(cid, indent): + lines.append(indent + node_form(sid[cid], label.get(cid) or cid, style.get(cid, ""))) + + for cid in by_container.get(None, []): + emit_node(cid, " ") + for cont, members in by_container.items(): + if cont is None: + continue + lines.append(f' subgraph {sid.get(cont, "g_" + cont)}["{esc(label.get(cont) or "")}"]') + for cid in members: + emit_node(cid, " ") + lines.append(" end") + + # Edges (only between leaves we emitted). + for c, _, _ in cells: + if c.get("edge") != "1": + continue + s, t = c.get("source"), c.get("target") + if s in sid and t in sid: + lbl = clean(c.get("value")) + arrow = f'-->|"{esc(lbl)}"|' if lbl else "-->" + lines.append(f" {sid[s]} {arrow} {sid[t]}") + return "\n".join(lines) + + +def main(): + ap = argparse.ArgumentParser(description="Convert a .drawio to Mermaid flowchart text.") + ap.add_argument("file") + ap.add_argument("-o", "--output", help="output path (default: stdout)") + ap.add_argument("--direction", default="TD", choices=["TD", "LR", "TB", "RL", "BT"]) + ap.add_argument("--fenced", action="store_true", help="wrap each graph in a ```mermaid fence") + args = ap.parse_args() + try: + root = ET.parse(args.file).getroot() + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {args.file}: {exc}") + pages = root.findall("diagram") or [root] + + blocks = [] + for i, page in enumerate(pages, 1): + graph = page_to_mermaid(page, args.direction) + name = page.get("name") + if len(pages) > 1 and name: + graph = f"%% Page {i}: {name}\n{graph}" + blocks.append(f"```mermaid\n{graph}\n```" if args.fenced else graph) + + text = ("\n\n".join(blocks)).rstrip() + "\n" + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output} ({len(pages)} page{'s' if len(pages) != 1 else ''})\n") + else: + sys.stdout.write(text) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/drawio2pptx.py b/skills/drawio-skill/scripts/drawio2pptx.py new file mode 100644 index 0000000..85b37f2 --- /dev/null +++ b/skills/drawio-skill/scripts/drawio2pptx.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python3 +"""Turn a .drawio into a PowerPoint deck — one slide per page. + +Exports each page of a (multi-page) .drawio to a PNG via the draw.io CLI and +lays them into a 16:9 .pptx, one page per slide, scaled to fit with the page +name as the slide title. A C4 model (Context / Container / Component) becomes a +ready-to-present deck; any multi-page diagram becomes a slide sequence. + + python3 drawio2pptx.py c4.drawio -o c4.pptx + python3 drawio2pptx.py architecture.drawio # -> architecture.pptx + +Needs the draw.io CLI (for the PNG export) and the `python-pptx` package +(`pip install python-pptx`) for writing the deck. Slides are 13.333in × 7.5in +(16:9); each image is centred and scaled to fit inside a small margin. + +Usage: python3 drawio2pptx.py [-o out.pptx] [--scale N] +""" +import argparse +import os +import struct +import subprocess +import sys +import tempfile +import xml.etree.ElementTree as ET + + +def page_names(path): + """Names of the pages, in order (None where unnamed).""" + try: + root = ET.parse(path).getroot() + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + diagrams = root.findall("diagram") + return [d.get("name") for d in diagrams] if diagrams else [None] + + +def png_size(path): + """(width, height) in pixels from a PNG's IHDR header.""" + with open(path, "rb") as f: + head = f.read(24) + return struct.unpack(">II", head[16:24]) + + +def export_page(drawio_file, index, out_png, scale): + """Export one page (1-based index) to PNG via the draw.io CLI.""" + r = subprocess.run(["drawio", "-x", "-f", "png", "--page-index", str(index), + "-s", str(scale), "-o", out_png, drawio_file], + capture_output=True) + return r.returncode == 0 and os.path.exists(out_png) + + +def main(): + ap = argparse.ArgumentParser(description="Export a .drawio to a PowerPoint deck (one slide per page).") + ap.add_argument("file") + ap.add_argument("-o", "--output", help="output .pptx (default: alongside input)") + ap.add_argument("--scale", type=float, default=2.0, help="PNG export scale (default 2)") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + try: + from pptx import Presentation + from pptx.util import Emu, Pt + except ImportError: + sys.exit("error: python-pptx is required (pip install python-pptx)") + + names = page_names(args.file) + out = args.output or os.path.splitext(args.file)[0] + ".pptx" + + prs = Presentation() + prs.slide_width = Emu(12192000) # 13.333in — 16:9 + prs.slide_height = Emu(6858000) # 7.5in + blank = prs.slide_layouts[6] # the built-in "Blank" layout + sw, sh = int(prs.slide_width), int(prs.slide_height) + margin = Emu(457200) # 0.5in + title_h = Emu(500000) + + made = 0 + with tempfile.TemporaryDirectory() as tmp: + for i, name in enumerate(names, 1): # draw.io --page-index is 1-based + png = os.path.join(tmp, f"page{i}.png") + if not export_page(args.file, i, png, args.scale): + sys.stderr.write(f"warning: page {i} export failed — skipped\n") + continue + slide = prs.slides.add_slide(blank) + top_pad = margin + if name: + box = slide.shapes.add_textbox(margin, Emu(180000), + Emu(sw - 2 * int(margin)), title_h) + tf = box.text_frame + tf.text = name + tf.paragraphs[0].runs[0].font.size = Pt(20) + tf.paragraphs[0].runs[0].font.bold = True + top_pad = Emu(180000) + title_h + + cw, ch = sw - 2 * int(margin), sh - int(top_pad) - int(margin) + pw, ph = png_size(png) + scale = min(cw / pw, ch / ph) # fit, preserve aspect + iw, ih = int(pw * scale), int(ph * scale) + left = Emu(int((sw - iw) / 2)) + top = Emu(int(top_pad) + int((ch - ih) / 2)) + slide.shapes.add_picture(png, left, top, width=Emu(iw), height=Emu(ih)) + made += 1 + + if not made: + sys.exit("error: no pages exported (is the draw.io CLI installed?)") + prs.save(out) + sys.stderr.write(f"wrote {out} ({made} slide{'s' if made != 1 else ''})\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/drawiodiff.py b/skills/drawio-skill/scripts/drawiodiff.py new file mode 100644 index 0000000..68f1e26 --- /dev/null +++ b/skills/drawio-skill/scripts/drawiodiff.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""Diff two .drawio diagrams into a colour-coded autolayout graph JSON. + +Compares an OLD and a NEW .drawio and emits a single graph where every node and +edge is tinted by what happened to it: + + added (in new only) -> green + removed (in old only) -> red, dashed + changed (matched, label moved) -> orange + same (matched, unchanged) -> grey + +The output is a normal graph JSON — feed it to autolayout.py for one clean, +freshly laid-out "what changed" diagram: + + python3 drawiodiff.py old.drawio new.drawio -o diff.json + python3 autolayout.py diff.json -o diff.drawio + +Nodes are matched by cell **id** (the default) — perfect for diagrams the +bundled importers generate, whose ids are stable semantic keys +(`aws_instance.web`, `shop-db-1`), so two snapshots line up exactly. This makes +it the natural companion to the live-infra importers: snapshot `terraform show +-json` / `docker inspect` / `kubectl get -o json` twice and diff the two to see +drift. For hand-drawn diagrams whose ids are random, pass `--by-label` to match +on the visible label text instead. + +Only leaf vertices and the edges between them are compared; container/group +cells and edge labels are skipped. The diff is a flat colour-coded view, so the +original icons/shapes are replaced by status colours (the label is kept). + +Usage: python3 drawiodiff.py [-o diff.json] + [--direction TB|LR] [--by-label] +""" +import argparse +import json +import sys +import xml.etree.ElementTree as ET + +STYLE = { + "added": "rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;", + "removed": "rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;dashed=1;", + "changed": "rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;", + "same": "rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;", +} +EDGE_STYLE = { + "added": "endArrow=classic;html=1;strokeColor=#82b366;strokeWidth=2;", + "removed": "endArrow=classic;html=1;strokeColor=#b85450;strokeWidth=2;dashed=1;", + "same": "endArrow=classic;html=1;strokeColor=#999999;", +} + + +def parse(path): + """Return (nodes, edges) for a .drawio: nodes {id: (label, style)} for leaf + vertices, edges {(source_id, target_id)}. Cells are flattened across pages; + UserObject/object wrappers are unwrapped (id on the wrapper, cell inside).""" + try: + tree = ET.parse(path) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + cells, labels = [], {} + for page in pages: + model = page.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + if (page.text or "").strip(): + sys.stderr.write(f"warning: {path}: a page is compressed, skipped\n") + continue + for child in root: + if child.tag == "mxCell": + cells.append(child) + labels[child.get("id")] = child.get("value") or "" + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + cells.append(inner) + labels[child.get("id")] = child.get("label") or child.get("value") or "" + parents = {c.get("parent") for c in cells} # ids that have children + nodes, edges = {}, set() + for c in cells: + cid = c.get("id") + if c.get("edge") == "1": + s, t = c.get("source"), c.get("target") + if s and t: + edges.add((s, t)) + elif c.get("vertex") == "1" and cid not in parents: # leaf vertices only + if "edgeLabel" in (c.get("style") or ""): + continue + g = c.find("mxGeometry") + if g is not None and g.get("relative") == "1": # edge-label child + continue + nodes[cid] = (labels.get(cid, ""), c.get("style") or "") + return nodes, edges + + +def main(): + ap = argparse.ArgumentParser(description="Diff two .drawio files -> autolayout graph JSON.") + ap.add_argument("old", help="baseline .drawio") + ap.add_argument("new", help="updated .drawio") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--by-label", action="store_true", + help="match nodes by visible label instead of cell id " + "(for hand-drawn diagrams with non-stable ids)") + args = ap.parse_args() + + old_n, old_e = parse(args.old) + new_n, new_e = parse(args.new) + + def keyed(nodes): + """Map match-key -> label. By id (default) the key is the cell id and the + value is its label; by label the key *is* the label.""" + if args.by_label: + return {lbl: lbl for lbl, _ in nodes.values()}, {i: lbl for i, (lbl, _) in nodes.items()} + return {i: lbl for i, (lbl, _) in nodes.items()}, {i: i for i in nodes} + + old_keys, old_id2key = keyed(old_n) + new_keys, new_id2key = keyed(new_n) + + nodes, counts = [], {"added": 0, "removed": 0, "changed": 0, "same": 0} + for key in sorted(set(old_keys) | set(new_keys)): + if key in old_keys and key not in new_keys: + status, label = "removed", old_keys[key] + elif key in new_keys and key not in old_keys: + status, label = "added", new_keys[key] + elif old_keys[key] != new_keys[key]: # matched, label moved + status, label = "changed", new_keys[key] + else: + status, label = "same", new_keys[key] + counts[status] += 1 + nodes.append({"id": key, "label": label or key, "style": STYLE[status], + "width": 160, "height": 60}) + + def edge_keys(edges, id2key): + out = set() + for s, t in edges: + if s in id2key and t in id2key: + out.add((id2key[s], id2key[t])) + return out + + old_ek, new_ek = edge_keys(old_e, old_id2key), edge_keys(new_e, new_id2key) + node_keys = {n["id"] for n in nodes} + edges = [] + for s, t in sorted(old_ek | new_ek): + if s not in node_keys or t not in node_keys: + continue + status = "same" if (s, t) in old_ek and (s, t) in new_ek else \ + ("added" if (s, t) in new_ek else "removed") + edges.append({"source": s, "target": t, "style": EDGE_STYLE[status]}) + + graph = {"direction": args.direction, "nodes": nodes, "edges": edges} + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + sys.stderr.write(f"+{counts['added']} added, -{counts['removed']} removed, " + f"~{counts['changed']} changed, ={counts['same']} unchanged\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/drawiohtml.py b/skills/drawio-skill/scripts/drawiohtml.py new file mode 100644 index 0000000..29bf3f1 --- /dev/null +++ b/skills/drawio-skill/scripts/drawiohtml.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Publish a .drawio as a single interactive HTML viewer. + +Exports every page to SVG via the draw.io CLI and inlines them into ONE +self-contained .html with pan (drag), zoom (wheel / buttons), page tabs, +node search, and working links — external links open normally and internal +page links ("data:page/id,…", e.g. a C4 model's drill-down) switch tabs +inside the viewer. Share the file with anyone: no draw.io, no server, +no external requests. + + python3 drawiohtml.py architecture.drawio -o architecture.html + python3 drawiohtml.py c4.drawio # -> c4.html, drill-down works + +Search matches node text (draw.io wraps every cell in ); +matches glow, Enter cycles through them and centres each. Internal page +links survive export by being rewritten to "#page-" fragments first +(draw.io drops raw data:page/id links from SVG). + +Usage: python3 drawiohtml.py [-o out.html] +""" +import argparse +import html +import json +import os +import re +import subprocess +import sys +import tempfile +import xml.etree.ElementTree as ET + +PAGE_LINK = "data:page/id," + + +def pages_of(path): + """[(id, name)] of the pages, in order.""" + try: + root = ET.parse(path).getroot() + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + return [(d.get("id") or f"p{i}", d.get("name") or f"Page {i + 1}") + for i, d in enumerate(root.findall("diagram"))] + + +def rewrite_page_links(tree): + """data:page/id,X links -> #page-X (fragments survive SVG export). Returns count.""" + n = 0 + for el in tree.getroot().iter(): + link = el.get("link") + if link and link.startswith(PAGE_LINK): + el.set("link", "#page-" + link[len(PAGE_LINK):]) + n += 1 + return n + + +def export_svg(drawio_file, index, out_svg): + """Export one page (1-based index) to SVG via the draw.io CLI.""" + r = subprocess.run(["drawio", "-x", "-f", "svg", "--embed-svg-images", + "--page-index", str(index), + "-o", out_svg, drawio_file], capture_output=True) + return r.returncode == 0 and os.path.exists(out_svg) + + +def strip_prolog(svg): + """Drop any XML declaration / doctype so the SVG can be inlined in HTML.""" + return re.sub(r"^\s*(<\?xml[^>]*\?>\s*|]*>\s*)*", "", svg) + + +def build_html(title, page_meta, svgs): + """One self-contained viewer page. page_meta = [(id, name)] aligned with svgs.""" + sections = "\n".join( + f'
{svg}
' + for (pid, _), svg in zip(page_meta, svgs)) + tabs = json.dumps([{"id": pid, "name": name} for pid, name in page_meta]) \ + .replace(" + +{html.escape(title)} +

{html.escape(title)}

+
+ + + + +
+
+{sections} +
+ +""" + + +def main(): + ap = argparse.ArgumentParser(description="Export a .drawio to a self-contained interactive HTML viewer.") + ap.add_argument("file") + ap.add_argument("-o", "--output", help="output .html (default: alongside input)") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + meta = pages_of(args.file) + if not meta: + sys.exit(f"error: no pages in {args.file}") + + tree = ET.parse(args.file) + relinked = rewrite_page_links(tree) + + svgs, kept = [], [] + with tempfile.TemporaryDirectory() as tmp: + src = args.file + if relinked: # export the rewritten copy instead + src = os.path.join(tmp, "relinked.drawio") + tree.write(src, encoding="utf-8", xml_declaration=False) + for i, (pid, name) in enumerate(meta, 1): # draw.io --page-index is 1-based + out = os.path.join(tmp, f"p{i}.svg") + if not export_svg(src, i, out): + sys.stderr.write(f"warning: page {i} ({name}) export failed — skipped\n") + continue + with open(out, encoding="utf-8") as f: + svgs.append(strip_prolog(f.read())) + kept.append((pid, name)) + + if not svgs: + sys.exit("error: no pages exported (is the draw.io CLI installed?)") + title = os.path.splitext(os.path.basename(args.file))[0] + out = args.output or os.path.splitext(args.file)[0] + ".html" + with open(out, "w", encoding="utf-8") as f: + f.write(build_html(title, kept, svgs)) + sys.stderr.write(f"wrote {out} ({len(svgs)} page{'s' if len(svgs) != 1 else ''}" + + (f", {relinked} drill-down link{'s' if relinked != 1 else ''}" if relinked else "") + + ")\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/edgeports.py b/skills/drawio-skill/scripts/edgeports.py new file mode 100644 index 0000000..987cbc9 --- /dev/null +++ b/skills/drawio-skill/scripts/edgeports.py @@ -0,0 +1,211 @@ +#!/usr/bin/env python3 +"""Assign edge connection points (ports) on an existing .drawio file. + +draw.io's floating connections attach every edge of a node to the middle of +whichever side faces the other endpoint. When a node has several connections +leaving the same side they all land on the same point, so the lines stack and +overlap — the usual complaint on swimlane / cross-functional flowcharts, where +handoff edges between lanes share long orthogonal corridors. + +This pass pins ``exitX/exitY`` and ``entryX/entryY`` instead: + +1. Resolve every vertex to absolute coordinates (through swimlane/container + parents, via ``validate.abs_rect``). +2. For each edge end, pick the side of the node that faces the other endpoint + (whichever of dx/dy dominates, measured centre-to-centre). +3. Group the ends by (node, side) and sort each group along the side by the far + endpoint's position across that axis. Sorting by the far endpoint is what + removes crossings: two edges leaving the same side keep their relative order + instead of swapping over each other. +4. Spread the group evenly over the side — k ends get slots 1/(k+1) .. k/(k+1). + +Only sides with 2+ ends are touched. Edges that already pin a port are left +alone, so hand-tuned geometry survives a re-run. Idempotent: running it twice +produces the same file. + +This is a *port* assignment, not a router — it fixes lines stacking at the +shape boundary, not an edge crossing an unrelated shape in the middle of its +run. For that, add waypoints (see references/xml-authoring.md). + +Usage: + python3 edgeports.py diagram.drawio # in place + python3 edgeports.py diagram.drawio -o routed.drawio + python3 edgeports.py diagram.drawio --dry-run # report only +""" + +import argparse +import importlib.util +import os +import sys +import xml.etree.ElementTree as ET + +_spec = importlib.util.spec_from_file_location( + "validate", os.path.join(os.path.dirname(os.path.abspath(__file__)), "validate.py")) +validate = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(validate) + +# Port coordinates per side. Each entry is (fixed_axis_value, varies_along_x). +# 'varies_along_x' says which coordinate the evenly-spaced slot fills in. +SIDES = { + "N": (0.0, True), # top edge: y=0, x varies + "S": (1.0, True), # bottom edge: y=1, x varies + "W": (0.0, False), # left edge: x=0, y varies + "E": (1.0, False), # right edge: x=1, y varies +} + + +def centre(r): + x, y, w, h = r + return (x + w / 2.0, y + h / 2.0) + + +def side_facing(src_rect, dst_rect): + """Which side of src faces dst: whichever of dx/dy dominates.""" + sx, sy = centre(src_rect) + dx_, dy_ = centre(dst_rect) + dx, dy = dx_ - sx, dy_ - sy + if abs(dx) >= abs(dy): + return "E" if dx >= 0 else "W" + return "S" if dy >= 0 else "N" + + +def has_port(style, end): + """True if the edge already pins this end's port (hand-tuned — leave it).""" + prefix = "exit" if end == "source" else "entry" + return (validate.style_num(style, prefix + "X") is not None + and validate.style_num(style, prefix + "Y") is not None) + + +def set_style(style, end, px, py): + """Return style with this end's port keys set, other keys order-preserved.""" + prefix = "exit" if end == "source" else "entry" + drop = {prefix + "X", prefix + "Y", prefix + "Dx", prefix + "Dy"} + parts = [p for p in (style or "").split(";") + if p and p.split("=", 1)[0] not in drop] + # Dx/Dy are perpendicular offsets in px; reset them so a re-run is stable. + parts += [f"{prefix}X={px:g}", f"{prefix}Y={py:g}", + f"{prefix}Dx=0", f"{prefix}Dy=0"] + return ";".join(parts) + ";" + + +def assign(cells, by_id): + """Compute {(edge_elem, end): (px, py)} for every end worth pinning.""" + rects = {} + for c in cells: + if c.get("vertex") == "1" and not validate.is_edge_label(c): + r = validate.abs_rect(c, by_id) + if r and not any(v != v for v in r): # NaN width/height guard + rects[c.get("id")] = r + + # Collect ends: one entry per (edge, end) whose node and peer are known. + groups = {} + for e in cells: + if e.get("edge") != "1": + continue + style = e.get("style") or "" + src, dst = e.get("source"), e.get("target") + if src not in rects or dst not in rects: + continue # dangling — validate.py's job + for end, me, peer in (("source", src, dst), ("target", dst, src)): + if has_port(style, end): + continue + side = side_facing(rects[me], rects[peer]) + groups.setdefault((me, side), []).append((e, end, rects[peer])) + + ports = {} + for (node_id, side), ends in groups.items(): + if len(ends) < 2: + continue # single edge: centre is fine + fixed, along_x = SIDES[side] + # Sort by the far endpoint's position across the axis we spread along. + # Tie-break on the other axis, then edge id, so the order is total and + # the output is deterministic. + ends.sort(key=lambda t: (centre(t[2])[0] if along_x else centre(t[2])[1], + centre(t[2])[1] if along_x else centre(t[2])[0], + t[0].get("id") or "")) + for i, (edge, end, _) in enumerate(ends): + slot = (i + 1) / float(len(ends) + 1) + ports[(edge, end)] = (slot, fixed) if along_x else (fixed, slot) + return ports + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("file", help="input .drawio") + ap.add_argument("-o", "--output", help="output path (default: edit in place)") + ap.add_argument("--dry-run", action="store_true", + help="report what would change, write nothing") + args = ap.parse_args() + + try: + tree = ET.parse(args.file) + except ET.ParseError as exc: + sys.exit(f"error: {args.file} is not parseable XML ({exc}). " + "Compressed .drawio files must be saved uncompressed first.") + + total = 0 + for model in tree.getroot().iter("mxGraphModel"): + cells = list(model.iter("mxCell")) + by_id = {c.get("id"): c for c in cells if c.get("id")} + ports = assign(cells, by_id) + for (edge, end), (px, py) in ports.items(): + edge.set("style", set_style(edge.get("style") or "", end, px, py)) + total += len(ports) + + if args.dry_run: + print(f"{total} edge end(s) would be pinned in {args.file}") + return + out = args.output or args.file + tree.write(out, encoding="utf-8", xml_declaration=False) + print(f"{total} edge end(s) pinned -> {out}") + + +def demo(): + """Self-check: three edges leaving one node's east side get distinct, + non-crossing ports, and a re-run is a no-op.""" + xml = """ + + + + + + + + + + + + + + + + """ + root = ET.fromstring(xml) + cells = list(root.iter("mxCell")) + by_id = {c.get("id"): c for c in cells if c.get("id")} + + # hub sits inside 'lane' (x=100), so its absolute x is 100, not 0. Without + # parent resolution every target would look like it was to the west. + assert validate.abs_rect(by_id["hub"], by_id)[0] == 100.0 + + ports = assign(cells, by_id) + exits = {e.get("id"): p for (e, end), p in ports.items() if end == "source"} + assert set(exits) == {"e1", "e2", "e3"}, exits # e4 pre-pinned, untouched + assert all(x == 1.0 for x, _ in exits.values()) # all leave the east side + ys = [exits[i][1] for i in ("e2", "e3", "e1")] # targets ordered top->bottom + assert ys == sorted(ys), ys # ports follow => no crossing + assert len(set(ys)) == 3, ys # and no two stack + + for (edge, end), (px, py) in ports.items(): + edge.set("style", set_style(edge.get("style"), end, px, py)) + assert not assign(cells, by_id), "second run must be a no-op" + print("ok") + + +if __name__ == "__main__": + if "--demo" in sys.argv: + demo() + else: + main() diff --git a/skills/drawio-skill/scripts/encode_drawio_url.py b/skills/drawio-skill/scripts/encode_drawio_url.py new file mode 100644 index 0000000..5523a1b --- /dev/null +++ b/skills/drawio-skill/scripts/encode_drawio_url.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +"""Encode a .drawio XML file into a diagrams.net browser URL. + +Used as the browser fallback when the draw.io desktop CLI is unavailable. +The diagram XML is carried in the URL fragment (after `#`), so nothing is +uploaded to any server. + +Two modes: + (default) read-only viewer -> https://viewer.diagrams.net/...#R + --edit editable editor -> https://app.diagrams.net/...#create= + +Usage: python3 encode_drawio_url.py [--edit] +""" +import base64 +import json +import sys +import urllib.parse +import zlib + + +def _deflate_b64(xml: str) -> str: + # draw.io's loader runs JS decodeURIComponent on the inflated string, so the + # XML MUST be percent-encoded (encodeURIComponent) BEFORE deflate — otherwise + # a literal `%` or any non-ASCII (e.g. CJK) label makes the browser throw + # "URI malformed" and the diagram never opens. encodeURIComponent leaves + # only A-Za-z0-9 and -_.!~*'() unescaped, which `quote` reproduces here. + pre = urllib.parse.quote(xml, safe="!~*'()") + c = zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS) + compressed = c.compress(pre.encode("utf-8")) + c.flush() + # Standard base64 (atob rejects url-safe -/_); strip newlines. + return base64.b64encode(compressed).decode("utf-8").replace("\n", "") + + +def encode(xml: str) -> str: + """Read-only viewer URL (mxGraph `#R` raw-inflate format).""" + return ( + "https://viewer.diagrams.net/?tags=%7B%7D&lightbox=1&edit=_blank#R" + + urllib.parse.quote(_deflate_b64(xml), safe="") + ) + + +def edit_url(xml: str) -> str: + """Editable editor URL — opens directly in the draw.io editor.""" + payload = json.dumps({"type": "xml", "compressed": True, "data": _deflate_b64(xml)}) + return ( + "https://app.diagrams.net/?grid=0&pv=0&border=10&edit=_blank#create=" + + urllib.parse.quote(payload, safe="") + ) + + +if __name__ == "__main__": + args = [a for a in sys.argv[1:] if a != "--edit"] + if len(args) != 1: + print("usage: encode_drawio_url.py [--edit] ", file=sys.stderr) + sys.exit(2) + with open(args[0], "r", encoding="utf-8") as f: + xml = f.read() + print(edit_url(xml) if "--edit" in sys.argv[1:] else encode(xml)) diff --git a/skills/drawio-skill/scripts/explain.py b/skills/drawio-skill/scripts/explain.py new file mode 100644 index 0000000..b2ad7ef --- /dev/null +++ b/skills/drawio-skill/scripts/explain.py @@ -0,0 +1,156 @@ +#!/usr/bin/env python3 +"""Read a .drawio and describe it as structured Markdown. + +The inverse of the skill's generators: instead of data -> diagram, this turns a +diagram -> documentation. It lists the components (grouped by their container / +swimlane / tier), the relations between them (edge labels become the relation +verb), and a per-page breakdown for multi-page files (e.g. a C4 model). Handy +for dropping an architecture summary into a README or PR, or for a text-only +description of a diagram someone handed you. + + python3 explain.py architecture.drawio # Markdown to stdout + python3 explain.py c4.drawio -o architecture.md + +Components are the leaf vertices; a vertex that contains others is treated as a +container and becomes a grouping heading. Relations read `source -> target`, +annotated with the edge label when present. A handful of common shapes are +named (data store, actor, decision, queue, cloud, and AWS/Azure/GCP/Kubernetes +vendor icons). UserObject/object wrappers are unwrapped; compressed pages are +reported but cannot be described (this skill always writes uncompressed XML). + +Usage: python3 explain.py [-o out.md] +""" +import argparse +import html +import re +import sys +import xml.etree.ElementTree as ET + +# style fragment -> human noun. First match wins; order matters (specific first). +SHAPE_TYPES = [ + ("mxgraph.aws", "AWS"), ("img/lib/azure", "Azure"), ("mxgraph.gcp", "GCP"), + ("mxgraph.kubernetes", "Kubernetes"), ("umlActor", "actor"), ("shape=actor", "actor"), + ("shape=cylinder", "data store"), ("shape=datastore", "data store"), + ("shape=cloud", "cloud"), ("rhombus", "decision"), ("mscae", "Azure"), + ("shape=process", "process"), ("shape=hexagon", "queue"), +] + + +def clean(text): + """Strip HTML tags/entities draw.io stores in labels; collapse whitespace.""" + if not text: + return "" + text = re.sub(r"", " ", text, flags=re.I) + text = re.sub(r"<[^>]+>", "", text) + return re.sub(r"\s+", " ", html.unescape(text)).strip() + + +def shape_of(style): + for frag, noun in SHAPE_TYPES: + if frag in (style or ""): + return noun + return None + + +def cells_of(page): + """[(cell, id, label)] for a page, unwrapping UserObject/object wrappers.""" + model = page.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + return None # compressed / empty page + out = [] + for child in root: + if child.tag == "mxCell": + out.append((child, child.get("id"), clean(child.get("value")))) + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + out.append((inner, child.get("id"), + clean(child.get("label") or child.get("value")))) + return out + + +def describe_page(page): + """Markdown body lines for one page (no page heading).""" + cells = cells_of(page) + if cells is None: + return ["_(compressed page — cannot describe)_"] + label = {cid: lbl for _, cid, lbl in cells} + style = {cid: (c.get("style") or "") for c, cid, _ in cells} + parents = {c.get("parent") for c, _, _ in cells if c.get("parent")} + + vertices = [(c, cid) for c, cid, _ in cells if c.get("vertex") == "1"] + containers = {cid for c, cid in vertices if cid in parents} # holds other cells + leaves = [(c, cid) for c, cid in vertices + if cid not in containers and "edgeLabel" not in style.get(cid, "")] + + # Group leaves by their container's label (else "Ungrouped"). + groups, order = {}, [] + for c, cid in leaves: + parent = c.get("parent") + gname = label.get(parent) or "" if parent in containers else "" + gname = gname or "Ungrouped" + if gname not in groups: + groups[gname] = [] + order.append(gname) + typ = shape_of(style.get(cid, "")) + name = label.get(cid) or f"(unlabeled {cid})" + groups[gname].append(f"{name}" + (f" _{typ}_" if typ else "")) + + lines = [f"### Components ({len(leaves)})", ""] + single = len(order) == 1 and order[0] == "Ungrouped" + for gname in order: + if not single: + lines.append(f"- **{gname}**") + lines += [f" - {item}" for item in groups[gname]] + else: + lines += [f"- {item}" for item in groups[gname]] + lines.append("") + + edges = [c for c, _, _ in cells if c.get("edge") == "1"] + rels = [] + for e in edges: + s, t = label.get(e.get("source")), label.get(e.get("target")) + if not s or not t: # dangling endpoint — skip + continue + verb = clean(e.get("value")) + rels.append(f"- {s} —{verb}→ {t}" if verb else f"- {s} → {t}") + lines.append(f"### Relations ({len(rels)})") + lines.append("") + lines += rels or ["_(none)_"] + lines.append("") + return lines + + +def main(): + ap = argparse.ArgumentParser(description="Describe a .drawio diagram as Markdown.") + ap.add_argument("file") + ap.add_argument("-o", "--output", help="output Markdown path (default: stdout)") + args = ap.parse_args() + try: + tree = ET.parse(args.file) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {args.file}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + + title = args.file.rsplit("/", 1)[-1].rsplit(".", 1)[0] + lines = [f"# {title}", ""] + for i, page in enumerate(pages, 1): + name = page.get("name") + if len(pages) > 1: + lines.append(f"## Page {i}: {name}" if name else f"## Page {i}") + lines.append("") + lines += describe_page(page) + + text = "\n".join(lines).rstrip() + "\n" + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/goimports.py b/skills/drawio-skill/scripts/goimports.py new file mode 100644 index 0000000..ae135c2 --- /dev/null +++ b/skills/drawio-skill/scripts/goimports.py @@ -0,0 +1,144 @@ +#!/usr/bin/env python3 +"""Extract a Go module's package-import graph as autolayout graph JSON. + +The Go counterpart to pyimports.py / jsimports.py. Reads the module path from +go.mod, walks the module, treats each directory of .go files as one package +(node = its import path), and records the intra-module package imports. Stdlib +and third-party imports are ignored. Transitive reduction is on by default so +the diagram stays readable. + + python3 goimports.py ./mymodule -o graph.json + python3 autolayout.py graph.json -o diagram.drawio + +Parsing is regex-based over `import` statements (single and block form), +which is enough for a structural package graph. *_test.go files and vendor/ +are skipped. + +Usage: python3 goimports.py [-o graph.json] [--direction TB|LR] + [--group] [--no-reduce] +""" +import argparse +import json +import os +import re +import subprocess +import sys + +MODULE = re.compile(r"^module\s+(\S+)", re.MULTILINE) +BLOCK = re.compile(r"import\s*\((.*?)\)", re.DOTALL) +SINGLE = re.compile(r'import\s+(?:[\w.]+\s+|_\s+)?"([^"]+)"') +QUOTED = re.compile(r'"([^"]+)"') + + +def module_path(root): + """Read the `module` path from go.mod at the module root, or None.""" + gomod = os.path.join(root, "go.mod") + if not os.path.exists(gomod): + return None + with open(gomod, encoding="utf-8", errors="ignore") as f: + m = MODULE.search(f.read()) + return m.group(1) if m else None + + +def discover(root, modpath): + """Map package import path -> list of .go files (one entry per directory).""" + root = os.path.abspath(root) + pkgs = {} + for dirpath, dirs, files in os.walk(root): + dirs[:] = [d for d in dirs + if d not in ("vendor", "testdata") and not d.startswith(".")] + gofiles = [os.path.join(dirpath, f) for f in files + if f.endswith(".go") and not f.endswith("_test.go")] + if not gofiles: + continue + rel = os.path.relpath(dirpath, root).replace(os.sep, "/") + ip = modpath if rel == "." else f"{modpath}/{rel}" + pkgs[ip] = gofiles + return pkgs + + +def imports_of(files, modpath, pkgs): + """Intra-module package import paths referenced by a package's files.""" + found = set() + for path in files: + try: + with open(path, encoding="utf-8", errors="ignore") as f: + src = f.read() + except OSError: + continue + specs = [] + for block in BLOCK.findall(src): + specs += QUOTED.findall(block) + specs += SINGLE.findall(src) + for spec in specs: + if (spec == modpath or spec.startswith(modpath + "/")) and spec in pkgs: + found.add(spec) + return found + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def main(): + ap = argparse.ArgumentParser(description="Go import graph -> autolayout graph JSON.") + ap.add_argument("module", help="module directory (contains go.mod)") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group nodes into containers by top-level package dir") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + args = ap.parse_args() + + modpath = module_path(args.module) + if not modpath: + sys.exit(f"error: no go.mod with a module path found in {args.module}") + pkgs = discover(args.module, modpath) + if not pkgs: + sys.exit(f"error: no Go packages found under {args.module}") + edges = sorted({(ip, t) for ip, files in pkgs.items() + for t in imports_of(files, modpath, pkgs) if t != ip}) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce(list(pkgs), edges) + # Drop the module prefix from labels for readability; ids stay full. + strip = modpath + "/" + label = lambda ip: ip[len(strip):] if ip.startswith(strip) else os.path.basename(ip) + + def node(ip): + d = {"id": ip, "label": label(ip)} + if args.group: + rest = label(ip).split("/") + if len(rest) > 1: # nested under a sub-package + d["group"] = "/".join(rest[:-1]) # full sub-package path -> nested boxes + return d + + graph = { + "direction": args.direction, + "nodes": [node(ip) for ip in pkgs], + "edges": [{"source": s, "target": t} for s, t in edges], + } + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(pkgs)} packages, {len(edges)} edges{note}\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/heatmap.py b/skills/drawio-skill/scripts/heatmap.py new file mode 100644 index 0000000..d65d463 --- /dev/null +++ b/skills/drawio-skill/scripts/heatmap.py @@ -0,0 +1,226 @@ +#!/usr/bin/env python3 +"""Colour a diagram by data — turn a .drawio into a metric heatmap. + +Reads a .drawio and a metrics file (CSV `key,value` or JSON `{key: value}`), +matches each metric key to a node by cell id or by its label text, and recolours +that node along a gradient from the lowest value to the highest. Optionally +scales node size by value (`--size`) and drops in a legend. The result is a new +.drawio you can export like any other — an architecture diagram that now shows +cost / latency / traffic / error-rate as a heat map. + + python3 heatmap.py architecture.drawio --metrics latency.csv -o hot.drawio + python3 heatmap.py architecture.drawio -m cost.json --palette cool --size + +Metrics match on cell id first, then on the (HTML-stripped) label, case- +insensitively. Unmatched nodes keep their original style. `--palette` picks the +colour ramp (heat|cool|warm), `--reverse` flips it (so low = hot). + +Usage: python3 heatmap.py --metrics [-o out.drawio] + [--palette heat|cool|warm] [--reverse] [--size] [--no-legend] +""" +import argparse +import csv +import html +import json +import os +import re +import sys +import xml.etree.ElementTree as ET + +# Sequential ramps as (low, mid, high) anchor colours; value is lerped across them. +PALETTES = { + "heat": ("#57bb8a", "#ffd666", "#e67c73"), # green -> yellow -> red + "cool": ("#deebf7", "#6baed6", "#08519c"), # light -> deep blue + "warm": ("#fff7bc", "#fec44f", "#d95f0e"), # pale -> deep amber +} + + +def clean(text): + """Strip the HTML tags/entities draw.io stores in labels; collapse whitespace.""" + if not text: + return "" + text = re.sub(r"", " ", text, flags=re.I) + text = re.sub(r"<[^>]+>", "", text) + return re.sub(r"\s+", " ", html.unescape(text)).strip() + + +def _num(x): + try: + return float(x) + except (TypeError, ValueError): + return None + + +def load_metrics(path): + """{key: float} from a JSON object/list or a CSV/TSV (first col key, last numeric col value).""" + with open(path, encoding="utf-8") as f: + text = f.read() + if path.lower().endswith(".json"): + data = json.loads(text) + if isinstance(data, dict): + return {str(k): _num(v) for k, v in data.items() if _num(v) is not None} + out = {} + for row in data: + key = row.get("id") or row.get("key") or row.get("name") or row.get("label") + val = _num(row.get("value", row.get("val", row.get("metric")))) + if key is not None and val is not None: + out[str(key)] = val + return out + delim = "\t" if text.splitlines() and "\t" in text.splitlines()[0] else "," + out = {} + for row in csv.reader(text.splitlines(), delimiter=delim): + if len(row) < 2: + continue + v = _num(row[-1]) + if v is not None: # skip header / non-numeric rows + out[row[0].strip()] = v + return out + + +def _rgb(c): + c = c.lstrip("#") + return tuple(int(c[i:i + 2], 16) for i in (0, 2, 4)) + + +def ramp(anchors, t): + """t in [0,1] -> #rrggbb across a 3-stop (low, mid, high) ramp.""" + lo, mid, hi = (_rgb(c) for c in anchors) + a, b, u = (lo, mid, t * 2) if t < 0.5 else (mid, hi, (t - 0.5) * 2) + return "#%02x%02x%02x" % tuple(round(a[i] + (b[i] - a[i]) * u) for i in range(3)) + + +def darker(hexcolor, f=0.6): + return "#%02x%02x%02x" % tuple(round(c * f) for c in _rgb(hexcolor)) + + +def set_style(style, fill, stroke): + """Replace/insert fillColor & strokeColor in a draw.io style string.""" + s = re.sub(r"(fillColor|strokeColor)=[^;]*;?", "", style or "").strip("; ") + return (s + ";" if s else "") + f"fillColor={fill};strokeColor={stroke};" + + +def vertices(root): + """Yield (cell, id, label) for every vertex, unwrapping UserObject/object wrappers.""" + for child in root: + if child.tag == "mxCell" and child.get("vertex") == "1": + yield child, child.get("id"), clean(child.get("value")) + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None and inner.get("vertex") == "1": + yield inner, child.get("id"), clean(child.get("label") or child.get("value")) + + +def scale_geom(cell, factor): + """Grow/shrink a cell about its centre by `factor`.""" + g = cell.find("mxGeometry") + if g is None: + return + w, h = _num(g.get("width")), _num(g.get("height")) + if w is None or h is None: + return + nw, nh = w * factor, h * factor + g.set("width", "%g" % nw) + g.set("height", "%g" % nh) + for attr, delta in (("x", (nw - w) / 2), ("y", (nh - h) / 2)): + v = _num(g.get(attr)) + if v is not None: + g.set(attr, "%g" % (v - delta)) + + +def color_for(val, lo, hi, anchors, reverse): + t = (val - lo) / (hi - lo) if hi > lo else 0.5 + fill = ramp(anchors, 1 - t if reverse else t) + return fill, darker(fill), t + + +def content_bounds(root): + """(min_x, min_y) of the page's top-level cells, so the legend sits clear of them.""" + xs, ys = [], [] + for child in root: + cell = child if child.tag == "mxCell" else child.find("mxCell") + if cell is None or cell.get("parent") != "1": # skip nested (relative-coord) cells + continue + g = cell.find("mxGeometry") + x, y = (_num(g.get("x")), _num(g.get("y"))) if g is not None else (None, None) + if x is not None and y is not None: + xs.append(x) + ys.append(y) + return (min(xs) if xs else 20, min(ys) if ys else 20) + + +def add_legend(root, anchors, lo, hi, reverse): + """Drop a small min/mid/max swatch legend just left of the page's content.""" + w, h = 90, 26 + cx, cy = content_bounds(root) + x0, y0 = cx - w - 40, cy # to the left of the diagram + + def cell(cid, value, style, gy, gh): + c = ET.SubElement(root, "mxCell", {"id": cid, "value": value, "style": style, + "vertex": "1", "parent": "1"}) + ET.SubElement(c, "mxGeometry", {"x": "%g" % x0, "y": "%g" % gy, + "width": "%g" % w, "height": "%g" % gh, "as": "geometry"}) + + cell("hm-title", "Heatmap", + "text;html=1;fontStyle=1;align=left;verticalAlign=middle;", y0, 20) + for i, val in enumerate((hi, (lo + hi) / 2, lo)): + fill, stroke, _ = color_for(val, lo, hi, anchors, reverse) + cell("hm-%d" % i, "%g" % val, + set_style("rounded=0;whiteSpace=wrap;html=1;", fill, stroke), + y0 + 24 + i * (h + 4), h) + + +def main(): + ap = argparse.ArgumentParser(description="Recolour a .drawio into a metric heatmap.") + ap.add_argument("file", help="input .drawio (uncompressed)") + ap.add_argument("-m", "--metrics", required=True, help="metrics .csv or .json") + ap.add_argument("-o", "--output", help="output .drawio (default: -heat.drawio)") + ap.add_argument("--palette", default="heat", choices=list(PALETTES)) + ap.add_argument("--reverse", action="store_true", help="flip the ramp (low = hot)") + ap.add_argument("--size", action="store_true", help="also scale node size by value") + ap.add_argument("--no-legend", dest="legend", action="store_false", help="skip the legend") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + if not os.path.isfile(args.metrics): + sys.exit(f"error: {args.metrics} not found") + metrics = load_metrics(args.metrics) + if not metrics: + sys.exit(f"error: no numeric metrics parsed from {args.metrics}") + low_map = {k.lower(): v for k, v in metrics.items()} + anchors = PALETTES[args.palette] + lo, hi = min(metrics.values()), max(metrics.values()) + + tree = ET.parse(args.file) + matched, first_root = 0, None + for diagram in tree.getroot().iter("diagram"): + model = diagram.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: # compressed / empty page + continue + if first_root is None: + first_root = root + for cell, cid, label in list(vertices(root)): + val = metrics.get(cid) + if val is None and label: + val = metrics.get(label, low_map.get(label.lower())) + if val is None: + continue + fill, stroke, t = color_for(val, lo, hi, anchors, args.reverse) + cell.set("style", set_style(cell.get("style"), fill, stroke)) + if args.size and hi > lo: + scale_geom(cell, 0.7 + 0.8 * t) + matched += 1 + + if args.legend and first_root is not None and matched: + add_legend(first_root, anchors, lo, hi, args.reverse) + + out = args.output or os.path.splitext(args.file)[0] + "-heat.drawio" + tree.write(out, encoding="utf-8", xml_declaration=False) + if matched == 0: + sys.stderr.write("warning: no nodes matched any metric key (check ids/labels)\n") + sys.stderr.write(f"wrote {out} ({matched}/{len(metrics)} metrics matched)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/jsimports.py b/skills/drawio-skill/scripts/jsimports.py new file mode 100644 index 0000000..6908ea8 --- /dev/null +++ b/skills/drawio-skill/scripts/jsimports.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""Extract a JS/TS project's module-import graph as autolayout graph JSON. + +The JavaScript/TypeScript counterpart to pyimports.py: walks a source +directory, scans each module for static and dynamic import specifiers +(`import`/`export ... from`, `require()`, `import()`), keeps only the +intra-project edges (relative specifiers that resolve to a file under the +root), and applies transitive reduction so the diagram stays readable. + + python3 jsimports.py src -o graph.json + python3 autolayout.py graph.json -o diagram.drawio + +Resolution is path-based: relative specifiers are resolved against the +importing file's directory, trying the .ts/.tsx/.js/.jsx/.mjs/.cjs extensions +and directory index files. Bare specifiers (node_modules packages such as +"react") are ignored. Scanning is regex-based rather than a full parser, so a +specifier inside a comment or string literal is counted in rare cases. + +Usage: python3 jsimports.py [-o graph.json] [--direction TB|LR] + [--group] [--no-reduce] +""" +import argparse +import json +import os +import re +import subprocess +import sys + +EXTS = (".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs") +SPEC = re.compile( + r"(?:import|export)\b[^'\";]*?\bfrom\s*['\"]([^'\"]+)['\"]" # import/export ... from "x" + r"|import\s*['\"]([^'\"]+)['\"]" # import "x" (side effect) + r"|require\s*\(\s*['\"]([^'\"]+)['\"]\s*\)" # require("x") + r"|import\s*\(\s*['\"]([^'\"]+)['\"]\s*\)" # import("x") dynamic +) + + +def modid(path, root): + """Module id = path relative to root, extension stripped, posix separators.""" + rel = os.path.relpath(path, root) + for ext in EXTS: + if rel.endswith(ext): + rel = rel[: -len(ext)] + break + return rel.replace(os.sep, "/") + + +def discover(root): + """Map module id -> absolute file path for every source file under root.""" + root = os.path.abspath(root) + modules = {} + for dirpath, dirs, files in os.walk(root): + dirs[:] = [d for d in dirs if d != "node_modules" and not d.startswith(".")] + for fn in files: + if fn.endswith(EXTS) and not fn.endswith(".d.ts"): + full = os.path.join(dirpath, fn) + modules[modid(full, root)] = full + return modules, root + + +def resolve(spec, importer, root, modules): + """Resolve a relative specifier to a known module id, or None (external).""" + if not spec.startswith("."): + return None + base = os.path.normpath(os.path.join(os.path.dirname(importer), spec)) + candidates = ([base + e for e in EXTS] + + [os.path.join(base, "index" + e) for e in EXTS] + + [base]) + for cand in candidates: + mid = modid(cand, root) + if mid in modules and modules[mid] != importer: + return mid + return None + + +def edges_of(mid, path, root, modules): + """Intra-project modules imported by module `mid`.""" + found = set() + try: + with open(path, encoding="utf-8", errors="ignore") as f: + src = f.read() + except OSError: + return found + for m in SPEC.finditer(src): + spec = m.group(1) or m.group(2) or m.group(3) or m.group(4) + target = resolve(spec, path, root, modules) + if target and target != mid: + found.add(target) + return found + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def common_dir(ids): + """Longest shared leading path segment across module ids (e.g. 'src/').""" + common = [] + for parts in zip(*[m.split("/") for m in ids]): + if len(set(parts)) == 1: + common.append(parts[0]) + else: + break + return "/".join(common) + "/" if common else "" + + +def main(): + ap = argparse.ArgumentParser(description="JS/TS import graph -> autolayout graph JSON.") + ap.add_argument("src", help="source directory") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group nodes into containers by top-level directory") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + args = ap.parse_args() + + modules, root = discover(args.src) + if not modules: + sys.exit(f"error: no JS/TS modules found under {args.src}") + edges = sorted({(m, t) for m, p in modules.items() + for t in edges_of(m, p, root, modules)}) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce(list(modules), edges) + strip = common_dir(list(modules)) + label = lambda m: (m[len(strip):] if strip and m.startswith(strip) else m) or m + + def node(m): + d = {"id": m, "label": label(m)} + if args.group: + rest = label(m).split("/") + if len(rest) > 1: # has a sub-directory + d["group"] = "/".join(rest[:-1]) # full directory path -> nested boxes + return d + + graph = { + "direction": args.direction, + "nodes": [node(m) for m in modules], + "edges": [{"source": s, "target": t} for s, t in edges], + } + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(modules)} modules, {len(edges)} edges{note}\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/k8simports.py b/skills/drawio-skill/scripts/k8simports.py new file mode 100644 index 0000000..0d4566b --- /dev/null +++ b/skills/drawio-skill/scripts/k8simports.py @@ -0,0 +1,242 @@ +#!/usr/bin/env python3 +"""Extract a Kubernetes manifest set's object graph as autolayout graph JSON. + +Reads manifest files (a directory, or explicit .yaml/.yml/.json paths), maps +each object kind to its official draw.io Kubernetes icon (mxgraph.kubernetes +set, resolved from the bundled shape index), and derives the reference edges +that make an architecture readable: + + Ingress -> Service (spec backend service names) + Service -> workload (selector labels match the pod template) + workload -> ConfigMap/Secret (env / envFrom / volumes) + workload -> PVC (persistentVolumeClaim volumes) + HPA -> target workload (scaleTargetRef) + +Workloads: Deployment, StatefulSet, DaemonSet, ReplicaSet, Job, CronJob, Pod. +Edges only land on objects that are themselves in the manifest set. The output +feeds autolayout.py: + + python3 k8simports.py ./manifests -o graph.json + python3 autolayout.py graph.json -o cluster.drawio + +JSON input (single object, or a `kind: List` as produced by +`kubectl get ... -o json`) parses with the stdlib alone; .yaml/.yml files +need PyYAML (`pip install pyyaml`). Pass `-` to read a live cluster snapshot +from stdin: `kubectl get all,ing,cm,secret,pvc -o json | k8simports.py -`. + +Usage: python3 k8simports.py [-o graph.json] + [--direction TB|LR] [--group] [--no-icons] +""" +import argparse +import glob +import gzip +import json +import os +import re +import sys + +# Object kind -> prIcon name inside the mxgraph.kubernetes.icon2 shape set. +KIND_ICON = { + "ClusterRole": "c-role", "ClusterRoleBinding": "crb", "ConfigMap": "cm", + "CronJob": "cronjob", "CustomResourceDefinition": "crd", "DaemonSet": "ds", + "Deployment": "deploy", "Endpoints": "ep", "HorizontalPodAutoscaler": "hpa", + "Ingress": "ing", "Job": "job", "Namespace": "ns", "NetworkPolicy": "netpol", + "Node": "node", "PersistentVolume": "pv", "PersistentVolumeClaim": "pvc", + "Pod": "pod", "ReplicaSet": "rs", "Role": "role", "RoleBinding": "rb", + "Secret": "secret", "Service": "svc", "ServiceAccount": "sa", + "StatefulSet": "sts", "StorageClass": "sc", +} +WORKLOADS = {"Deployment", "StatefulSet", "DaemonSet", "ReplicaSet", "Job", "CronJob", "Pod"} +_INDEX = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "data", + "shape-index.json.gz") + + +def icon_styles(): + """prIcon name -> (style, w, h) from the bundled official shape index.""" + with gzip.open(_INDEX, "rt", encoding="utf-8") as f: + shapes = json.load(f) + out = {} + for s in shapes: + st = s["style"] + m = re.search(r"prIcon=([\w-]+)", st) + # Skip the kubernetesLabel=1 variants (they paint the kind name into + # the icon; our node label already names the object below it). + if "mxgraph.kubernetes.icon2" in st and m and "kubernetesLabel" not in st: + out.setdefault(m.group(1), (st, s["w"], s["h"])) + return out + + +def load_manifests(paths): + """Parse all given files into a list of k8s objects.""" + files = [] + for p in paths: + if os.path.isdir(p): + for ext in ("yaml", "yml", "json"): + files.extend(glob.glob(os.path.join(p, "**", f"*.{ext}"), recursive=True)) + else: + files.append(p) + objs = [] + for path in sorted(set(files)): + if path == "-": # `kubectl get ... -o json | k8simports.py -` + docs = [json.loads(sys.stdin.read())] + elif path.endswith(".json"): + with open(path, encoding="utf-8") as f: + docs = [json.loads(f.read())] + else: + with open(path, encoding="utf-8") as f: + text = f.read() + try: + import yaml + except ImportError: + sys.exit(f"error: {path} is YAML but PyYAML is not installed " + "(pip install pyyaml) — or feed JSON from `kubectl get ... -o json`") + docs = list(yaml.safe_load_all(text)) + for doc in docs: + if not isinstance(doc, dict): + continue + if doc.get("kind") == "List": + objs.extend(i for i in doc.get("items", []) if isinstance(i, dict)) + else: + objs.append(doc) + return [o for o in objs if o.get("kind") and o.get("metadata", {}).get("name")] + + +def pod_spec(obj): + spec = obj.get("spec") or {} + if obj["kind"] == "Pod": + return spec + if obj["kind"] == "CronJob": + spec = (spec.get("jobTemplate") or {}).get("spec") or {} + return (spec.get("template") or {}).get("spec") or {} + + +def pod_labels(obj): + if obj["kind"] == "Pod": + return (obj.get("metadata") or {}).get("labels") or {} + spec = obj.get("spec") or {} + if obj["kind"] == "CronJob": + spec = (spec.get("jobTemplate") or {}).get("spec") or {} + return ((spec.get("template") or {}).get("metadata") or {}).get("labels") or {} + + +def mounted_refs(pspec): + """(kind, name) pairs a pod spec references via env/envFrom/volumes.""" + refs = set() + for c in (pspec.get("containers") or []) + (pspec.get("initContainers") or []): + for e in c.get("env") or []: + vf = e.get("valueFrom") or {} + for key, kind in (("configMapKeyRef", "ConfigMap"), ("secretKeyRef", "Secret")): + if vf.get(key, {}).get("name"): + refs.add((kind, vf[key]["name"])) + for e in c.get("envFrom") or []: + for key, kind in (("configMapRef", "ConfigMap"), ("secretRef", "Secret")): + if e.get(key, {}).get("name"): + refs.add((kind, e[key]["name"])) + for v in pspec.get("volumes") or []: + if v.get("configMap", {}).get("name"): + refs.add(("ConfigMap", v["configMap"]["name"])) + if v.get("secret", {}).get("secretName"): + refs.add(("Secret", v["secret"]["secretName"])) + if v.get("persistentVolumeClaim", {}).get("claimName"): + refs.add(("PersistentVolumeClaim", v["persistentVolumeClaim"]["claimName"])) + return refs + + +def ingress_backends(obj): + """Service names referenced by an Ingress (networking.k8s.io/v1 + legacy).""" + names, stack = set(), [obj.get("spec") or {}] + while stack: + cur = stack.pop() + if isinstance(cur, dict): + svc = cur.get("service") + if isinstance(svc, dict) and svc.get("name"): + names.add(svc["name"]) + if isinstance(cur.get("serviceName"), str): + names.add(cur["serviceName"]) + stack.extend(cur.values()) + elif isinstance(cur, list): + stack.extend(cur) + return names + + +def main(): + ap = argparse.ArgumentParser(description="Kubernetes manifests -> autolayout graph JSON.") + ap.add_argument("paths", nargs="+", help="manifest files and/or directories") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", help="group objects by namespace") + ap.add_argument("--no-icons", action="store_true", + help="plain boxes instead of official Kubernetes icons") + args = ap.parse_args() + + objs = load_manifests(args.paths) + if not objs: + sys.exit("error: no Kubernetes objects found (need kind + metadata.name)") + + def key(obj): + meta = obj.get("metadata") or {} + return (meta.get("namespace") or "", obj["kind"], meta["name"]) + + by_key = {key(o): o for o in objs} + icons = {} if args.no_icons else icon_styles() + + edges = set() + for k, obj in by_key.items(): + ns, kind, _ = k + + def link(tkind, tname): + if (ns, tkind, tname) in by_key and (ns, tkind, tname) != k: + edges.add((k, (ns, tkind, tname))) + + if kind == "Ingress": + for svc in ingress_backends(obj): + link("Service", svc) + elif kind == "Service": + sel = (obj.get("spec") or {}).get("selector") or {} + if sel: + for tk, target in by_key.items(): + if (tk[0] == ns and tk[1] in WORKLOADS + and sel.items() <= pod_labels(target).items()): + edges.add((k, tk)) + elif kind in WORKLOADS: + for tkind, tname in mounted_refs(pod_spec(obj)): + link(tkind, tname) + elif kind == "HorizontalPodAutoscaler": + ref = (obj.get("spec") or {}).get("scaleTargetRef") or {} + if ref.get("kind") and ref.get("name"): + link(ref["kind"], ref["name"]) + + def nid(k): + ns, kind, name = k + return f"{ns + '/' if ns else ''}{kind}/{name}" + + nodes = [] + for k in by_key: + ns, kind, name = k + node = {"id": nid(k), "label": name} + icon = icons.get(KIND_ICON.get(kind, "")) + if icon: + node.update(style=icon[0], width=icon[1], height=icon[2]) + else: + node["label"] = f"{name}\n{kind}" + if args.group and ns: + node["group"] = ns + nodes.append(node) + + graph = {"direction": args.direction, "nodes": nodes, + "edges": [{"source": nid(s), "target": nid(t)} for s, t in sorted(edges)]} + if icons: + # Icon labels render below the shape — reserve extra layout spacing. + graph.update(ranksep=0.7, nodesep=0.6) + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + sys.stderr.write(f"{len(nodes)} objects, {len(edges)} edges\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/openapiimports.py b/skills/drawio-skill/scripts/openapiimports.py new file mode 100644 index 0000000..842c94f --- /dev/null +++ b/skills/drawio-skill/scripts/openapiimports.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +"""Turn an OpenAPI / Swagger spec into an API diagram as autolayout graph JSON. + +Reads an OpenAPI 3 or Swagger 2 spec (JSON, or YAML with PyYAML) and emits one +node per operation — coloured by HTTP method — plus one node per component +schema, with edges from each operation to the schemas it references (request / +response bodies) and between schemas that nest one another. Feeds autolayout.py: + + python3 openapiimports.py openapi.yaml -o graph.json + python3 autolayout.py graph.json -o api.drawio + +Operations are grouped by their first `tag` (falling back to the first path +segment) with `--group`; `--no-schemas` drops the data-model nodes to show just +the endpoint surface. `$ref`s are resolved to their final name; only schemas +defined under components/definitions become nodes, so external refs are ignored. + +Usage: python3 openapiimports.py [-o graph.json] + [--direction TB|LR] [--group] [--no-schemas] +""" +import argparse +import json +import os +import sys + +METHODS = ("get", "post", "put", "patch", "delete", "head", "options", "trace") +# HTTP method -> (fill, stroke). GET reads, POST creates, PUT/PATCH update, DELETE removes. +METHOD_STYLE = { + "get": ("#dae8fc", "#6c8ebf"), "post": ("#d5e8d4", "#82b366"), + "put": ("#ffe6cc", "#d79b00"), "patch": ("#ffe6cc", "#d79b00"), + "delete": ("#f8cecc", "#b85450"), +} +OTHER_STYLE = ("#f5f5f5", "#666666") +SCHEMA_STYLE = "rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" +OP_EDGE = "edgeStyle=orthogonalEdgeStyle;html=1;rounded=0;fontSize=10;endArrow=open;" +REF_EDGE = ("edgeStyle=orthogonalEdgeStyle;html=1;rounded=0;fontSize=10;" + "dashed=1;endArrow=open;strokeColor=#9673a6;") + + +def load_spec(path): + """Parse the spec: JSON directly, YAML (or ambiguous) via PyYAML if present.""" + with open(path, encoding="utf-8") as f: + text = f.read() + if path.lower().endswith((".yaml", ".yml")): + try: + import yaml + except ImportError: + sys.exit("error: spec is YAML but PyYAML is not installed " + "(pip install pyyaml) — or convert the spec to JSON") + return yaml.safe_load(text) + try: + return json.loads(text) + except json.JSONDecodeError: + try: + import yaml + except ImportError: + sys.exit("error: could not parse spec as JSON (install PyYAML to read YAML)") + return yaml.safe_load(text) + + +def find_refs(obj): + """Yield the final name of every $ref anywhere inside a spec fragment.""" + if isinstance(obj, dict): + for k, v in obj.items(): + if k == "$ref" and isinstance(v, str): + yield v.split("/")[-1] + else: + yield from find_refs(v) + elif isinstance(obj, list): + for item in obj: + yield from find_refs(item) + + +def method_style(method): + fill, stroke = METHOD_STYLE.get(method, OTHER_STYLE) + return ("rounded=1;whiteSpace=wrap;html=1;align=left;spacingLeft=6;" + f"fillColor={fill};strokeColor={stroke};") + + +def build(spec, group, no_schemas, direction): + """Spec dict -> autolayout graph JSON dict.""" + paths = spec.get("paths") or {} + # components.schemas (OpenAPI 3) or definitions (Swagger 2) + schemas = (spec.get("components") or {}).get("schemas") or spec.get("definitions") or {} + want_schemas = bool(schemas) and not no_schemas + sid = {name: f"S:{name}" for name in schemas} + + nodes, edges, seen = [], [], set() + + def add_edge(src, dst, style): + if src != dst and (src, dst) not in seen: + seen.add((src, dst)) + edges.append({"source": src, "target": dst, "style": style, "label": ""}) + + i = 0 + for path, item in paths.items(): + if not isinstance(item, dict): + continue + for method in METHODS: + op = item.get(method) + if not isinstance(op, dict): + continue + oid = f"op{i}" + i += 1 + summary = (op.get("summary") or op.get("operationId") or "").strip() + head = f"{method.upper()} {path}" + nodes.append({ + "id": oid, "label": head + (f"\n{summary}" if summary else ""), + "style": method_style(method), + "width": max(160, 8 * len(head) + 20), "height": 40, + **({"group": (op.get("tags") or [path.strip('/').split('/')[0] or "root"])[0]} + if group else {}), + }) + if want_schemas: + for ref in set(find_refs(op)): + if ref in sid: + add_edge(oid, sid[ref], OP_EDGE) + + if want_schemas: + for name, schema in schemas.items(): + fields = schema.get("properties") if isinstance(schema, dict) else None + count = len(fields) if fields else 0 + nodes.append({ + "id": sid[name], + "label": name + (f"\n({count} field{'s' if count != 1 else ''})" if count else ""), + "style": SCHEMA_STYLE, "width": max(140, 9 * len(name) + 20), "height": 40, + **({"group": "schemas"} if group else {}), + }) + for ref in set(find_refs(schema)): + if ref in sid: + add_edge(sid[name], sid[ref], REF_EDGE) + + return {"direction": direction, "nodes": nodes, "edges": edges} + + +def main(): + ap = argparse.ArgumentParser(description="OpenAPI/Swagger spec -> API diagram graph JSON.") + ap.add_argument("spec", help="OpenAPI 3 / Swagger 2 spec (.json or .yaml)") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="LR", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", help="group operations by tag") + ap.add_argument("--no-schemas", action="store_true", + help="show only endpoints, omit schema nodes and edges") + args = ap.parse_args() + + if not os.path.isfile(args.spec): + sys.exit(f"error: {args.spec} not found") + spec = load_spec(args.spec) or {} + if not (spec.get("paths")): + sys.exit("error: no paths found (is this an OpenAPI/Swagger spec?)") + + graph = build(spec, args.group, args.no_schemas, args.direction) + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + ops = sum(1 for n in graph["nodes"] if n["id"].startswith("op")) + sys.stderr.write(f"{ops} operations, {len(graph['nodes']) - ops} schemas, " + f"{len(graph['edges'])} edges\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/prdiff.py b/skills/drawio-skill/scripts/prdiff.py new file mode 100644 index 0000000..ae0fd83 --- /dev/null +++ b/skills/drawio-skill/scripts/prdiff.py @@ -0,0 +1,191 @@ +#!/usr/bin/env python3 +"""Render base/head/diff PNGs for every .drawio changed between two git refs. + +For each `.drawio` that differs between `--base` and `--head`, exports the +base and head pages as PNGs via the draw.io CLI, and — for files present on +both sides — chains `drawiodiff.py` -> `autolayout.py` -> CLI export into a +third colour-coded diff PNG. Added/removed files just get the one side that +exists. Emits a Markdown report with one section per changed file (status + +image links) and a summary count, suitable for a PR comment or CI job +summary; pair with `.github/actions/drawio-diff/`. + + python3 prdiff.py --base origin/main --head HEAD -o drawio-pr/report.md + +Missing draw.io CLI degrades gracefully: the Markdown still lists every +changed file, just without images (a review comment listing the files is +still useful). Missing git, or `--repo` not a git repository, is fatal. + +Usage: python3 prdiff.py --base [--head ] [--repo ] [--out-dir ] [-o report.md] +""" +import argparse +import os +import shutil +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) + + +def changed_drawios(base, head, repo): + """List of (path, status) for .drawio files that differ between base and head. + + status is "added", "removed", or "modified" (renames/copies count as + modified, keyed on the new path). Shells to `git diff --name-status`. + """ + try: + r = subprocess.run( + ["git", "-C", repo, "diff", "--name-status", f"{base}..{head}", "--", "*.drawio"], + capture_output=True, text=True) + except FileNotFoundError: + sys.exit("error: not a git repo / git not found (git not on PATH)") + if r.returncode != 0: + sys.exit(f"error: not a git repo / git not found: {r.stderr.strip()}") + entries = [] + for line in r.stdout.splitlines(): + if not line.strip(): + continue + parts = line.split("\t") + code, path = parts[0], parts[-1] + status = "added" if code.startswith("A") else "removed" if code.startswith("D") else "modified" + entries.append((path, status)) + return entries + + +def git_show_file(repo, ref, path, dest): + """Write the blob at ref:path (in repo) to dest. False if it doesn't exist there.""" + r = subprocess.run(["git", "-C", repo, "show", f"{ref}:{path}"], capture_output=True) + if r.returncode != 0: + return False + with open(dest, "wb") as f: + f.write(r.stdout) + return True + + +def export_png(src_drawio, out_png): + """CLI-export page 1 of src_drawio to out_png. True on success.""" + r = subprocess.run(["drawio", "-x", "-f", "png", "--page-index", "1", "-o", out_png, src_drawio], + capture_output=True) + return r.returncode == 0 and os.path.exists(out_png) + + +def export_diff_png(base_drawio, head_drawio, out_png, tmp): + """drawiodiff.py -> autolayout.py -> CLI export a coloured diff PNG. True on success.""" + diff_json = os.path.join(tmp, "diff.json") + diff_drawio = os.path.join(tmp, "diff.drawio") + r1 = subprocess.run([sys.executable, os.path.join(HERE, "drawiodiff.py"), + base_drawio, head_drawio, "-o", diff_json], capture_output=True) + if r1.returncode != 0 or not os.path.exists(diff_json): + return False + r2 = subprocess.run([sys.executable, os.path.join(HERE, "autolayout.py"), + diff_json, "-o", diff_drawio], capture_output=True) + if r2.returncode != 0 or not os.path.exists(diff_drawio): + return False + return export_png(diff_drawio, out_png) + + +def build_entry(repo, base, head, path, status, out_dir, drawio_available): + """One render_markdown entry: fetch both sides, export whatever PNGs it can.""" + entry = {"path": path, "status": status} + if not drawio_available: + entry["skipped"] = True + return entry + slug = path.replace("/", "__") + with tempfile.TemporaryDirectory() as tmp: + base_drawio = os.path.join(tmp, "base.drawio") + head_drawio = os.path.join(tmp, "head.drawio") + have_base = git_show_file(repo, base, path, base_drawio) + have_head = git_show_file(repo, head, path, head_drawio) + if have_base: + p = os.path.join(out_dir, f"{slug}.base.png") + if export_png(base_drawio, p): + entry["base_png"] = p + if have_head: + p = os.path.join(out_dir, f"{slug}.head.png") + if export_png(head_drawio, p): + entry["head_png"] = p + if have_base and have_head: + p = os.path.join(out_dir, f"{slug}.diff.png") + if export_diff_png(base_drawio, head_drawio, p, tmp): + entry["diff_png"] = p + return entry + + +def render_markdown(entries, out_dir): + """Pure: Markdown PR report from prdiff entries. No I/O, no CLI. + + entries: list of {"path", "status", "base_png"?, "head_png"?, "diff_png"?, + "skipped"?} — image paths (if any) are made relative to out_dir for the + Markdown links. "skipped" means the draw.io CLI was unavailable. + """ + counts = {"added": 0, "removed": 0, "modified": 0} + for e in entries: + counts[e["status"]] = counts.get(e["status"], 0) + 1 + lines = [ + "# draw.io diagram changes", + "", + f"{len(entries)} file(s) changed: +{counts.get('added', 0)} added, " + f"-{counts.get('removed', 0)} removed, ~{counts.get('modified', 0)} modified", + ] + if not entries: + lines.append("") + lines.append("No `.drawio` files changed.") + return "\n".join(lines) + "\n" + + def rel(png): + return os.path.relpath(png, out_dir).replace(os.sep, "/") if png else None + + for e in entries: + lines.append("") + lines.append(f"## {e['path']} ({e['status']})") + if e.get("skipped"): + lines.append("") + lines.append("_draw.io CLI not available — images skipped._") + continue + base_r, head_r, diff_r = rel(e.get("base_png")), rel(e.get("head_png")), rel(e.get("diff_png")) + lines.append("") + if base_r: + lines.append(f"![base]({base_r})") + if head_r: + lines.append(f"![head]({head_r})") + if diff_r: + lines.append(f"![diff]({diff_r})") + if not (base_r or head_r or diff_r): + lines.append("_no image produced._") + return "\n".join(lines) + "\n" + + +def main(): + ap = argparse.ArgumentParser(description="Render PNGs + a Markdown report for .drawio files " + "changed between two git refs.") + ap.add_argument("--base", required=True, help="base git ref/sha") + ap.add_argument("--head", default="HEAD", help="head git ref/sha (default HEAD)") + ap.add_argument("--repo", default=".", help="path to the git repo (default: current directory)") + ap.add_argument("--out-dir", default="drawio-pr", help="directory for exported PNGs (default: ./drawio-pr)") + ap.add_argument("-o", "--output", help="write the Markdown report here (default: stdout)") + args = ap.parse_args() + + changed = changed_drawios(args.base, args.head, args.repo) + if not changed: + sys.stderr.write("no .drawio files changed\n") + + drawio_available = shutil.which("drawio") is not None + if not drawio_available and changed: + sys.stderr.write("warning: draw.io CLI not found - image export skipped, " + "Markdown will list files only (is the draw.io CLI installed?)\n") + os.makedirs(args.out_dir, exist_ok=True) + + entries = [build_entry(args.repo, args.base, args.head, path, status, args.out_dir, drawio_available) + for path, status in changed] + report = render_markdown(entries, args.out_dir) + + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(report) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(report) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/pyclasses.py b/skills/drawio-skill/scripts/pyclasses.py new file mode 100644 index 0000000..ac69036 --- /dev/null +++ b/skills/drawio-skill/scripts/pyclasses.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python3 +"""Extract a Python project's class-inheritance graph as autolayout graph JSON. + +A finer-grained companion to pyimports.py: instead of module->module imports, +it emits one node per class and an edge from each subclass to the project base +classes it extends. With --group, classes are boxed by their module (nested by +sub-package), so the result reads as an auto-generated class hierarchy. + + python3 pyclasses.py myproject --group -o graph.json + python3 autolayout.py graph.json -o diagram.drawio + +Only inheritance is resolved (statically reliable); base classes are matched +by name, preferring a class in the same module. External bases (object, +Exception, third-party) are ignored. This is a *class structure* view, not a +function-level call graph — static call resolution in Python is unreliable, so +that is deliberately out of scope. + +Usage: python3 pyclasses.py [-o graph.json] [--direction TB|LR] + [--group] [--no-reduce] +""" +import argparse +import ast +import json +import os +import re +import subprocess +import sys + + +def discover(root): + """Map dotted module name -> file path; qualify with the package name when + root is itself a package (mirrors pyimports.py).""" + root = os.path.abspath(root) + base = os.path.basename(root) if os.path.exists(os.path.join(root, "__init__.py")) else "" + modules = {} + for dirpath, _, files in os.walk(root): + for fn in files: + if not fn.endswith(".py"): + continue + rel = os.path.relpath(os.path.join(dirpath, fn), root)[:-3] + parts = rel.split(os.sep) + if parts[-1] == "__init__": + parts = parts[:-1] + parts = ([base] + parts) if base else parts + if parts: + modules[".".join(parts)] = os.path.join(dirpath, fn) + return modules, base + + +def base_name(node): + """Simple name of a base-class expression (`Foo` or `pkg.Foo` -> 'Foo').""" + if isinstance(node, ast.Name): + return node.id + if isinstance(node, ast.Attribute): + return node.attr + return None + + +def classes_in(module, path): + """Top-level classes of a module: list of (qualified_id, simple_name, [base names]).""" + try: + with open(path, encoding="utf-8") as f: + tree = ast.parse(f.read(), filename=path) + except SyntaxError: + return [] + out = [] + for node in tree.body: + if isinstance(node, ast.ClassDef): + bases = [b for b in (base_name(b) for b in node.bases) if b] + out.append((f"{module}.{node.name}", node.name, bases)) + return out + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def main(): + ap = argparse.ArgumentParser(description="Python class-inheritance graph -> autolayout graph JSON.") + ap.add_argument("project", help="package or project directory") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="box classes by their module (nested by sub-package)") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + args = ap.parse_args() + + modules, base = discover(args.project) + classes = {} # qualified id -> (module, bases) + by_name = {} # simple name -> [qualified ids] + for mod, path in modules.items(): + for cid, name, bases in classes_in(mod, path): + classes[cid] = (mod, bases) + by_name.setdefault(name, []).append(cid) + if not classes: + sys.exit(f"error: no classes found under {args.project}") + + def resolve(name, module): + cands = by_name.get(name, []) + same = [c for c in cands if classes[c][0] == module] + if same: + return same[0] # prefer a class in the same module + return cands[0] if len(cands) == 1 else None # else only if unambiguous + + edges = set() + for cid, (mod, bases) in classes.items(): + for b in bases: + target = resolve(b, mod) + if target and target != cid: + edges.add((cid, target)) + edges = sorted(edges) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce(list(classes), edges) + + strip = base + "." if base else "" + short = lambda m: m[len(strip):] if strip and m.startswith(strip) else m + + def node(cid): + # No hard-coded colour: autolayout tints nodes by their group (module), + # so a grouped class hierarchy reads as coloured-by-module. + d = {"id": cid, "label": cid.rsplit(".", 1)[1]} + if args.group: + mod = classes[cid][0] + path = short(mod).replace(".", "/") # module path -> nested boxes + if path: + d["group"] = path + return d + + graph = { + "direction": args.direction, + "nodes": [node(cid) for cid in classes], + "edges": [{"source": s, "target": t} for s, t in edges], + } + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(classes)} classes, {len(edges)} inheritance edges{note}\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/pyimports.py b/skills/drawio-skill/scripts/pyimports.py new file mode 100644 index 0000000..6a498bb --- /dev/null +++ b/skills/drawio-skill/scripts/pyimports.py @@ -0,0 +1,155 @@ +#!/usr/bin/env python3 +"""Extract a Python project's module-import graph as autolayout graph JSON. + +Walks a package/project directory, parses each module with `ast`, builds the +intra-project import edges, and (by default) applies transitive reduction so +the diagram stays readable instead of becoming a hairball. The output feeds +autolayout.py: + + python3 pyimports.py myproject -o graph.json + python3 autolayout.py graph.json -o diagram.drawio + +Transitive reduction uses Graphviz `tred` (drops edges implied by a longer +path); pass --no-reduce to keep every import edge. Only intra-project imports +are kept — third-party and stdlib imports are ignored. + +Usage: python3 pyimports.py [-o graph.json] [--direction TB|LR] [--no-reduce] +""" +import argparse +import ast +import json +import os +import re +import subprocess +import sys + + +def discover(root): + """Map dotted module name -> file path for every .py under root, plus the + package prefix. If root is itself a package (has __init__.py), module names + are qualified with its name so the project's own absolute imports resolve.""" + root = os.path.abspath(root) + base = os.path.basename(root) if os.path.exists(os.path.join(root, "__init__.py")) else "" + modules = {} + for dirpath, _, files in os.walk(root): + for fn in files: + if not fn.endswith(".py"): + continue + rel = os.path.relpath(os.path.join(dirpath, fn), root)[:-3] # strip .py + parts = rel.split(os.sep) + if parts[-1] == "__init__": + parts = parts[:-1] # package = its dir + parts = ([base] + parts) if base else parts + if parts: + modules[".".join(parts)] = os.path.join(dirpath, fn) + return modules, base + + +def resolve(name, current, modules): + """Resolve a dotted name to the longest known module prefix (or None).""" + parts = name.split(".") if name else [] + while parts: + cand = ".".join(parts) + if cand in modules and cand != current: + return cand + parts = parts[:-1] + return None + + +def edges_of(name, path, modules): + """Intra-project modules imported by `name`.""" + pkg = name if path.endswith("__init__.py") else name.rsplit(".", 1)[0] if "." in name else "" + found = set() + try: + with open(path, encoding="utf-8") as f: + tree = ast.parse(f.read(), filename=path) + except SyntaxError: + return found + for node in ast.walk(tree): + if isinstance(node, ast.Import): # import a.b.c + for alias in node.names: + target = resolve(alias.name, name, modules) + if target: + found.add(target) + elif isinstance(node, ast.ImportFrom): # from a.b import c + if node.level: # relative: climb level-1 packages + base = pkg.split(".") if pkg else [] + base = base[: len(base) - (node.level - 1)] + prefix = ".".join(base) + mod = f"{prefix}.{node.module}" if prefix and node.module else (node.module or prefix) + else: + mod = node.module or "" + target = resolve(mod, name, modules) + if target: + found.add(target) + for alias in node.names: # `from pkg import submodule` + sub = f"{mod}.{alias.name}" if mod else alias.name + target = resolve(sub, name, modules) + if target: + found.add(target) + return found + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def main(): + ap = argparse.ArgumentParser(description="Python import graph -> autolayout graph JSON.") + ap.add_argument("project", help="package or project directory") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group nodes into containers by sub-package") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + args = ap.parse_args() + + modules, base = discover(args.project) + if not modules: + sys.exit(f"error: no .py modules found under {args.project}") + edges = sorted({(name, t) for name, path in modules.items() + for t in edges_of(name, path, modules)}) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce(list(modules), edges) + # Drop the shared package prefix from labels for readability; ids stay full. + strip = base + "." if base else "" + label = lambda m: m[len(strip):] if strip and m.startswith(strip) else m + + def node(m): + d = {"id": m, "label": label(m)} + if args.group: + rest = label(m).split(".") + if len(rest) > 1: # nested under a sub-package + d["group"] = "/".join(rest[:-1]) # full sub-package path -> nested boxes + return d + + graph = { + "direction": args.direction, + "nodes": [node(m) for m in modules], + "edges": [{"source": s, "target": t} for s, t in edges], + } + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(modules)} modules, {len(edges)} edges{note}\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/raster2drawio.py b/skills/drawio-skill/scripts/raster2drawio.py new file mode 100644 index 0000000..bb88615 --- /dev/null +++ b/skills/drawio-skill/scripts/raster2drawio.py @@ -0,0 +1,196 @@ +#!/usr/bin/env python3 +"""De-rasterize an image-extracted graph (JSON) into an editable .drawio. + +Turns a whiteboard photo, legacy PNG, or Visio screenshot into an editable +diagram: Claude's own vision reads the image and extracts a JSON description +of the nodes/edges (the workflow is documented in +references/derasterize.md); this script turns that JSON into `.drawio` XML, +honoring the coordinates, labels, shapes, and colors Claude read off the +image. + + python3 raster2drawio.py graph.json -o out.drawio + +Input JSON: + {"nodes": [{"id": "n1", "label": "API Gateway", "x": 120, "y": 60, + "w": 160, "h": 60, "shape": "rect", + "fill": "#dae8fc", "stroke": "#6c8ebf"}, + {"id": "n2", "label": "Auth DB", "x": 360, "y": 60, + "shape": "cylinder"}], + "edges": [{"source": "n1", "target": "n2", "label": "HTTPS", + "dashed": false, "arrow": true}]} + +Only "id" is required per node; label defaults to id, w/h default to +120/60, shape defaults to "rect" (choices: rect, rounded, ellipse, +rhombus/diamond, cylinder, parallelogram, cloud, hexagon), fill/stroke +default to the skill's palette blue. Edge "arrow" defaults to true +(endArrow=none when false); "dashed" defaults to false. + +If ANY node is missing x or y, positions are not guessed: the graph is +handed to autolayout.py (shelled out, requires Graphviz `dot`) to place it, +and a note is written to stderr. + +Usage: python3 raster2drawio.py [-o out.drawio] +""" +import argparse +import json +import os +import subprocess +import sys +import tempfile +from xml.sax.saxutils import escape + +DEFAULT_W, DEFAULT_H = 120, 60 +DEFAULT_FILL, DEFAULT_STROKE = "#dae8fc", "#6c8ebf" +SHAPES = { + "rect": "whiteSpace=wrap;html=1;", + "rounded": "rounded=1;whiteSpace=wrap;html=1;", + "ellipse": "ellipse;whiteSpace=wrap;html=1;", + "rhombus": "rhombus;whiteSpace=wrap;html=1;", + "diamond": "rhombus;whiteSpace=wrap;html=1;", + "cylinder": "shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;size=15;", + "parallelogram": "shape=parallelogram;whiteSpace=wrap;html=1;", + "cloud": "ellipse;shape=cloud;whiteSpace=wrap;html=1;", + "hexagon": "shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;", +} +EDGE_BASE = "edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;" + + +def attr(value): + # Newlines in labels become so draw.io renders a line break (a raw + # newline inside an XML attribute is normalized to a space by parsers). + return escape(str(value), {'"': """, "\n": " "}) + + +def node_style(node): + base = SHAPES.get(node.get("shape", "rect"), SHAPES["rect"]) + fill = node.get("fill", DEFAULT_FILL) + stroke = node.get("stroke", DEFAULT_STROKE) + return f"{base}fillColor={fill};strokeColor={stroke};" + + +def edge_style(edge): + style = EDGE_BASE + if edge.get("dashed"): + style += "dashed=1;" + if edge.get("arrow") is False: + style += "endArrow=none;" + return style + + +def to_drawio(nodes, edges): + """Direct build: every node already has x/y. Mirrors autolayout.py's + to_drawio() string-building, without the dot layout pass.""" + cells = [] + for node in nodes: + w, h = node.get("w", DEFAULT_W), node.get("h", DEFAULT_H) + cells.append( + f' \n' + f' \n' + f" " + ) + for i, edge in enumerate(edges): + cells.append( + f' \n' + f' \n' + f" " + ) + return ( + "\n" + ' \n' + ' \n' + " \n" + ' \n' + ' \n' + + "\n".join(cells) + + "\n \n \n \n\n" + ) + + +def build_autolayout_graph(nodes, edges): + """Same graph, in autolayout.py's input shape (positions dropped — + dot will compute fresh ones for every node).""" + return { + "direction": "TB", + "nodes": [ + {"id": n["id"], "label": n.get("label", n["id"]), "style": node_style(n), + "width": n.get("w", DEFAULT_W), "height": n.get("h", DEFAULT_H)} + for n in nodes + ], + "edges": [ + {"source": e["source"], "target": e["target"], "label": e.get("label", ""), + "style": edge_style(e)} + for e in edges + ], + } + + +def run_autolayout(graph): + """Shell out to the sibling autolayout.py; return the .drawio XML text.""" + here = os.path.dirname(os.path.abspath(__file__)) + autolayout = os.path.join(here, "autolayout.py") + fd, graph_path = tempfile.mkstemp(suffix=".json") + try: + with os.fdopen(fd, "w", encoding="utf-8") as f: + json.dump(graph, f) + r = subprocess.run([sys.executable, autolayout, graph_path], + capture_output=True, text=True) + finally: + os.unlink(graph_path) + if r.returncode != 0: + sys.exit(f"error: autolayout.py failed: {r.stderr.strip()}") + return r.stdout + + +def main(): + ap = argparse.ArgumentParser( + description="Convert an image-extracted graph JSON into an editable .drawio.") + ap.add_argument("input", help="graph JSON file, or - for stdin") + ap.add_argument("-o", "--output", help="output .drawio path (default: stdout)") + args = ap.parse_args() + + if args.input == "-": + raw = sys.stdin.read() + else: + try: + with open(args.input, encoding="utf-8") as f: + raw = f.read() + except OSError as exc: + sys.exit(f"error: cannot read {args.input}: {exc}") + try: + graph = json.loads(raw) + except json.JSONDecodeError as exc: + sys.exit(f"error: invalid JSON: {exc}") + + nodes = graph.get("nodes") or [] + edges = graph.get("edges") or [] + if not nodes: + sys.exit("error: no nodes in input") + for n in nodes: + if "id" not in n: + sys.exit("error: every node needs an 'id'") + for e in edges: + if "source" not in e or "target" not in e: + sys.exit("error: every edge needs 'source' and 'target'") + + if any(n.get("x") is None or n.get("y") is None for n in nodes): + xml = run_autolayout(build_autolayout_graph(nodes, edges)) + sys.stderr.write( + "note: some nodes had no x/y — positions were auto-placed via autolayout.py\n") + else: + xml = to_drawio(nodes, edges) + + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(xml) + sys.stderr.write(f"wrote {args.output} ({len(nodes)} nodes, {len(edges)} edges)\n") + else: + sys.stdout.write(xml) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/relabel.py b/skills/drawio-skill/scripts/relabel.py new file mode 100644 index 0000000..33a6bef --- /dev/null +++ b/skills/drawio-skill/scripts/relabel.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python3 +"""Swap every label in a .drawio via a mapping — layout, styles, ids untouched. + +The main use-case is language variants of one diagram (EN <-> CN docs figures): +extract the labels, translate the values, apply the map — the geometry never +moves, so both variants stay pixel-identical except for the text. + + python3 relabel.py diagram.drawio --extract -o labels.json # step 1 + # edit labels.json: keep keys, replace each value with the new text + python3 relabel.py diagram.drawio --map labels.json -o diagram_cn.drawio + +Extract emits an identity JSON map {"label": "label", ...} of every non-empty +vertex/edge label, UserObject label, and page name, in document order. Apply +replaces each label that exactly matches a map key (raw string, HTML markup +included) and reports what matched. Unmapped labels stay unchanged; map keys +that matched nothing are listed on stderr so translations don't silently miss. + +Usage: relabel.py (--extract | --map ) [-o ] +""" +import argparse +import json +import os +import sys +import xml.etree.ElementTree as ET + + +def label_slots(tree): + """Yield (element, attribute) for every label-bearing slot in the file.""" + for diagram in tree.getroot().iter("diagram"): + if diagram.get("name"): + yield diagram, "name" + model = diagram.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: # compressed page — can't edit + sys.stderr.write("warning: skipping compressed page " + f"'{diagram.get('name', '?')}' (open+save in draw.io to decompress)\n") + continue + for child in root: + if child.tag == "mxCell": + if child.get("value"): + yield child, "value" + elif child.tag in ("UserObject", "object"): + if child.get("label"): + yield child, "label" + inner = child.find("mxCell") + if inner is not None and inner.get("value"): + yield inner, "value" + + +def main(): + p = argparse.ArgumentParser(description="Extract or swap .drawio labels via a JSON map.") + p.add_argument("file", help="input .drawio") + mode = p.add_mutually_exclusive_group(required=True) + mode.add_argument("--extract", action="store_true", + help="dump an identity label map as JSON") + mode.add_argument("--map", metavar="JSON", dest="mapfile", + help="JSON map {old label: new label} to apply") + p.add_argument("-o", "--output", help="output path (default: stdout for --extract, " + "-relabel.drawio for --map)") + args = p.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + tree = ET.parse(args.file) + + if args.extract: + seen = {} + for el, attr in label_slots(tree): + seen.setdefault(el.get(attr), el.get(attr)) + out = json.dumps(seen, ensure_ascii=False, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(out + "\n") + sys.stderr.write(f"wrote {args.output} ({len(seen)} labels)\n") + else: + print(out) + return + + with open(args.mapfile, encoding="utf-8") as f: + mapping = json.load(f) + if not isinstance(mapping, dict): + sys.exit("error: map file must be a JSON object {old: new}") + + matched, used = 0, set() + for el, attr in label_slots(tree): + old = el.get(attr) + if old in mapping: + el.set(attr, str(mapping[old])) + matched += 1 + used.add(old) + + out = args.output or os.path.splitext(args.file)[0] + "-relabel.drawio" + tree.write(out, encoding="utf-8", xml_declaration=False) + unused = [k for k in mapping if k not in used] + if unused: + sys.stderr.write("warning: %d map key(s) matched no label: %s\n" + % (len(unused), ", ".join(repr(k)[:60] for k in unused[:10]))) + sys.stderr.write(f"wrote {out} ({matched} labels replaced)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/repair_png.py b/skills/drawio-skill/scripts/repair_png.py new file mode 100644 index 0000000..7e95b7c --- /dev/null +++ b/skills/drawio-skill/scripts/repair_png.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +"""Repair truncated IEND chunk in draw.io -e PNG exports (issue #8). + +draw.io's CLI emits -e PNGs with the 4-byte IEND length field but missing +the 8 bytes of "IEND" type + CRC. Strict PNG decoders and vision APIs +(Anthropic included) reject the file with 400 "Could not process image". +SVG/PDF are unaffected. + +Usage: python3 repair_png.py + +Idempotent: the endswith(IEND) guard makes this a no-op once draw.io +fixes the bug upstream, so it's safe to run unconditionally after every +-e PNG export. +""" +import sys + +IEND = b"\x00\x00\x00\x00IEND\xaeB`\x82" + + +def repair(path: str) -> bool: + with open(path, "rb") as f: + data = f.read() + if data.endswith(IEND): + return False + if data.endswith(b"\x00\x00\x00\x00"): + data = data[:-4] + with open(path, "wb") as f: + f.write(data + IEND) + return True + + +if __name__ == "__main__": + if len(sys.argv) != 2: + print("usage: repair_png.py ", file=sys.stderr) + sys.exit(2) + if repair(sys.argv[1]): + print(f"repaired {sys.argv[1]}") diff --git a/skills/drawio-skill/scripts/restyle.py b/skills/drawio-skill/scripts/restyle.py new file mode 100644 index 0000000..023ebb3 --- /dev/null +++ b/skills/drawio-skill/scripts/restyle.py @@ -0,0 +1,161 @@ +#!/usr/bin/env python3 +"""Re-theme an EXISTING .drawio with a style preset — layout and shapes untouched. + +Style presets (styles/schema.json) normally apply at generation time; this is +the post-processor for diagrams that already exist: "make this dark", "apply my +corporate style to last week's diagram". + + python3 restyle.py diagram.drawio --preset dark + python3 restyle.py diagram.drawio --preset ~/.drawio-skill/styles/corp.json -o out.drawio + +What it changes, per the preset application rules in references/style-presets.md: +- Every vertex fill/stroke is remapped to the preset palette. Each existing + fillColor is matched to its nearest palette slot by hue (grey/low-saturation + -> neutral), so same-colored nodes stay same-colored in the new theme. +- font.fontFamily on every vertex; existing fontSize values are kept (they + encode hierarchy). +- extras: fontColor (vertices + text cells), edgeColor (edge stroke + label), + sketch=1, globalStrokeWidth, page background on . +Edge ROUTING styles and shape keywords are left alone — rewriting them would +break existing waypoints and geometry. fillColor=none is structural (lanes, +transparent containers) and is never replaced. + +Usage: restyle.py --preset [-o ] +""" +import argparse +import colorsys +import json +import os +import re +import sys +import xml.etree.ElementTree as ET + +SKILL_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +# Canonical hue (degrees) of each palette slot in the built-in conventions. +SLOT_HUES = {"primary": 210, "success": 120, "warning": 50, "accent": 30, + "danger": 0, "secondary": 280} +SLOT_ORDER = ["primary", "success", "warning", "accent", "danger", "neutral", "secondary"] + + +def find_preset(name): + """Resolve a preset name/path to its JSON dict (user dir, then built-ins).""" + candidates = [name] if name.endswith(".json") else [ + os.path.expanduser(f"~/.drawio-skill/styles/{name.lower()}.json"), + os.path.join(SKILL_DIR, "styles", "built-in", f"{name.lower()}.json"), + ] + for path in candidates: + if os.path.isfile(path): + with open(path, encoding="utf-8") as f: + return json.load(f) + builtin = os.path.join(SKILL_DIR, "styles", "built-in") + known = sorted(f[:-5] for f in os.listdir(builtin) if f.endswith(".json")) + sys.exit(f"error: preset '{name}' not found (built-ins: {', '.join(known)})") + + +def hue_slot(hexcolor, palette): + """Nearest non-null palette slot for an existing fill color, by hue.""" + r, g, b = (int(hexcolor[i:i + 2], 16) / 255 for i in (1, 3, 5)) + h, l, s = colorsys.rgb_to_hls(r, g, b) + if s < 0.15 or l > 0.97 or l < 0.03: # grey / near-white / near-black + slot = "neutral" + else: + deg = h * 360 + slot = min(SLOT_HUES, key=lambda k: min(abs(deg - SLOT_HUES[k]), + 360 - abs(deg - SLOT_HUES[k]))) + if palette.get(slot): + return slot + for k in SLOT_ORDER: # fallback ladder + if palette.get(k): + return k + sys.exit("error: preset palette has no non-null slots") + + +def get_key(style, key): + m = re.search(rf"(?:^|;){key}=([^;]*)", style) + return m.group(1) if m else None + + +def set_keys(style, **kv): + """Replace/insert style keys, dropping existing occurrences first.""" + for key in kv: + style = re.sub(rf"(?:^|;){key}=[^;]*", "", style) + style = style.strip("; ") + tail = ";".join(f"{k}={v}" for k, v in kv.items() if v is not None) + return (style + ";" if style else "") + tail + ";" + + +def main(): + p = argparse.ArgumentParser(description="Apply a style preset to an existing .drawio.") + p.add_argument("file", help="input .drawio") + p.add_argument("--preset", required=True, help="preset name (user or built-in) or JSON path") + p.add_argument("-o", "--output", help="output path (default: -.drawio)") + args = p.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + preset = find_preset(args.preset) + palette, extras, font = preset["palette"], preset.get("extras", {}), preset["font"] + + vertex_extra = {"fontFamily": font["fontFamily"]} + if extras.get("fontColor"): + vertex_extra["fontColor"] = extras["fontColor"] + if extras.get("sketch"): + vertex_extra["sketch"] = "1" + if extras.get("globalStrokeWidth") not in (None, 1): + vertex_extra["strokeWidth"] = "%g" % extras["globalStrokeWidth"] + + tree = ET.parse(args.file) + slot_map, n_vert, n_edge = {}, 0, 0 + for diagram in tree.getroot().iter("diagram"): + model = diagram.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + sys.stderr.write(f"warning: skipping compressed page '{diagram.get('name', '?')}'\n") + continue + if extras.get("background"): + model.set("background", extras["background"]) + for child in root: + cell = child if child.tag == "mxCell" else child.find("mxCell") + if cell is None: + continue + style = cell.get("style") or "" + if cell.get("edge") == "1": + kv = {} + if extras.get("edgeColor"): + # labelBackgroundColor=none: the default white label box is + # unreadable under a light edgeColor on dark backgrounds + kv.update(strokeColor=extras["edgeColor"], fontColor=extras["edgeColor"], + labelBackgroundColor="none") + if extras.get("sketch"): + kv["sketch"] = "1" + if extras.get("globalStrokeWidth") not in (None, 1): + kv["strokeWidth"] = "%g" % extras["globalStrokeWidth"] + if kv: + cell.set("style", set_keys(style, **kv)) + n_edge += 1 + continue + if cell.get("vertex") != "1": + continue + kv = dict(vertex_extra) + fill = get_key(style, "fillColor") + if fill and re.fullmatch(r"#[0-9A-Fa-f]{6}", fill): + slot = slot_map.setdefault(fill.lower(), hue_slot(fill.lower(), palette)) + pair = palette[slot] + kv.update(fillColor=pair["fillColor"], strokeColor=pair["strokeColor"]) + elif fill is None: + # No fillColor -> draw.io default white fill. Keep its default + # dark text: extras.fontColor would be unreadable on white. + kv.pop("fontColor", None) + cell.set("style", set_keys(style, **kv)) + n_vert += 1 + + out = args.output or "%s-%s.drawio" % (os.path.splitext(args.file)[0], + preset.get("name", "restyled")) + tree.write(out, encoding="utf-8", xml_declaration=False) + remap = ", ".join(f"{c}->{s}" for c, s in sorted(slot_map.items())) + sys.stderr.write(f"wrote {out} ({n_vert} vertices, {n_edge} edges restyled" + + (f"; colors: {remap}" if remap else "") + ")\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/runbook.py b/skills/drawio-skill/scripts/runbook.py new file mode 100644 index 0000000..99ac123 --- /dev/null +++ b/skills/drawio-skill/scripts/runbook.py @@ -0,0 +1,241 @@ +#!/usr/bin/env python3 +"""Turn a flowchart / decision-tree .drawio into a click-through HTML runbook. + +Parses the nodes and edges out of a .drawio and infers a node "type" from its +shape style (ellipse -> start/end, rhombus -> decision, parallelogram -> io, +else process). The ellipse with no incoming edges is taken as the start node. +The output is a single self-contained HTML page: the current node's text +front and center, one button per outgoing edge (labeled with the edge's +choice text, or "Continue" when a node has a single unlabeled successor), a +breadcrumb trail of visited nodes, Back/Restart controls, and an "end" state +on terminal nodes (no outgoing edges). No draw.io CLI is needed -- the XML is +read and the HTML is built directly, so the whole script is testable without +any external tool. + + python3 runbook.py triage.drawio -o triage.html + +Usage: python3 runbook.py [-o out.html] +""" +import argparse +import html +import json +import os +import sys +import xml.etree.ElementTree as ET + + +def parse(path): + """Return (nodes, edges, start_id). + + nodes: {id: {"label": str, "type": "start"|"end"|"decision"|"io"|"process"}} + edges: [{"source": id, "target": id, "label": str}, ...] in document order. + Cells are flattened across pages; UserObject/object wrappers are unwrapped + (id on the wrapper, cell inside) -- mirrors drawiodiff.py parse(). + """ + try: + tree = ET.parse(path) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {path}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + cells, labels = [], {} + for page in pages: + model = page.find("mxGraphModel") + root = model.find("root") if model is not None else None + if root is None: + if (page.text or "").strip(): + sys.stderr.write(f"warning: {path}: a page is compressed, skipped\n") + continue + for child in root: + if child.tag == "mxCell": + cells.append(child) + labels[child.get("id")] = child.get("value") or "" + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + cells.append(inner) + labels[child.get("id")] = child.get("label") or child.get("value") or "" + + parents = {c.get("parent") for c in cells} # ids that have children + order, styles, edges = [], {}, [] + for c in cells: + cid = c.get("id") + if c.get("edge") == "1": + s, t = c.get("source"), c.get("target") + if s and t: + edges.append({"source": s, "target": t, "label": labels.get(cid, "")}) + elif c.get("vertex") == "1" and cid not in parents: # leaf vertices only + style = c.get("style") or "" + if "edgeLabel" in style: + continue + g = c.find("mxGeometry") + if g is not None and g.get("relative") == "1": # edge-label child + continue + order.append(cid) + styles[cid] = style + + indeg = {i: 0 for i in order} + outdeg = {i: 0 for i in order} + for e in edges: + if e["source"] in outdeg: + outdeg[e["source"]] += 1 + if e["target"] in indeg: + indeg[e["target"]] += 1 + + nodes = {} + for nid in order: + style = styles[nid] + if "ellipse" in style: + ntype = "end" if outdeg[nid] == 0 and indeg[nid] > 0 else "start" + elif "rhombus" in style: + ntype = "decision" + elif "parallelogram" in style: + ntype = "io" + else: + ntype = "process" + nodes[nid] = {"label": labels.get(nid, ""), "type": ntype} + + edges = [e for e in edges if e["source"] in nodes and e["target"] in nodes] + + # Start node: the ellipse with no incoming edges; else the unique in-degree-0 + # node; else the first node in document order. Warn to stderr if ambiguous. + ellipse_zero_in = [nid for nid in order if "ellipse" in styles[nid] and indeg[nid] == 0] + if len(ellipse_zero_in) == 1: + start_id = ellipse_zero_in[0] + elif len(ellipse_zero_in) > 1: + sys.stderr.write("warning: multiple ellipse nodes with in-degree 0; picking the first\n") + start_id = ellipse_zero_in[0] + else: + zero_in = [nid for nid in order if indeg[nid] == 0] + if len(zero_in) == 1: + start_id = zero_in[0] + elif len(zero_in) > 1: + sys.stderr.write("warning: no unique in-degree-0 node; picking the first\n") + start_id = zero_in[0] + elif order: + sys.stderr.write("warning: no start node found by heuristics; using the first node\n") + start_id = order[0] + else: + start_id = None + return nodes, edges, start_id + + +def build_html(title, nodes, edges, start_id): + """One self-contained click-through page. nodes: {id:{label,type}}; edges: + [{source,target,label}, ...]; start_id: node id to begin the walk at.""" + adjacency = {} + for e in edges: + adjacency.setdefault(e["source"], []).append({"target": e["target"], "label": e["label"]}) + payload = json.dumps({"nodes": nodes, "edges": adjacency, "start": start_id}).replace(" + +{html.escape(title)} +

{html.escape(title)}

+
+
+
+

+

+
+

End of path -- nothing more to check.

+
+
+ + +
+
+ +""" + + +def main(): + ap = argparse.ArgumentParser(description="Turn a flowchart .drawio into a click-through HTML runbook.") + ap.add_argument("file") + ap.add_argument("-o", "--output", help="output .html (default: alongside input)") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + nodes, edges, start_id = parse(args.file) + if not nodes: + sys.exit(f"error: no nodes found in {args.file}") + if start_id is None: + sys.exit(f"error: no start node found in {args.file}") + + title = os.path.splitext(os.path.basename(args.file))[0] + out = args.output or os.path.splitext(args.file)[0] + ".html" + with open(out, "w", encoding="utf-8") as f: + f.write(build_html(title, nodes, edges, start_id)) + sys.stderr.write(f"wrote {out} ({len(nodes)} nodes, {len(edges)} edges)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/rustimports.py b/skills/drawio-skill/scripts/rustimports.py new file mode 100644 index 0000000..df17b84 --- /dev/null +++ b/skills/drawio-skill/scripts/rustimports.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +"""Extract a Rust crate's module-use graph as autolayout graph JSON. + +The Rust counterpart to pyimports.py / jsimports.py / goimports.py. Treats each +.rs file as a module (path-derived: src/foo/bar.rs -> module foo::bar; main.rs / +lib.rs / mod.rs name the enclosing module), and records intra-crate `use` edges +resolved through Rust's path roots: + + use crate::a::b::C; -> edge to module a::b + use super::sibling; -> resolved against the current module's parent + use self::child::Item;-> resolved against the current module + use other_crate::...; / use std::...; -> external, ignored + +Brace groups (`use crate::a::{B, C};`, `use crate::{a, b};`) are expanded. +Transitive reduction is on by default so the diagram stays readable. + + python3 rustimports.py ./mycrate --group -o graph.json + python3 autolayout.py graph.json -o diagram.drawio + +Parsing is regex-based, not a full parser: inline `mod { ... }` blocks are not +split out, `#[cfg]`-gated modules are always included, and 2015-edition bare +intra-crate paths (without `crate::`) are not resolved. + +Usage: python3 rustimports.py [-o graph.json] [--direction TB|LR] + [--group] [--no-reduce] +""" +import argparse +import json +import os +import re +import subprocess +import sys + +USE = re.compile(r"\buse\s+([^;]+);") + + +def crate_name(root): + cargo = os.path.join(root, "Cargo.toml") + if os.path.exists(cargo): + with open(cargo, encoding="utf-8", errors="ignore") as f: + m = re.search(r'(?m)^\s*name\s*=\s*"([^"]+)"', f.read()) + if m: + return m.group(1) + return "crate" + + +def discover(root): + """Map module path (tuple of segments; () is the crate root) -> file path.""" + root = os.path.abspath(root) + src = os.path.join(root, "src") if os.path.isdir(os.path.join(root, "src")) else root + modules = {} + for dirpath, dirs, files in os.walk(src): + dirs[:] = [d for d in dirs if d != "target" and not d.startswith(".")] + for fn in files: + if not fn.endswith(".rs"): + continue + parts = os.path.relpath(os.path.join(dirpath, fn), src)[:-3].split(os.sep) + if parts[-1] == "mod": + parts = parts[:-1] + if len(parts) == 1 and parts[0] in ("main", "lib"): + parts = [] # crate root + modules[tuple(parts)] = os.path.join(dirpath, fn) + return modules, src + + +def split_top(inner): + """Split a brace group on top-level commas, ignoring nested braces.""" + out, depth, cur = [], 0, "" + for ch in inner: + if ch == "{": + depth += 1 + elif ch == "}": + depth -= 1 + if ch == "," and depth == 0: + out.append(cur) + cur = "" + else: + cur += ch + if cur.strip(): + out.append(cur) + return out + + +def base_segments(prefix, current): + """Classify a `use` path prefix into intra-crate base segments, or None.""" + segs = [s for s in (p.strip() for p in prefix.split("::")) if s] + if not segs: + return None + if segs[0] == "crate": + return segs[1:] + if segs[0] == "self": + return list(current) + segs[1:] + if segs[0] == "super": + n = 0 + while segs and segs[0] == "super": + n += 1 + segs = segs[1:] + if n > len(current): + return None # climbs above the crate root + return list(current)[: len(current) - n] + segs + return None # std / external crate + + +def resolve(parts, modules, current): + """Longest known module prefix of `parts` (a tuple), or None.""" + if not parts: + return () if () in modules and () != tuple(current) else None + p = list(parts) + while p: + if tuple(p) in modules and tuple(p) != tuple(current): + return tuple(p) + p = p[:-1] + return None + + +def edges_of(current, path, modules): + """Intra-crate module paths used by the module at `current`.""" + found = set() + try: + with open(path, encoding="utf-8", errors="ignore") as f: + src = f.read() + except OSError: + return found + for stmt in USE.findall(src): + if "{" in stmt: + prefix = stmt[: stmt.index("{")] + inner = stmt[stmt.index("{") + 1: stmt.rindex("}")] if "}" in stmt else "" + leaves = split_top(inner) + else: + prefix, leaves = stmt, [None] + base = base_segments(prefix, current) + if base is None: + continue + for leaf in leaves: + segs = list(base) + if leaf: + first = leaf.strip().split("::")[0].split()[0] + if first and first not in ("self", "*"): + segs.append(first) + target = resolve(tuple(segs), modules, current) + if target is not None and target != current: + found.add(target) + return found + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def main(): + ap = argparse.ArgumentParser(description="Rust module-use graph -> autolayout graph JSON.") + ap.add_argument("crate", help="crate directory (contains Cargo.toml and/or src/)") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="box modules by their parent module path (nested)") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + args = ap.parse_args() + + modules, _ = discover(args.crate) + if not modules: + sys.exit(f"error: no .rs modules found under {args.crate}") + name = crate_name(args.crate) + mid = lambda parts: name if not parts else "::".join(parts) + edges = sorted({(mid(m), mid(t)) for m, path in modules.items() + for t in edges_of(m, path, modules)}) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce([mid(m) for m in modules], edges) + + def node(parts): + d = {"id": mid(parts), "label": name if not parts else parts[-1]} + if args.group and len(parts) > 1: + d["group"] = "/".join(parts[:-1]) # parent module path -> nested boxes + return d + + graph = { + "direction": args.direction, + "nodes": [node(m) for m in modules], + "edges": [{"source": s, "target": t} for s, t in edges], + } + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(modules)} modules, {len(edges)} edges{note}\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/seqlayout.py b/skills/drawio-skill/scripts/seqlayout.py new file mode 100644 index 0000000..b5b8c33 --- /dev/null +++ b/skills/drawio-skill/scripts/seqlayout.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Deterministic sequence-diagram layout: message list JSON -> .drawio XML. + +Sequence diagrams are the easiest type to get wrong by hand-placing +coordinates (lifelines, activation bars and message arrows all share exact +x/y math) and the least suited to Graphviz — but their geometry is pure +arithmetic: participants split the x axis, messages advance the y axis. +This script computes all of it, using the same official styles as +references/diagram-types.md (umlLifeline shapes, block/open arrows). + + python3 seqlayout.py seq.json -o diagram.drawio + +Input JSON: + { + "title": "Login flow", # optional page name + "participants": [ + {"id": "u", "label": "User", "actor": true}, + {"id": "s", "label": "Server"} # order = left-to-right order + ], + "messages": [ + {"from": "u", "to": "s", "label": "POST /login"}, # sync (solid, filled arrow) + {"from": "s", "to": "s", "label": "validate()"}, # self message + {"from": "s", "to": "u", "label": "200 OK", "return": true}, # return (grey dashed) + {"from": "u", "to": "s", "label": "notify", "async": true}, # async (dashed, open arrow) + {"note": "token cached", "over": "s"} # note beside a lifeline + ] + } + +Activation bars are automatic: a sync/async message opens a bar on the +target, a return message closes the sender's bar, and bars still open at the +end run to the bottom. Override per message with "activate": false (don't +open on target) or "deactivate": true (close the sender's bar after this +message). Arrows attach to the bar edge when a bar is active, else to the +lifeline. Fragments (alt/loop/opt frames) are out of scope — add them in +draw.io afterwards. + +Usage: python3 seqlayout.py [-o diagram.drawio] +""" +import argparse +import json +import sys +from xml.sax.saxutils import escape + +LIFELINE_W, HEADER_H = 100, 40 +BAR_W = 10 +TOP, ROW, SELF_ROW, NOTE_ROW, BOTTOM_PAD = 40, 50, 70, 60, 40 +MIN_SPACING = 200 + +LIFELINE = ("shape=umlLifeline;perimeter=lifelinePerimeter;whiteSpace=wrap;html=1;" + "container=1;dropTarget=0;collapsible=0;recursiveResize=0;outlineConnect=0;" + f"portConstraint=eastwest;size={HEADER_H};") +# Actor lifelines render a stick figure in the header — anchor the name at +# the header bottom (white-backed) so figure and label don't overlap. +ACTOR = LIFELINE + ("participant=umlActor;verticalAlign=bottom;" + "spacingBottom=-14;labelBackgroundColor=#ffffff;") +BAR = "html=1;points=[];perimeter=orthogonalPerimeter;outlineConnect=0;fillColor=#ffffff;" +NOTE = ("shape=note;whiteSpace=wrap;html=1;size=14;fillColor=#fff2cc;" + "strokeColor=#d6b656;") +SYNC = "html=1;verticalAlign=bottom;endArrow=block;curved=0;rounded=0;" +ASYNC = "html=1;verticalAlign=bottom;endArrow=open;dashed=1;curved=0;rounded=0;" +RETURN = ("html=1;verticalAlign=bottom;endArrow=open;dashed=1;curved=0;rounded=0;" + "strokeColor=#999999;fontColor=#999999;") + + +def attr(value): + return escape(str(value), {'"': """, "\n": " "}) + + +def frac(y, top, height): + return round(max(0.0, min(1.0, (y - top) / height)), 4) + + +def layout(spec): + parts = spec["participants"] + if not parts: + sys.exit("error: no participants") + order = {p["id"]: i for i, p in enumerate(parts)} + if len(order) != len(parts): + sys.exit("error: duplicate participant ids") + + # x axis: uniform spacing, widened if any label needs it (~7px/char). + spacing = max([MIN_SPACING] + [7 * len(str(p.get("label", p["id"]))) + 80 for p in parts]) + spacing = -(-spacing // 10) * 10 # snap up to the grid + cx = {p["id"]: TOP + i * spacing + LIFELINE_W // 2 for i, p in enumerate(parts)} + + # y axis: walk the messages once, assigning each row a y position and + # tracking one open activation bar per participant ({pid: start_y}). + y = TOP + HEADER_H + 50 + rows, open_bar, bars = [], {}, [] # bars: (pid, y0, y1) + + def close(pid, at): + if pid in open_bar: + bars.append((pid, open_bar.pop(pid), at)) + + for i, m in enumerate(spec.get("messages", [])): + if "note" in m: + rows.append(("note", m, y)) + y += NOTE_ROW + continue + src, dst = m["from"], m["to"] + if src not in order or dst not in order: + sys.exit(f"error: message {i} references unknown participant") + is_return = m.get("return", False) + if src == dst: + rows.append(("self", m, y)) + y += SELF_ROW + continue + rows.append(("msg", m, y)) + if is_return: + close(src, y) # returning ends the caller's work + elif m.get("activate", True) and dst not in open_bar: + open_bar[dst] = y # call starts work on the target + if m.get("deactivate"): + close(src, y) + y += ROW + height = y + BOTTOM_PAD - TOP + for pid in list(open_bar): + close(pid, TOP + height - BOTTOM_PAD) + + cells = [] + for p in parts: + style = ACTOR if p.get("actor") else LIFELINE + cells.append( + f' \n' + f' \n' + " ") + + # Activation bars: children of their lifeline (coordinates relative to it). + bar_of = {} # pid -> list of (y0, y1, cell_id) + for n, (pid, y0, y1) in enumerate(bars): + bid = f"act{n}" + bar_of.setdefault(pid, []).append((y0, y1, bid)) + cells.append( + f' \n' + f' \n' + " ") + + def anchor(pid, my, side): + """(cell_id, exitX-style fragment values) for a message endpoint: the + activation bar's edge when one is active at this y, else the lifeline.""" + for y0, y1, bid in bar_of.get(pid, []): + if y0 <= my <= y1: + return bid, (1 if side == "right" else 0), frac(my, y0, y1 - y0) + return pid, 0.5, frac(my, TOP, height) + + for i, (kind, m, my) in enumerate(rows): + if kind == "note": + pid = m.get("over") + if pid not in cx: + sys.exit(f"error: note {i} is over unknown participant {pid!r}") + w = max(120, 7 * len(str(m["note"])) + 30) + cells.append( + f' \n' + f' \n' + " ") + continue + src, dst = m["from"], m["to"] + style = RETURN if m.get("return") else ASYNC if m.get("async") else SYNC + if kind == "self": + sid, sx, sy = anchor(src, my, "right") + _, tx, ty = anchor(src, my + 30, "right") + loop_x = cx[src] + 60 + cells.append( + f' \n' + f' \n' + f' ' + f'' + f'\n' + " \n" + " ") + continue + rightward = order[src] < order[dst] + sid, sx, sy = anchor(src, my, "right" if rightward else "left") + tid, tx, ty = anchor(dst, my, "left" if rightward else "right") + cells.append( + f' \n' + f' \n' + " ") + + name = attr(spec.get("title", "Sequence")) + return ( + f'\n \n' + ' \n' + " \n" + ' \n' + ' \n' + + "\n".join(cells) + + "\n \n \n \n\n" + ) + + +def main(): + ap = argparse.ArgumentParser(description="Sequence-diagram JSON -> draw.io XML.") + ap.add_argument("input", help="sequence JSON file") + ap.add_argument("-o", "--output", help="output .drawio path (default: stdout)") + args = ap.parse_args() + with open(args.input, encoding="utf-8") as f: + spec = json.load(f) + xml = layout(spec) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(xml) + print(f"wrote {args.output} ({len(spec['participants'])} participants, " + f"{len(spec.get('messages', []))} messages)", file=sys.stderr) + else: + sys.stdout.write(xml) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/shapesearch.py b/skills/drawio-skill/scripts/shapesearch.py new file mode 100644 index 0000000..f296565 --- /dev/null +++ b/skills/drawio-skill/scripts/shapesearch.py @@ -0,0 +1,162 @@ +#!/usr/bin/env python3 +"""Search 10k+ official draw.io shapes for their exact style strings. + +Resolves a keyword query (e.g. "aws lambda", "uml actor", "k8s pod") to the +matching palette shapes so a diagram can use the real draw.io `style=` string +instead of a hand-guessed one. Covers AWS / Azure / GCP / Cisco / Kubernetes / +UML / BPMN / P&ID / electrical / flowchart / network / general shape sets. + +Based on the search in jgraph/drawio-mcp (Apache-2.0): tag map with exact + +Soundex matching, strict AND first, scored OR fallback. The matched set is +identical to upstream; the one addition is a tiebreaker that, among shapes with +the same tag score, prefers ones whose title contains the query terms verbatim +(so "dynamodb" returns the shape titled "DynamoDB", not a neighbor merely tagged +with it). The bundled index (data/shape-index.json.gz) is the upstream draw.io +shape data — see data/SHAPE-INDEX-NOTICE.md. + +Usage: + python3 shapesearch.py "aws lambda" [--limit N] [--json] +""" +import argparse +import gzip +import json +import os +import re +import sys + +INDEX = os.path.join(os.path.dirname(__file__), "..", "data", "shape-index.json.gz") +_SOUNDEX_MAP = "01230120022455012603010202" # A..Z digit codes +_TRAIL = re.compile(r"\.*\d*$") # strip trailing digits/dots before soundex + + +def soundex(name): + if not name: + return "" + s = [name[0].upper()] + si = 1 + for ch in name[1:]: + c = ord(ch.upper()) - 65 + if 0 <= c <= 25 and _SOUNDEX_MAP[c] != "0": + code = _SOUNDEX_MAP[c] + if code != s[si - 1]: + s.append(code) + si += 1 + if si > 3: + break + s += ["0"] * (4 - len(s)) + return "".join(s[:4]) + + +def build_tag_map(shapes): + """tag (and its Soundex) -> set of shape indices.""" + tag_map = {} + for i, shape in enumerate(shapes): + raw = shape.get("tags") + if not raw: + continue + seen = set() + for token in re.sub(r"[/,()]", " ", raw.lower()).split(" "): + if len(token) < 2 or token in seen: + continue + seen.add(token) + tag_map.setdefault(token, set()).add(i) + sx = soundex(_TRAIL.sub("", token)) + if sx and sx != token and sx not in seen: + seen.add(sx) + tag_map.setdefault(sx, set()).add(i) + return tag_map + + +def split_compound(token): + """'pid2misc' -> ['pid','misc']; 'discInst' -> ['disc','inst'].""" + spaced = re.sub(r"([a-z])([A-Z])", r"\1 \2", token) + spaced = re.sub(r"([a-zA-Z])(\d)", r"\1 \2", spaced) + spaced = re.sub(r"(\d)([a-zA-Z])", r"\1 \2", spaced) + return [p for p in spaced.lower().split() if len(p) >= 2] + + +def match_term(tag_map, term): + exact = set(tag_map.get(term, set())) + phonetic = set() + sx = soundex(_TRAIL.sub("", term)) + if sx and sx != term: + phonetic = {i for i in tag_map.get(sx, set()) if i not in exact} + return exact, phonetic + + +def search(shapes, tag_map, query, limit): + if not query: + return [] + terms, seen = [], set() + for raw in query.lower().split(): + subs = split_compound(raw) or ([raw] if len(raw) >= 2 else []) + for t in subs: + if t not in seen: + seen.add(t) + terms.append(t) + if not terms: + return [] + + term_matches = [match_term(tag_map, t) for t in terms] + + # Strict AND across all terms first. + and_set = None + for exact, phonetic in term_matches: + combined = exact | phonetic + and_set = combined if and_set is None else (and_set & combined) + if not and_set: + break + + # Score: +1.0 exact, +0.5 Soundex-only, per term. AND results if any, else OR. + scores = {} + pool = and_set if and_set else None + for exact, phonetic in term_matches: + for idx in exact: + if pool is None or idx in pool: + scores[idx] = scores.get(idx, 0) + 1.0 + for idx in phonetic: + if (pool is None or idx in pool) and idx not in exact: + scores[idx] = scores.get(idx, 0) + 0.5 + + # Rank by tag score desc, then by how many query terms appear verbatim in the + # title, then casefolded title, then index. The title-hit tiebreak (our one + # addition over upstream) only reorders *within* an equal tag-score group, so + # e.g. the shape literally titled "DynamoDB" ranks above a neighbor that is + # merely tagged `dynamodb` (like "Attribute"). The trailing index keeps ties + # deterministic. + term_set = set(terms) + + def title_hits(idx): + toks = set(re.split(r"[^a-z0-9]+", shapes[idx].get("title", "").casefold())) + return len(term_set & toks) + + ranked = sorted(scores, key=lambda i: (-scores[i], -title_hits(i), + shapes[i].get("title", "").casefold(), i)) + return [{"style": shapes[i]["style"], "w": shapes[i]["w"], + "h": shapes[i]["h"], "title": shapes[i]["title"]} for i in ranked[:limit]] + + +def main(): + ap = argparse.ArgumentParser(description="Search official draw.io shapes for their style strings.") + ap.add_argument("query", help='keywords, e.g. "aws lambda" or "uml actor"') + ap.add_argument("--limit", type=int, default=10) + ap.add_argument("--json", action="store_true", help="emit JSON instead of a table") + args = ap.parse_args() + + if not os.path.exists(INDEX): + sys.exit(f"error: shape index not found at {INDEX}") + with gzip.open(INDEX, "rt", encoding="utf-8") as f: + shapes = json.load(f) + + results = search(shapes, build_tag_map(shapes), args.query, args.limit) + if not results: + sys.exit(f"no shapes matched {args.query!r}") + if args.json: + print(json.dumps(results, indent=2, ensure_ascii=False)) + else: + for r in results: + print(f"{r['title']} ({r['w']}x{r['h']})\n {r['style']}") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/sqlerd.py b/skills/drawio-skill/scripts/sqlerd.py new file mode 100644 index 0000000..11e29fd --- /dev/null +++ b/skills/drawio-skill/scripts/sqlerd.py @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 +"""Extract an ER diagram from SQL DDL as autolayout graph JSON. + +Parses ``CREATE TABLE`` statements (regex + paren matching — no SQL library), +one node per table listing its columns with PK/FK markers, and one +crow's-foot edge per foreign key (many side at the referencing table). +The output feeds autolayout.py: + + python3 sqlerd.py schema.sql -o graph.json + python3 autolayout.py graph.json -o erd.drawio + +Understood per table: column name + type, inline ``PRIMARY KEY`` / +``REFERENCES tab(col)``, table-level ``PRIMARY KEY (...)`` and +``[CONSTRAINT x] FOREIGN KEY (col) REFERENCES tab(col)``. Quoted identifiers +("t", `t`, [t]) and ``schema.table`` prefixes are normalized; edges land only +on tables defined in the scanned files. Dialect-specific clauses beyond that +(partitioning, generated columns, …) are simply ignored — worst case a column +line is skipped, never a wrong edge. + +Usage: python3 sqlerd.py [-o graph.json] + [--direction TB|LR] [--group] [--no-types] +""" +import argparse +import glob +import json +import os +import re +import sys + +TABLE_STYLE = ("rounded=0;whiteSpace=wrap;html=1;align=left;verticalAlign=top;" + "spacingLeft=6;spacingTop=4;fillColor=#dae8fc;strokeColor=#6c8ebf;") +# orthogonalEdgeStyle (not entityRelationEdgeStyle) so the edge honours the +# obstacle-avoiding waypoints dot computed; ER arrows give the crow's foot. +ER_EDGE = ("edgeStyle=orthogonalEdgeStyle;html=1;rounded=0;fontSize=11;" + "labelBackgroundColor=#ffffff;" + "startArrow=ERmany;startFill=0;endArrow=ERone;endFill=0;") + +_COMMENT = re.compile(r"/\*.*?\*/|--[^\n]*", re.S) +_CREATE = re.compile(r"\bCREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?([\w.\"`\[\]]+)\s*\(", + re.I) +_FK = re.compile(r"FOREIGN\s+KEY\s*\(([^)]+)\)\s*REFERENCES\s+([\w.\"`\[\]]+)\s*(?:\(([^)]+)\))?", + re.I) +_PK = re.compile(r"PRIMARY\s+KEY\s*\(([^)]+)\)", re.I) +_INLINE_REF = re.compile(r"\bREFERENCES\s+([\w.\"`\[\]]+)", re.I) +_SKIP = re.compile(r"^\s*(CONSTRAINT|UNIQUE|CHECK|KEY|INDEX|FULLTEXT|SPATIAL|EXCLUDE|LIKE)\b", + re.I) + + +def ident(raw): + """Normalize an identifier: strip quoting, keep the last dotted part.""" + name = raw.strip().strip('"`[]').split(".")[-1].strip('"`[]') + return name.lower() + + +def split_columns(body): + """Split a CREATE TABLE body on top-level commas.""" + items, depth, cur = [], 0, [] + for ch in body: + if ch == "(": + depth += 1 + elif ch == ")": + depth -= 1 + if ch == "," and depth == 0: + items.append("".join(cur).strip()) + cur = [] + else: + cur.append(ch) + if "".join(cur).strip(): + items.append("".join(cur).strip()) + return items + + +def parse_tables(text): + """{table: {"schema", "columns": [(name, type)], "pks": set, "fks": [(col, table)]}}""" + text = _COMMENT.sub("", text) + tables = {} + for m in _CREATE.finditer(text): + raw_name = m.group(1) + depth, i = 1, m.end() + while i < len(text) and depth: + if text[i] == "(": + depth += 1 + elif text[i] == ")": + depth -= 1 + i += 1 + body = text[m.end():i - 1] + name = ident(raw_name) + parts = raw_name.strip().strip('"`[]').split(".") + schema = ident(parts[-2]) if len(parts) > 1 else "" + cols, pks, fks = [], set(), [] + for item in split_columns(body): + fk = _FK.search(item) + if fk: + for col in fk.group(1).split(","): + fks.append((ident(col), ident(fk.group(2)))) + continue + pk = _PK.search(item) + if pk and _SKIP.match(item) is None and item.upper().lstrip().startswith("PRIMARY"): + pks.update(ident(c) for c in pk.group(1).split(",")) + continue + if _SKIP.match(item): + continue + toks = item.split() + if len(toks) < 2: + continue + col, ctype = ident(toks[0]), toks[1].rstrip(",") + cols.append((col, ctype)) + if re.search(r"\bPRIMARY\s+KEY\b", item, re.I): + pks.add(col) + ref = _INLINE_REF.search(item) + if ref: + fks.append((col, ident(ref.group(1)))) + tables[name] = {"schema": schema, "columns": cols, "pks": pks, "fks": fks} + return tables + + +def main(): + ap = argparse.ArgumentParser(description="SQL DDL -> ER diagram graph JSON.") + ap.add_argument("path", help=".sql file or directory containing .sql files") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", help="group tables by schema") + ap.add_argument("--no-types", action="store_true", + help="list column names only (hide the SQL types)") + args = ap.parse_args() + + files = ([args.path] if os.path.isfile(args.path) + else sorted(glob.glob(os.path.join(args.path, "**", "*.sql"), recursive=True))) + tables = {} + for path in files: + with open(path, encoding="utf-8") as f: + tables.update(parse_tables(f.read())) + if not tables: + sys.exit(f"error: no CREATE TABLE statements found under {args.path}") + + nodes, edges = [], [] + for name, t in tables.items(): + fk_cols = {c for c, _ in t["fks"]} + lines = [name] + for col, ctype in t["columns"]: + mark = "PK " if col in t["pks"] else "FK " if col in fk_cols else "" + lines.append(f"{mark}{col}" + ("" if args.no_types else f": {ctype}")) + width = max(160, -(-max(7 * len(l) + 30 for l in lines) // 10) * 10) + height = -(-(30 + 20 * len(t["columns"])) // 10) * 10 + node = {"id": name, "label": "\n".join(lines), "style": TABLE_STYLE, + "width": width, "height": height} + if args.group and t["schema"]: + node["group"] = t["schema"] + nodes.append(node) + for col, ref in t["fks"]: + if ref in tables and ref != name: + edges.append({"source": name, "target": ref, "label": col, + "style": ER_EDGE}) + + graph = {"direction": args.direction, "nodes": nodes, "edges": edges} + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + sys.stderr.write(f"{len(nodes)} tables, {len(edges)} foreign keys\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/svgflow.py b/skills/drawio-skill/scripts/svgflow.py new file mode 100644 index 0000000..ece652e --- /dev/null +++ b/skills/drawio-skill/scripts/svgflow.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +"""Make a diagram's edges *flow* — an animated data-flow SVG. + +Exports a .drawio to SVG (or takes an .svg directly) and turns every edge into a +marching-ants animation: dashes travel along each connector in the direction of +the arrow, so the diagram shows data/flow moving through it. The result is a +single self-contained .svg that loops forever in any browser — nice for a README +(GitHub renders SVG), a docs page, or a slide background. + + python3 svgflow.py architecture.drawio -o architecture-flow.svg + python3 svgflow.py already-exported.svg -o flow.svg + +Edges are found by draw.io's own marker: connector *lines* carry +`pointer-events="stroke"` (shape outlines and arrowheads use `="all"`), so only +the real edges animate — arrowheads and shapes stay put. `--speed` sets seconds +per cycle, `--dash` the dash pattern, `--reverse` flips the flow direction. + +Usage: python3 svgflow.py [-o out.svg] + [--speed SEC] [--dash "6 4"] [--reverse] +""" +import argparse +import os +import re +import subprocess +import sys +import tempfile + +EDGE_PATH = re.compile(r'()[^>])*pointer-events="stroke"(?:(?!/?>)[^>])*/?>)') + + +def to_svg(path): + """Return SVG text for a .drawio (export via CLI) or .svg (read directly).""" + if path.lower().endswith(".svg"): + with open(path, encoding="utf-8") as f: + return f.read() + with tempfile.TemporaryDirectory() as tmp: + out = os.path.join(tmp, "d.svg") + r = subprocess.run(["drawio", "-x", "-f", "svg", "--embed-svg-images", "-o", out, path], + capture_output=True) + if r.returncode != 0 or not os.path.exists(out): + sys.exit("error: draw.io SVG export failed (is the draw.io CLI installed?)") + with open(out, encoding="utf-8") as f: + return f.read() + + +def animate(svg, speed, dash, reverse): + """Tag edge paths and inject the flow keyframes. Returns (svg, edge_count).""" + svg, n = EDGE_PATH.subn(r'\1class="dio-flow" \2', svg) + # One dash+gap of travel per cycle => seamless loop. Reverse flips the sign. + period = sum(float(x) for x in dash.split()) or 10 + offset = period if reverse else -period + style = (f"") + svg = re.sub(r"(]*>)", r"\1" + style, svg, count=1) + return svg, n + + +def main(): + ap = argparse.ArgumentParser(description="Animate a diagram's edges into a flowing SVG.") + ap.add_argument("file", help=".drawio (exported to SVG) or an .svg") + ap.add_argument("-o", "--output", help="output .svg (default: -flow.svg)") + ap.add_argument("--speed", type=float, default=1.2, help="seconds per flow cycle (default 1.2)") + ap.add_argument("--dash", default="6 4", help='dash pattern, e.g. "6 4" (default)') + ap.add_argument("--reverse", action="store_true", help="flow toward the source") + args = ap.parse_args() + + if not os.path.isfile(args.file): + sys.exit(f"error: {args.file} not found") + svg = to_svg(args.file) + if " element found in the exported output") + svg, n = animate(svg, args.speed, args.dash, args.reverse) + if n == 0: + sys.stderr.write("warning: no edges found to animate " + "(a diagram with no connectors?)\n") + + out = args.output or os.path.splitext(args.file)[0] + "-flow.svg" + with open(out, "w", encoding="utf-8") as f: + f.write(svg) + sys.stderr.write(f"wrote {out} ({n} edge{'s' if n != 1 else ''} animated)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/tfimports.py b/skills/drawio-skill/scripts/tfimports.py new file mode 100644 index 0000000..13a4d2a --- /dev/null +++ b/skills/drawio-skill/scripts/tfimports.py @@ -0,0 +1,262 @@ +#!/usr/bin/env python3 +"""Extract a Terraform configuration's resource graph as autolayout graph JSON. + +Parses ``.tf`` files with a small regex + brace-matching pass (no HCL library +needed), builds resource-reference edges (``aws_iam_role.lambda.arn`` inside +another resource's body -> edge), and resolves each resource type to its +official draw.io cloud icon via the bundled shape index — AWS (aws4 set), +Azure (azure2 set) and GCP (Google Cloud icon set). The output feeds +autolayout.py: + + python3 tfimports.py ./infra -o graph.json + python3 autolayout.py graph.json -o infra.drawio + +Nodes are the ``resource`` and ``module`` blocks declared in the scanned files; +data sources, variables, locals and providers are ignored. A reference is any +``type.name`` / ``module.name`` token in a resource body that matches a +declared node — attribute chains (``aws_s3_bucket.logs.arn``), ``"${...}"`` +interpolations and ``depends_on`` entries all count. Transitive reduction +(Graphviz ``tred``) keeps big graphs readable; ``--no-reduce`` keeps every +edge. Heredoc bodies with unbalanced braces are the one known parse limit. + +Usage: python3 tfimports.py [-o graph.json] + [--direction TB|LR] [--group] [--no-reduce] [--no-icons] +""" +import argparse +import glob +import importlib.util +import json +import os +import re +import subprocess +import sys + +# provider prefix of the resource type -> (icon query prefix, style predicate). +# The predicate pins results to the modern shape set for that cloud — a bare +# keyword search happily returns another vendor's icon (e.g. "kubernetes +# deployment" -> Azure Arc), so set filtering is what makes resolution safe. +PROVIDERS = { + "aws": ("aws", lambda st: "mxgraph.aws4" in st), + "azurerm": ("azure", lambda st: "img/lib/azure2" in st), + "azuread": ("azure", lambda st: "img/lib/azure2" in st), + "google": ("gcp", lambda st: "editableCssRules" in st), +} + +# Resource types whose derived query ("aws lambda function") misses or mis-hits +# the intended icon; values are the query that finds it. Keep alphabetical. +QUERY_OVERRIDES = { + "aws_alb": "aws elastic load balancing", + "aws_apigatewayv2_api": "aws api gateway", + "aws_autoscaling_group": "aws ec2 auto scaling", + "aws_cloudwatch_log_group": "aws cloudwatch", + "aws_db_instance": "aws rds", + "aws_dynamodb_table": "aws dynamodb", + "aws_ecr_repository": "aws elastic container registry", + "aws_ecs_cluster": "aws elastic container service", + "aws_ecs_service": "aws elastic container service", + "aws_ecs_task_definition": "aws elastic container service", + "aws_efs_file_system": "aws elastic file system", + "aws_eks_cluster": "aws elastic kubernetes service", + "aws_elasticache_cluster": "aws elasticache", + "aws_iam_policy": "aws identity and access management", + "aws_instance": "aws ec2", + "aws_kms_key": "aws key management service", + "aws_lambda_function": "aws lambda", + "aws_lb": "aws elastic load balancing", + "aws_rds_cluster": "aws aurora", + "aws_s3_bucket": "aws simple storage service", + "aws_secretsmanager_secret": "aws secrets manager", + "aws_sfn_state_machine": "aws step functions", + "aws_sns_topic": "aws simple notification service", + "aws_sqs_queue": "aws simple queue service", + "azurerm_app_service": "azure app services", + "azurerm_application_gateway": "azure application gateways", + "azurerm_cosmosdb_account": "azure cosmos db", + "azurerm_kubernetes_cluster": "azure kubernetes services", + "azurerm_linux_function_app": "azure function apps", + "azurerm_linux_virtual_machine": "azure virtual machine", + "azurerm_linux_web_app": "azure app services", + "azurerm_mssql_database": "azure sql database", + "azurerm_mssql_server": "azure sql database", + "azurerm_servicebus_namespace": "azure service bus", + "azurerm_storage_account": "azure storage accounts", + "azurerm_virtual_network": "azure virtual networks", + "azurerm_windows_function_app": "azure function apps", + "azurerm_windows_virtual_machine": "azure virtual machine", + "azurerm_windows_web_app": "azure app services", + "google_cloudfunctions2_function": "gcp cloud functions", + "google_cloudfunctions_function": "gcp cloud functions", + "google_compute_instance": "gcp compute engine", + "google_container_cluster": "gcp kubernetes engine", + "google_redis_instance": "gcp memorystore", + "google_sql_database_instance": "gcp cloud sql", + "google_storage_bucket": "gcp cloud storage", +} + +_COMMENT = re.compile(r"/\*.*?\*/|(?:#|//)[^\n]*", re.S) +_BLOCK = re.compile(r'^[ \t]*(resource|module)[ \t]+"([\w.-]+)"(?:[ \t]+"([\w.-]+)")?[ \t]*\{', re.M) +_REF = re.compile(r"\b([a-z][a-z0-9_]*\.[A-Za-z_][A-Za-z0-9_-]*)") + + +def parse_blocks(text): + """Yield (kind, label1, label2, body) for resource/module blocks.""" + text = _COMMENT.sub("", text) + for m in _BLOCK.finditer(text): + depth, i = 1, m.end() + while i < len(text) and depth: + if text[i] == "{": + depth += 1 + elif text[i] == "}": + depth -= 1 + i += 1 + yield m.group(1), m.group(2), m.group(3), text[m.end():i - 1] + + +def load_shapesearch(): + path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "shapesearch.py") + spec = importlib.util.spec_from_file_location("shapesearch", path) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +class IconResolver: + """Resolve a Terraform resource type to an official draw.io icon style.""" + + def __init__(self): + import gzip + self.ss = load_shapesearch() + with gzip.open(self.ss.INDEX, "rt", encoding="utf-8") as f: + self.shapes = json.load(f) + self.tag_map = self.ss.build_tag_map(self.shapes) + self.cache = {} + + def _and_styles(self, words): + """Style strings of shapes whose tags match EVERY query word. + shapesearch.search() falls back to scored OR matching when the AND set + is empty — fine interactively, but here a partial match means a visibly + wrong icon, so results outside this set are rejected and the caller's + back-off handles the miss (a plain box beats a wrong icon).""" + idxs = None + for t in words: + exact, phonetic = self.ss.match_term(self.tag_map, t) + s = exact | phonetic + idxs = s if idxs is None else idxs & s + if not idxs: + return set() + return {self.shapes[i]["style"] for i in idxs} + + def resolve(self, rtype): + if rtype in self.cache: + return self.cache[rtype] + provider = rtype.split("_", 1)[0] + hit = None + if provider in PROVIDERS: + prefix, want = PROVIDERS[provider] + words = (QUERY_OVERRIDES.get(rtype) or + f"{prefix} {rtype.split('_', 1)[1].replace('_', ' ')}").split() + # Back off one trailing word at a time: "aws lambda event source + # mapping" eventually matches on "aws lambda". + while len(words) > 1 and hit is None: + allowed = self._and_styles(words) + good = [r for r in + self.ss.search(self.shapes, self.tag_map, " ".join(words), 40) + if r["style"] in allowed and want(r["style"]) + and "group" not in r["style"].lower()] + # Prefer aws4 service icons (resIcon=) over scenario glyphs. + hit = next((r for r in good if "resIcon=" in r["style"]), None) or \ + (good[0] if good else None) + words = words[:-1] + if hit and max(hit["w"], hit["h"]) < 44: + # Some sets (GCP) ship tiny nominal sizes; scale up so the icon + # is not dwarfed by its label. aspect=fixed keeps the ratio. + f = 48 / max(hit["w"], hit["h"]) + hit = dict(hit, w=round(hit["w"] * f), h=round(hit["h"] * f)) + self.cache[rtype] = hit + return hit + + +def transitive_reduce(nodes, edges): + """Drop edges implied by a longer path, via Graphviz `tred`.""" + idx = {n: i for i, n in enumerate(nodes)} + dot = "digraph{" + "".join(f"{idx[s]}->{idx[t]};" for s, t in edges) + "}" + try: + out = subprocess.run(["tred"], input=dot, capture_output=True, + text=True, check=True).stdout + except (FileNotFoundError, subprocess.CalledProcessError) as exc: + sys.stderr.write(f"warning: tred unavailable, keeping all edges ({exc})\n") + return edges + rev = {i: n for n, i in idx.items()} + return [(rev[int(a)], rev[int(b)]) for a, b in re.findall(r"(\d+)\s*->\s*(\d+)", out)] + + +def main(): + ap = argparse.ArgumentParser(description="Terraform resource graph -> autolayout graph JSON.") + ap.add_argument("path", help=".tf file or directory containing .tf files") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group resources into containers by service (aws_s3_* -> s3)") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + ap.add_argument("--no-icons", action="store_true", + help="plain boxes instead of official cloud icons") + args = ap.parse_args() + + files = ([args.path] if os.path.isfile(args.path) + else sorted(glob.glob(os.path.join(args.path, "**", "*.tf"), recursive=True))) + blocks = [] + for path in files: + with open(path, encoding="utf-8") as f: + blocks.extend(parse_blocks(f.read())) + if not blocks: + sys.exit(f"error: no resource/module blocks found under {args.path}") + + declared = {} # node id -> (rtype or None, name, body) + for kind, l1, l2, body in blocks: + nid = f"{l1}.{l2}" if kind == "resource" else f"module.{l1}" + declared[nid] = (l1 if kind == "resource" else None, l2 or l1, body) + + edges = sorted({(nid, ref) for nid, (_, _, body) in declared.items() + for ref in _REF.findall(body) if ref in declared and ref != nid}) + raw = len(edges) + if not args.no_reduce: + edges = transitive_reduce(list(declared), edges) + + resolver = None if args.no_icons else IconResolver() + unmatched = [] + nodes = [] + for nid, (rtype, name, _) in declared.items(): + node = {"id": nid, "label": name} + icon = resolver.resolve(rtype) if resolver and rtype else None + if icon: + node.update(style=icon["style"], width=icon["w"], height=icon["h"]) + else: + # No icon: keep the type visible on the box (second line). + node["label"] = f"{name}\n{rtype}" if rtype else f"module {name}" + if rtype: + unmatched.append(rtype) + if args.group and rtype and "_" in rtype: + node["group"] = rtype.split("_")[1] + nodes.append(node) + + graph = {"direction": args.direction, "nodes": nodes, + "edges": [{"source": s, "target": t} for s, t in edges]} + if resolver: + # Icon labels render below the shape — reserve extra layout spacing. + graph.update(ranksep=0.7, nodesep=0.6) + text = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(text) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(text) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(nodes)} nodes, {len(edges)} edges{note}\n") + if unmatched: + sys.stderr.write("no icon for: " + ", ".join(sorted(set(unmatched))) + "\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/tfstate.py b/skills/drawio-skill/scripts/tfstate.py new file mode 100644 index 0000000..33496eb --- /dev/null +++ b/skills/drawio-skill/scripts/tfstate.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python3 +"""Draw the cloud resources ACTUALLY deployed, from `terraform show -json`. + +Where tfimports.py reads the *declared* config (`.tf` files), this reads the +*real* state: what Terraform recorded as provisioned. It is provider-agnostic +(the JSON is uniform across AWS / Azure / GCP), expands `count`/`for_each` into +their real instances, keeps module nesting, and reuses tfimports' icon resolver +so every resource shows its official cloud icon. The output feeds autolayout.py: + + terraform show -json | python3 tfstate.py - -o graph.json + python3 autolayout.py graph.json -o deployed.drawio + +Input is the JSON `terraform show -json` prints — from live state (no argument) +or a saved plan (`terraform show -json plan.tfplan`) — as a file path or `-` for +stdin. Nodes are the managed resource instances (data sources are ignored); +edges come from the dependencies Terraform recorded in state (`depends_on`). +`--group` boxes resources by their module; `--no-icons` forces plain boxes. + +Usage: terraform show -json | python3 tfstate.py - [-o graph.json] + [--direction TB|LR] [--group] [--no-reduce] [--no-icons] +""" +import argparse +import importlib.util +import json +import os +import sys + + +def load_tfimports(): + path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "tfimports.py") + spec = importlib.util.spec_from_file_location("tfimports", path) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +def walk_module(mod, out): + """Collect (address, type, name, index, module_path, depends_on) for every + managed resource, recursing into child modules.""" + addr = mod.get("address", "") # "" for root, else module.x[...] + for r in mod.get("resources") or []: + if r.get("mode") == "data": + continue + out.append((r.get("address"), r.get("type"), r.get("name"), + r.get("index"), addr, r.get("depends_on") or [])) + for child in mod.get("child_modules") or []: + walk_module(child, out) + + +def main(): + ap = argparse.ArgumentParser(description="`terraform show -json` -> autolayout graph JSON.") + ap.add_argument("input", help="`terraform show -json` output file, or - for stdin") + ap.add_argument("-o", "--output", help="output JSON path (default: stdout)") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("--group", action="store_true", + help="group resources into containers by module") + ap.add_argument("--no-reduce", action="store_true", + help="keep every edge (skip transitive reduction)") + ap.add_argument("--no-icons", action="store_true", + help="plain boxes instead of official cloud icons") + args = ap.parse_args() + + if args.input == "-": + text = sys.stdin.read() + else: + with open(args.input, encoding="utf-8") as f: + text = f.read() + try: + data = json.loads(text) + except json.JSONDecodeError as exc: + sys.exit(f"error: input is not valid JSON ({exc}) — feed `terraform show -json`") + # State: top-level "values"; saved plan: "planned_values". + root = ((data.get("values") or data.get("planned_values") or {}).get("root_module")) or {} + resources = [] + walk_module(root, resources) + if not resources: + sys.exit("error: no managed resources found in the Terraform state/plan") + + addresses = {r[0] for r in resources} + + def targets(dep): + """Instance addresses a depends_on entry names. State records the + un-indexed address (`aws_subnet.this`) for a resource with several + instances (`aws_subnet.this[0]`), so expand by prefix too.""" + if dep in addresses: + return {dep} + return {a for a in addresses if a.startswith(dep + "[")} + + edges = sorted({(addr, t) for addr, _, _, _, _, deps in resources + for dep in deps for t in targets(dep) if t != addr}) + + tf = load_tfimports() + raw = len(edges) + if not args.no_reduce and edges: + edges = tf.transitive_reduce(list(addresses), edges) + + resolver = None if args.no_icons else tf.IconResolver() + unmatched, nodes = [], [] + for addr, rtype, name, index, mpath, _ in resources: + label = name if index is None else f"{name}[{index}]" + node = {"id": addr, "label": label} + icon = resolver.resolve(rtype) if resolver and rtype else None + if icon: + node.update(style=icon["style"], width=icon["w"], height=icon["h"]) + else: + node["label"] = f"{label}\n{rtype}" if rtype else label + if rtype: + unmatched.append(rtype) + if args.group and mpath: + node["group"] = mpath + nodes.append(node) + + graph = {"direction": args.direction, "nodes": nodes, + "edges": [{"source": s, "target": t} for s, t in edges]} + if resolver: + # Icon labels render below the shape — reserve extra layout spacing. + graph.update(ranksep=0.7, nodesep=0.6) + out = json.dumps(graph, indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(out) + sys.stderr.write(f"wrote {args.output}\n") + else: + sys.stdout.write(out) + note = "" if args.no_reduce else f" (reduced from {raw})" + sys.stderr.write(f"{len(nodes)} resources, {len(edges)} edges{note}\n") + if unmatched: + sys.stderr.write("no icon for: " + ", ".join(sorted(set(unmatched))) + "\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/timelapse.py b/skills/drawio-skill/scripts/timelapse.py new file mode 100644 index 0000000..ebcd05a --- /dev/null +++ b/skills/drawio-skill/scripts/timelapse.py @@ -0,0 +1,257 @@ +#!/usr/bin/env python3 +"""Animate how a codebase's architecture grew, across its git history. + +Walks the git history of a directory, re-runs one of the bundled importers at +each sampled commit (the tree is pulled with ``git archive`` — the working copy +is never touched), lays each out and exports a PNG frame, then assembles a +single self-contained HTML player (frames embedded as base64, play / step +controls, no external files or CDNs). Open it in any browser to watch the +modules and edges appear over time. + + python3 timelapse.py skills/drawio-skill/scripts --importer pyimports + # -> architecture-evolution.html + +The importer is any of the bundled graph extractors (pyimports, jsimports, +goimports, rustimports, pyclasses, tfimports, k8simports, composeimports, +sqlerd); it is run against the archived directory with the same positional +"path" argument they all take, so point the path at the project/package/infra +root the importer expects. Extra importer flags pass through via +``--importer-args`` (e.g. ``--importer-args "--group"``). + +Commits that touched the directory are sampled evenly (always keeping the first +and last) down to ``--max-frames``; a commit where the importer finds nothing +(the path did not exist yet) is skipped. Needs git, the importer's requirements, +Graphviz (autolayout) and the draw.io CLI — the same tools the importers use. + +Usage: python3 timelapse.py [--importer NAME] [--importer-args STR] + [--max-frames N] [-o out.html] [--direction TB|LR] [--keep-frames] +""" +import argparse +import base64 +import io +import json +import os +import subprocess +import sys +import tarfile +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +IMPORTERS = {"pyimports", "jsimports", "goimports", "rustimports", "pyclasses", + "tfimports", "k8simports", "composeimports", "sqlerd"} + + +def git(root, *args): + """Run a git command in `root`; return (returncode, stdout_bytes).""" + p = subprocess.run(["git", "-C", root, *args], capture_output=True) + return p.returncode, p.stdout + + +def sample_indices(total, n): + """Evenly-spaced indices across range(total), always incl. first and last. + + For total <= n every index is kept; otherwise n indices are picked so the + first (0) and last (total-1) are always present and the rest are spread + uniformly between them. + """ + if total <= 0: + return [] + if total <= n or n <= 1: + return list(range(total)) + return sorted({round(i * (total - 1) / (n - 1)) for i in range(n)}) + + +def history(root, subpath): + """Chronological [(hash, iso_date, subject), ...] of commits touching subpath.""" + code, out = git(root, "log", "--format=%H%x09%aI%x09%s", "--", subpath or ".") + if code != 0: + return [] + rows = [] + for line in out.decode("utf-8", "replace").splitlines(): + parts = line.split("\t", 2) + if len(parts) == 3: + rows.append(tuple(parts)) + return list(reversed(rows)) # oldest first + + +def extract_tree(root, commit, subpath, dest): + """Extract subpath of `commit` into dest via git archive. False if absent.""" + code, tar_bytes = git(root, "archive", commit, subpath or ".") + if code != 0 or not tar_bytes: + return False + with tarfile.open(fileobj=io.BytesIO(tar_bytes)) as tf: + tf.extractall(dest) # git-authored tar; trusted + return True + + +def build_frame(importer, importer_args, work_path, direction, tmp): + """Importer -> autolayout -> PNG for one commit. Returns (png_bytes, n, e) or None.""" + graph_json = os.path.join(tmp, "graph.json") + imp = subprocess.run( + [sys.executable, os.path.join(HERE, importer + ".py"), work_path, + "-o", graph_json, *importer_args], + capture_output=True) + if imp.returncode != 0 or not os.path.exists(graph_json): + return None + with open(graph_json, encoding="utf-8") as f: + graph = json.loads(f.read()) + if not graph.get("nodes"): + return None + graph["direction"] = direction + with open(graph_json, "w", encoding="utf-8") as f: + f.write(json.dumps(graph)) + drawio = os.path.join(tmp, "frame.drawio") + lay = subprocess.run( + [sys.executable, os.path.join(HERE, "autolayout.py"), graph_json, "-o", drawio], + capture_output=True) + if lay.returncode != 0 or not os.path.exists(drawio): + return None + png = os.path.join(tmp, "frame.png") + exp = subprocess.run(["drawio", "-x", "-f", "png", "--width", "1600", + "-o", png, drawio], capture_output=True) + if exp.returncode != 0 or not os.path.exists(png): + return None + with open(png, "rb") as f: + png_data = f.read() + return png_data, len(graph["nodes"]), len(graph["edges"]) + + +def build_html(frames, title): + """Self-contained HTML player for the frame list.""" + data = [{"img": "data:image/png;base64," + base64.b64encode(png).decode(), + "hash": h[:9], "date": d[:10], "subj": s, "n": n, "e": e} + for png, h, d, s, n, e in frames] + peak = max((f["n"] for f in data), default=1) or 1 + payload = json.dumps(data).replace(" + +{title} +

{title}

+
+
architecture frame
+
+ + · + + +
+
+
+ + + + +
+
+""" + + +def main(): + ap = argparse.ArgumentParser(description="Git-history architecture time-lapse -> HTML player.") + ap.add_argument("path", help="directory to visualize (inside a git repo)") + ap.add_argument("--importer", default="pyimports", + help="bundled importer to run at each commit (default pyimports)") + ap.add_argument("--importer-args", default="", + help="extra args passed to the importer, e.g. \"--group\"") + ap.add_argument("--max-frames", type=int, default=10, help="sample down to N commits") + ap.add_argument("--direction", default="TB", choices=["TB", "LR"]) + ap.add_argument("-o", "--output", default="architecture-evolution.html") + ap.add_argument("--keep-frames", action="store_true", + help="also write the PNG frames next to the HTML") + args = ap.parse_args() + + importer = args.importer[:-3] if args.importer.endswith(".py") else args.importer + if importer not in IMPORTERS: + sys.exit(f"error: unknown importer {importer!r} (choose one of: " + + ", ".join(sorted(IMPORTERS)) + ")") + if not os.path.isdir(args.path): + sys.exit(f"error: {args.path} is not a directory") + code, top = git(args.path, "rev-parse", "--show-toplevel") + if code != 0: + sys.exit(f"error: {args.path} is not inside a git repository") + root = top.decode().strip() + subpath = os.path.relpath(os.path.abspath(args.path), root) + if subpath == ".": + subpath = "" + + commits = history(root, subpath) + if not commits: + sys.exit(f"error: no commits touch {args.path}") + picked = [commits[i] for i in sample_indices(len(commits), args.max_frames)] + imp_args = args.importer_args.split() + + frames = [] + for n, (h, date, subj) in enumerate(picked, 1): + sys.stderr.write(f"[{n}/{len(picked)}] {h[:9]} {subj[:50]}\n") + with tempfile.TemporaryDirectory() as tmp: + if not extract_tree(root, h, subpath, tmp): + continue + work = os.path.join(tmp, subpath) if subpath else tmp + frame = build_frame(importer, imp_args, work, args.direction, tmp) + if frame is None: + sys.stderr.write(" (importer found nothing — skipped)\n") + continue + png, nn, ee = frame + frames.append((png, h, date, subj, nn, ee)) + if args.keep_frames: + fp = f"{os.path.splitext(args.output)[0]}-frame{len(frames):02d}.png" + with open(fp, "wb") as f: + f.write(png) + + if not frames: + sys.exit("error: no frames produced (importer found nothing in any commit)") + title = f"Architecture evolution — {os.path.basename(os.path.abspath(args.path))}" + with open(args.output, "w", encoding="utf-8") as f: + f.write(build_html(frames, title)) + sys.stderr.write(f"wrote {args.output} ({len(frames)} frames)\n") + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/tubemap.py b/skills/drawio-skill/scripts/tubemap.py new file mode 100644 index 0000000..4c1bd14 --- /dev/null +++ b/skills/drawio-skill/scripts/tubemap.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +"""Restyle a graph as a London-Underground-style metro map (Tube-Map Mode). + +Input JSON describes coloured *lines* (each an ordered list of station ids) and the +*stations* they pass through, placed on an integer grid. The script snaps stations to +a pixel grid, routes every line segment octilinearly (horizontal / vertical / 45° +diagonal, inserting one bend when two stations are not already aligned), draws thick +coloured line strokes, marks interchange stations as white-fill black-ring circles and +regular stops as small white circles, and labels each station — the classic tube-map +look — as an editable `.drawio`. + + python3 tubemap.py metro.json -o metro.drawio + +Input schema (see references/tubemap.md for the full authoring guide): + + { + "stations": { + "": {"label": "...", "gx": , "gy": , "interchange": } + }, + "lines": [ + {"name": "...", "color": "#rrggbb"?, "stations": ["", "", ...]} + ] + } + +Keep consecutive stations on a line horizontally, vertically, or 45°-diagonally aligned +for the cleanest routing; any other offset gets one automatic diagonal-then-straight +bend. A line with no "color" is assigned one from the default tube palette by order. + +Usage: python3 tubemap.py [-o out.drawio] [--grid N] +""" +import argparse +import json +import sys + +# Default palette (approx. real tube-line colours), cycled for lines lacking a "color". +TUBE_PALETTE = [ + "#0098d4", # blue + "#007d32", # green + "#e1251b", # red + "#ee7c0e", # orange + "#9b0056", # magenta + "#00a4a7", # teal + "#ffce00", # yellow + "#894e24", # brown +] + + +def esc(s): + return (s.replace("&", "&").replace("<", "<").replace(">", ">") + .replace('"', """)) + + +def octilinear_waypoints(x1, y1, x2, y2): + """Waypoints so the path is horizontal, vertical, or 45°: diagonal then straight. + + Returns [] when the two points are already octilinearly aligned, else a single bend + point (run the 45° diagonal for the shorter delta, then a straight axis segment). + """ + dx, dy = x2 - x1, y2 - y1 + if dx == 0 or dy == 0 or abs(dx) == abs(dy): + return [] + sx = 1 if dx > 0 else -1 + sy = 1 if dy > 0 else -1 + d = min(abs(dx), abs(dy)) + if abs(dx) > abs(dy): # diagonal first, then horizontal into the target + return [(x1 + sx * d, y2)] + return [(x2, y1 + sy * d)] # diagonal first, then vertical into the target + + +def build(data, grid=110): + """Build the tube-map `.drawio` XML string from the parsed metro description.""" + stations = data.get("stations", {}) + lines = data.get("lines", []) + if not stations: + sys.exit("error: no stations in input") + for ln in lines: + for sid in ln.get("stations", []): + if sid not in stations: + sys.exit(f"error: line {ln.get('name', '?')!r} references unknown " + f"station id {sid!r}") + + ox = oy = 80 + G = grid + + def px(sid): + s = stations[sid] + return ox + int(s["gx"]) * G, oy + int(s["gy"]) * G + + maxx = ox + max(int(s["gx"]) for s in stations.values()) * G + 220 + maxy = oy + max(int(s["gy"]) for s in stations.values()) * G + 120 + lw = max(8, G // 12) + + out = ['', + '', + ' ', + f' ', + ' '] + + nid = [2] + + def cid(): + c = nid[0] + nid[0] += 1 + return c + + # 1) Line strokes first, so station markers sit on top of them. + for i, ln in enumerate(lines): + col = ln.get("color") or TUBE_PALETTE[i % len(TUBE_PALETTE)] + sts = ln.get("stations", []) + for a, b in zip(sts, sts[1:]): + x1, y1 = px(a) + x2, y2 = px(b) + wps = octilinear_waypoints(x1, y1, x2, y2) + arr = "" + if wps: + pts = "".join(f'' for wx, wy in wps) + arr = f'{pts}' + out.append( + f' ' + f'' + f'' + f'{arr}') + + # 2) Station markers + labels. + for sid, s in stations.items(): + x, y = px(sid) + label = esc(str(s.get("label", sid))) + if s.get("interchange"): + r = lw + 6 + marker = (f'ellipse;fillColor=#ffffff;strokeColor=#111111;strokeWidth=3;' + f'html=1;') + else: + r = lw - 1 + marker = (f'ellipse;fillColor=#ffffff;strokeColor=#555555;strokeWidth=2;' + f'html=1;') + out.append( + f' ') + out.append( + f' ' + f'') + + out.append(' ') + return "\n".join(out), len(stations), len(lines) + + +def main(): + ap = argparse.ArgumentParser(description="Restyle a graph as a metro / tube map.") + ap.add_argument("input", help="metro JSON (or - for stdin)") + ap.add_argument("-o", "--output", help="output .drawio (default: stdout)") + ap.add_argument("--grid", type=int, default=110, help="grid pitch in px (default 110)") + args = ap.parse_args() + + if args.input == "-": + raw = sys.stdin.read() + else: + with open(args.input, encoding="utf-8") as f: + raw = f.read() + try: + data = json.loads(raw) + except json.JSONDecodeError as exc: + sys.exit(f"error: bad JSON in {args.input}: {exc}") + + xml, n_st, n_ln = build(data, grid=args.grid) + if args.output: + with open(args.output, "w", encoding="utf-8") as f: + f.write(xml) + sys.stderr.write(f"wrote {args.output} ({n_st} stations, {n_ln} lines)\n") + else: + sys.stdout.write(xml) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/scripts/validate.py b/skills/drawio-skill/scripts/validate.py new file mode 100644 index 0000000..be9cf61 --- /dev/null +++ b/skills/drawio-skill/scripts/validate.py @@ -0,0 +1,340 @@ +#!/usr/bin/env python3 +"""Deterministic structural linter for .drawio files. + +Catches the class of mistakes a vision self-check is slow and unreliable at: +dangling edge endpoints, duplicate or reserved ids, broken parent references, +and (as warnings) off-grid geometry, overlapping sibling nodes, and edge +routing defects. Runs without launching draw.io, so it is a fast pre-check +before the visual review step. + + python3 validate.py diagram.drawio + +Edge routing checks (warnings): an edge segment crossing a non-incident leaf +vertex ("routes through vertex"), and two edges crossing each other ("edges X +and Y cross") — the two defects the SKILL.md step-5 self-check looks for +("Edge-shape overlap", "Stacked edges"), but caught here deterministically. + +Routing is only knowable from the XML when an edge carries explicit waypoints +(````) — exactly the hand-routed case the SKILL.md tells +authors to use to route around shapes. Edges with no waypoints are auto-routed +by draw.io at render time (the path is not stored), so they are NOT geometry- +checked here, keeping these warnings free of false positives. Endpoints honour +``exitX/exitY``/``entryX/entryY`` when present, else the node centre, and +absolute positions are resolved through parent containers. + +Exit status is non-zero when any error (or, with --strict, any warning) is +found, so it can gate a workflow. Compressed (non-XML) diagram pages are +skipped with a warning — this skill always writes uncompressed XML. + +Usage: python3 validate.py [--strict] +""" +import argparse +import sys +import xml.etree.ElementTree as ET + +RESERVED = {"0", "1"} + + +def rect(cell): + """Return (x, y, w, h) floats for a cell's geometry, or None if absent/bad. + + x/y default to 0 when omitted: draw.io treats a missing position as the + origin, and container-managed children (table rows, swimlane/UML-class + lines under tableLayout) legitimately omit x/y while keeping width/height. + Only width/height are required to be present and numeric. + """ + g = cell.find("mxGeometry") + if g is None: + return None + try: + return (float(g.get("x", "0")), float(g.get("y", "0")), + float(g.get("width", "nan")), float(g.get("height", "nan"))) + except ValueError: + return None + + +def is_edge_label(cell): + """True for a draw.io edge label / relative-positioned child vertex. + + These legitimately omit width/height: their position is given relative to a + parent edge (style ``edgeLabel``) or via ``relative="1"`` geometry. Treating + them as normal vertices wrongly flags them as missing/invalid geometry. + """ + if "edgeLabel" in (cell.get("style") or ""): + return True + g = cell.find("mxGeometry") + return g is not None and g.get("relative") == "1" + + +def overlap(a, b): + ax, ay, aw, ah = a + bx, by, bw, bh = b + return ax < bx + bw and bx < ax + aw and ay < by + bh and by < ay + ah + + +# --- Edge routing geometry ------------------------------------------------- +# +# These helpers reason about edge paths. They only apply to edges with explicit +# waypoints (the route is otherwise computed by draw.io at render time and not +# stored in the XML), so the checks never guess an auto-routed path. + +def style_num(style, key): + """Return float value of ``key=`` in a draw.io style string, or None.""" + for part in (style or "").split(";"): + if part.startswith(key + "="): + try: + return float(part.split("=", 1)[1]) + except ValueError: + return None + return None + + +def abs_rect(cell, by_id): + """Absolute (x, y, w, h) of a vertex, summing parent-container offsets. + + Children of a container use coordinates relative to the container origin, so + an edge spanning containers needs absolute positions to be compared. + """ + r = rect(cell) + if r is None or any(v != v for v in r): + return None + x, y, w, h = r + parent, seen = cell.get("parent"), set() + while parent and parent in by_id and parent not in seen: + seen.add(parent) + p = by_id[parent] + if p.get("vertex") == "1": + pr = rect(p) + if pr and not any(v != v for v in pr): + x += pr[0] + y += pr[1] + parent = p.get("parent") + return (x, y, w, h) + + +def endpoint(edge, end, by_id): + """Absolute (x, y) where ``edge`` meets its source/target vertex. + + Honours exitX/exitY (source) and entryX/entryY (target) if the style pins + them; otherwise the vertex centre. Returns None if the vertex is unresolved. + """ + vid = edge.get(end) + if not vid or vid not in by_id: + return None + box = abs_rect(by_id[vid], by_id) + if box is None: + return None + x, y, w, h = box + style = edge.get("style") or "" + fx = style_num(style, "exitX" if end == "source" else "entryX") + fy = style_num(style, "exitY" if end == "source" else "entryY") + return (x + (fx if fx is not None else 0.5) * w, + y + (fy if fy is not None else 0.5) * h) + + +def edge_waypoints(edge): + """Explicit waypoints of an edge as [(x, y), ...].""" + g = edge.find("mxGeometry") + if g is None: + return [] + arr = g.find("Array") + if arr is None: + return [] + pts = [] + for pt in arr.findall("mxPoint"): + px, py = pt.get("x"), pt.get("y") + if px is not None and py is not None: + try: + pts.append((float(px), float(py))) + except ValueError: + pass + return pts + + +def edge_route(edge, by_id): + """Absolute polyline [(x, y), ...] for a waypointed edge, or None. + + Returns None when the edge has no explicit waypoints (auto-routed; path + unknown) or an endpoint cannot be resolved. + """ + waypoints = edge_waypoints(edge) + if not waypoints: + return None + s, t = endpoint(edge, "source", by_id), endpoint(edge, "target", by_id) + if s is None or t is None: + return None + return [s] + waypoints + [t] + + +def _orient(a, b, c): + v = (b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0]) + return 0 if abs(v) < 1e-9 else (1 if v > 0 else -1) + + +def segments_cross(p1, p2, p3, p4): + """True if segments p1p2 and p3p4 properly cross (interior intersection). + + Proper crossing only: collinear overlap and shared-endpoint touches return + False, so edges meeting at a common node or grazing a corner are not flagged. + """ + o1, o2 = _orient(p1, p2, p3), _orient(p1, p2, p4) + o3, o4 = _orient(p3, p4, p1), _orient(p3, p4, p2) + return o1 != o2 and o3 != o4 and 0 not in (o1, o2, o3, o4) + + +def _point_in_rect(p, box, eps=1e-6): + x, y, w, h = box + return x + eps < p[0] < x + w - eps and y + eps < p[1] < y + h - eps + + +def route_hits_rect(points, box): + """True if a polyline enters a rectangle's interior or crosses a border.""" + x, y, w, h = box + corners = [(x, y), (x + w, y), (x + w, y + h), (x, y + h)] + borders = list(zip(corners, corners[1:] + corners[:1])) + for a, b in zip(points, points[1:]): + if _point_in_rect(a, box) or _point_in_rect(b, box): + return True + if any(segments_cross(a, b, c, d) for c, d in borders): + return True + return False + + +def routes_cross(pa, pb): + """True if any segment of polyline pa properly crosses any of pb.""" + for a1, a2 in zip(pa, pa[1:]): + for b1, b2 in zip(pb, pb[1:]): + if segments_cross(a1, a2, b1, b2): + return True + return False + + +def geometry_warnings(cells, ids, parents): + """Edge-through-vertex and edge-crossing warnings for waypointed edges.""" + warns = [] + routed = [] # (edge_id, polyline, {source, target}) + for c in cells: + if c.get("edge") == "1": + pts = edge_route(c, ids) + if pts: + routed.append((c.get("id"), pts, + {c.get("source"), c.get("target")})) + # Edge routes through an unrelated leaf vertex (containers wrap children, so + # an edge legitimately traverses them — restrict to leaves, as overlap does). + leaves = [(c.get("id"), abs_rect(c, ids)) for c in cells + if c.get("vertex") == "1" and c.get("id") not in parents + and not is_edge_label(c)] + leaves = [(vid, box) for vid, box in leaves if box] + for eid, pts, ends in routed: + for vid, box in leaves: + if vid not in ends and route_hits_rect(pts, box): + warns.append(f"edge {eid!r} routes through vertex {vid!r}") + # Edge-edge crossings (both routes known). + for i in range(len(routed)): + for j in range(i + 1, len(routed)): + (ia, pa, _), (ib, pb, _) = routed[i], routed[j] + if routes_cross(pa, pb): + warns.append(f"edges {ia!r} and {ib!r} cross") + return warns + + +def check_page(diagram): + """Return (errors, warnings) for one page.""" + name = diagram.get("name", "?") + model = diagram.find("mxGraphModel") + if model is None: + if (diagram.text or "").strip(): + return [], [f"page {name!r}: compressed, skipped (cannot lint)"] + return [f"page {name!r}: no "], [] + root = model.find("root") + # Normalize UserObject/object wrappers (used for links & metadata): the id + # lives on the wrapper, geometry/style on the inner mxCell — fold the two + # into one cell so edges referencing the wrapper id resolve. + cells = [] + for child in (root if root is not None else []): + if child.tag == "mxCell": + cells.append(child) + elif child.tag in ("UserObject", "object"): + inner = child.find("mxCell") + if inner is not None: + inner.set("id", child.get("id", "")) + cells.append(inner) + errors, warns = [], [] + ids = {} + for c in cells: + cid = c.get("id") + if cid in ids: + errors.append(f"duplicate id {cid!r}") + ids[cid] = c + parents = {c.get("parent") for c in cells} # ids that have children + for c in cells: + cid, parent = c.get("id"), c.get("parent") + is_v, is_e = c.get("vertex") == "1", c.get("edge") == "1" + if parent is not None and parent not in ids: + errors.append(f"cell {cid!r} parent {parent!r} does not exist") + for end in ("source", "target"): + ref = c.get(end) + if ref and ref not in ids: + errors.append(f"edge {cid!r} {end} {ref!r} does not exist") + if (is_v or is_e) and cid in RESERVED: + errors.append(f"cell {cid!r} reuses reserved id 0/1") + if is_v and not is_edge_label(c): + r = rect(c) + if r is None or any(v != v for v in r): # None or NaN + errors.append(f"vertex {cid!r} has missing/invalid geometry") + else: + x, y, w, h = r + if w <= 0 or h <= 0: + warns.append(f"vertex {cid!r} non-positive size {w:g}x{h:g}") + if x < 0 or y < 0: + warns.append(f"vertex {cid!r} negative position ({x:g},{y:g})") + # Sibling overlap: only leaf vertices (containers legitimately wrap children). + boxes = [(c.get("id"), c.get("parent"), rect(c)) for c in cells + if c.get("vertex") == "1" and c.get("id") not in parents and rect(c) + and not any(v != v for v in rect(c))] + for i in range(len(boxes)): + for j in range(i + 1, len(boxes)): + (ia, pa, ra), (ib, pb, rb) = boxes[i], boxes[j] + if pa == pb and overlap(ra, rb): + warns.append(f"vertices {ia!r} and {ib!r} overlap") + warns += geometry_warnings(cells, ids, parents) + return errors, warns + + +def main(): + ap = argparse.ArgumentParser(description="Lint a .drawio file for structural errors.") + ap.add_argument("file") + ap.add_argument("--strict", action="store_true", help="treat warnings as failure too") + ap.add_argument("--score", action="store_true", + help="also print a readability score (lower is better) — " + "useful for comparing layout variants of the same graph") + args = ap.parse_args() + try: + tree = ET.parse(args.file) + except (ET.ParseError, OSError) as exc: + sys.exit(f"error: cannot parse {args.file}: {exc}") + pages = tree.getroot().findall("diagram") or [tree.getroot()] + errors, warns = [], [] + for page in pages: + e, w = check_page(page) + errors += e + warns += w + for w in warns: + print(f"warning: {w}") + for e in errors: + print(f"error: {e}") + print(f"{len(errors)} error(s), {len(warns)} warning(s)") + if args.score: + # Weighted by how badly each defect hurts readability. Comparable only + # across variants of the SAME graph (same nodes/edges). + through = sum(1 for w in warns if "routes through" in w) + cross = sum(1 for w in warns if " cross" in w) + olap = sum(1 for w in warns if " overlap" in w) + print(f"score: {20 * through + 10 * cross + 5 * olap} " + f"({through} through-vertex, {cross} crossings, {olap} overlaps)") + if errors or (args.strict and warns): + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/skills/drawio-skill/styles/built-in/colorblind-safe.json b/skills/drawio-skill/styles/built-in/colorblind-safe.json new file mode 100644 index 0000000..612703d --- /dev/null +++ b/skills/drawio-skill/styles/built-in/colorblind-safe.json @@ -0,0 +1,49 @@ +{ + "$schema": "../schema.json", + "name": "colorblind-safe", + "version": 1, + "default": false, + "source": { "type": "built-in" }, + "confidence": "high", + "palette": { + "primary": { "fillColor": "#ccedff", "strokeColor": "#0072b2" }, + "success": { "fillColor": "#ccfff1", "strokeColor": "#009e73" }, + "warning": { "fillColor": "#fff8cc", "strokeColor": "#8f7c00" }, + "accent": { "fillColor": "#ffefcc", "strokeColor": "#e69f00" }, + "danger": { "fillColor": "#ffe3cc", "strokeColor": "#d55e00" }, + "neutral": { "fillColor": "#e6e6e6", "strokeColor": "#555555" }, + "secondary": { "fillColor": "#f1dae7", "strokeColor": "#cc79a7" } + }, + "roles": { + "service": "primary", + "database": "success", + "queue": "warning", + "gateway": "accent", + "error": "danger", + "external": "neutral", + "security": "secondary" + }, + "shapes": { + "service": "rounded=1", + "database": "shape=cylinder3", + "queue": "rounded=1", + "decision": "rhombus", + "external": "rounded=1;dashed=1", + "container": "swimlane;startSize=30" + }, + "font": { + "fontFamily": "Helvetica", + "fontSize": 12, + "titleFontSize": 14, + "titleBold": true + }, + "edges": { + "style": "edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1", + "arrow": "endArrow=classic;endFill=1", + "dashedFor": ["external"] + }, + "extras": { + "sketch": false, + "globalStrokeWidth": 2 + } +} diff --git a/skills/drawio-skill/styles/built-in/corporate.json b/skills/drawio-skill/styles/built-in/corporate.json new file mode 100644 index 0000000..1b3f289 --- /dev/null +++ b/skills/drawio-skill/styles/built-in/corporate.json @@ -0,0 +1,49 @@ +{ + "$schema": "../schema.json", + "name": "corporate", + "version": 1, + "default": false, + "source": { "type": "built-in" }, + "confidence": "high", + "palette": { + "primary": { "fillColor": "#e3f2fd", "strokeColor": "#1565c0" }, + "success": { "fillColor": "#e8f5e9", "strokeColor": "#2e7d32" }, + "warning": { "fillColor": "#fff9c4", "strokeColor": "#f57c00" }, + "accent": { "fillColor": "#fff3e0", "strokeColor": "#e65100" }, + "danger": { "fillColor": "#ffebee", "strokeColor": "#c62828" }, + "neutral": { "fillColor": "#eceff1", "strokeColor": "#455a64" }, + "secondary": { "fillColor": "#f3e5f5", "strokeColor": "#6a1b9a" } + }, + "roles": { + "service": "primary", + "database": "success", + "queue": "warning", + "gateway": "accent", + "error": "danger", + "external": "neutral", + "security": "secondary" + }, + "shapes": { + "service": "rounded=0", + "database": "shape=cylinder3", + "queue": "rounded=0", + "decision": "rhombus", + "external": "rounded=0;dashed=1", + "container": "swimlane;startSize=30" + }, + "font": { + "fontFamily": "Arial", + "fontSize": 11, + "titleFontSize": 13, + "titleBold": true + }, + "edges": { + "style": "edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1", + "arrow": "endArrow=classic;endFill=1", + "dashedFor": ["optional", "async"] + }, + "extras": { + "sketch": false, + "globalStrokeWidth": 1 + } +} diff --git a/skills/drawio-skill/styles/built-in/dark.json b/skills/drawio-skill/styles/built-in/dark.json new file mode 100644 index 0000000..5ad6b7a --- /dev/null +++ b/skills/drawio-skill/styles/built-in/dark.json @@ -0,0 +1,52 @@ +{ + "$schema": "../schema.json", + "name": "dark", + "version": 1, + "default": false, + "source": { "type": "built-in" }, + "confidence": "high", + "palette": { + "primary": { "fillColor": "#004870", "strokeColor": "#33b6ff" }, + "success": { "fillColor": "#007052", "strokeColor": "#33ffc7" }, + "warning": { "fillColor": "#5a4916", "strokeColor": "#d7b85b" }, + "accent": { "fillColor": "#705100", "strokeColor": "#ffc633" }, + "danger": { "fillColor": "#502220", "strokeColor": "#c4716e" }, + "neutral": { "fillColor": "#383838", "strokeColor": "#999999" }, + "secondary": { "fillColor": "#3d2c45", "strokeColor": "#a182b0" } + }, + "roles": { + "service": "primary", + "database": "success", + "queue": "warning", + "gateway": "accent", + "error": "danger", + "external": "neutral", + "security": "secondary" + }, + "shapes": { + "service": "rounded=1", + "database": "shape=cylinder3", + "queue": "rounded=1", + "decision": "rhombus", + "external": "rounded=1;dashed=1", + "container": "swimlane;startSize=30" + }, + "font": { + "fontFamily": "Helvetica", + "fontSize": 12, + "titleFontSize": 14, + "titleBold": true + }, + "edges": { + "style": "edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1", + "arrow": "endArrow=classic;endFill=1", + "dashedFor": ["external"] + }, + "extras": { + "sketch": false, + "globalStrokeWidth": 1, + "background": "#1e1e1e", + "fontColor": "#f0f0f0", + "edgeColor": "#bbbbbb" + } +} diff --git a/skills/drawio-skill/styles/built-in/default.json b/skills/drawio-skill/styles/built-in/default.json new file mode 100644 index 0000000..a202de1 --- /dev/null +++ b/skills/drawio-skill/styles/built-in/default.json @@ -0,0 +1,49 @@ +{ + "$schema": "../schema.json", + "name": "default", + "version": 1, + "default": false, + "source": { "type": "built-in" }, + "confidence": "high", + "palette": { + "primary": { "fillColor": "#dae8fc", "strokeColor": "#6c8ebf" }, + "success": { "fillColor": "#d5e8d4", "strokeColor": "#82b366" }, + "warning": { "fillColor": "#fff2cc", "strokeColor": "#d6b656" }, + "accent": { "fillColor": "#ffe6cc", "strokeColor": "#d79b00" }, + "danger": { "fillColor": "#f8cecc", "strokeColor": "#b85450" }, + "neutral": { "fillColor": "#f5f5f5", "strokeColor": "#666666" }, + "secondary": { "fillColor": "#e1d5e7", "strokeColor": "#9673a6" } + }, + "roles": { + "service": "primary", + "database": "success", + "queue": "warning", + "gateway": "accent", + "error": "danger", + "external": "neutral", + "security": "secondary" + }, + "shapes": { + "service": "rounded=1", + "database": "shape=cylinder3", + "queue": "rounded=1", + "decision": "rhombus", + "external": "rounded=1;dashed=1", + "container": "swimlane;startSize=30" + }, + "font": { + "fontFamily": "Helvetica", + "fontSize": 12, + "titleFontSize": 14, + "titleBold": true + }, + "edges": { + "style": "edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1", + "arrow": "endArrow=classic;endFill=1", + "dashedFor": [] + }, + "extras": { + "sketch": false, + "globalStrokeWidth": 1 + } +} diff --git a/skills/drawio-skill/styles/built-in/handdrawn.json b/skills/drawio-skill/styles/built-in/handdrawn.json new file mode 100644 index 0000000..1485a5a --- /dev/null +++ b/skills/drawio-skill/styles/built-in/handdrawn.json @@ -0,0 +1,49 @@ +{ + "$schema": "../schema.json", + "name": "handdrawn", + "version": 1, + "default": false, + "source": { "type": "built-in" }, + "confidence": "high", + "palette": { + "primary": { "fillColor": "#ffe4b5", "strokeColor": "#b8651e" }, + "success": { "fillColor": "#def0dc", "strokeColor": "#5c8a49" }, + "warning": { "fillColor": "#fff4cc", "strokeColor": "#b8901a" }, + "accent": { "fillColor": "#ffd9b3", "strokeColor": "#c25100" }, + "danger": { "fillColor": "#ffcdbf", "strokeColor": "#a53d3d" }, + "neutral": { "fillColor": "#f5e6d3", "strokeColor": "#8b7355" }, + "secondary": { "fillColor": "#e6d7e8", "strokeColor": "#7b4397" } + }, + "roles": { + "service": "primary", + "database": "success", + "queue": "warning", + "gateway": "accent", + "error": "danger", + "external": "neutral", + "security": "secondary" + }, + "shapes": { + "service": "rounded=1", + "database": "shape=cylinder3", + "queue": "rounded=1", + "decision": "rhombus", + "external": "rounded=1;dashed=1", + "container": "swimlane;startSize=30" + }, + "font": { + "fontFamily": "Helvetica", + "fontSize": 12, + "titleFontSize": 14, + "titleBold": true + }, + "edges": { + "style": "edgeStyle=orthogonalEdgeStyle;curved=1;rounded=1;orthogonalLoop=1;jettySize=auto;html=1", + "arrow": "endArrow=classic;endFill=1", + "dashedFor": ["optional"] + }, + "extras": { + "sketch": true, + "globalStrokeWidth": 2 + } +} diff --git a/skills/drawio-skill/styles/schema.json b/skills/drawio-skill/styles/schema.json new file mode 100644 index 0000000..0d4dedf --- /dev/null +++ b/skills/drawio-skill/styles/schema.json @@ -0,0 +1,264 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://github.com/Agents365-ai/drawio-skill/styles/schema.json", + "title": "drawio-skill preset", + "type": "object", + "required": [ + "name", + "version", + "palette", + "roles", + "shapes", + "font", + "edges" + ], + "additionalProperties": false, + "properties": { + "$schema": { + "type": "string" + }, + "name": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9_-]*$" + }, + "version": { + "type": "integer", + "const": 1 + }, + "default": { + "type": "boolean" + }, + "confidence": { + "type": "string", + "enum": [ + "low", + "medium", + "high" + ] + }, + "source": { + "type": "object", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "enum": [ + "xml", + "image", + "built-in", + "hand-authored" + ] + }, + "path": { + "type": "string" + }, + "extracted_at": { + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$" + } + } + }, + "palette": { + "type": "object", + "additionalProperties": false, + "required": [ + "primary", + "success", + "warning", + "accent", + "danger", + "neutral", + "secondary" + ], + "properties": { + "primary": { + "$ref": "#/$defs/colorPair" + }, + "success": { + "$ref": "#/$defs/colorPair" + }, + "warning": { + "$ref": "#/$defs/colorPair" + }, + "accent": { + "$ref": "#/$defs/colorPair" + }, + "danger": { + "$ref": "#/$defs/colorPair" + }, + "neutral": { + "$ref": "#/$defs/colorPair" + }, + "secondary": { + "$ref": "#/$defs/colorPair" + } + } + }, + "roles": { + "type": "object", + "additionalProperties": false, + "properties": { + "service": { + "$ref": "#/$defs/slotName" + }, + "database": { + "$ref": "#/$defs/slotName" + }, + "queue": { + "$ref": "#/$defs/slotName" + }, + "gateway": { + "$ref": "#/$defs/slotName" + }, + "error": { + "$ref": "#/$defs/slotName" + }, + "external": { + "$ref": "#/$defs/slotName" + }, + "security": { + "$ref": "#/$defs/slotName" + } + } + }, + "shapes": { + "type": "object", + "additionalProperties": { + "type": "string" + }, + "properties": { + "service": { + "type": "string" + }, + "database": { + "type": "string" + }, + "queue": { + "type": "string" + }, + "decision": { + "type": "string" + }, + "external": { + "type": "string" + }, + "container": { + "type": "string" + } + } + }, + "font": { + "type": "object", + "additionalProperties": false, + "required": [ + "fontFamily", + "fontSize" + ], + "properties": { + "fontFamily": { + "type": "string" + }, + "fontSize": { + "type": "integer", + "minimum": 8, + "maximum": 36 + }, + "titleFontSize": { + "type": "integer", + "minimum": 8, + "maximum": 48 + }, + "titleBold": { + "type": "boolean" + } + } + }, + "edges": { + "type": "object", + "additionalProperties": false, + "required": [ + "style", + "arrow" + ], + "properties": { + "style": { + "type": "string" + }, + "arrow": { + "type": "string" + }, + "dashedFor": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "extras": { + "type": "object", + "additionalProperties": false, + "properties": { + "sketch": { + "type": "boolean" + }, + "globalStrokeWidth": { + "type": "number", + "minimum": 0.5, + "maximum": 6 + }, + "background": { + "type": "string", + "pattern": "^#[0-9A-Fa-f]{6}$" + }, + "fontColor": { + "type": "string", + "pattern": "^#[0-9A-Fa-f]{6}$" + }, + "edgeColor": { + "type": "string", + "pattern": "^#[0-9A-Fa-f]{6}$" + } + } + } + }, + "$defs": { + "colorPair": { + "oneOf": [ + { + "type": "null" + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "fillColor", + "strokeColor" + ], + "properties": { + "fillColor": { + "type": "string", + "pattern": "^#[0-9A-Fa-f]{6}$" + }, + "strokeColor": { + "type": "string", + "pattern": "^#[0-9A-Fa-f]{6}$" + } + } + } + ] + }, + "slotName": { + "type": "string", + "enum": [ + "primary", + "success", + "warning", + "accent", + "danger", + "neutral", + "secondary" + ] + } + } +} diff --git a/skills/frontend-design/LICENSE.txt b/skills/frontend-design/LICENSE.txt new file mode 100644 index 0000000..f433b1a --- /dev/null +++ b/skills/frontend-design/LICENSE.txt @@ -0,0 +1,177 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS diff --git a/skills/frontend-design/SKILL.md b/skills/frontend-design/SKILL.md new file mode 100644 index 0000000..decdff4 --- /dev/null +++ b/skills/frontend-design/SKILL.md @@ -0,0 +1,55 @@ +--- +name: frontend-design +description: Guidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. +license: Complete terms in LICENSE.txt +--- + +# Frontend Design + +Approach this as the design lead at a small studio known for giving every client a visual identity that could not be mistaken for anyone else's. This client has already rejected proposals that felt templated, and is paying for a distinctive point of view: make deliberate, opinionated choices about palette, typography, and layout that are specific to this brief, and take one real aesthetic risk you can justify. + +## Ground it in the subject + +If the brief does not pin down what the product or subject is, pin it yourself before designing: name one concrete subject, its audience, and the page's single job, and state your choice. If there's any information in your memory about the human's preferences, context about what they're building, or designs you've made before – use that as a hint. The subject's own world, its materials, instruments, artifacts, and vernacular, is where distinctive choices come from. Build with the brief's real content and subject matter throughout. + +## Design principles + +For web designs, the hero is a thesis. Open with the most characteristic thing in the subject's world, in whatever form makes sense for it: a headline, an image, an animation, a live demo, an interactive moment. Be deliberate with your choice: a big number with a small label, supporting stats, and a gradient accent is the template answer, only use if that's truly the best option. + +Typography carries the personality of the page. Pair the display and body faces deliberately, not the same families you would reach for on any other project, and set a clear type scale with intentional weights, widths, and spacing. Make the type treatment itself a memorable part of the design, not a neutral delivery vehicle for the content. + +Structure is information. Structural devices, numbering, eyebrows, dividers, labels, should encode something true about the content, not decorate it. Many generic designs use numbered markers (01 / 02 / 03), but that's only appropriate if the content actually is a sequence - like a real process or a typed timeline where order carries information the reader needs. Question if choices like numbered markers actually make sense before incorporating them. + +Leverage motion deliberately. Think about where and if animation can serve the subject: a page-load sequence, a scroll-triggered reveal, hover micro-interactions, ambient atmosphere. An orchestrated moment usually lands harder than scattered effects; choose what the direction calls for. However, sometimes less is more, and extra animation contributes to the feeling that the design is AI-generated. + +Match complexity to the vision. Maximalist directions need elaborate execution; minimal directions need precision in spacing, type, and detail. Elegance is executing the chosen vision well. + +Consider written content carefully. Often a design brief may not contain real content, and it's up to you to come up with copy. Copy can make a design feel as templated as the design itself. See the below section on writing for more guidance. + +## Process: brainstorm, explore, plan, critique, build, critique again + +For calibration: AI-generated design right now clusters around three looks: (1) a warm cream background (near #F4F1EA) with a high-contrast serif display and a terracotta accent; (2) a near-black background with a single bright acid-green or vermilion accent; (3) a broadsheet-style layout with hairline rules, zero border-radius, and dense newspaper-like columns. All three are legitimate for some briefs, but they are defaults rather than choices, and they appear regardless of subject. Where the brief pins down a visual direction, follow it exactly — the brief's own words always win, including when it asks for one of these looks. Where it leaves an axis free, don't spend that freedom on one of these defaults. Just like a human designer who's hired, there's often a careful balance between doing what you're good at and taking each project as a chance to experiment and learn. + +Work in two passes. First, brainstorm a short design plan based on the human's design brief: create a compact token system with color, type, layout, and signature. Color: describe the palette as 4–6 named hex values. Type: the typefaces for 2+ roles (a characterful display face that's used with restraint, a complementary body face, and a utility face for captions or data if needed). Layout: a layout concept, using one-sentence prose descriptions and ASCII wireframes to ideate and compare. Signature: the single unique element this page will be remembered by that embodies the brief in an appropriate way. + +Then review that plan against the brief before building: if any part of it reads like the generic default you would produce for any similar page (work through a similar prompt to see if you arrive somewhere similar) rather than a choice made for this specific brief — revise that part, say what you changed and why. Only after you've confirmed the relative uniqueness of your design plan should you start to write the code, following the revised plan exactly and deriving every color and type decision from it. + +When writing the code, be careful of structuring your CSS selector specificities. It's easy to generate CSS classes that cancel each other out (especially with a type-based selector like .section and a element-based selector like .cta). This can happen often with paddings/margins between sections. + +Try to do a lot of this planning and iteration in your thinking, and only show ideas to the user when you have higher confidence it'll delight them. + +## Restraint and self-critique + +Spend your boldness in one place. Let the signature element be the one memorable thing, keep everything around it quiet and disciplined, and cut any decoration that does not serve the brief. Not taking a risk can be a risk itself! Build to a quality floor without announcing it: responsive down to mobile, visible keyboard focus, reduced motion respected. Critique your own work as you build, taking screenshots if your environment supports it – a picture is worth 1000 tokens. Consider Chanel's advice: before leaving the house, take a look in the mirror and remove one accessory. Human creators have memory and always try to do something new, so if you have a space to quickly jot down notes about what you've tried, it can help you in future passes. + +## More on writing in design + +Words appear in a design for one reason: to make it easier to understand, and therefore easier to use. They are design material, not decoration. Bring the same intentionality to copy that you would bring to spacing and color. Before writing anything, ask what the design needs to say, and how it can best be said to help the person navigate the experience. + +Write from the end user's side of the screen. Name things by what people control and recognize, never by how the system is built. A person manages notifications, not webhook config. Describe what something does in plain terms rather than selling it. Being specific is always better than being clever. + +Use active voice as default. A control should say exactly what happens when it's used: "Save changes," not "Submit." An action keeps the same name through the whole flow, so the button that says "Publish" produces a toast that says "Published." The vocabulary of an interface is the signposting for someone navigating the product. Cohesion and consistency are how people learn their way around. + +Treat failure and emptiness as moments for direction, not mood. Explain what went wrong and how to fix it, in the interface's voice rather than a person's. Errors don't apologize, and they are never vague about what happened. An empty screen is an invitation to act. + +Keep the register conversational and tuned: plain verbs, sentence case, no filler, with tone matched to the brand and the audience. Let each element do exactly one job. A label labels, an example demonstrates, and nothing quietly does double duty. diff --git a/skills/ln-24-architecture-auditor/SKILL.md b/skills/ln-24-architecture-auditor/SKILL.md new file mode 100644 index 0000000..e34c501 --- /dev/null +++ b/skills/ln-24-architecture-auditor/SKILL.md @@ -0,0 +1,127 @@ +--- +name: ln-24-architecture-auditor +description: "Audits implemented architecture fitness, boundaries, contracts, dependencies, and configuration ownership. Use for system structure; not for current-state documentation, diagrams, or plan review." +--- + +# Architecture Auditor + +**Goal:** Perform a read-only audit of the architecture the system actually executes. Evaluate whether structure, dependencies, contracts, and cross-component ownership fit current product needs without rewarding pattern names or speculative modernization. Judge where atomicity and resource ownership belong; leave local query, transaction, and data-resource correctness to a persistence-focused review. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict, decision, and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred tool | Use it when | Fallback | +|---|---|---|---| +| Physical and declared architecture | Native file listing, manifests, build files, configuration, and architecture documents | Establishing modules, packages, domains, layers, entrypoints, and deployment units | Targeted repository map from known entrypoints | +| Symbols and dependency topology | Language server, compiler metadata, or host-native code intelligence | Tracing imports, calls, implementations, routes, events, and cycles | Narrow search plus direct inspection of definitions and consumers | +| Runtime wiring | Registration code, dependency injection, routing, startup configuration, and safe runtime diagnostics | A component may exist but not be discoverable or connected | Static trace with explicit uncertainty | +| Historical intent | Git history, blame, and decision records | A current exception or parallel mechanism may have a still-valid reason | Current behavior and documented constraints remain authoritative | +| Architectural fitness | Current official framework and platform documentation | A finding depends on supported extension, lifecycle, configuration, or boundary behavior | Primary-source web research; otherwise mark `UNVERIFIED` | +| Quantitative structure | Existing dependency, cycle, complexity, or package-analysis commands | The repository already defines reliable structural analysis | Reproducible static inventory and call-path evidence | + +Use diagrams only when they clarify a relationship that prose cannot. Do not generate a diagram as a substitute for evidence, and do not modify code or architecture documents during the audit. + +## Evidence Rules + +- Executable dependencies, runtime wiring, and public contracts outweigh intended diagrams or folder names. +- A cycle or cross-layer call is a finding only when it creates a concrete change, ownership, testing, deployment, or failure cost. +- Pattern compliance is not a goal by itself; evaluate fitness against product complexity, team workflow, and operational constraints. +- Framework convention and generated wiring require framework-aware verification before being labeled leakage or dead code. +- Modernization is justified only by a present defect or measurable simplification, not novelty. +- Explicit repository boundary rules define intended constraints; documentation explains them; inference from folder names is low-confidence evidence and must not create a violation by itself. +- A prior audit baseline separates new, resolved, and accepted debt. It does not make an active correctness or security risk disappear. +- Shared system-design baseline, current-state, target-design, decision, diagram, and migration artifacts are optional intent evidence. Their absence is not a defect by itself, and their presence never outranks executable behavior for the implemented state. + +## Checklist + +### 1. Discover the Actual Architecture + +- [ ] Read repository instructions, architecture documents, manifests, entrypoints, deployment definitions, and configuration ownership rules. +- [ ] Discover shared architecture artifacts by repository convention, distinguish a system-design baseline from a prior audit baseline, and classify each as current, proposed, accepted, superseded, stale, contradictory, or `UNKNOWN`; do not require any particular artifact path. +- [ ] Map packages, modules, domains, layers, processes, data stores, queues, external systems, and public interfaces in scope. +- [ ] Record ownership and independent build, deploy, scale, and failure boundaries; do not infer a service boundary from a directory or process name alone. +- [ ] Identify the dominant organizing model and any competing models: layer-first, domain-first, service boundaries, plugin boundaries, or framework conventions. +- [ ] Trace representative critical flows from entrypoint through orchestration, domain behavior, persistence or integration, and observable outcome. +- [ ] Compare documented current state, target state, accepted decisions, and active migration phase with executable structure; record drift and authority conflicts without assuming documentation describes what actually runs. +- [ ] Inspect Git state so current work and unrelated user changes are not misclassified as established architecture. +- [ ] Keep the audit read-only and disclose any permitted diagnostic caches or generated analysis artifacts. + +### 2. Audit Pattern Fitness and Ownership + +- [ ] Identify major patterns from code behavior rather than names and score each on problem fit, completeness, consistency, and current maintenance cost. +- [ ] Check whether abstractions remove real volatility or merely move straightforward code behind interfaces, factories, registries, or generic layers. +- [ ] Check layer direction, domain ownership, orchestration depth, side-effect boundaries, and whether policy remains separated from infrastructure detail. +- [ ] Trace where cross-component transactions, sessions, connections, streams, processes, subscriptions, and background work are owned; report boundary ambiguity without duplicating local lifecycle or transaction-correctness analysis. +- [ ] For each state-changing critical flow, identify the atomicity owner and how partial failure is prevented, retried idempotently, or compensated across stores and messages. +- [ ] Check read-named or pure-looking interfaces for hidden writes, broad side effects, network calls, or lifecycle ownership that violates their contract. +- [ ] Find parallel architectural mechanisms, partially completed migrations, compatibility paths with no consumer, and extension points with no credible variation. +- [ ] Check whether failure handling and retries sit at the layer that owns the operation rather than being duplicated or swallowed across layers. + +### 3. Audit Contracts and Dependencies + +- [ ] Inspect public API, service, event, command, and persistence boundaries for stable input/output models plus explicit error, nullability, idempotency, and compatibility contracts. +- [ ] Check entity or framework-type leakage, missing boundary models, boolean-mode APIs, excessive parameters, unstable serialization, and inconsistent naming across layers. +- [ ] Build module or package dependency direction using resolved internal edges; account for aliases, re-exports, generated code, reflection, registries, plugins, and runtime loading before declaring an edge absent. +- [ ] Apply configured forbidden/allowed dependency rules first; if rules are only inferred, report the inferred model and confidence instead of presenting it as policy. +- [ ] Identify cycles, forbidden imports, unstable dependency direction, excessive fan-in or fan-out, and isolated islands without treating a metric threshold as a finding until it predicts a concrete cost. +- [ ] Trace cycle and coupling findings to concrete effects on change radius, initialization, testing, deployment, ownership, or failure propagation. +- [ ] Check that producers and consumers agree on event names, schemas, versions, delivery semantics, ordering, and registration. +- [ ] Check physical structure for domain cohesion, framework placement, junk drawers, duplicate module roots, orphan packages, and files whose location hides ownership. +- [ ] Check configuration boundaries for typed settings, composition-root ownership, precedence and override semantics, startup validation, scattered environment reads, secret ownership, and leakage into domain behavior. +- [ ] Verify runtime discovery: routes, handlers, jobs, commands, plugins, middleware, serializers, and dependency bindings must be registered and reachable. + +### 4. Evaluate Evolution and Alternatives + +- [ ] Identify current architecture pain using repository evidence: repeated change sets, fragile tests, broad blast radius, duplicate mechanisms, release coupling, or incident-prone ownership. +- [ ] If a system-design baseline exists, verify its source and freshness, then compare confirmed constraints with actual behavior. +- [ ] If a prior audit baseline exists, compare new, resolved, and retained findings; continue to report accepted or retained risks when their impact remains material. +- [ ] Research external pattern or framework behavior only when it can confirm a capability, limitation, lifecycle rule, or supported simplification. +- [ ] Compare the current shape with the simplest credible alternative, including migration risk, compatibility, rollback, team impact, and operational cost. +- [ ] Prefer incremental boundary repair when a rewrite or new pattern would create more transitional complexity than it removes. +- [ ] Reject recommendations that require speculative scale, unsupported future variants, or replacement of working conventions without a demonstrated defect. +- [ ] Include a migration sequence only when the recommendation cannot be applied safely as one bounded change. + +### 5. Validate Findings and Report + +- [ ] Verify structural findings through at least one dependency path, call path, registration path, public contract, or reproducible analysis result. +- [ ] Filter generated code, framework conventions, deliberate adapters, test-only architecture, and documented exceptions before confirming a violation. +- [ ] Apply a materiality and acceptable-alternative gate to every candidate. Require a concrete correctness, security, ownership, deployment, change-amplification, or recurring maintenance impact at the system's evidenced scale. Reject nitpicks, pattern preference, theoretical purity, generic best practice, speculative scale, and a merely different architecture when the current tradeoff is reasonable; when several shapes work, require the boundary outcome or constraint rather than one preferred pattern. +- [ ] Put a directly relevant Markdown practice link in every finding's required resolution: prefer current official documentation or a specification, and use reputable primary engineering material only when official sources do not resolve the tradeoff. Open and verify the source; it must support the proposed architectural mechanism, not merely the defect category. Reject search-result links, generic best-practice articles, and decorative citations. +- [ ] Classify findings as `P0`-`P3` based on correctness, security, change amplification, deployment coupling, and recurring maintenance cost. +- [ ] Include affected boundaries, evidence, concrete consequence, why the current compromise is not acceptable, migration risk, and the smallest safe next step for every finding while allowing equivalent target shapes. +- [ ] Order recommendations by prerequisite and risk reduction, separating immediate correctness fixes from optional evolution. +- [ ] Use `BLOCKED` when required runtime wiring, boundary evidence, or an authoritative contract cannot be verified without a credible fallback; use `FAIL` for an evidenced unresolved correctness or security boundary defect, unsafe ownership ambiguity, or `P0/P1` structural risk; use `CONCERNS` only for material non-blocking change amplification, and `PASS` only when no evidenced architecture defect creates material cost or risk. +- [ ] Return the verdict with the actual architecture map, fitness assessment, findings, limitations, and residual structural risks. + +## Output Contract + +```markdown +# Architecture Audit + +**Verdict:** PASS | CONCERNS | FAIL | BLOCKED + +## Actual architecture +- Modules, boundaries, entrypoints, and external systems +- Critical flows and runtime wiring +- Observed current state, intended target state, active transition state, and evidenced drift + +## Fitness summary +| Area | Status | Evidence | +|---|---|---| +| Pattern fitness and ownership | PASS / CONCERNS / FAIL | ... | +| Contracts and boundaries | PASS / CONCERNS / FAIL | ... | +| Dependency topology | PASS / CONCERNS / FAIL | ... | +| Physical structure and configuration | PASS / CONCERNS / FAIL | ... | + +## Findings +| Priority | Problem | Evidence and justification | Required resolution | +|---|---|---|---| +| P0 / P1 / P2 / P3 | Concrete architectural defect | Boundary and evidence, material consequence at evidenced scale, migration context, and why the current tradeoff is not acceptable | Smallest safe outcome or boundary correction, migration and rollback constraints, and a verified `[practice reference](URL)` to official or primary engineering guidance; allow equivalent valid target shapes | + +Use `None` when no candidate survives the evidence, materiality, fitness, and acceptable-alternative gates. + +## Evolution order and residual risks +Prerequisite-aware recommendations, accepted exceptions, and blind spots. +``` diff --git a/skills/ln-71-system-design-baseline-builder/SKILL.md b/skills/ln-71-system-design-baseline-builder/SKILL.md new file mode 100644 index 0000000..6116e8e --- /dev/null +++ b/skills/ln-71-system-design-baseline-builder/SKILL.md @@ -0,0 +1,106 @@ +--- +name: ln-71-system-design-baseline-builder +description: "Creates a project baseline of architecture drivers and constraints. Use before design or planning; not for target design, plan review, implementation, or architecture audit." +--- + +# System Design Baseline Builder + +**Goal:** Create or update one durable source of truth for the project's architecture-driving requirements and constraints. Change only the approved architecture document; do not design the solution, review a plan, audit implementation, edit product code, or invent missing targets. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Repository rules and document conventions | Native file reads plus focused search | User-provided convention with an explicit limitation | +| Existing requirements and architecture artifacts | Narrow repository search and direct reads | Conversation evidence marked with its source | +| Current workload or service evidence | Metrics, dashboards, logs, manifests, or checked-in reports | Mark `UNKNOWN`; never manufacture production numbers | +| Current external limits or standards | Official documentation or specifications | Mark the claim `UNVERIFIED` | +| Document mutation | Minimal patch to the approved Markdown artifact | Return `BLOCKED` if no safe writable path is authorized | + +Use external research only when a time-sensitive fact changes a constraint. Do not browse for values that must come from product owners, operators, the repository, or measured workload. + +## Artifact Rules + +- Prefer an existing unambiguous architecture-requirements document. +- Otherwise use `docs/architecture/system-design-baseline.md`. +- Read before writing, preserve unrelated content, and update facts in place instead of creating parallel truth. +- Classify applicability separately as `APPLICABLE` or `NOT_APPLICABLE`, with evidence for exclusions. +- Rank each applicable item as `DRIVER`, `SUPPORTING`, or `INFORMATIONAL`. +- Classify evidence separately as `CONFIRMED`, `ASSUMED`, or `UNKNOWN`. +- Record source, owner, confirmation date, and review trigger for each architecture driver. +- Separate observed current values, required targets, hard limits, and future evolution triggers. +- Use measurable quality-attribute scenarios; avoid words such as "fast", "scalable", or "secure" without a response measure. +- Treat the baseline as versioned project knowledge, not an immutable promise. + +## Checklist + +### 1. Establish Scope and Destination + +- [ ] Resolve the project, business outcome, intended readers, approved documentation scope, and language. +- [ ] Read applicable repository instructions and inspect Git state so unrelated changes remain untouched. +- [ ] Search for existing requirement, architecture, SLO, recovery, security, cost, and ownership documents. +- [ ] Select one canonical artifact: reuse a clear equivalent or choose the default path; explain why no duplicate will be created. +- [ ] Return `BLOCKED` if the destination is ambiguous and choosing one could split project truth. + +### 2. Build the Evidence Ledger + +- [ ] Extract confirmed business goals, actors, critical journeys, scope, non-goals, and decision horizon. +- [ ] Record sources for current workload, data volume, service behavior, platform limits, and existing commitments. +- [ ] Separate repository facts from stakeholder choices and estimates. +- [ ] Detect contradictions between documents, code, configuration, and stated requirements; preserve both claims until resolved. +- [ ] Ask only for choices whose absence materially changes architecture; mark all other gaps `UNKNOWN`. + +### 3. Define and Prioritize Architecture Drivers + +- [ ] **Business and scope:** Record actors, critical journeys, business horizon, scope, non-goals, and externally committed outcomes. +- [ ] **Demand and data scale:** Record current and target users, rates, concurrency, payloads, growth, retention, and forecast horizon where relevant. +- [ ] **User-observable service quality:** Define SLIs and SLOs for availability, latency, throughput, error rate, correctness, or freshness with measurement windows. +- [ ] **Data semantics and recovery:** Define consistency, ordering, idempotency, reconciliation, durability, backup, RTO, RPO, and acceptable data loss at affected boundaries. +- [ ] **Security, privacy, and compliance:** Define trust boundaries, data classification, residency, access, audit, and destructive-action constraints. +- [ ] **Operations and economics:** Define ownership, operational capacity, cost envelope, supported regions, delivery cadence, and platform or vendor constraints. +- [ ] **Evolution:** Record thresholds, business events, or evidence that justify revisiting an assumption, target, or deferred capability. +- [ ] Separate applicability, criticality, and evidence status; do not use `UNKNOWN` to mean unimportant or `NOT_APPLICABLE`. +- [ ] Prioritize the few scenarios most likely to shape architecture and express each as source/stimulus/environment/artifact/response/measure. + +### 4. Write the Baseline + +- [ ] Create or update the artifact with: identity and status; business context; scope and non-goals; critical scenarios; workload and data; quality targets; recovery; consistency; security; cost and operations; constraints; assumptions and unknowns; review triggers. +- [ ] Give every material parameter its theme, applicability, criticality, evidence status, value or range, source, owner, as-of date, and review trigger. +- [ ] Keep calculations reproducible and label estimates separately from observed measurements. +- [ ] Link shared architecture artifacts only by repository path or document title; never require a particular workflow or tool. +- [ ] Preserve historical context needed to understand changed requirements instead of silently rewriting prior commitments. + +### 5. Validate and Report + +- [ ] Re-read the written artifact and verify that no unknown was converted into a confident fact. +- [ ] Check that targets are measurable, internally consistent, and proportionate to the evidenced business horizon. +- [ ] Check that every architecture-critical gap has an owner or exact next evidence action. +- [ ] Confirm no product code, tests, unrelated documents, or external systems changed. +- [ ] Use `READY` only when the baseline is usable for decisions and no material unknown lacks a safe handling rule; use `INCOMPLETE` for a useful artifact with consequential open drivers; use `BLOCKED` when scope, authority, or destination prevents safe creation. + +## Output Contract + +```markdown +# System Design Baseline + +**Verdict:** READY | INCOMPLETE | BLOCKED +**Artifact:** path + +## Established drivers +- Prioritized architecture-driving scenarios +- Applicable business, demand, quality, data, security, operational, economic, and evolution constraints + +## Driver register +| Theme | Parameter | Applicability | Criticality | Evidence status | Value or measure | Source and owner | Review trigger | +|---|---|---|---|---|---|---|---| + +## Changes made +- Created or updated sections +- Preserved conventions and related artifacts + +## Residual risks +Only constraints that can still reverse an architecture decision. +``` diff --git a/skills/ln-72-current-architecture-documenter/SKILL.md b/skills/ln-72-current-architecture-documenter/SKILL.md new file mode 100644 index 0000000..5f432a3 --- /dev/null +++ b/skills/ln-72-current-architecture-documenter/SKILL.md @@ -0,0 +1,103 @@ +--- +name: ln-72-current-architecture-documenter +description: "Documents implemented current-state architecture from repository evidence. Use for onboarding or migration baselines; not for target design, audit verdicts, or code changes." +--- + +# Current Architecture Documenter + +**Goal:** Produce a trustworthy snapshot of the architecture implemented in the checked-out repository. Document what exists and how it behaves; do not score it, prescribe a target architecture, repair code, or turn intended diagrams into facts. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Snapshot identity and worktree state | Git status, branch, remote, and HEAD | Record the supplied snapshot as `UNVERIFIED` | +| Structure and configuration | Native listing, search, manifests, and direct file reads | Narrow manual inspection | +| Symbols, dependencies, and consumers | Language intelligence or resolved dependency tooling | Search definitions, registrations, imports, and callers | +| Runtime and deployment topology | Entrypoints, IaC, containers, CI, configuration, and runtime evidence | Mark deployment relationships `UNKNOWN` | +| Document mutation | Minimal patch to the approved architecture document | Return `BLOCKED` if no writable path is authorized | + +Prefer local evidence over remote repository state. A path, diagram, or naming convention is a lead until executable wiring or an authoritative contract confirms it. + +## Artifact Rules + +- Reuse a clear current-state architecture document; otherwise use `docs/architecture/current-state.md`. +- Anchor the document to remote, branch, HEAD, worktree state, and observation date. +- Cite material claims with file paths, symbols, commands, or configuration keys. +- Label claims `OBSERVED`, `DOCUMENTED`, `INFERRED`, or `UNKNOWN`. +- Separate actual structure from intended target design and from audit findings. +- Map the whole system first, then deepen only the two or three areas needed to explain critical behavior. +- Preserve contradictions and uncertainty instead of resolving them by preference. +- Prefer responsibility-oriented descriptions over exhaustive file inventories. +- Record the evidence cutoff so readers can distinguish uninspected scope from genuine absence. +- Keep volatile counts or inventories only when they affect architectural understanding. + +## Checklist + +### 1. Establish the Documentation Contract + +- [ ] Resolve repository scope, intended readers, approved destination, required depth, and language. +- [ ] Read repository instructions and record snapshot identity plus dirty-worktree limitations. +- [ ] Search for existing current-state, baseline, target-design, decision, diagram, and deployment artifacts. +- [ ] Reuse an unambiguous current-state document or select the default path without duplicating project knowledge. +- [ ] Keep the run read-only except for the approved architecture document. + +### 2. Map the System Breadth + +- [ ] Identify languages, frameworks, package roots, generated surfaces, and canonical build or run commands from manifests and CI. +- [ ] Identify users, external systems, entrypoints, applications, services, processes, workers, scheduled jobs, and deployment units. +- [ ] Map major domains or modules, responsibilities, ownership, and dependency direction. +- [ ] Map data stores, caches, queues, files, external APIs, schemas, and systems of record. +- [ ] Record public interfaces, runtime discovery, registration, configuration composition, and environment boundaries. +- [ ] Record build, deploy, scale, and failure boundaries without inferring independence from directory names. + +### 3. Trace Critical Behavior + +- [ ] Select representative critical flows based on business importance and architectural reach. +- [ ] Trace each flow from actor or trigger through entrypoint, runtime coordination, domain behavior, persistence or integration, and observable outcome. +- [ ] Record synchronous and asynchronous hops, transaction ownership, consistency, retry, timeout, idempotency, and error propagation where evidenced. +- [ ] Describe deployment, startup, shutdown, health, observability, and recovery paths visible from repository evidence. +- [ ] Deepen only the hardest two or three subsystems; keep ordinary implementation detail out of the architecture document. + +### 4. Write the Current-State Artifact + +- [ ] Write snapshot identity, system context, component inventory, responsibilities, dependency and data flow, runtime topology, critical flows, deployment, ownership, and evidence index. +- [ ] Include the minimum useful diagrams inline or link existing diagram artifacts by path. +- [ ] Distinguish observed implementation from documented intent and explicitly list drift or contradictions without assigning severity. +- [ ] Mark remote-only, runtime-only, organizational, or production facts `UNKNOWN` when local evidence cannot establish them. +- [ ] Preserve existing manually maintained context unless repository evidence disproves it; document the contradiction when it does. + +### 5. Verify and Report + +- [ ] Open every cited file or symbol and remove unsupported claims. +- [ ] Confirm the map explains how critical behavior is discovered, executed, persisted, and deployed. +- [ ] Confirm abstraction levels are not mixed and names remain consistent across prose and diagrams. +- [ ] Confirm no target recommendation, audit verdict, product code, test, or external state was introduced. +- [ ] Use `DOCUMENTED` when the snapshot is evidence-backed and useful; use `INCONCLUSIVE` when material topology remains unknown; use `BLOCKED` when repository identity, scope, or destination cannot be established safely. + +## Output Contract + +```markdown +# Current Architecture Documentation + +**Verdict:** DOCUMENTED | INCONCLUSIVE | BLOCKED +**Artifact:** path +**Snapshot:** remote, branch, HEAD, worktree state, observed date + +## Architecture mapped +- Context, modules, runtime and deployment units +- Data, interfaces, ownership, and critical flows + +## Evidence limitations +| Claim or area | Status | Evidence inspected | Exact next action | +|---|---|---|---| + +## Changes made +- Created or updated sections and diagrams + +## Residual unknowns +Facts that require runtime, organizational, or external confirmation. +``` diff --git a/skills/ln-73-system-design-proposal-builder/SKILL.md b/skills/ln-73-system-design-proposal-builder/SKILL.md new file mode 100644 index 0000000..655f743 --- /dev/null +++ b/skills/ln-73-system-design-proposal-builder/SKILL.md @@ -0,0 +1,105 @@ +--- +name: ln-73-system-design-proposal-builder +description: "Creates a decision-complete target system design from requirements and constraints. Use before implementation planning; not for requirements baselines, reviews, audits, or code changes." +--- + +# System Design Proposal Builder + +**Goal:** Create a proportionate, evidence-backed target system design that turns requirements into explicit boundaries, contracts, data flow, failure behavior, operations, and tradeoffs. Change only the approved design document; do not implement, audit, or approve the delivery. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Requirements and constraints | Approved requirements, baseline, decisions, and direct stakeholder input | Mark material gaps and ask the smallest decision question | +| Current implementation and conventions | Repository search, manifests, entrypoints, and architecture artifacts | Treat as greenfield only when the user or repository establishes that fact; otherwise mark current state `UNKNOWN` and return `REVISE` or `BLOCKED` when the gap can change boundaries, compatibility, or migration | +| External capabilities and limits | Current official documentation and specifications | Mark claims `UNVERIFIED`; avoid vendor-dependent commitment | +| Estimates | Reproducible arithmetic from sourced workload assumptions | Use ranges and sensitivity; never present estimates as measurements | +| Document mutation | Minimal patch to the approved target-design artifact | Return `BLOCKED` if scope or path is unsafe | + +Use patterns as candidate solutions, not goals. Introduce infrastructure only when a requirement, failure mode, ownership boundary, or measured horizon pays for its lifecycle cost. + +## Artifact Rules + +- Reuse a clear target-design document; otherwise use `docs/architecture/target-design.md`. +- Read available baseline, current-state, decision, interface, diagram, and migration artifacts by path; none is mandatory. +- Label facts, assumptions, estimates, proposed decisions, and unresolved choices separately. +- Start with requirements and estimates, then domains and contracts, HLD, and only the critical LLD. +- Consider at least two credible alternatives for consequential decisions, including the simplest option. +- Prefer reversible choices and a modular monolith unless evidence justifies independent service boundaries. +- Do not silently change an accepted decision; record the conflict and required governance action. + +## Checklist + +### 1. Frame the Design + +- [ ] Resolve business outcome, actors, critical journeys, scope, non-goals, decision horizon, and intended readers. +- [ ] Read repository instructions and inspect relevant architecture artifacts and current implementation. +- [ ] Extract functional requirements and measurable quality drivers, preserving their source and status. +- [ ] Identify architecture-critical unknowns and ask only questions whose answers change the target shape. +- [ ] Return `BLOCKED` when a required business boundary or safety constraint cannot be responsibly assumed. + +### 2. Estimate Before Choosing Components + +- [ ] Estimate average and peak request or event rates, concurrency, payload and bandwidth, storage growth, retention, and recovery volume where relevant. +- [ ] Show formulas, ranges, growth horizon, and assumptions; identify the variables that can reverse a choice. +- [ ] Identify likely first bottlenecks and explicit thresholds for deferred scaling mechanisms. +- [ ] Separate availability, latency, durability, consistency, security, cost, and operability requirements from implementation preferences. +- [ ] Reject speculative scale and list complex mechanisms intentionally deferred. + +### 3. Define Domains, Data, and Contracts + +- [ ] Map business capabilities, domains or modules, ownership, invariants, and allowed dependency direction. +- [ ] Define systems of record, data models at architecture depth, lifecycle, retention, consistency, and transaction boundaries. +- [ ] Define public APIs, events, commands, schemas, errors, idempotency, ordering, versioning, and compatibility expectations. +- [ ] Define trust boundaries, identities, authorization, sensitive data, secrets, abuse controls, and audit needs proportionate to risk. +- [ ] Keep framework and vendor details outside the core model unless they are genuine constraints. + +### 4. Build HLD and Critical LLD + +- [ ] Describe system context, deployable units, stores, queues, external systems, responsibilities, and labeled data flows. +- [ ] Trace success, overload, dependency failure, partial failure, retry, timeout, degradation, recovery, and cancellation for critical journeys. +- [ ] Deep-dive only the two or three components with the highest correctness, scale, security, or reversibility risk. +- [ ] Define observability, SLI measurement points, health, deployment strategy, rollback, backup, and operator actions. +- [ ] Define ownership, team impact, cost drivers, and operational burden for the proposed topology. + +### 5. Decide and Validate + +- [ ] Compare credible alternatives against requirements, estimates, failure behavior, complexity, cost, migration, and future triggers. +- [ ] State selected and rejected options with consequences, sensitivity points, and assumptions that would reopen the decision. +- [ ] Identify significant decisions that deserve their own compact decision records without requiring another workflow. +- [ ] Define architecture acceptance evidence: contract checks, load or failure experiments, security validation, recovery proof, and observability signals. +- [ ] Outline current-to-target implications and compatibility needs without expanding into a full implementation plan. + +### 6. Write and Report + +- [ ] Write context, drivers, estimates, domains, contracts, HLD, critical LLD, failure and operations model, security, alternatives, decisions, validation, open questions, and evolution triggers. +- [ ] Preserve existing content outside the approved scope and link shared artifacts only by document path or title. +- [ ] Re-read the proposal for unsupported facts, hidden decisions, mixed abstraction, and unjustified machinery. +- [ ] Confirm no code, tests, delivery plan, audit result, or external state changed. +- [ ] Use `READY` only when the design is decision-complete enough for implementation planning; use `REVISE` for material but solvable gaps; use `BLOCKED` when required intent, evidence, authority, or destination is unavailable. + +## Output Contract + +```markdown +# System Design Proposal + +**Verdict:** READY | REVISE | BLOCKED +**Artifact:** path + +## Target design +- Requirements, estimates, boundaries, contracts, HLD, and critical LLD + +## Decisions and tradeoffs +| Decision | Selected option | Alternatives | Evidence | Reopen trigger | +|---|---|---|---|---| + +## Validation and transition +- Required evidence, compatibility, rollout, rollback, and observability + +## Open decisions and residual risks +Only items that can still change implementation planning. +``` diff --git a/skills/ln-74-architecture-decision-recorder/SKILL.md b/skills/ln-74-architecture-decision-recorder/SKILL.md new file mode 100644 index 0000000..f550d7c --- /dev/null +++ b/skills/ln-74-architecture-decision-recorder/SKILL.md @@ -0,0 +1,103 @@ +--- +name: ln-74-architecture-decision-recorder +description: "Records one architecture decision with context, alternatives, tradeoffs, and consequences. Use for a significant choice; not for broad design, audit, or implementation." +--- + +# Architecture Decision Recorder + +**Goal:** Preserve the context, forces, alternatives, decision, and consequences of one architecturally significant choice in a compact durable record. Change only approved decision documentation; do not design the whole system, approve a decision silently, delete history, audit code, or implement the choice. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Existing decision convention | Repository search and direct document reads | Use the default path and compact format | +| Decision drivers | Requirements, architecture artifacts, implementation evidence, and stakeholder statements | Mark unsupported drivers `UNKNOWN` | +| Alternatives and external claims | Repository evidence plus current official sources | Mark time-sensitive claims `UNVERIFIED` | +| Sequence and supersession | Existing filenames, indexes, and decision links | Return `BLOCKED` rather than reuse a number | +| Document mutation | Minimal patch to one approved decision record and necessary supersession links | Return `BLOCKED` if authority or path is unclear | + +One record captures one decision. If the request contains independent decisions with different drivers or lifecycles, split them only with explicit approval. + +## Artifact Rules + +- Reuse the repository's established ADR convention when one exists. +- Otherwise use `docs/architecture/decisions/NNNN-.md` with the next unused monotonic number. +- Never renumber, overwrite, or delete historical records. +- Default a new record to `Proposed`. +- Use `Accepted` only after explicit confirmation from an authorized decision-maker. +- Mark a replaced record `Superseded` and link both directions; preserve its original content. +- Link shared requirements and architecture artifacts only by path or title. +- Record positive, negative, and neutral consequences without advocacy language. +- Keep the record short enough to review as a single decision. +- Preserve the rationale a future maintainer needs to reconsider it safely. +- Label retrospective records explicitly; do not imply that documentation created after implementation was prior approval. +- Separate evidence needed before acceptance from monitoring required after adoption. +- Prefer stable repository references over conversation, branch-local, or ephemeral links. + +## Checklist + +### 1. Establish the Decision Contract + +- [ ] Resolve the exact decision, scope, owner or deciders, affected system, and why the choice is architecturally significant. +- [ ] Read repository instructions, Git state, and existing decision conventions. +- [ ] Search for duplicate, conflicting, deprecated, or superseding decisions before allocating a new record. +- [ ] Confirm the request is one decision rather than a broad design or implementation plan. +- [ ] Resolve the approved status; default to `Proposed` when acceptance is not explicit. + +### 2. Gather Context and Forces + +- [ ] State the current context and problem in value-neutral language. +- [ ] Extract business drivers, quality attributes, constraints, assumptions, and decision horizon from available evidence. +- [ ] Identify affected boundaries, contracts, data, security, operations, cost, ownership, and migration implications. +- [ ] Separate present facts from forecasts and preferences. +- [ ] Record contradictions or missing evidence that could change the choice. + +### 3. Evaluate Alternatives + +- [ ] Include the status quo and the simplest credible option unless they are demonstrably infeasible. +- [ ] Include materially different alternatives rather than cosmetic variants. +- [ ] Compare options against the same drivers: correctness, quality targets, complexity, reversibility, cost, operations, team fit, and evolution. +- [ ] State why each rejected alternative loses in this context without claiming universal inferiority. +- [ ] Record sensitivity or review triggers that would make a rejected option preferable later. + +### 4. Record the Decision + +- [ ] Write title, status, date, deciders or owner, context, drivers, considered options, decision, consequences, validation, and review triggers. +- [ ] State the decision in active, testable language and name what remains deliberately undecided. +- [ ] Record positive, negative, and neutral consequences plus accepted risks. +- [ ] Link affected requirements, designs, diagrams, interfaces, migration documents, or issues by stable repository reference. +- [ ] For supersession, update status and cross-links without erasing prior rationale. + +### 5. Validate and Report + +- [ ] Confirm the record contains one decision and can be understood without conversation history. +- [ ] Confirm every consequential claim has evidence, a named assumption, or an explicit owner. +- [ ] Confirm `Accepted` was not assigned without explicit authority. +- [ ] Confirm no architecture, code, tests, task tracker, or external system changed beyond approved decision documentation. +- [ ] Use `RECORDED` when the record and status are valid; use `INCOMPLETE` when material context or authority remains open; use `BLOCKED` for ambiguous scope, unsafe numbering, conflicting ownership, or no writable destination. + +## Output Contract + +```markdown +# Architecture Decision Record + +**Verdict:** RECORDED | INCOMPLETE | BLOCKED +**Artifact:** path +**Decision status:** Proposed | Accepted | Deprecated | Superseded + +## Decision captured +- Context, selected option, and decisive drivers + +## Alternatives and consequences +- Rejected options, accepted costs, and review triggers + +## Links changed +- New record and any supersession links + +## Open authority or evidence +Only items that prevent acceptance or could reverse the decision. +``` diff --git a/skills/ln-75-architecture-diagram-builder/SKILL.md b/skills/ln-75-architecture-diagram-builder/SKILL.md new file mode 100644 index 0000000..cd34d53 --- /dev/null +++ b/skills/ln-75-architecture-diagram-builder/SKILL.md @@ -0,0 +1,103 @@ +--- +name: ln-75-architecture-diagram-builder +description: "Creates evidence-backed current or target architecture diagrams when the diagram is the primary deliverable. Not for UI design, architecture audit, or invented structure." +--- + +# Architecture Diagram Builder + +**Goal:** Create the smallest set of understandable, evidence-backed diagrams needed to communicate current or proposed architecture. Change only approved architecture documentation; do not invent relationships, perform visual product design, replace prose evidence, audit fitness, or edit implementation. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Architecture evidence | Repository files, runtime wiring, IaC, contracts, and approved artifacts | User-provided model with `UNVERIFIED` labels | +| Relationship tracing | Language intelligence, dependency tools, and focused search | Direct inspection of producers, consumers, and registrations | +| Diagram format | Existing repository convention and renderer | Mermaid in Markdown, then plain ASCII | +| Syntax verification | Repository renderer, parser, or preview | Manual fence, identifier, and relationship inspection | +| Document mutation | Minimal patch to approved diagram artifacts | Return `BLOCKED` if path or evidence boundary is unsafe | + +Diagrams communicate a model; executable behavior remains authoritative for current state. Keep current, target, and transition views visibly distinct. + +## Artifact Rules + +- Reuse an existing diagram convention or use `docs/architecture/diagrams/.md`. +- Prefer Markdown with Mermaid for text-reviewable source; use ASCII when Mermaid is unsupported. +- Choose only views that answer a named audience question. +- For static structure, start with system context and container views; add component depth only when it changes a decision. +- Use sequence or dynamic views for critical runtime interactions and failure paths. +- Use data-flow views for stores, sensitive data, trust boundaries, and transformations. +- Use deployment views for runtime nodes, regions, networks, scaling, and failover. +- Give every diagram a title, scope, audience, legend, element descriptions, and labeled relationships. +- Keep diagram source reviewable in version control. +- Split views when one diagram needs multiple unrelated stories. +- Never use color as the only carrier of meaning. +- Use stable element identifiers and concise display labels so revisions produce reviewable diffs. +- Keep detailed evidence beside the diagram rather than crowding nodes and relationships. +- Preserve an understandable existing notation; introduce a new notation only when it answers the audience question better. + +## Checklist + +### 1. Establish the Diagram Contract + +- [ ] Resolve audience, question, current or target state, scope, approved destination, and required notation. +- [ ] Read repository instructions, relevant architecture artifacts, and existing diagram conventions. +- [ ] Select the minimum useful view or views; reject diagrams that add no relationship clarity. +- [ ] Define the evidence boundary and label user-supplied or proposed elements separately. +- [ ] Keep the run read-only except for approved architecture diagram documentation. + +### 2. Build the Architecture Model + +- [ ] Identify people, software systems, deployable containers, components, stores, queues, and external dependencies relevant to the question. +- [ ] Record responsibility, type, technology when decision-relevant, owner when known, and current/target status for each element. +- [ ] Resolve relationship direction, label, protocol or data, synchronicity, and trust or network boundary where relevant. +- [ ] Trace runtime discovery and registration before including current-state routes, handlers, jobs, plugins, or consumers. +- [ ] Mark uncertain elements or relationships `UNKNOWN` rather than completing the picture aesthetically. + +### 3. Select and Draw Views + +- [ ] Create a system-context view when readers need system scope and external actors. +- [ ] Create a container or deployment view when readers need responsibilities, deployability, stores, or operational topology. +- [ ] Create a component view only for a complex area whose internal boundaries change understanding. +- [ ] Create sequence or dynamic views for critical success, failure, retry, timeout, recovery, or migration interactions. +- [ ] Create data-flow or trust-boundary views when security, privacy, residency, or system-of-record questions require them. +- [ ] Avoid mixing abstraction levels in one view unless the exception is explicit and necessary. + +### 4. Make the Diagram Self-Describing + +- [ ] Add title, diagram type, scope, current/target marker, intended audience, and observation or proposal date. +- [ ] Add a legend for shapes, colors, line styles, abbreviations, and uncertainty markers. +- [ ] Label every relationship with intent or data; avoid generic arrows and unexplained acronyms. +- [ ] Keep names consistent with code, contracts, and shared architecture documents. +- [ ] Add compact evidence notes or links sufficient to trace current-state claims. + +### 5. Verify and Report + +- [ ] Validate syntax with the repository renderer or perform a complete manual syntax inspection. +- [ ] Check readability at normal rendering size and split overloaded views rather than shrinking labels. +- [ ] Verify each current-state element and relationship against repository evidence. +- [ ] Confirm no UI design, audit verdict, code, tests, or external state changed. +- [ ] Use `READY` when diagrams are valid, scoped, and evidenced; use `INCONCLUSIVE` when material relationships remain unknown; use `BLOCKED` when scope, evidence, format, or destination prevents a trustworthy diagram. + +## Output Contract + +```markdown +# Architecture Diagrams + +**Verdict:** READY | INCONCLUSIVE | BLOCKED +**Artifacts:** paths + +## Views created or updated +| View | State | Audience question | Evidence basis | +|---|---|---|---| + +## Verification +- Syntax or render check +- Current-state relationship checks + +## Unknowns and residual risks +Only missing relationships or rendering limits that affect interpretation. +``` diff --git a/skills/ln-76-architecture-migration-planner/SKILL.md b/skills/ln-76-architecture-migration-planner/SKILL.md new file mode 100644 index 0000000..050c679 --- /dev/null +++ b/skills/ln-76-architecture-migration-planner/SKILL.md @@ -0,0 +1,105 @@ +--- +name: ln-76-architecture-migration-planner +description: "Plans a reversible architecture migration with compatibility, data movement, rollout, and rollback. Use for current-to-target transitions; not execution, generic planning, or delivery review." +--- + +# Architecture Migration Planner + +**Goal:** Create a safe, reversible transition from an evidenced current architecture to an explicit target architecture. Change only the approved migration document; do not execute migrations, edit product code, build a generic task plan, approve delivery, or hide irreversible steps. + +**Execution contract:** Treat the ordered checkbox workflow below as this skill's Definition of Done. Work through every item in order, and mark it complete only when its action and required evidence are complete. `N/A`, skipped, unavailable, or delegated items remain incomplete. +Before returning, apply this skill's verdict and approval rules to every incomplete item and prepend **Checklist: X/Y complete**
**Incomplete: None | section/item — reason; outcome impact; exact next action**; list every incomplete item. + +## Tool Routing + +| Need | Preferred capability | Fallback | +|---|---|---| +| Current and target states | Repository evidence plus approved architecture artifacts | Explicit user-provided states with limitations | +| Consumers and compatibility | Language intelligence, schema tools, config search, telemetry, and direct inspection | Conservative inventory marked `UNVERIFIED` | +| Data scale and runtime risk | Production metrics, checked-in reports, migrations, and workload evidence | Ranges with validation gates before execution | +| External migration constraints | Official vendor migration and compatibility guidance | Mark dependent phases `BLOCKED` or `UNVERIFIED` | +| Document mutation | Minimal patch to the approved migration-plan artifact | Return `BLOCKED` when authority or path is unclear | + +A migration phase must leave the system in a supported state. Additive and reversible steps precede cutover; destructive cleanup follows verified zero use. + +## Artifact Rules + +- Reuse a clear architecture migration document; otherwise use `docs/architecture/migration-plan.md`. +- Treat current-state, target-design, baseline, decisions, diagrams, interfaces, and telemetry as optional shared evidence. +- Keep the plan architectural: phases, compatibility, data, topology, gates, rollback, ownership, and removal. +- Leave file-level implementation tasks to downstream planning. +- Separate preparation, coexistence, migration, cutover, stabilization, and removal. +- Never describe rollback as "revert" when data or external effects are not reversible. +- Give every destructive step explicit approval, backup, restoration, and zero-consumer evidence requirements. + +## Checklist + +### 1. Establish the Transition Contract + +- [ ] Resolve migration scope, business outcome, current state, target state, non-goals, deadline or horizon, owners, and approved destination. +- [ ] Read repository instructions, Git state, relevant architecture artifacts, migrations, deployment configuration, and compatibility policies. +- [ ] Verify that current and target states are specific enough to compute a gap; return `BLOCKED` rather than invent either state. +- [ ] Identify protected user journeys, invariants, SLOs, recovery objectives, compliance duties, and change windows. +- [ ] Keep the run read-only except for the approved migration document. + +### 2. Build the Gap and Dependency Map + +- [ ] Inventory affected modules, deployables, data stores, schemas, APIs, events, configuration, infrastructure, and operational procedures. +- [ ] Inventory internal and external consumers, owners, versions, traffic, data volume, and evidence quality. +- [ ] Map current-to-target changes in boundaries, ownership, contracts, data, runtime topology, observability, and failure behavior. +- [ ] Identify shared mutable resources, sequencing dependencies, long-running work, mixed-version windows, and irreversible effects. +- [ ] Record unknown consumers or usage as migration risks; absence of search results is not zero usage. + +### 3. Design Compatibility and Data Safety + +- [ ] Define old/new contract compatibility, version negotiation, adapters, dual-read or dual-write behavior, and deprecation policy where relevant. +- [ ] Use expand/migrate/contract for schema and data-shape changes; keep additive and destructive operations in separate releases. +- [ ] Define backfill selection, batching, throttling, idempotency, checkpoints, retries, reconciliation, and correctness oracle. +- [ ] Define source of truth during coexistence and conflict handling for concurrent writes. +- [ ] Define backup, restore, RPO/RTO impact, privacy, retention, and audit evidence for data movement. + +### 4. Build Reversible Phases + +- [ ] Define preparation, shadow or coexistence, progressive migration, cutover, stabilization, and old-path removal as independently verifiable phases. +- [ ] For every phase, state prerequisites, changed architecture state, owner, entry gate, observable success, abort condition, rollback or roll-forward action, and exit evidence. +- [ ] Define feature flags, routing controls, canary cohorts, rate limits, maintenance windows, and blast-radius controls where justified. +- [ ] Define metrics, logs, traces, reconciliation reports, dashboards, alerts, and SLO gates needed before traffic or data movement. +- [ ] Keep old and new versions interoperable through realistic deployment ordering and rollback windows. + +### 5. Plan Cutover and Removal + +- [ ] Define go/no-go authority, communication, freeze conditions, exact cutover control, and immediate verification. +- [ ] Define rollback boundaries separately for code, configuration, traffic, schema, and already-migrated data. +- [ ] Require measured zero use, migrated consumers, retention expiry, and explicit approval before destructive removal. +- [ ] List old code paths, contracts, flags, adapters, jobs, data, infrastructure, dashboards, and documentation to remove. +- [ ] Define post-cutover observation period, ownership handoff, incident response, and closure evidence. + +### 6. Write and Report + +- [ ] Write transition summary, state gap, dependencies, compatibility, data plan, phased sequence, gates, observability, rollback, removal, owners, assumptions, and open decisions. +- [ ] Link shared artifacts by stable repository path or title without requiring a particular workflow. +- [ ] Re-read every phase for unsupported zero-downtime, zero-loss, consumer, capacity, or reversibility claims. +- [ ] Confirm no migration, code, test, deployment, task tracker, or external change was executed. +- [ ] Use `READY` only when phases are safely executable inputs to implementation planning; use `REVISE` for material compatibility, data, gate, or rollback gaps; use `BLOCKED` when current state, target state, authority, or safety evidence is unavailable. + +## Output Contract + +```markdown +# Architecture Migration Plan + +**Verdict:** READY | REVISE | BLOCKED +**Artifact:** path + +## State transition +- Current state, target state, gap, consumers, and invariants + +## Phases +| Phase | Entry gate | Change | Success evidence | Abort condition | Rollback or roll-forward | +|---|---|---|---|---|---| + +## Data, compatibility, and removal +- Coexistence, migration, reconciliation, cutover, and zero-use proof + +## Open decisions and residual risks +Only items that can change safety, ordering, or reversibility. +```