docs: unwrap hard-wrapped Markdown to one line per paragraph
Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. Reflow all tracked non-vendor Markdown (plus vendor/AGENTS.md) so each prose paragraph is a single line; soft-wrapping is the editor's job. Fenced code, tables, and list structure are preserved (wrapped list items fold to one line per bullet). Documents the convention in AGENTS.md.
This commit is contained in:
@@ -1,10 +1,6 @@
|
||||
# Cookbook: adding an LLM adapter
|
||||
|
||||
How to connect a new model provider. Reference implementations:
|
||||
`packages/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm-pi-ai`
|
||||
(wrapping an LLM library). Read the `StreamChunk` doc in
|
||||
`packages/llm/src/types.ts` first — it records the protocol conventions both
|
||||
adapters were verified against.
|
||||
How to connect a new model provider. Reference implementations: `packages/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm-pi-ai` (wrapping an LLM library). Read the `StreamChunk` doc in `packages/llm/src/types.ts` first — it records the protocol conventions both adapters were verified against.
|
||||
|
||||
## The shape
|
||||
|
||||
@@ -22,54 +18,26 @@ export function apply(ctx: Context, config: Config) {
|
||||
}
|
||||
```
|
||||
|
||||
Registration is effect-based (HMR-safe); one adapter per model name —
|
||||
duplicates throw. Secrets are cordis-native: schemastery Config with env
|
||||
fallbacks, fed from cordis.yml via `!!js process.env.MY_KEY`. Never read
|
||||
ad-hoc key files in code.
|
||||
Registration is effect-based (HMR-safe); one adapter per model name — duplicates throw. Secrets are cordis-native: schemastery Config with env fallbacks, fed from cordis.yml via `!!js process.env.MY_KEY`. Never read ad-hoc key files in code.
|
||||
|
||||
## Protocol obligations (the contract two implementations verified)
|
||||
|
||||
- Emit `usage` BEFORE `finish`; emit NOTHING after `finish`. The robust way:
|
||||
buffer finish/usage until the provider's end-of-stream marker, then flush
|
||||
(handles providers that send trailing usage-only chunks).
|
||||
- Tool-call `arguments` are RAW JSON strings end-to-end; stream fragments as
|
||||
`argumentsDelta`. If your provider hands back parsed objects, re-stringify
|
||||
at `block-end`.
|
||||
- Allocate block `index`es in first-seen stream order; reuse the index for
|
||||
every delta of the same block.
|
||||
- Errors have exactly two sanctioned paths: THROW from `stream()` (transport
|
||||
and protocol failures — use `LlmError` with a stable code), or end the
|
||||
stream with `finish {kind: 'error' | 'aborted'}` (provider in-band
|
||||
failures). Consumers handle both; pick per failure class and document it.
|
||||
- Emit `usage` BEFORE `finish`; emit NOTHING after `finish`. The robust way: buffer finish/usage until the provider's end-of-stream marker, then flush (handles providers that send trailing usage-only chunks).
|
||||
- Tool-call `arguments` are RAW JSON strings end-to-end; stream fragments as `argumentsDelta`. If your provider hands back parsed objects, re-stringify at `block-end`.
|
||||
- Allocate block `index`es in first-seen stream order; reuse the index for every delta of the same block.
|
||||
- Errors have exactly two sanctioned paths: THROW from `stream()` (transport and protocol failures — use `LlmError` with a stable code), or end the stream with `finish {kind: 'error' | 'aborted'}` (provider in-band failures). Consumers handle both; pick per failure class and document it.
|
||||
- Honor `options.signal` (pass it to fetch / your SDK).
|
||||
- `prefill` and other unsupported `GenerateOptions` fields: throw
|
||||
`LlmError(..., 'UNSUPPORTED')` rather than silently dropping.
|
||||
- `prefill` and other unsupported `GenerateOptions` fields: throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping.
|
||||
|
||||
Provider-specific request knobs (thinking modes, effort levels) belong in
|
||||
the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays
|
||||
provider-neutral.
|
||||
Provider-specific request knobs (thinking modes, effort levels) belong in the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays provider-neutral.
|
||||
|
||||
## Structure that worked
|
||||
|
||||
Split the adapter into testable stages (llm-deepseek's layout): wire types
|
||||
(`types.ts`, coverage-exempt) → request serializer → SSE/transport parser →
|
||||
chunk-translation state machine → a thin adapter class wiring them. Each
|
||||
stage gets its own unit suite.
|
||||
Split the adapter into testable stages (llm-deepseek's layout): wire types (`types.ts`, coverage-exempt) → request serializer → SSE/transport parser → chunk-translation state machine → a thin adapter class wiring them. Each stage gets its own unit suite.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit: mock the provider, not the harness.** A scripted `node:http`
|
||||
server speaking the provider's wire format covers happy paths, every error
|
||||
status, malformed payloads, premature closes, and aborts — no network, and
|
||||
it drives the 100% per-file coverage gate. Works for SDK-backed adapters
|
||||
too (point the SDK's baseURL at the mock).
|
||||
- **Hostile framing tests.** Split stream payloads at arbitrary byte
|
||||
positions (including mid-UTF-8) — real networks do.
|
||||
- **E2E: `tests/*.e2e.ts`** under `yarn test:e2e`, gated with
|
||||
`describe.skipIf(!process.env.MY_KEY)` so CI (no secrets) stays green.
|
||||
Cover each model × each provider mode you map (thinking on/off, effort
|
||||
levels), a tool-call round trip INCLUDING the follow-up turn with results
|
||||
in history, and loose assertions only (substring/structure, bounded
|
||||
maxTokens — real models are nondeterministic).
|
||||
- Register the e2e file pattern in `knip.json` (per-workspace `entry`
|
||||
override) or knip flags it unused.
|
||||
- **Unit: mock the provider, not the harness.** A scripted `node:http` server speaking the provider's wire format covers happy paths, every error status, malformed payloads, premature closes, and aborts — no network, and it drives the 100% per-file coverage gate. Works for SDK-backed adapters too (point the SDK's baseURL at the mock).
|
||||
- **Hostile framing tests.** Split stream payloads at arbitrary byte positions (including mid-UTF-8) — real networks do.
|
||||
- **E2E: `tests/*.e2e.ts`** under `yarn test:e2e`, gated with `describe.skipIf(!process.env.MY_KEY)` so CI (no secrets) stays green. Cover each model × each provider mode you map (thinking on/off, effort levels), a tool-call round trip INCLUDING the follow-up turn with results in history, and loose assertions only (substring/structure, bounded maxTokens — real models are nondeterministic).
|
||||
- Register the e2e file pattern in `knip.json` (per-workspace `entry` override) or knip flags it unused.
|
||||
|
||||
Reference in New Issue
Block a user