Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: address model experience review

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-12 02:55:26 +08:00
parent a08485fc80
commit b5cd511f35
64 changed files with 443 additions and 410 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
packages/llm/llm/README.md
View File

@@ -2,13 +2,6 @@
Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin.
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Provider request transport | This service adds no system text, schema, or message. It routes the already-assembled frozen `GenerateOptions` to one adapter, while `llm/stream` listeners may cache, retry, or replace the stream without mutating that request. | Zero direct context tokens. The selected adapter and provider tokenizer determine billing, cache accounting, and serialization overhead for the existing content. |
| Streamed model output | Text, reasoning, and tool-call chunks are exposed to the loop, which decides what becomes retained assistant history. | Output usage is provider-reported; later input cost arises only after the loop records assembled content. |
## Service: `LlmService` (ctx key: `llm`)
An adapter registry plus a single streaming call surface, interceptable via a waterfall event.
@@ -55,6 +48,13 @@ Every product adapter must identify the application on every provider HTTP reque
Two adapters implement `LlmAdapter` against this vocabulary, deliberately built on different internals to keep the contract honest (see [the twin LLM adapters](../../../docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md)): [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) (hand-rolled fetch/SSE) and [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) (via `@earendil-works/pi-ai`). The pair pinned down the `StreamChunk` conventions now documented in `types.ts` (usage before finish, raw-string tool arguments, the two sanctioned error paths).
## Model Experience
| Context surface | What the model sees | Token effect |
|---|---|---|
| Provider request transport | This service adds no system text, schema, or message. It routes the already-assembled frozen `GenerateOptions` to one adapter, while `llm/stream` listeners may cache, retry, or replace the stream without mutating that request. | Zero direct context tokens. The selected adapter and provider tokenizer determine billing, cache accounting, and serialization overhead for the existing content. |
| Streamed model output | Text, reasoning, and tool-call chunks are exposed to the loop, which decides what becomes retained assistant history. | Output usage is provider-reported; later input cost arises only after the loop records assembled content. |
## Known Limitations and Deferred Work
- **No retry/caching/rate-limit layer ships** — `llm/stream` is the intended wrap seam and has no production listener, so provider 429/5xx failures surface immediately.