docs: address model experience review
This commit is contained in:
Tianyi Cui
@@ -4,12 +4,6 @@ Dev-mode event-contract invariants and session-log freeze. A pure-listener plugi
|
||||
|
||||
**Off in production.** Enable it in tests and the demos, where a contract violation should fail loudly. It costs nothing when not registered, and doubles as executable documentation of the event taxonomy — the assertions *are* the contract.
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| None | The plugin observes and validates session events, agent states, and frozen model requests; it does not rewrite a prompt, schema, message, or stream. An invariant failure aborts the faulty turn instead of adding guidance. | Zero model tokens when checks pass; a failure prevents or ends a request rather than contributing context. |
|
||||
|
||||
## Plugin
|
||||
|
||||
A functional plugin — register the module namespace (this is what loading by name in `cordis.yml` does):
|
||||
@@ -60,6 +54,12 @@ A `DeepReadonly<SessionEvent>` is high type-noise across every log consumer, and
|
||||
|
||||
A seeded/forked session arrives with events already in its log (the `Session` constructor copies the seed without emitting `session/event`). On `session/created` the plugin replays the existing log through the checker and freezes those entries, so seeded history is held to the same contract.
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| None | The plugin observes and validates session events, agent states, and frozen model requests; it does not rewrite a prompt, schema, message, or stream. An invariant failure aborts the faulty turn instead of adding guidance. | Zero model tokens when checks pass; a failure prevents or ends a request rather than contributing context. |
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The request-reconstructability assertion covers loop-built requests only** — hand-built one-shots (e.g. compaction's summarize call) carry no live `sessionId` marker and are skipped.
|
||||
|
||||
@@ -4,12 +4,6 @@ A replay LLM plugin for keyless snapshot tests. It installs a single `llm/stream
|
||||
|
||||
Its consumer is the ACP snapshot harness in `examples/acp-agent`, which loads this plugin (via `cordis.snapshot.yml`) in place of a real LLM adapter. The package exists so its derive/parse/replay logic falls under the per-file 100% coverage gate on `packages/*/src` (the same logic, while it lived under `examples/`, was outside the gate).
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| Keyless test stream | The real loop still assembles its normal system prompt, tools, prefix, and history, but no provider model receives them. Recorded assistant chunks are replayed as the response and then enter later history exactly like live output. | Zero billed or tokenizer-evaluated model tokens. Fixture output creates deterministic retained test context for later replay steps. |
|
||||
|
||||
## How the fixture works
|
||||
|
||||
The fixture IS the persisted session log (`<scenario>/session.jsonl`). Its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call's chunk sequence (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`", done by the snapshot harness — this plugin does not record. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}` (the harness pins that content in one scenario and scrubs the rest); replay is indifferent — derivation reads only `assistant/chunk` events and the line-0 session header.
|
||||
@@ -50,6 +44,12 @@ Replay keys every call by its calling session id (`GenerateOptions.sessionId`, s
|
||||
|
||||
Named `name` / `inject` / `Config` / `apply`, with **no default export**: the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default would collapse the module to the bare function and drop the `inject` namespace (see [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)).
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| Keyless test stream | The real loop still assembles its normal system prompt, tools, prefix, and history, but no provider model receives them. Recorded assistant chunks are replayed as the response and then enter later history exactly like live output. | Zero billed or tokenizer-evaluated model tokens. Fixture output creates deterministic retained test context for later replay steps. |
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **First-call-order script binding assumes sequential delegation** — a cut that runs sibling subagents concurrently (or a compaction summarize call landing mid-run) would bind live sessions to recorded scripts non-deterministically; a stronger keying is deferred until such a scenario exists (`XXX(concurrent-subagents)`).
|
||||
|
||||
@@ -4,12 +4,6 @@ A scripted `SubagentProvider` for testing the [subagent seam](../../subagent/sub
|
||||
|
||||
It lets a test drive `ctx.subagents` and the model-facing `dsh-tool-subagent` through the **real cordis Loader / export path**, exercising provider registration, start-time capability validation, the run lifecycle (`result` / `cancel` / `dispose`), and the structured-output branch — all deterministically and keylessly.
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| Deterministic test result | No child model runs. When exercised through `dsh-tool-subagent`, the parent sees the mock provider's configured reply or stop-reason error, and structured tests receive the configured object. | Zero child-model tokens. Only the scripted final result is added to the parent test history. |
|
||||
|
||||
## Usage
|
||||
|
||||
Load it as a plugin (functional shape: `name`/`inject`/`Config`/`apply`, no default). Config (all optional):
|
||||
@@ -25,6 +19,12 @@ Load it as a plugin (functional shape: `name`/`inject`/`Config`/`apply`, no defa
|
||||
|
||||
A `cancel()` issued before `result` settles flips the stop reason to `aborted`, so the cancellation path is observable.
|
||||
|
||||
## Model Experience
|
||||
|
||||
| Context surface | What the model sees | Token effect |
|
||||
|---|---|---|
|
||||
| Deterministic test result | No child model runs. When exercised through `dsh-tool-subagent`, the parent sees the mock provider's configured reply or stop-reason error, and structured tests receive the configured object. | Zero child-model tokens. Only the scripted final result is added to the parent test history. |
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Scripted provider only** — it does not run a model, create a child agent, or exercise real prompt/tool-loop behavior.
|
||||
|
||||
Reference in New Issue
Block a user