feature(session): session surface

docs/adr/0019-session-surface.md

@@ -0,0 +1,63 @@
+# ADR 0019: Session surface — a linked list over the event log for LLM message derivation
+
+Status: accepted (2026-06-17)
+
+## Context
+
+The `Session` event log is the single source of truth ([ADR 0003](0003-event-sourced-sessions.md)), but the only view over it was `deriveMessages()` — a linear scan that filtered and transformed raw events into `Message[]`. This creates problems for session-history-manipulating plugins (compaction, tool-call result pruning, etc.). Without a central mechanism, each plugin would need to wrap `agent/request` to rewrite the message list — a pattern that suffers from listener-ordering fragility, provides no durable record of what was changed, and forces repeated changes to the core `deriveMessages()` whenever a new manipulation is added. A central hub in the `session` package, with a provenance-recording mechanism and enough flexibility for future plugins to manipulate session history through a stable API, lays a solid foundation for plugin development.
+
+## Decision
+
+Add a **surface** — a derived, cached linked list of "surface nodes" (the subset of events that produce LLM messages) — maintained by `surfaceOp` markers in the event log.
+
+### Two new top-level fields on `SessionEvent`
+
+Every `SessionEvent` gains two optional fields (structural metadata, like `seq`/`time`):
+
+- **`sourceEventSeqs?: number[]`** — seq numbers of events that are provenance sources (e.g., the `assistant/chunk` seqs that built an `assistant/message`, or the surface nodes shadowed by a compaction marker). Provenance is a core design principle; without it, the replace-range operation cannot be validated on replay.
+- **`surfaceOp?: SurfaceOp`** — how this event entered the surface. Absent for non-surface events.
+
+### SurfaceOp: two operations
+
+```ts
+export type SurfaceOp =
+  | 'append'                                    // normal tail append
+  | { op: 'replace'; start: number; end: number }  // shadow [start, end] inclusive
+```
+
+1. **Append** — add a new node to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`. The loop passes `surfaceOp: 'append'` on all such appends, and `sourceEventSeqs` where applicable (e.g., `assistant/message` records its `assistant/chunk` sources; `tool/result` records its `tool/call` source).
+
+2. **Replace** — remove nodes from `start` through `end` (both inclusive) and insert a new node in their place. Both `start` and `end` must be valid surface node seqs in the current surface; `start === end` replaces a single node. The node's `sourceEventSeqs` must contain every shadowed surface node. The shadowed events remain in the log but are no longer on the surface.
+
+The both-ends-inclusive design was chosen over half-open `[start, endExclusive)` because the surface is a doubly-linked list — both ends are naturally named by node seqs, and single-node replacement (`start === end`) is a common case that reads naturally with inclusive semantics.
+
+### SurfaceManager: delta-based, not full rebuild
+
+A `SurfaceManager` class (private to `Session`) maintains the cached linked list. It tracks `_lastProcessedSeq` and processes only the **delta** (new events since the last access) rather than rescanning the entire log. Because the log is append-only, prior events never change — full rebuild is only needed after a wholesale log replacement (e.g., seeding).
+
+Why delta processing? The naive approach (a dirty flag + full rebuild on every access) would be O(N²) over a session's lifetime — every single-event append triggers a complete scan of all prior events. Delta processing is O(1) when no new events and O(new events) when new events arrive.
+
+`deriveMessages()` uses the surface when surface markers exist, falling back to the existing linear scan for sessions without markers (backward compatibility).
+
+### Persistence
+
+The new fields are serialized as top-level JSON properties. The JSONL backend requires zero changes — `JSON.stringify`/`JSON.parse` preserve everything transparently. The SQLite backend adds two nullable TEXT columns (`source_event_seqs`, `surface_op`) with an `ALTER TABLE` migration (SCHEMA_VERSION 1 → 2). The session format `version` stays at 1 — the new fields are optional and backward-compatible.
+
+### Crash recovery
+
+The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls after a crash. These closers carry `surfaceOp: 'append'` and `sourceEventSeqs` pointing to the orphaned `tool/call` event, so the rehydrated surface is valid.
+
+### Invariants
+
+The dev-mode invariants plugin validates: `sourceEventSeqs` references (non-empty, no duplicates, references earlier events, references known seqs) and `surfaceOp` (replace start ≤ end).
+
+## Consequences
+
+- **`packages/session`**: New `surface.ts` (`SurfaceManager`), new types (`SurfaceOp`, `SurfaceAppendOpts`), new fields on `SessionEvent`, modified `append()` (third optional `SurfaceAppendOpts` param), refactored `deriveMessages()` (surface path + legacy fallback), surface-aware `repair.ts`.
+- **`packages/agent-loop`**: All surface-capable appends pass surface opts. Chunk seqs are collected for `assistant/message` provenance; `tool/call` seqs are captured for `tool/result` provenance.
+- **`packages/session-persistence-sqlite`**: Schema migration v1 → v2 (two new nullable TEXT columns).
+- **`packages/invariants`**: Surface-related validation rules.
+- **`packages/session-persistence-jsonl`**: No changes required.
+- **`packages/session-persistence`**: Abstract interface unchanged.
+
+The surface is the foundation for future compaction: a compaction plugin appends a new event (e.g., `compaction/marker`, added to `SessionEventMap` via declaration merging) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed nodes. Replay preserves the compaction decision deterministically.

docs/adr/README.md

@@ -30,3 +30,4 @@ Do NOT write an ADR for: a mechanical or local choice (a variable name, a one-fi
 | [0016](0016-pnpm-over-yarn.md) | pnpm as the package manager instead of Yarn 4 | accepted |
 | [0017](0017-turn-enclosure-invariant.md) | Every session event is enclosed in a turn | accepted |
 | [0018](0018-session-persistence.md) | Session persistence as an abstract service over `SessionEvent` | accepted |
+| [0019](0019-session-surface.md) | Session surface — a linked list over the event log for LLM message derivation | accepted |