Commit Graph

993 Commits

Author SHA1 Message Date
Tianyi Cui
23fb1febcd chore: keep the type-equiv manifest in its one-line-per-entry format
The previous commit rewrote the whole file through a JSON pretty-printer,
reformatting every existing entry; restore the established compact style with
the four new entries appended to the tools.md group.
2026-07-07 21:17:52 +08:00
Tianyi Cui
0e0f3b2f19 review: acquire the structured runtime per run, not per backend
The Codex simplification concern plus the duplication comment on the spawn
apply, resolved by deletion: the backend-lifetime holds are gone, so the
runtime registers at the first structured run and disposes when the last
settles — a deployment that never passes outputSchema carries no always-on
global state, and there is no per-backend acquisition block left to extract.

The driver spec now drives an INLINE spawn-shaped provider over
startInProcessRun, which removes the spawn/fork devDependencies (the
test-only workspace cycle); plugin-level structured coverage moves to the
backends' own specs (capture through the shipped plugin, mid-run backend
unload, seeded fork capture). tools.md, the driver README, and both backend
READMEs describe the run-scoped lifetime; the module-graph regenerates
without the cycle edges.
2026-07-07 21:09:02 +08:00
Tianyi Cui
280233ba78 fix review finding: the capture commits only on the final post-execute accept
The cross-seam blocker: structured_output recorded its value in the tool
BODY, before tools/post-execute could block the call — a PostToolUse hook's
block turned the logged result into isError while readResult still returned
structured success and the continuation veto ended the turn.

Two-phase commit: the body validates and STAGES (RunState.pending); a fourth
runtime listener on tools/post-execute — prepend, so await next() returns the
composed final decision — promotes the stage to captured only on an accepted
call, and clears it on every path. A block now yields a consistent pair: the
model and log see the isError feedback, the run settles error with no
structured value, and the turn continues so the model can react. Regressions:
block denies the capture end-to-end; accept-with-replacement still commits.
2026-07-07 21:08:12 +08:00
Tianyi Cui
d1b52a063b fix review findings: own-property and plain-JSON discipline in the schema subset
Three Codex findings on json-schema.ts, one discipline:

- required-declared and every value check now use Object.hasOwn — 'in' let
  inherited names (toString) satisfy required, dodge additionalProperties:
  false, and validate a declared property against the value's prototype
  member instead of a carried one
- isObjectLike now means PLAIN JSON object (proto chain of at most one link,
  realm-agnostic): a Date annotation or a Map-as-properties no longer passes
  structurally and serializes lossily — they fail loud as subset violations
- startInProcessRun asserts BEFORE the defensive structuredClone, so a
  hostile schema fails as OutputSchemaError, never a raw DataCloneError

Also the type-equiv catalog gap: tools.md gains the structured-output subset
vocabulary (4 blocks) with matching manifest entries. The driver index also
drops the runtime internals from its public re-export (runs acquire it
internally; no external consumer remains — see the following commit).
2026-07-07 21:07:54 +08:00
Tianyi Cui
8c8189844f docs: regenerate the config catalog after the master merge
Master's generated config catalog (#188, flattened paths #191) now records
plugin Configs; the nudge removal dropped structuredNudgeRetries from both
backends, so the regenerated catalog loses those rows.
2026-07-07 09:44:18 +08:00
Tianyi Cui
08ff3e404f Merge remote-tracking branch 'origin/master' into structured-output-subagent-seam
# Conflicts:
#	packages/hooks/hooks-claude/tests/bridge.spec.ts
2026-07-07 09:41:14 +08:00
Tianyi Cui
5a91893b86 test: drain the detached SubagentStart continuation before the bridge spec ends
The markers are touched mid-script, so runPoint's continuation chain (child
exit → merge → the listener's detached .then) can still be in flight when the
marker poll resolves; a vitest worker that exits first leaves the inject
condition's short-circuit path uncounted. Observed as a CI-only 99.03%
branch-coverage flake on hooks-claude — surfaced by this branch shifting suite
timing, latent before it. Two macrotask rounds pin the path deterministically.
2026-07-07 09:37:50 +08:00
Tianyi Cui
dc5656fab3 Merge pull request #194 from deepseek-harness/worktree-snapshot-pin-request-header
Snapshot tests: pin request-header content in one scenario, tokenize the rest
2026-07-07 09:18:58 +08:00
Tianyi Cui
17f89642ae Merge branch 'master' into worktree-snapshot-pin-request-header 2026-07-07 09:16:28 +08:00
Tianyi Cui
09dbc06079 Merge pull request #193 from deepseek-harness/worktree-hooks-dispose-quiescence
fix(hooks): drain detached hook runs on bridge dispose
2026-07-07 09:15:31 +08:00
Tianyi Cui
b907c20213 review: drop the structured-output nudge; FIXME the context-global registry constraint
Two human review directives:

- No re-prompt. A structured child that finishes a turn cleanly without
  calling structured_output settles error to the parent immediately —
  readResult already carried that mapping; the nudge loop only delayed it.
  Deletes the loop, its cancellation-window guard, STRUCTURED_OUTPUT_NUDGE,
  and the structuredNudgeRetries Config on both backends.

- FIXME in the structured module doc: per-agent/per-session tool registry and
  prompt assembly would dissolve the final-assembly enforcement dance (the
  placeholder tool, the swap, the strip, the global-registration lifetime).
2026-07-07 09:14:48 +08:00
Tianyi Cui
2d3f75c77b Merge branch 'master' into worktree-hooks-dispose-quiescence 2026-07-07 09:13:00 +08:00
Tianyi Cui
25fc167165 Merge branch 'master' into worktree-snapshot-pin-request-header 2026-07-07 09:09:27 +08:00
Tianyi Cui
515d04339b Address review round 2: preserve delta insert arity; reject unpinned header-deltas
Residuals from the Codex re-review:

1. A system delta's insert was flattened to one token, so deltas differing
   only in inserted-line count compared equal. Now one {{system}} token per
   inserted line — position AND extent survive, content does not.

2. The live uniformity guard folded only request/header snapshots, so a
   mid-run header CHANGE (request/header-delta) could diverge from the pin
   invisibly. Non-pinning runs now assert zero header-delta events: a
   scenario that legitimately changes its header mid-run exists to show
   that change, so it must pin (fail-loud until it does).
2026-07-07 01:26:25 +08:00
Tianyi Cui
a0d8f33b29 Address review: structure-preserving delta scrub, live header-uniformity guard, RFC style
Codex review findings on the pinned-header change:

1. scrubRequestHeaders flattened a request/header-delta's whole
   system/tools payload to one token, so two meaningfully different deltas
   compared equal. Now the structural facts survive — keepStart/keepEnd
   line positions, added/removed/changed tool NAMES — and only the bulk
   (inserted prompt lines, schema bodies) is tokenized.

2. The one-pin design rested on an unasserted premise (all sessions
   compose the same header). Every non-pinning scenario now asserts, live,
   that each request/header its run produces equals the pinned fixture's
   header (both sides normalized against their own volatile values), so a
   session-dependent header fails loud until it gets its own pin.
   Verified the guard bites: perturbing the pinned fixture's prompt fails
   a non-pinned scenario with the intended message.

3. RFC de-slopped per docs/AGENTS.md: no PR reference, no SHOULD
   spec-speak; Decision/Verification/Consequences updated for 1 and 2.
2026-07-07 00:51:55 +08:00
Tianyi Cui
81cbc3daa4 Merge pull request #191 from deepseek-harness/catalog-flatten
docs: flatten single-file catalog dirs to docs/tool-catalog.md and docs/persistence-catalog.md
2026-07-07 00:30:36 +08:00
Tianyi Cui
0c568201c9 Merge remote-tracking branch 'origin/master' into catalog-flatten 2026-07-07 00:29:12 +08:00
Tianyi Cui
215ff9a4e6 Merge pull request #188 from deepseek-harness/config-catalog
feat: generated plugin config catalog (docs/config-catalog.md)
2026-07-07 00:27:59 +08:00
Tianyi Cui
a520965f09 fix review findings: post-capture tool calls denied; schema snapshotted at start
Two bot findings on the structured runtime:

- Terminal means terminal WITHIN the step: a model response listing
  structured_output before further tool calls executed those calls after the
  final answer was accepted (the turn-continuation veto only fires at step
  end). A third runtime listener now denies every call for a captured agent
  at the tools/pre-execute gate — dispatch skipped, isError result naming the
  contract. Calls preceding the capture in the same response are untouched.

- The output schema is structuredClone'd before the subset assertion: the
  caller keeps its reference, so asserting and attaching the original let a
  post-start() mutation drift the enforced schema away from the asserted one.
  The clone pins assertion, model-visible parameters, and validation to one
  value.
2026-07-07 00:11:55 +08:00
Tianyi Cui
d45e470192 fix: retarget the i18n pairing exclusions to the flattened catalog paths
The manifest excluded docs/tool-catalog/ and docs/persistence-catalog/ as
directories; the trailing slash is a deliberate path boundary, so the
flattened single files no longer matched and lost their generated-doc
classification — the gate would have accepted a hand-added .zh.md twin it
must reject. The entries now name the files.
2026-07-07 00:08:55 +08:00
Tianyi Cui
4e2da27201 Merge branch 'config-catalog' into catalog-flatten 2026-07-07 00:06:19 +08:00
Tianyi Cui
a4152705ef fix review finding: abort tests must prove the hook process is gone
Codex review of the draft: the abort-on-dispose tests asserted only that
dispose() resolved and no warning fired — a regression back to untracked
fire-and-forget runs would pass both. The hook script now records its
shell PID before sleeping, and after dispose the test asserts
process.kill(pid, 0) throws: the drain resolves only after the killed
run settled (the child reaped), so ESRCH-by-dispose-return is
deterministic under the correct implementation, while an alive or
unreaped process fails it. Verified the discriminator by gutting drain()
locally: exactly these two tests fail.
2026-07-07 00:02:10 +08:00
Tianyi Cui
f0ef280848 fix: identity-check name resolutions across the pasted closure; exclude the catalog from i18n pairing
Review findings on the config catalog:

The transitive-paste closure deduplicated by bare name, so two DIFFERENT
package-local declarations sharing a name would silently collapse to the
first one pasted — and a reference satisfied by the wrong declaration is a
wrong catalog, not just a thin one. Every resolution is now identity-checked
by source pointer: reaching the same declaration again is benign, while a
name that resolves to two distinct declarations, to conflicting imports, or
to a declaration in one file and an import in another is a violation — a
verbatim fence has a single flat namespace, so the fix is a rename at the
source. A spec case pins the two-declarations collision.

docs/config-catalog.md also joins the translation-pairing exclusions: it is
a generated English-only doc, and without the entry the gate would accept a
hand-added .zh.md twin it must reject.
2026-07-07 00:00:34 +08:00
Tianyi Cui
1fd9e2a9ab Pin request-header content in one snapshot scenario
Every session.jsonl fixture embedded the full composed system prompt and
complete tool-schema list in its request/header event (~8 KB on one line,
identical across the suite), so any prompt or tool-schema edit forced a
re-record or hand-edit of every fixture — see the dynamic-workflows PR for
the churn pattern this removes.

Now exactly one scenario (text-turn, flagged pinsHeader) commits and
compares that content verbatim; every other fixture stores and compares it
as {{system}}/{{tools}} tokens via the new pure scrubRequestHeaders
normalizer (applied to both compare sides and to record-mode writes, so a
re-record cannot reintroduce the content). request/header-delta payloads
are scrubbed the same way; config/reason stay verbatim — a model swap
SHOULD churn every fixture, a prompt edit should not. Replay is unaffected:
script derivation reads only assistant/chunk events.

Fixture meta-guards enforce the split: non-pinning fixtures must be fixed
points of the scrub, the pinning fixture must not be, and exactly one
scenario pins. Committed fixtures migrated through the same function.

Docs: pinned-header RFC (implemented/testing), base snapshot RFC + testing
policy + llm-replay module doc/README updated.
2026-07-06 23:41:20 +08:00
Tianyi Cui
82dfc62620 Merge branch 'config-catalog' into catalog-flatten 2026-07-06 23:38:40 +08:00
Tianyi Cui
fe815c9586 Merge remote-tracking branch 'origin/master' into config-catalog
# Conflicts:
#	docs/rfc/INDEX.md
2026-07-06 23:31:58 +08:00
Tianyi Cui
74502fa8c2 Structured output on the subagent seam: schema subset, capture runtime, spawn/fork support
Carved out of #170 per review feedback — the foundation the workflow tool
builds on, now standing alone on master:

- dsh-tools: the structured-output JSON Schema subset (StructuredOutputSchema,
  assertSupportedOutputSchema, validateStructuredValue) — rejects loud outside
  the enforced subset, listing every violation
- dsh-subagent: SubagentStartRequest.outputSchema / SubagentResult.structured
  become a real capability; the service rejects a schema'd request whose
  provider lacks it
- dsh-subagent-inprocess: the shared structured runtime — one global
  structured_output capture tool, a prepend final-assembly listener that
  strips the placeholder for plain agents and swaps in the run's own schema
  (plus the calling instruction as a trailing section) for structured
  children, an agent/turn-continuation veto once captured, and the
  capture/nudge loop in the run driver (structuredNudgeRetries, cancellation
  honored mid-nudge); lifetime refcounted by backends and live runs
- subagent-spawn / subagent-fork flip outputSchema: true

One deliberate divergence from the #170 revision: the backends do NOT add
'tools' to their plugin inject. Doing so deferred their apply past the todo
plugin, and the delegation tool mirrors provider lifecycle — so the
model-visible tool order of every existing prompt changed, invalidating every
recorded snapshot fixture. The runtime now gates its capture-tool registration
on tools availability itself (sync when live, a scoped inject fiber when the
Loader starts the backend first), keeping this PR byte-invisible to existing
transcripts: all 35 snapshot scenarios pass against master's fixtures
unchanged.
2026-07-06 23:29:08 +08:00
Tianyi Cui
9ca31ab193 fix(hooks): drain detached hook runs on bridge dispose
The emit-shaped hook points (SessionStart, SubagentStart, SubagentStop)
run fire-and-forget: no seam awaits the run chain, so disposing a bridge
could strand a live hook process and let a late continuation inject into
a disposed context. The floating continuation also made the coverage
gate racy: the only coverage of the SubagentStart continuation's
no-context branch arm rode on an un-awaited .then, and on a loaded CI
runner the fork's per-file coverage snapshot beat it — master run
28798191671 failed the 100% branch gate on hooks-claude/src/index.ts at
99.03% (uncovered line 336) with the identical tree passing the PR run
three minutes earlier.

New shared primitive createDetachedRuns() in dsh-hook-protocol: a bridge
tracks each detached run chain, passes the tracker's abort signal to
runHook, and registers drain() as its effect disposer — drain aborts
still-running hook processes (a kill via the bash seam, not a wait out
to the 10-minute default hook timeout), then resolves once every chain
has settled. fiber.dispose() resolving now means the bridge's detached
work is quiescent (docs/defensive-patterns.md).

The subagent marker test disposes the bridge as its sync point, so the
formerly racy branch arm is executed deterministically before the file's
coverage snapshot; new tests pin abort-on-dispose promptness for both
bridges and the tracker's settle/drain contract in hook-protocol.
2026-07-06 23:27:54 +08:00
Tianyi Cui
6d9a4f1258 Merge branch 'config-catalog' into catalog-flatten
# Conflicts:
#	docs/config-catalog.md
#	scripts/gen-config-catalog.ts
2026-07-06 23:16:29 +08:00
Tianyi Cui
fb3f3a69ec fix: extend the schema cross-check to nested key paths; scope the catalog framing
Review findings on the config catalog:

The schema-subset check compared only top-level z.object keys against
top-level type members, so a nested loader-accepted key (agents[].id,
agentOptions.model, capabilities.*) missing from the declared type would
pass the gate unseen. The walk now collects nested object/array
compositions as key paths and resolves each against the declared config
type — through interfaces (heritage included), aliases, literals,
intersections, unions, arrays, indexed access, Partial-style wrappers, and
type references across package-local and workspace imports (re-export
chains included). The check stays presence-only and one-directional, and
only a definite miss is a violation: a path crossing a type the walk
cannot enumerate (an external package's) is skipped, never mis-reported.
The recursion guard applies at named declarations only — a structural
first child shares its span start with its parent, so a span-keyed guard
on every node mistakes ordinary descent for a cycle and silently turns
definite misses into unknowns.

The page and RFC framing also overstated the catalog as the exact
cordis.yml-settable surface: the paste is the plugin's full declared
config type, and a field the runtime schema deliberately excludes (the ACP
bridge's test-injected stream) is a runtime-only seam its own JSDoc marks.
Both now say so.

Five spec cases pin the new behavior: nested hidden key, workspace
intersection via star re-export, Partial wrapper, indexed-access
composition, and the external-type unknown path.
2026-07-06 23:06:03 +08:00
Tianyi Cui
116e2c45a8 docs: flatten single-file catalog dirs to docs/tool-catalog.md and docs/persistence-catalog.md
A directory holding exactly one file adds a path level for nothing;
cordis-catalog/ keeps its folder because it holds two sibling pages
(events.md + services.md), matching config-catalog.md which was born flat.

docs/tool-catalog/tools.md -> docs/tool-catalog.md and
docs/persistence-catalog/log-events.md -> docs/persistence-catalog.md; their
generators' OUT paths and in-page relative links drop one directory level,
and every citation repo-wide (docs, RFCs, package READMEs, generator
headers) is updated. graph-atlas.md, config-catalog.md, and the doc graphs
regenerate with the new paths.
2026-07-06 22:26:06 +08:00
Tianyi Cui
0606cd559c Merge pull request #175 from deepseek-harness/codex/parallelize-ci
ci: parallelize GitHub workflows
2026-07-06 22:15:13 +08:00
Tianyi Cui
8458530298 Merge branch 'master' into codex/parallelize-ci 2026-07-06 22:12:44 +08:00
Wenlu Wang
654d192c47 Merge pull request #182 from deepseek-harness/codex/gate-cadence-docs
docs: clarify local gate cadence
2026-07-06 22:10:19 +08:00
Tianyi Cui
b0c7eadd23 feat: generated plugin config catalog (docs/config-catalog.md)
scripts/gen-config-catalog.ts walks every packages/<group>/<pkg> entry with
the TypeScript compiler API and emits docs/config-catalog.md: per loadable
plugin, the verbatim config declaration (JSDoc included) its apply/constructor
receives in a ts config-catalog fence, the inject requirements, resolved links
for every referenced type (package-local types pasted transitively, other
plugins' config types as intra-page anchors, LINK_MAP names to
core-data-structures, workspace types to source), and terse classification
lists for config-free plugins, abstract seams, and libraries — classification
is total, so a new package cannot go undocumented.

The walk enforces per-field JSDoc prose on every pasted declaration and
statically cross-checks the schemastery schema (z.object keys, z.intersect
composition across packages): every schema-validated key must be a declared
member of the config type. One violation existed repo-wide — the agents[].id
field in dsh-agent-loop — fixed by adding its JSDoc (which shifts the
cordis-catalog services page's source pointers; regenerated).

verify-config-catalog (--check) joins doc-sync; doc-typecheck learns the
ts config-catalog fence; gen-cordis-catalog exports its JSDoc/pointer helpers
and LINK_MAP for reuse. Negative-path spec in
packages/core/agent-core/tests/gen-config-catalog.spec.ts mirrors the
gen-cordis-catalog spec. Decision record:
docs/rfc/implemented/process/2026-07-06-generated-config-catalog.md (includes
the deliberate acceptance of README ## Config overlap).
2026-07-06 21:57:17 +08:00
Tianyi Cui
9df79534a6 Merge branch 'master' into codex/gate-cadence-docs 2026-07-06 21:55:48 +08:00
Tianyi Cui
c57c7527a5 Merge branch 'master' into codex/parallelize-ci 2026-07-06 21:54:07 +08:00
Tianyi Cui
1c999804d8 Merge pull request #184 from deepseek-harness/codex/clarify-doc-budget-guidance
Clarify doc budget guidance
2026-07-06 20:46:31 +08:00
kingwl
e48a344f6a docs: keep gate cadence guidance agent-only 2026-07-06 17:08:50 +08:00
Hypatia May
41e1955ace Clarify condensing in doc budget guidance 2026-07-06 16:31:18 +08:00
Hypatia May
72166dad03 Clarify doc budget guidance 2026-07-06 16:11:56 +08:00
kingwl
e1b85da182 docs: trim daily command wording 2026-07-06 13:45:17 +08:00
kingwl
7af8f98080 docs: clarify local gate cadence 2026-07-06 13:45:17 +08:00
Tianyi Cui
cbc9c26366 Merge pull request #179 from deepseek-harness/worktree-reconstructable-requests
Make every LLM request a pure function of the session log
2026-07-06 13:37:20 +08:00
Tianyi Cui
39f2898502 fix review findings: last stale compact README line; stale-mark the Code Mode proposal
The compact-basic README's opening line still described summarization
as routed through the agent request pipeline — corrected to the direct
one-shot llm/stream contract the rest of the package documents. The
proposed Code Mode RFC gets a premise-stale note: it names the old
full-request agent/request seam, which now shapes call config only —
re-map onto the log channels and system-prompt/assemble before
implementing from it.
2026-07-06 04:22:37 +08:00
Tianyi Cui
8bd80e5e9b fix review findings: one frozen seed through the waterfall; stale agent/request docs
Codex diff review, round 1, two (A) findings:

- The agent/request fallback resolved the RAW seed object — on later
  steps the session's cached header fold — so a delegating listener
  (await next(), mutate, return) could rewrite the fold in place and
  the change would compare as already-baseline: no delta logged, the
  persisted log unable to reconstruct the request (the dev invariant
  would fire on the divergence, but the log would still lie). One
  structuredClone'd, deep-frozen seed now serves both the listener
  chain and the fallback — in-place shaping after delegation throws —
  and Session.requestHeader() freezes its fold on update, so the leak
  class is unrepresentable from either side. Pinned by a loop-level
  delegating-mutator test.
- Doc sweep for the old contract: agent README's event row (mutate
  GenerateOptions / tool filtering → frozen config seed, replacement
  out, logged header), compact-basic's module JSDoc (summarize routed
  through agent/request → direct one-shot at llm/stream), and
  architecture.md's event-domain line (request mutation → call-config
  shaping).
2026-07-06 04:17:29 +08:00
Tianyi Cui
23ed47ba75 chore: knip learns agent-loop's e2e entry (ported from #162 with request-cache.e2e.ts) 2026-07-06 04:01:56 +08:00
Tianyi Cui
c58e5adee7 test: close the coverage gaps the gate found
surface.replaceGeneration gets its direct-access test (the getter folds
a pending delta itself, not only via a nodes read); the reconstruction
theorem drives maxTokens and stop through the config waterfall; the
compact-basic envelope drops a dead conditional — config.maxTokens is
required and validated positive, so this backend's envelope always
carries the cap (the return type's optionality exists for overriding
subclasses).
2026-07-06 03:55:30 +08:00
Tianyi Cui
c0808d5126 docs: the governing principle — every LLM request is reconstructable from the session log
The reconstructability RFC is the principle's home: model-visible ⟺
logged in both forms, the mechanism (boundary derivation + header
fold), the enforcement (write-time round-trip guard, the dev
invariant), the corollaries ranked (prefix-cache stability first), the
MiniCode lineage with the provenance arrow inverted, and the
alternatives it beat — including the stateful transmission client
whose three-design archaeology lives in PR #162.

Placements per the one-home-per-fact taxonomy: a standing-order line in
root AGENTS.md (with displacement trims to stay inside the 1,575-word
ceiling), the principle statement in architecture.md § Session Log and
its Turn Flow lines (condensed to the ratcheted 1,630 ceiling), the
request-envelope section in core-data-structures/core.md with the
LlmCallConfig paste, both review-requested FIXMEs
(FIXME(call-config-shape) beside the type, FIXME(catalog-verbs) at the
catalog's drift-gate note), cookbook rows redirected off agent/request
(tool filtering → system-prompt/assemble, plan-mode prompt → sections/
inject()), and the llm/stream JSDoc stating the frozen-request
contract. RFC index and all generated catalogs regenerated.
2026-07-06 03:49:35 +08:00
Tianyi Cui
7376346246 test: re-record snapshot goldens — every transcript gains its request/header event
One re-record after the header events landed: recorded scenarios
re-harvested against the live API; the three fs-writing fixtures are
relativized (this recording's model happened to emit absolute
file_path arguments, which only round-trip through replay when the
path is cwd-independent) and, with the two never-re-recorded authored
scenarios (error-finish, cancel), rewritten in the normalized
authored-fixture form the harness documents — each now carrying the
request/header snapshot the loop logs before its first dispatch.
Keyless replay verified green across all 35 scenarios.
2026-07-06 03:39:48 +08:00