diff --git a/docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md b/docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md index 0ecf3e0e78..9e3e406cdf 100644 --- a/docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md +++ b/docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md @@ -81,7 +81,7 @@ Resolved targets must expose at least three concepts: - An opaque `targetKey`, used for stale guards and file-state lookup. The local backend might use a realpath-like key; a remote backend might use a workspace URI or file id. Consumers must not parse or assume this is a local absolute path. - A `displayPath`, used for model/UI-facing output. It may be a local absolute path, workspace-relative path, or remote URI depending on the backend. -Read and mutation results must include an opaque file `version`. A local backend can use mtime/size or a hash-like token; a remote backend can use a revision id. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. +Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files. Both own regular-file checks, bounded line/output handling is NOT theirs — line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`), which reads through `ctx.fs` and renders the model-facing window. The provider owns UTF-8 decoding and binary/NUL rejection; it does not know about line windows or views. diff --git a/docs/rfc/implemented/feature/2026-06-24-workspace-context.md b/docs/rfc/implemented/feature/2026-06-24-workspace-context.md index 31372959cf..54b513c082 100644 --- a/docs/rfc/implemented/feature/2026-06-24-workspace-context.md +++ b/docs/rfc/implemented/feature/2026-06-24-workspace-context.md @@ -60,7 +60,7 @@ There is intentionally no watcher. Detection occurs at the next successful struc `maxBytes` is required and applies separately to a rendered baseline or one dynamic reconciliation batch; there is no implicit or unbounded render budget. Non-positive and non-finite values disable loading. When content exceeds the budget, broader files are omitted before the most-specific file is truncated. A visible `Workspace instruction budget ...` notice names omitted and truncated paths and byte counts, and output never exceeds the configured bytes. -`maxSourceBytes` is a positive per-file cap with a 1 MiB default. The loader checks reported size before reading and still consumes content through `streamText()` with a running UTF-8 byte count, so missing/stale metadata cannot force an unbounded allocation. An oversized winning candidate is unavailable rather than a reason to fall through to another same-directory name. The plugin deliberately keeps no process-wide content cache: every reconciliation observes the current bounded text, then computes the SHA-1 used by visible structured duplicate-suppression state. +`maxSourceBytes` is a positive per-file cap with a 1 MiB default. The loader checks reported size before reading and still consumes content through `streamText()` with a running UTF-8 byte count, so missing/stale metadata cannot force an unbounded allocation. An oversized winning candidate is unavailable rather than a reason to fall through to another same-directory name. The plugin deliberately keeps no process-wide cache and never retains instruction prose. It keeps only `{ path, version, digest }` per effective scope in a `WeakMap>`: a matching provider `FsVersion` plus matching effective prompt state skips the read, while a changed version triggers a bounded read and SHA-1 confirmation. SHA-1 remains the cross-provider content identity persisted in visible structured metadata; provider versions are only an in-memory invalidation fast path. Cache transitions for model-visible changes commit only when the corresponding context survives the complete tool-result policy chain. ## Alternatives considered diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 545c4c96e7..563ace0f50 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -13,7 +13,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## Behavior - **`resolve(path, opts?)`** — a relative `path` resolves against `opts.cwd` when the caller supplies one (the model-facing tools pass the calling agent's session cwd — see [the per-session cwd RFC](../../../docs/rfc/implemented/architecture/2026-07-02-fs-per-session-cwd.md)), else `config.cwd` (default `process.cwd()`); an absolute `path` ignores both. `opts.signal` is checked before and after local resolution, while a remote sibling backend may use it to abort its round-trip. The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. -- **`stat`** — returns `FsInfo` (`version` = `mtimeMs:size`, `type` of `file`/`directory`/`other`, byte `size`) or `undefined` when the target is absent. +- **`stat`** — returns `FsInfo` (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`) or `undefined` when the target is absent. - **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing. - **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`. - **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index a70f82ad17..6f3bb6d2ae 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -21,7 +21,7 @@ import { randomUUID } from 'node:crypto' import { createReadStream } from 'node:fs' import { chmod, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises' -import type { Dirent, Stats } from 'node:fs' +import type { BigIntStats, Dirent, Stats } from 'node:fs' import { basename, dirname, join, resolve } from 'node:path' import { TextDecoder } from 'node:util' import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' @@ -76,9 +76,9 @@ async function readFileAbortable(absolutePath: string, verb: 'read' | 'edit', si } } -/** Opaque version token from a stat: millisecond mtime plus byte size. */ -function versionOf(info: Stats): FsVersion { - return FsVersion(`${info.mtimeMs}:${info.size}`) +/** Opaque version token from high-resolution identity and freshness metadata. */ +function versionOf(info: BigIntStats): FsVersion { + return FsVersion(`${info.dev}:${info.ino}:${info.size}:${info.mtimeNs}:${info.ctimeNs}`) } /** @@ -176,18 +176,21 @@ export async function resolveLocalTarget(cwd: string, path: string): Promise Promise): Promise { +async function probeStats( + absolutePath: string, + readStats: (path: string) => Promise, +): Promise { try { return await readStats(absolutePath) } catch (error: unknown) { @@ -206,9 +209,14 @@ async function probeStats(absolutePath: string, readStats: (path: string) => Pro * @returns the metadata, or null when the path — or a parent segment — does not exist. */ export async function probe(absolutePath: string): Promise { - const info = await probeStats(absolutePath, stat) + const info = await probeStats(absolutePath, path => stat(path, { bigint: true })) if (!info) return null - return { version: versionOf(info), mode: info.mode & 0o777, type: pathType(info), size: info.size } + return { + version: versionOf(info), + mode: Number(info.mode & 0o777n), + type: pathType(info), + size: Number(info.size), + } } /** @@ -217,9 +225,14 @@ export async function probe(absolutePath: string): Promise { * @returns path-entry metadata, or null when the entry is absent. */ export async function probeNoFollow(absolutePath: string): Promise { - const info = await probeStats(absolutePath, lstat) + const info = await probeStats(absolutePath, path => lstat(path, { bigint: true })) if (!info) return null - return { version: versionOf(info), mode: info.mode & 0o777, type: pathLinkType(info), size: info.size } + return { + version: versionOf(info), + mode: Number(info.mode & 0o777n), + type: pathLinkType(info), + size: Number(info.size), + } } // --- Directory listing --- diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index ba3daad488..e30cfbbd74 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -7,7 +7,7 @@ */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, writeFile, unlink } from 'node:fs/promises' +import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from 'cordis' @@ -99,6 +99,20 @@ describe('stat', () => { expect(await fs.stat(await fs.resolve('missing.txt'))).toBeUndefined() }) + it('changes version after a same-size rewrite even when mtime is restored', async () => { + const path = join(dir, 'same-size.txt') + await writeFile(path, 'first') + const target = await fs.resolve(path) + const beforeInfo = await stat(path) + const beforeVersion = await versionOf(target) + + await fs.writeText(target, 'other') + await utimes(path, beforeInfo.atime, beforeInfo.mtime) + + expect((await stat(path)).size).toBe(beforeInfo.size) + expect(await versionOf(target)).not.toBe(beforeVersion) + }) + it('honors a pre-aborted signal', async () => { await expect(fs.stat(await fs.resolve('a.txt'), AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) }) @@ -327,9 +341,6 @@ describe('writeText', () => { await writeFile(join(dir, 'a.txt'), 'v1') const target = await fs.resolve('a.txt') const before = await versionOf(target) - // Change the byte length so the mtimeMs:size token provably differs (a - // same-size same-tick rewrite can collide — the documented version-token - // limitation; not what this test is about). const outcome = await fs.writeText(target, 'a much longer replacement body', { kind: 'replaceIfVersion', version: before }) expect(outcome.version).not.toBe(before) expect(outcome.version).toBe(await versionOf(target)) diff --git a/packages/fs/fs/src/types.ts b/packages/fs/fs/src/types.ts index c0bbeb677c..a46f564909 100644 --- a/packages/fs/fs/src/types.ts +++ b/packages/fs/fs/src/types.ts @@ -41,16 +41,17 @@ export function FsTargetKey(key: string): FsTargetKey { /** * Opaque file-version token — the freshness token a write/edit guards against. - * The local backend derives it from mtime+size; a remote backend might use a - * revision id. The policy layer records it for stale checks; consumers may - * display related metadata but MUST NOT interpret this token. + * The local backend derives it from high-resolution stat identity and freshness + * fields; a remote backend might use a revision id. The policy layer records it + * for stale checks; consumers may display related metadata but MUST NOT + * interpret this token. */ export type FsVersion = Branded<'FsVersion'> /** * Brand a string as an {@link FsVersion}. For backend use only — a consumer * never manufactures a version, it receives one from `stat`/write/edit outcomes. - * @param v - the backend's raw version string (the local backend derives it from mtime+size). + * @param v - the backend's raw version string. * @returns the same string, branded; no validation is performed. */ export function FsVersion(v: string): FsVersion { diff --git a/packages/prompt/workspace-context/README.md b/packages/prompt/workspace-context/README.md index 61384a4d80..ee2c75c4e0 100644 --- a/packages/prompt/workspace-context/README.md +++ b/packages/prompt/workspace-context/README.md @@ -48,7 +48,7 @@ The core `context/message` envelope is disabled for these messages because the p Model-visible text contains no hidden state markers. Each dynamic context event instead carries JSON metadata with a versioned list of `{ action, scope, path, previousPath?, digest? }` changes. On every relevant tool touch, the plugin reconstructs loaded state from its visible session events and overlays a short in-memory pending window for context present on the immutable top-level `tools/result` but not yet appended by the loop. Nested Code Mode results stage pending changes under the outer execution token for same-run duplicate suppression; the outer result rolls that state back and recommits only contexts that survived outer policy. -An unchanged path and SHA-1 content digest is not injected again. Resume works because visible metadata is persisted in the session log. Compaction re-arms a scope after its context event leaves the visible surface. A removal is a tombstone, so a later candidate reappearance is loaded again. Only changes actually rendered within the byte budget enter metadata and pending state; an omitted change remains eligible for a later touch. +An unchanged path and SHA-1 content digest is not injected again. A per-session, per-scope metadata cache stores only `{ path, version, digest }`: when the provider's opaque `FsVersion` and the effective visible state both match, reconciliation skips the content read; a changed version triggers a bounded read and SHA-1 confirmation before any model-visible update. Resume works because SHA-1 state is persisted in the session log, while an empty in-memory version cache merely causes one confirming read. Compaction re-arms a scope after its context event leaves the visible surface even when the cached version is unchanged. A removal is a tombstone, so a later candidate reappearance is loaded again. Only model-visible changes actually rendered within the byte budget enter metadata, pending state, and the version cache; an omitted change remains eligible for a later touch, while a same-digest version refresh updates metadata only. The frozen baseline itself is not rewritten mid-instance. Its initial path/digest map is retained as comparison state; the next successful filesystem touch appends any baseline replacement or removal. A resumed loop recomposes the current baseline and also reconciles still-visible dynamic scopes during prefix composition. There is no file watcher, so an on-disk change becomes visible at the next successful `read`, `write`, or `edit` touch, or when a resumed loop composes its prefix. @@ -72,7 +72,7 @@ The user-global file is always `$DSH_HOME/AGENTS.md`; the candidate list only co Rendering preserves the most specific instruction files first. It drops whole broader files before truncating the most-specific file and emits a visible `Workspace instruction budget ...` notice naming omitted and truncated paths. The rendered bytes never exceed `maxBytes`. -Instruction content is read through `streamText()` under `maxSourceBytes`, even when provider metadata omits size or a file grows after its metadata probe. An oversized file is ignored without falling through to a lower-priority same-directory candidate; during dynamic reconciliation it is temporarily unavailable rather than removed. The plugin keeps no process-wide prose cache: every reconciliation observes current content, computes its SHA-1 only after the bounded read, and relies on persisted structured metadata for duplicate suppression. +Instruction content is read through `streamText()` under `maxSourceBytes`, even when provider metadata omits size or a file grows after its metadata probe. An oversized file is ignored without falling through to a lower-priority same-directory candidate; during dynamic reconciliation it is temporarily unavailable rather than removed. The plugin keeps no process-wide cache and never caches instruction prose. Its session-local scope cache uses provider versions only as a fast invalidation signal; after invalidation, SHA-1 over the bounded read remains the cross-provider content identity stored in structured session metadata. ## Non-goals diff --git a/packages/prompt/workspace-context/src/files.ts b/packages/prompt/workspace-context/src/files.ts index 5ba24a6c57..caf84e5ea0 100644 --- a/packages/prompt/workspace-context/src/files.ts +++ b/packages/prompt/workspace-context/src/files.ts @@ -7,7 +7,7 @@ import { createReadStream } from 'node:fs' import { lstat, stat } from 'node:fs/promises' import { dirname, isAbsolute, join, relative, resolve } from 'node:path' -import type { FileSystem, FsInfo, FsPathInfo, FsTarget } from '@deepseek-ai/dsh-fs' +import type { FileSystem, FsInfo, FsPathInfo, FsTarget, FsVersion } from '@deepseek-ai/dsh-fs' import { DEFAULT_DSH_HOME_DISPLAY, defaultDshHome } from '@deepseek-ai/dsh-paths' import { resolveConfig, resolveDiscoveryConfig, type ResolvedConfig } from './config.ts' import { renderWorkspaceContext, type RenderedWorkspaceContext } from './render.ts' @@ -21,11 +21,21 @@ export interface InstructionFile { /** An instruction file whose UTF-8 content was read successfully. */ export interface LoadedInstructionFile extends InstructionFile { content: string + /** Provider freshness token when the file was loaded through `ctx.fs`. */ + version?: FsVersion } interface DiscoveredInstructionFile extends InstructionFile { target?: FsTarget size?: number + version?: FsVersion +} + +/** Provider metadata for a winning scope candidate before its content is read. */ +export interface ProbedInstructionFile extends InstructionFile { + target: FsTarget + version: FsVersion + size?: number } interface DiscoverOptions { @@ -49,7 +59,7 @@ export interface RenderedInstructionSet { /** Tri-state scope probe that distinguishes confirmed absence from provider failure. */ export type ScopeInstructionProbe = - | { kind: 'present'; file: LoadedInstructionFile } + | { kind: 'present'; file: ProbedInstructionFile } | { kind: 'absent' } | { kind: 'unavailable' } @@ -75,14 +85,14 @@ async function fsStatFile( path: string, fileSystem: FileSystem, signal?: AbortSignal, -): Promise<{ target: FsTarget; size?: number } | undefined> { +): Promise<{ target: FsTarget; size?: number; version: FsVersion } | undefined> { try { const pathInfo = await fileSystem.lstat(path, undefined, signal) if (pathInfo?.type !== 'file') return undefined const target = await fileSystem.resolve(path, signalOptions(signal)) const info = await fileSystem.stat(target, signal) if (info?.type !== 'file') return undefined - return { target, ...info.size === undefined ? {} : { size: info.size } } + return { target, version: info.version, ...info.size === undefined ? {} : { size: info.size } } } catch { signal?.throwIfAborted() // Provider absence and discovery races are both non-fatal. @@ -94,7 +104,7 @@ async function statFile( path: string, fileSystem?: FileSystem, signal?: AbortSignal, -): Promise<{ target?: FsTarget; size?: number } | undefined> { +): Promise<{ target?: FsTarget; size?: number; version?: FsVersion } | undefined> { return fileSystem === undefined ? nodeStatFile(path, signal) : fsStatFile(path, fileSystem, signal) } @@ -316,7 +326,14 @@ export async function loadBaselineInstructionSet( const loaded: LoadedInstructionFile[] = [] for (const file of discovered) { const content = await readBounded(file, config.maxSourceBytes, fileSystem, options.signal) - if (content !== undefined) loaded.push({ absolutePath: file.absolutePath, displayPath: file.displayPath, content }) + if (content !== undefined) { + loaded.push({ + absolutePath: file.absolutePath, + displayPath: file.displayPath, + content, + ...file.version === undefined ? {} : { version: file.version }, + }) + } } if (loaded.length === 0) return undefined const rendered = renderWorkspaceContext(loaded, { maxBytes: config.maxBytes }) @@ -329,11 +346,11 @@ export async function loadBaselineInstructionSet( * @param scope - `user-global`, `.`, or a project-relative directory. * @param projectRoot - project root used to resolve and display project scopes. * @param resolved - normalized plugin configuration. - * @param fileSystem - provider used for no-follow probing and reading. - * @param signal - cancellation for provider probes and streaming. - * @returns present content, confirmed absence, or temporary unavailability. + * @param fileSystem - provider used for no-follow probing. + * @param signal - cancellation for provider probes. + * @returns present metadata, confirmed absence, or temporary unavailability. */ -export async function loadScopeInstruction( +export async function probeScopeInstruction( scope: string, projectRoot: string, resolved: ResolvedConfig, @@ -364,19 +381,42 @@ export async function loadScopeInstruction( return { kind: 'unavailable' } } if (info?.type !== 'file') return { kind: 'unavailable' } - const discovered: DiscoveredInstructionFile = { + const file: ProbedInstructionFile = { absolutePath, displayPath: scope === 'user-global' ? userGlobalDisplayPath(resolved.dshHome) : relativeDisplay(projectRoot, absolutePath), target, + version: info.version, ...info.size === undefined ? {} : { size: info.size }, } - const content = await readBounded(discovered, resolved.maxSourceBytes, fileSystem, signal) - if (content === undefined) return { kind: 'unavailable' } - return { kind: 'present', file: { absolutePath, displayPath: discovered.displayPath, content } } + return { kind: 'present', file } } return { kind: 'absent' } } +/** + * Read one already-probed scope candidate under the configured source cap. + * @param file - winning provider candidate and its metadata snapshot. + * @param maxSourceBytes - maximum UTF-8 bytes accepted from the source. + * @param fileSystem - provider used for the streaming read. + * @param signal - cancellation for provider streaming. + * @returns loaded content with the probed version, or undefined when unavailable. + */ +export async function readScopeInstruction( + file: ProbedInstructionFile, + maxSourceBytes: number, + fileSystem: FileSystem, + signal?: AbortSignal, +): Promise { + const content = await readBounded(file, maxSourceBytes, fileSystem, signal) + if (content === undefined) return undefined + return { + absolutePath: file.absolutePath, + displayPath: file.displayPath, + content, + version: file.version, + } +} + function userGlobalDisplayPath(dshHome: string): string { return dshHome === resolve(defaultDshHome()) ? `${DEFAULT_DSH_HOME_DISPLAY}/AGENTS.md` : '$DSH_HOME/AGENTS.md' } diff --git a/packages/prompt/workspace-context/src/index.ts b/packages/prompt/workspace-context/src/index.ts index 9b6f8bf826..600f449c6b 100644 --- a/packages/prompt/workspace-context/src/index.ts +++ b/packages/prompt/workspace-context/src/index.ts @@ -16,13 +16,17 @@ import type { PostToolDecision, ToolExecution, ToolExecutionResult, ToolExecutio import { Config, resolveConfig, type ResolvedConfig } from './config.ts' import { loadBaselineInstructionSet } from './files.ts' import { - baselineInstructionChanges, + applyInstructionVersionUpdates, + baselineInstructionState, commitPendingInstructionContexts, dynamicInstructionContext, name, reconcileInstructionContext, + retainedInstructionVersionUpdates, rollbackPendingInstructionChanges, workspaceContextMessage, + type InstructionVersionCache, + type InstructionVersionUpdate, type PendingInstructionChange, } from './state.ts' import type { WorkspaceInstructionChange } from './render.ts' @@ -43,7 +47,13 @@ export function apply(ctx: Context, config: Config): void { const resolved: ResolvedConfig = resolveConfig(config) const pendingNestedChanges = new WeakMap>() const baselineInstructionStates = new WeakMap>() - const pendingByParent = new Map() + const instructionVersions: InstructionVersionCache = new WeakMap() + const pendingVersionUpdates = new Map() + const pendingByParent = new Map() ctx.on('agent/session-prefix', async (agent: Agent, _prefix, signal, next): Promise => { const rest = await next() @@ -61,22 +71,26 @@ export function apply(ctx: Context, config: Config): void { instructionFileCandidates: resolved.instructionFileCandidates, signal, }, fileSystem) - baselineInstructionStates.set(agent.session, baselineInstructionChanges(instructions?.included ?? [])) + const baseline = baselineInstructionState(instructions?.included ?? []) + baselineInstructionStates.set(agent.session, baseline.changes) + instructionVersions.set(agent.session, baseline.versions) const update = await reconcileInstructionContext( agent, resolved, pendingNestedChanges, baselineInstructionStates, + instructionVersions, fileSystem, { includeBaselineScopes: false, signal }, ) if (update !== undefined) { - agent.inject(update.content, { - source: update.source, - envelope: update.envelope, - meta: update.meta, + agent.inject(update.context.content, { + source: update.context.source, + envelope: update.context.envelope, + meta: update.context.meta, }) + applyInstructionVersionUpdates(agent.session, update.versionUpdates, instructionVersions) } if (instructions === undefined || instructions.rendered.text.length === 0) return rest return [workspaceContextMessage(instructions.rendered.text), ...rest] @@ -97,33 +111,41 @@ export function apply(ctx: Context, config: Config): void { if (downstream.kind === 'block') return downstream const fileSystem = ctx.get('fs') if (fileSystem === undefined) return downstream - const context = await dynamicInstructionContext( + const update = await dynamicInstructionContext( exec.agent, exec, result, resolved, pendingNestedChanges, baselineInstructionStates, + instructionVersions, fileSystem, ) - if (context === undefined) return downstream + if (update === undefined) return downstream + pendingVersionUpdates.set(exec.token, update.versionUpdates) return { kind: 'accept', ...downstream.content !== undefined ? { content: downstream.content } : {}, - additionalContexts: [context, ...downstream.additionalContexts ?? []], + additionalContexts: [update.context, ...downstream.additionalContexts ?? []], } }) ctx.on('tools/result', (exec: ToolExecution, result: ToolExecutionResult) => { + const ownVersionUpdates = pendingVersionUpdates.get(exec.token) ?? [] + pendingVersionUpdates.delete(exec.token) if (exec.parent !== undefined) { if (exec.agent === undefined) return // Child contexts participate in duplicate suppression within one composite // run, but remain provisional until the parent reaches its final policy. const changes = commitPendingInstructionContexts(exec.agent, result.additionalContexts, pendingNestedChanges) if (changes.length === 0) return + const versionUpdates = retainedInstructionVersionUpdates(ownVersionUpdates, changes) const staged = pendingByParent.get(exec.parent) - if (staged === undefined) pendingByParent.set(exec.parent, { agent: exec.agent, changes }) - else staged.changes.push(...changes) + if (staged === undefined) pendingByParent.set(exec.parent, { agent: exec.agent, changes, versionUpdates }) + else { + staged.changes.push(...changes) + staged.versionUpdates.push(...versionUpdates) + } return } @@ -135,6 +157,12 @@ export function apply(ctx: Context, config: Config): void { rollbackPendingInstructionChanges(staged.agent, staged.changes, pendingNestedChanges) } if (exec.agent === undefined) return - commitPendingInstructionContexts(exec.agent, result.additionalContexts, pendingNestedChanges) + const committed = commitPendingInstructionContexts(exec.agent, result.additionalContexts, pendingNestedChanges) + const stagedVersionUpdates = staged?.versionUpdates ?? [] + const versionUpdates = retainedInstructionVersionUpdates( + [...stagedVersionUpdates, ...ownVersionUpdates], + committed, + ) + applyInstructionVersionUpdates(exec.agent.session, versionUpdates, instructionVersions) }) } diff --git a/packages/prompt/workspace-context/src/state.ts b/packages/prompt/workspace-context/src/state.ts index 9a3e87d7f7..7db2b89986 100644 --- a/packages/prompt/workspace-context/src/state.ts +++ b/packages/prompt/workspace-context/src/state.ts @@ -6,8 +6,8 @@ import type { Agent, HookContext } from '@deepseek-ai/dsh-agent' import type { Message } from '@deepseek-ai/dsh-llm' -import type { JsonValue } from '@deepseek-ai/dsh-session' -import type { FileSystem } from '@deepseek-ai/dsh-fs' +import type { JsonValue, Session } from '@deepseek-ai/dsh-session' +import type { FileSystem, FsVersion } from '@deepseek-ai/dsh-fs' import type { ToolExecution, ToolExecutionResult } from '@deepseek-ai/dsh-tools' import type { ResolvedConfig } from './config.ts' import { instructionContentSha1 } from './digest.ts' @@ -15,7 +15,8 @@ import { ancestorChain, descendantDirsBetween, findProjectRoot, - loadScopeInstruction, + probeScopeInstruction, + readScopeInstruction, relativeDisplay, type LoadedInstructionFile, } from './files.ts' @@ -37,6 +38,28 @@ export interface PendingInstructionChange { afterSeq: number } +/** Per-scope metadata cache; instruction prose is deliberately not retained. */ +export interface InstructionVersionState { + path: string + version: FsVersion + digest: string +} + +/** Session-isolated fast-path state keyed by logical instruction scope. */ +export type InstructionVersionCache = WeakMap> + +/** A cache transition coupled to the model-visible change that authorizes it. */ +export interface InstructionVersionUpdate { + change: WorkspaceInstructionChange + state?: InstructionVersionState +} + +/** Rendered reconciliation plus cache transitions awaiting final policy. */ +export interface ReconciledInstructionContext { + context: WorkspaceHookContext + versionUpdates: InstructionVersionUpdate[] +} + /** Plugin-owned raw context with required replay metadata. */ export interface WorkspaceHookContext extends HookContext { envelope: 'raw' @@ -103,7 +126,11 @@ function workspaceInstructionChanges(meta: JsonValue | undefined): WorkspaceInst } function sameInstructionChange(a: WorkspaceInstructionChange, b: WorkspaceInstructionChange): boolean { - return a.action === b.action && a.scope === b.scope && a.path === b.path && a.digest === b.digest + return a.action === b.action + && a.scope === b.scope + && a.path === b.path + && a.previousPath === b.previousPath + && a.digest === b.digest } function visibleInstructionChanges( @@ -128,20 +155,72 @@ function visibleInstructionChanges( } /** - * Convert retained baseline files into scope/path/digest comparison state. + * Convert retained baseline files into comparison and metadata-cache state. * @param files - baseline files that survived rendering. - * @returns latest baseline state keyed by logical scope. + * @returns latest baseline changes and provider versions keyed by logical scope. */ -export function baselineInstructionChanges(files: LoadedInstructionFile[]): Map { - return new Map(files.map((file) => { +export function baselineInstructionState(files: LoadedInstructionFile[]): { + changes: Map + versions: Map +} { + const changes = new Map() + const versions = new Map() + for (const file of files) { + const digest = instructionContentSha1(file.content) const change: WorkspaceInstructionChange = { action: 'set', scope: scopeForDisplayPath(file.displayPath), path: file.displayPath, - digest: instructionContentSha1(file.content), + digest, } - return [change.scope, change] - })) + changes.set(change.scope, change) + if (file.version !== undefined) { + versions.set(change.scope, { path: file.displayPath, version: file.version, digest }) + } + } + return { changes, versions } +} + +function versionStatesFor(session: Session, cache: InstructionVersionCache): Map { + let states = cache.get(session) + if (states === undefined) { + states = new Map() + cache.set(session, states) + } + return states +} + +/** + * Keep only cache updates whose model-visible changes survived final policy. + * @param updates - proposed updates from one or more reconciliations. + * @param committedChanges - transitions retained on the authoritative result. + * @returns updates authorized by an exact retained transition. + */ +export function retainedInstructionVersionUpdates( + updates: readonly InstructionVersionUpdate[], + committedChanges: readonly WorkspaceInstructionChange[], +): InstructionVersionUpdate[] { + return updates.filter(update => committedChanges.some(change => sameInstructionChange(update.change, change))) +} + +/** + * Apply authorized metadata-cache transitions without retaining instruction prose. + * @param session - owning session. + * @param updates - ordered set/delete transitions. + * @param cache - session-isolated metadata cache. + */ +export function applyInstructionVersionUpdates( + session: Session, + updates: readonly InstructionVersionUpdate[], + cache: InstructionVersionCache, +): void { + if (updates.length === 0) return + const states = versionStatesFor(session, cache) + for (const update of updates) { + if (update.state === undefined) states.delete(update.change.scope) + else states.set(update.change.scope, update.state) + } + if (states.size === 0) cache.delete(session) } function pendingChangesFor( @@ -217,18 +296,20 @@ function relativeScope(projectRoot: string, dir: string): string { * @param resolved - normalized plugin configuration. * @param pendingBySession - short pending window before returned context is logged. * @param baselineBySession - frozen baseline comparison state per session. + * @param versionCache - per-session scope metadata used to skip unchanged reads. * @param fileSystem - provider used for current file probes. * @param options - touched path and whether baseline scopes should be checked. - * @returns a structured context update, or undefined when state is unchanged/unavailable. + * @returns rendered context plus deferred cache updates, or undefined when unchanged/unavailable. */ export async function reconcileInstructionContext( agent: Agent, resolved: ResolvedConfig, pendingBySession: WeakMap>, baselineBySession: WeakMap>, + versionCache: InstructionVersionCache, fileSystem: FileSystem, options: { touchedPath?: string; includeBaselineScopes: boolean; signal?: AbortSignal }, -): Promise { +): Promise { const session = agent.session const pending = pendingChangesFor(session, pendingBySession) const visible = visibleInstructionChanges(agent, pending) @@ -247,57 +328,74 @@ export async function reconcileInstructionContext( for (const dir of descendantDirsBetween(cwd, options.touchedPath)) scopes.add(relativeScope(projectRoot, dir)) } - const current = new Map() - const unavailable = new Set() + const versions = versionStatesFor(session, versionCache) const seenAbsolutePaths = new Set() - for (const scope of scopes) { - const probe = await loadScopeInstruction(scope, projectRoot, resolved, fileSystem, options.signal) - if (probe.kind === 'unavailable') { - unavailable.add(scope) - continue - } - if (probe.kind === 'absent') continue - const { file } = probe - if (seenAbsolutePaths.has(file.absolutePath)) continue - seenAbsolutePaths.add(file.absolutePath) - current.set(scope, file) - } - const items: ChangeRenderItem[] = [] + const versionUpdates: InstructionVersionUpdate[] = [] for (const scope of scopes) { - if (unavailable.has(scope)) continue const previous = effective.get(scope) - const file = current.get(scope) - if (file === undefined) { - if (previous !== undefined && previous.action !== 'remove') { - items.push({ - change: { action: 'remove', scope, path: previous.path }, - file: { absolutePath: `removed:${scope}`, displayPath: previous.path, content: '' }, - }) + const probe = await probeScopeInstruction(scope, projectRoot, resolved, fileSystem, options.signal) + if (probe.kind === 'unavailable') continue + if (probe.kind === 'absent') { + if (previous === undefined || previous.action === 'remove') { + versions.delete(scope) + continue } + const change: WorkspaceInstructionChange = { action: 'remove', scope, path: previous.path } + items.push({ + change, + file: { absolutePath: `removed:${scope}`, displayPath: previous.path, content: '' }, + }) + versionUpdates.push({ change }) continue } + const { file: probedFile } = probe + if (seenAbsolutePaths.has(probedFile.absolutePath)) continue + seenAbsolutePaths.add(probedFile.absolutePath) + const cached = versions.get(scope) + if ( + cached !== undefined + && cached.path === probedFile.displayPath + && cached.version === probedFile.version + && previous !== undefined + && previous.action !== 'remove' + && previous.path === cached.path + && previous.digest === cached.digest + ) continue + + const file = await readScopeInstruction(probedFile, resolved.maxSourceBytes, fileSystem, options.signal) + if (file === undefined) continue const currentDigest = instructionContentSha1(file.content) - if (previous !== undefined && previous.action !== 'remove' && previous.path === file.displayPath && previous.digest === currentDigest) continue + const nextVersion: InstructionVersionState = { + path: file.displayPath, + version: probedFile.version, + digest: currentDigest, + } + if (previous !== undefined && previous.action !== 'remove' && previous.path === file.displayPath && previous.digest === currentDigest) { + versions.set(scope, nextVersion) + continue + } const action = previous === undefined || previous.action === 'remove' ? 'set' : 'replace' const previousPath = action === 'replace' && previous !== undefined && previous.path !== file.displayPath ? previous.path : undefined - items.push({ - change: { - action, - scope, - path: file.displayPath, - ...previousPath === undefined ? {} : { previousPath }, - digest: currentDigest, - }, - file, - }) + const change: WorkspaceInstructionChange = { + action, + scope, + path: file.displayPath, + ...previousPath === undefined ? {} : { previousPath }, + digest: currentDigest, + } + items.push({ change, file }) + versionUpdates.push({ change, state: nextVersion }) } if (items.length === 0) return undefined const rendered = renderInstructionChanges(items, resolved.maxBytes) if (rendered.text.length === 0 || rendered.changes.length === 0) return undefined - return workspaceContextHook(rendered.text, rendered.changes) + return { + context: workspaceContextHook(rendered.text, rendered.changes), + versionUpdates: retainedInstructionVersionUpdates(versionUpdates, rendered.changes), + } } /** @@ -308,8 +406,9 @@ export async function reconcileInstructionContext( * @param resolved - normalized plugin configuration. * @param pendingNestedChanges - per-session pending transition maps. * @param baselineInstructionStates - retained baseline comparison state. + * @param versionCache - per-session scope metadata used to skip unchanged reads. * @param fileSystem - provider used for current file probes. - * @returns a structured context update, or undefined for irrelevant/failed/unchanged calls. + * @returns rendered context plus deferred cache updates, or undefined for irrelevant/failed/unchanged calls. */ export async function dynamicInstructionContext( agent: Agent | undefined, @@ -318,13 +417,14 @@ export async function dynamicInstructionContext( resolved: ResolvedConfig, pendingNestedChanges: WeakMap>, baselineInstructionStates: WeakMap>, + versionCache: InstructionVersionCache, fileSystem: FileSystem, -): Promise { +): Promise { if (agent === undefined || result.isError) return undefined const touchedPath = filePathFromExecution(exec) if (touchedPath === undefined) return undefined return reconcileInstructionContext( - agent, resolved, pendingNestedChanges, baselineInstructionStates, fileSystem, + agent, resolved, pendingNestedChanges, baselineInstructionStates, versionCache, fileSystem, { touchedPath, includeBaselineScopes: baselineInstructionStates.has(agent.session), diff --git a/packages/prompt/workspace-context/tests/workspace-context.spec.ts b/packages/prompt/workspace-context/tests/workspace-context.spec.ts index 321da53b84..d57b6c4c51 100644 --- a/packages/prompt/workspace-context/tests/workspace-context.spec.ts +++ b/packages/prompt/workspace-context/tests/workspace-context.spec.ts @@ -31,6 +31,7 @@ import { renderWorkspaceContext, } from '@deepseek-ai/dsh-workspace-context' import { + baselineInstructionState, commitPendingInstructionContexts, rollbackPendingInstructionChanges, type PendingInstructionChange, @@ -46,7 +47,7 @@ async function write(path: string, content: string): Promise { } class RecordingFileSystem extends FileSystem { - entries = new Map() + entries = new Map() lstatTypes = new Map() throwOnStat = new Set() omitSizes = new Set() @@ -68,7 +69,7 @@ class RecordingFileSystem extends FileSystem { const entry = this.entries.get(target.targetKey) if (entry === undefined) return undefined const info: FsInfo = { - version: FsVersion(`v:${target.targetKey}`), + version: entry.version ?? FsVersion(`v:${target.targetKey}:${entry.type}:${entry.content ?? ''}`), type: entry.type, } if (entry.content !== undefined && !this.omitSizes.has(target.targetKey)) info.size = Buffer.byteLength(entry.content, 'utf8') @@ -1062,7 +1063,7 @@ describe('workspace context request injection', () => { expect(derivedText(agent)).toContain('ctx.fs rule') expect(derivedText(agent)).not.toContain('node fs rule') - expect(fs.readTargets).toEqual([join(root, 'AGENTS.md'), join(root, 'AGENTS.md')]) + expect(fs.readTargets).toEqual([join(root, 'AGENTS.md')]) } finally { await rm(root, { recursive: true, force: true }) await rm(home, { recursive: true, force: true }) @@ -1084,7 +1085,7 @@ describe('workspace context request injection', () => { await composeBaselinePrefix(ctx, agent) expect(derivedText(agent)).toContain('provider-only rule') - expect(fs.readTargets).toEqual([join(root, 'AGENTS.md'), join(root, 'AGENTS.md')]) + expect(fs.readTargets).toEqual([join(root, 'AGENTS.md')]) } finally { await ctx.fiber.dispose() await rm(dirname(root), { recursive: true, force: true }) @@ -1490,6 +1491,22 @@ describe('workspace context request injection', () => { }) describe('dynamic nested workspace context injection', () => { + it('builds persisted digest state without inventing a provider version', () => { + const state = baselineInstructionState([{ + absolutePath: '/repo/AGENTS.md', + displayPath: 'AGENTS.md', + content: 'root rule', + }]) + + const change = state.changes.get('.') + expect(change).toMatchObject({ + action: 'set', + path: 'AGENTS.md', + }) + expect(change?.digest).toMatch(/^[a-f0-9]{40}$/) + expect(state.versions).toEqual(new Map()) + }) + it('propagates the tool execution signal into dynamic filesystem reconciliation', async () => { const root = join(await tempRepo(), 'virtual-repo') const home = join(await tempRepo(), 'virtual-home') @@ -1647,6 +1664,113 @@ describe('dynamic nested workspace context injection', () => { } }) + it('skips instruction content reads while provider version and effective state are unchanged', async () => { + const root = join(await tempRepo(), 'virtual-repo') + const home = join(await tempRepo(), 'virtual-home') + const ctx = new Context() + try { + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(RecordingFileSystem) + const fs = ctx.fs as RecordingFileSystem + const instructionPath = join(root, 'pkg/AGENTS.md') + fs.entries.set(join(root, '.git'), { type: 'directory' }) + fs.entries.set(instructionPath, { type: 'file', content: 'nested package rule' }) + fs.entries.set(join(root, 'pkg/file.txt'), { type: 'file', content: 'hello' }) + await ctx.plugin(ToolFs) + await ctx.plugin(workspaceContext, { dshHome: home, maxBytes: 65536 }) + const agent = stubAgent(root) + + const first = await ctx.tools.execute({ + callId: CallId('read-before-version-fast-path'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent, + }) + appendAdditionalContexts(agent, first) + const second = await ctx.tools.execute({ + callId: CallId('read-with-version-fast-path'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent, + }) + + expect(first.additionalContexts).toBeDefined() + expect(second.additionalContexts).toBeUndefined() + expect(fs.readTargets.filter(path => path === instructionPath)).toHaveLength(1) + } finally { + await ctx.fiber.dispose() + await rm(dirname(root), { recursive: true, force: true }) + await rm(dirname(home), { recursive: true, force: true }) + } + }) + + it('re-reads a changed provider version, then refreshes metadata when SHA-1 is unchanged', async () => { + const root = join(await tempRepo(), 'virtual-repo') + const home = join(await tempRepo(), 'virtual-home') + const ctx = new Context() + try { + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(RecordingFileSystem) + const fs = ctx.fs as RecordingFileSystem + const instructionPath = join(root, 'pkg/AGENTS.md') + fs.entries.set(join(root, '.git'), { type: 'directory' }) + fs.entries.set(instructionPath, { type: 'file', content: 'same package rule', version: FsVersion('revision-1') }) + fs.entries.set(join(root, 'pkg/file.txt'), { type: 'file', content: 'hello' }) + await ctx.plugin(ToolFs) + await ctx.plugin(workspaceContext, { dshHome: home, maxBytes: 65536 }) + const agent = stubAgent(root) + + const first = await ctx.tools.execute({ + callId: CallId('read-before-same-digest-version-change'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent, + }) + appendAdditionalContexts(agent, first) + fs.entries.set(instructionPath, { type: 'file', content: 'same package rule', version: FsVersion('revision-2') }) + const afterVersionChange = await ctx.tools.execute({ + callId: CallId('read-after-same-digest-version-change'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent, + }) + const afterRefresh = await ctx.tools.execute({ + callId: CallId('read-after-version-cache-refresh'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent, + }) + + expect(afterVersionChange.additionalContexts).toBeUndefined() + expect(afterRefresh.additionalContexts).toBeUndefined() + expect(fs.readTargets.filter(path => path === instructionPath)).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await rm(dirname(root), { recursive: true, force: true }) + await rm(dirname(home), { recursive: true, force: true }) + } + }) + + it('isolates instruction version caches between sessions that touch the same scope', async () => { + const root = join(await tempRepo(), 'virtual-repo') + const home = join(await tempRepo(), 'virtual-home') + const ctx = new Context() + try { + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(RecordingFileSystem) + const fs = ctx.fs as RecordingFileSystem + const instructionPath = join(root, 'pkg/AGENTS.md') + fs.entries.set(join(root, '.git'), { type: 'directory' }) + fs.entries.set(instructionPath, { type: 'file', content: 'shared path, separate sessions' }) + fs.entries.set(join(root, 'pkg/file.txt'), { type: 'file', content: 'hello' }) + await ctx.plugin(ToolFs) + await ctx.plugin(workspaceContext, { dshHome: home, maxBytes: 65536 }) + + const first = await ctx.tools.execute({ + callId: CallId('read-from-first-session'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent: stubAgent(root), + }) + const second = await ctx.tools.execute({ + callId: CallId('read-from-second-session'), name: 'read', arguments: { file_path: 'pkg/file.txt' }, agent: stubAgent(root), + }) + + expect(first.additionalContexts).toBeDefined() + expect(second.additionalContexts).toBeDefined() + expect(fs.readTargets.filter(path => path === instructionPath)).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await rm(dirname(root), { recursive: true, force: true }) + await rm(dirname(home), { recursive: true, force: true }) + } + }) + it('replaces previously loaded instructions when the same file content changes', async () => { const root = await tempRepo() const home = await tempRepo()