Merge remote-tracking branch 'origin/master' into feat/read-image-context
# Conflicts: # docs/config-catalog.i18n.yaml # docs/config-catalog.md # docs/config-catalog.zh.md # packages/support/llm-replay/src/index.ts # packages/support/llm-replay/tests/llm-replay.spec.ts
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/context/workspace-context/README.md
|
||||
README.md: 7ab21bbf8c72f8424bc8d4fdad9153c7ed8bb7e9
|
||||
README.zh.md: eb807bfc0611854d54eda3ff26c97c6af51da529
|
||||
README.md: 1ae47bef728a81c61f0db637872c93321a7bc687
|
||||
README.zh.md: 8087412057b3e5924764df179ae293db5699f9ae
|
||||
|
||||
@@ -50,7 +50,7 @@ The plugin owns the complete `<system-reminder>` framing, and every injected `us
|
||||
|
||||
Model-visible text contains no hidden state markers. Each baseline or dynamic context event instead carries a typed `workspace-instructions` source with a list of `{ action, scope, path, digest? }` changes; a complete baseline also carries `baseline: true` and a `baselineIdentity` derived from normalized discovery, precedence, project-root, and budget configuration. A matching durable `user/message` confirms a queued baseline and its candidate versions. An entering pre-step waits for every queued projection, folds newly composed context into its final batch immediately after the claimed messages, and removes the pending inbox copy; rejection keeps the current context queued. If a listener rewrites away a claimed workspace message without entering its replacement, a later boundary recomposes the current context. Nested results aggregate successful file touches under their parent execution token, including when a later composite result is blocked; the top-level result transfers those touches either to the currently open session step or directly to the per-agent projection queue. A `step/end` releases its staged touches only after that boundary is in durable history, and serialized projections reconcile against visible session events plus the current inbox before replacing the single pending workspace context.
|
||||
|
||||
An unchanged path and SHA-1 content digest is not injected again. A per-session, per-scope provider cache stores only `{ path, version, digest, trimmedDigest }`: when the provider's opaque `FsVersion` and the effective visible state both match, reconciliation skips the content read; a changed version triggers a bounded read and SHA-1 confirmation before any model-visible update. The `trimmedDigest` — SHA-1 over the whitespace-trimmed content — is the per-directory duplicate key, so an unchanged file can still be removed when an earlier candidate converges on its content. Resume works because SHA-1 state is persisted in the typed source, while an empty in-memory version cache merely causes one confirming read. Compaction re-arms a scope after its context event leaves the visible surface even when the cached version is unchanged. A removal is a tombstone, so a later candidate reappearance is loaded again. Only model-visible changes actually rendered within the byte budget enter the source, pending state, and version cache; an omitted change remains eligible for a later touch, while a same-digest version refresh updates only the provider cache.
|
||||
An unchanged path and SHA-1 content digest is not injected again. A per-session, per-scope provider cache stores only `{ path, version, digest, trimmedDigest }`: when the provider's opaque `FsVersion` and the effective visible state both match, reconciliation skips the content read; a changed version triggers a bounded read and SHA-1 confirmation before any model-visible update. The `trimmedDigest` — SHA-1 over the whitespace-trimmed content — is the per-directory duplicate key, so an unchanged file can still be removed when an earlier candidate converges on its content. Resume works because SHA-1 state is persisted in the typed source, while an empty in-memory version cache merely causes one confirming read. Compaction re-arms a scope after its context event leaves the visible surface even when the cached version is unchanged. A removal is a tombstone, so a later candidate reappearance is loaded again. A model-visible change enters the source, pending state, and version cache only when its file-specific section retains at least one content byte, or when its original content is genuinely empty. Partial truncation records the complete-content digest once any content byte survives; truncation to zero remains eligible for a later touch, while a same-digest version refresh updates only the provider cache. A baseline may still publish its budget diagnostic with an empty change list. A dynamic batch with no committed change is not injected at all, and a later touch retries it.
|
||||
|
||||
The initial baseline event itself is not rewritten. Its typed changes remain authoritative only while that event is in the visible session surface. When compaction shadows the event, the next entering pre-step composes the current baseline and records it in the same request; a successful filesystem touch can instead re-add an unchanged baseline scope or append its replacement or removal. The in-memory scope marker and provider-version cache only select and accelerate probes. At the first pre-step after resume or hot remount, a compatible visible baseline is retained and compared with the files retained by the current complete rendering. Unchanged and budget-omitted files append nothing; offline additions, edits, removals, and files leaving the retained budget set append `set`, `replace`, or `remove` transitions. An incompatible visible baseline is superseded by one complete current baseline, including an explicit empty baseline when no candidate remains. There is no file watcher, so an on-disk change becomes visible at the next successful `read`, `write`, or `edit` touch, when a resumed session reconciles its baseline, or when an entering pre-step restores a shadowed baseline.
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ These instructions apply to work under `packages/app`. Use them as guidance when
|
||||
|
||||
模型可见文本不含隐藏状态标记。每个基线或动态上下文事件改为携带带类型的 `workspace-instructions` 来源,其中包含 `{ action, scope, path, digest? }` 变更列表;完整基线还会携带 `baseline: true`,以及从规范化的发现、优先级、项目根目录和预算配置派生的 `baselineIdentity`。匹配的持久 `user/message` 会确认已排队基线及其候选版本。进入步骤的 pre-step 会等待所有已排队投影完成,再把新组合的上下文折入最终批次,位置紧随已领取的消息,并移除 inbox 中仍待处理的副本;若被拒绝,当前上下文则继续排队。若监听器改写掉已领取的 workspace 消息,又没有让替代消息进入,后续边界会重新组合当前上下文。即使后续复合结果被拦截,成功的嵌套文件 touch 也会聚合到父级执行 token 下;顶层结果会将这些 touch 交给当前打开的会话步骤,或直接交给逐 agent 投影队列。`step/end` 只会在自身边界进入持久历史后释放其暂存的 touch;串行投影会根据可见会话事件和当前 inbox 协调状态,再替换唯一一条待处理工作区上下文。
|
||||
|
||||
路径与 SHA-1 内容 digest 都未变时,不会重复注入。每会话、每 scope 提供方 cache 只存储 `{ path, version, digest, trimmedDigest }`:当提供方的不透明 `FsVersion` 与有效可见状态都匹配时,对账会跳过内容读取;版本改变会在任何模型可见更新之前触发有界读取与 SHA-1 确认。`trimmedDigest` 是针对去除空白后内容的 SHA-1,也是每目录重复 key,因此较早候选文件与某个未更改文件的内容收敛后,后者仍可被移除。恢复可行,因为 SHA-1 状态持久化在带类型的来源中,而空的内存版本 cache 只会导致一次确认读取。压缩(compaction)会在 scope 的上下文事件离开可见表层后重新启用它,即使缓存版本未变。移除是 tombstone,因此候选文件之后重新出现时会重新加载。只有在字节预算内实际渲染的模型可见变更才会进入来源、pending 状态和版本 cache;已省略变更仍可在后续 touch 处理,而相同 digest 的版本刷新只更新提供方 cache。
|
||||
路径与 SHA-1 内容 digest 都未变时,不会重复注入。每会话、每 scope 提供方 cache 只存储 `{ path, version, digest, trimmedDigest }`:当提供方的不透明 `FsVersion` 与有效可见状态都匹配时,对账会跳过内容读取;版本改变会在任何模型可见更新之前触发有界读取与 SHA-1 确认。`trimmedDigest` 是针对去除空白后内容的 SHA-1,也是每目录重复 key,因此较早候选文件与某个未更改文件的内容收敛后,后者仍可被移除。恢复可行,因为 SHA-1 状态持久化在带类型的来源中,而空的内存版本 cache 只会导致一次确认读取。压缩(compaction)会在 scope 的上下文事件离开可见表层后重新启用它,即使缓存版本未变。移除是 tombstone,因此候选文件之后重新出现时会重新加载。模型可见变更只有在对应文件专属段落保留至少一个内容字节,或原始内容确实为空时,才会进入来源、pending 状态和版本 cache。只要任一内容字节保留下来,部分截断就会记录完整内容的 digest;截断到零字节则仍可在后续 touch 处理,而相同 digest 的版本刷新只更新提供方 cache。基线即使带空变更列表,仍可发布字节预算诊断。动态批次若没有可提交变更,则完全不注入,并在后续 touch 时重试。
|
||||
|
||||
初始基线事件自身不会被改写。其带类型的变更仅在该事件仍位于可见会话表层时才是权威状态。当压缩遮蔽该事件时,下一次进入步骤的 pre-step 会组合当前基线,并在同一请求中记录它;也可以改由一次成功的文件系统 touch 重新添加未变的基线 scope,或追加其替换或移除。内存中的 scope 标记和提供方版本 cache 只负责选择探测对象并加速探测。恢复或插件热重挂后的第一次 pre-step 会保留兼容的可见基线,并将它与当前完整渲染所保留的文件进行比较。未变化和被预算省略的文件不追加任何内容;agent 离线期间新增、编辑、移除或不再属于预算保留集的文件会追加 `set`、`replace` 或 `remove` 转换。不兼容的可见基线会被一条完整的当前基线取代;如果没有候选文件,这条当前基线会是显式空基线。没有文件 watcher,因此磁盘变更会在下一次成功 `read`、`write` 或 `edit` touch 时可见,也会在恢复后的会话对账其基线时,或进入步骤的 pre-step 恢复被遮蔽的基线时可见。
|
||||
|
||||
|
||||
@@ -12,7 +12,13 @@ import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import { dshHomeDisplay } from '@deepseek-ai/dsh-paths'
|
||||
import { resolveConfig, resolveDiscoveryConfig, type ResolvedConfig } from './config.ts'
|
||||
import { trimmedInstructionDigest } from './digest.ts'
|
||||
import { decodeScopeKey, renderWorkspaceContext, USER_GLOBAL_DIRECTORY, USER_GLOBAL_FILE, type RenderedWorkspaceContext } from './render.ts'
|
||||
import {
|
||||
decodeScopeKey,
|
||||
renderWorkspaceInstructionSet,
|
||||
type RenderedWorkspaceContext,
|
||||
USER_GLOBAL_DIRECTORY,
|
||||
USER_GLOBAL_FILE,
|
||||
} from './render.ts'
|
||||
|
||||
/** An instruction candidate identified by absolute and model-facing paths. */
|
||||
export interface InstructionFile {
|
||||
@@ -64,7 +70,6 @@ export interface RenderedInstructionSet {
|
||||
/** Candidates retained by content deduplication and byte budgeting. */
|
||||
included: LoadedInstructionFile[]
|
||||
}
|
||||
|
||||
/** Tri-state scope probe that distinguishes confirmed absence from provider failure. */
|
||||
export type ScopeInstructionProbe =
|
||||
| { kind: 'present'; file: ProbedInstructionFile }
|
||||
@@ -420,26 +425,26 @@ export async function loadBaselineInstructionSet(
|
||||
const deduped = dedupInstructionFilesByDirectory(loaded)
|
||||
if (deduped.length === 0) {
|
||||
if (options.replacePreviousBaseline !== true) return undefined
|
||||
const { rendered, included } = renderWorkspaceInstructionSet([], {
|
||||
maxBytes: config.maxBytes,
|
||||
replacePreviousBaseline: true,
|
||||
})
|
||||
return {
|
||||
rendered: renderWorkspaceContext([], {
|
||||
maxBytes: config.maxBytes,
|
||||
replacePreviousBaseline: true,
|
||||
}),
|
||||
rendered,
|
||||
observed: [],
|
||||
included: [],
|
||||
included,
|
||||
}
|
||||
}
|
||||
const rendered = renderWorkspaceContext(deduped, {
|
||||
const { rendered, included } = renderWorkspaceInstructionSet(deduped, {
|
||||
maxBytes: config.maxBytes,
|
||||
...options.replacePreviousBaseline === undefined
|
||||
? {}
|
||||
: { replacePreviousBaseline: options.replacePreviousBaseline },
|
||||
})
|
||||
const omitted = new Set(rendered.omitted.map(file => file.absolutePath))
|
||||
return {
|
||||
rendered,
|
||||
observed: loaded,
|
||||
included: deduped.filter(file => !omitted.has(file.absolutePath)),
|
||||
included,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -32,6 +32,17 @@ export interface RenderedWorkspaceContext {
|
||||
truncated: TruncatedInstruction[]
|
||||
}
|
||||
|
||||
interface RenderedInstructionContext extends RenderedWorkspaceContext {
|
||||
/**
|
||||
* Original files semantically represented by rendered section text. This is
|
||||
* not the complement of `omitted`: a truncated file may be represented here
|
||||
* and in `truncated`, while a notice-only file appears in neither. A genuinely
|
||||
* empty file counts when its heading survives because that heading conveys
|
||||
* that the instruction exists and has no content.
|
||||
*/
|
||||
represented: LoadedInstructionFile[]
|
||||
}
|
||||
|
||||
/** Structured dynamic state persisted outside model-visible prompt prose. */
|
||||
export interface WorkspaceInstructionChange {
|
||||
action: 'set' | 'replace' | 'remove'
|
||||
@@ -56,11 +67,15 @@ function byteLength(value: string): number {
|
||||
}
|
||||
|
||||
function truncateUtf8(value: string, maxBytes: number): string {
|
||||
let truncated = Buffer.from(value, 'utf8').subarray(0, Math.max(0, maxBytes)).toString('utf8')
|
||||
while (byteLength(truncated) > maxBytes) {
|
||||
truncated = truncated.slice(0, -1)
|
||||
const bytes = Buffer.from(value, 'utf8')
|
||||
if (bytes.length <= maxBytes) return value
|
||||
let end = Math.max(0, Math.trunc(maxBytes))
|
||||
// If the first excluded byte is a UTF-8 continuation byte, the budget cut
|
||||
// through that code point. Back up to its lead byte and exclude it too.
|
||||
while (end > 0 && (bytes.readUInt8(end) & 0xc0) === 0x80) {
|
||||
end -= 1
|
||||
}
|
||||
return truncated
|
||||
return bytes.subarray(0, end).toString('utf8')
|
||||
}
|
||||
|
||||
function escapeInstructionFrameBody(body: string): string {
|
||||
@@ -143,6 +158,16 @@ function additionalSectionText(file: LoadedInstructionFile): string {
|
||||
|
||||
const BASELINE_RENDER_STYLE: RenderStyle = { intro: WORKSPACE_CONTEXT_INTRO, section: sectionText }
|
||||
|
||||
function baselineRenderStyle(files: LoadedInstructionFile[], replacePreviousBaseline: boolean | undefined): RenderStyle {
|
||||
if (replacePreviousBaseline !== true) return BASELINE_RENDER_STYLE
|
||||
return {
|
||||
...BASELINE_RENDER_STYLE,
|
||||
intro: files.length === 0
|
||||
? EMPTY_REPLACEMENT_WORKSPACE_CONTEXT_INTRO
|
||||
: REPLACEMENT_WORKSPACE_CONTEXT_INTRO,
|
||||
}
|
||||
}
|
||||
|
||||
function changedSectionText(item: ChangeRenderItem): string {
|
||||
const { change, file } = item
|
||||
if (change.action === 'set') return additionalSectionText(file)
|
||||
@@ -178,13 +203,12 @@ export function renderInstructionChanges(
|
||||
},
|
||||
}
|
||||
const rendered = renderInstructionContext(items.map(item => item.file), maxBytes, style)
|
||||
const omitted = new Set(rendered.omitted.map(file => file.absolutePath))
|
||||
const represented = new Set(rendered.represented.map(file => file.absolutePath))
|
||||
return {
|
||||
text: rendered.text,
|
||||
// TODO(rendered-change-proof): retain a transition only when its semantic
|
||||
// notice survived rendering; a tiny compact budget can currently return
|
||||
// unrelated notice text while still committing the full state transition.
|
||||
changes: items.filter(item => !omitted.has(item.file.absolutePath)).map(item => item.change),
|
||||
changes: items
|
||||
.filter(item => represented.has(item.file.absolutePath))
|
||||
.map(item => item.change),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -252,47 +276,75 @@ function renderInstructionContext(
|
||||
files: LoadedInstructionFile[],
|
||||
maxBytes: number,
|
||||
style: RenderStyle,
|
||||
): RenderedWorkspaceContext {
|
||||
if (maxBytes <= 0 || !Number.isFinite(maxBytes)) return { text: '', omitted: files, truncated: [] }
|
||||
): RenderedInstructionContext {
|
||||
if (maxBytes <= 0 || !Number.isFinite(maxBytes)) {
|
||||
return { text: '', omitted: files, truncated: [], represented: [] }
|
||||
}
|
||||
|
||||
const fullText = buildInstructionText(files, maxBytes, [], [], style)
|
||||
if (byteLength(fullText) <= maxBytes) return { text: fullText, omitted: [], truncated: [] }
|
||||
if (byteLength(fullText) <= maxBytes) {
|
||||
return { text: fullText, omitted: [], truncated: [], represented: files }
|
||||
}
|
||||
|
||||
for (let start = 1; start < files.length; start += 1) {
|
||||
const included = files.slice(start)
|
||||
const omitted = files.slice(0, start).map(file => ({ absolutePath: file.absolutePath, displayPath: file.displayPath }))
|
||||
const suffixText = buildInstructionText(included, maxBytes, omitted, [], style)
|
||||
if (byteLength(suffixText) <= maxBytes) return { text: suffixText, omitted, truncated: [] }
|
||||
if (byteLength(suffixText) <= maxBytes) return { text: suffixText, omitted, truncated: [], represented: included }
|
||||
}
|
||||
|
||||
const mostSpecific = files.at(-1)
|
||||
/* v8 ignore next -- callers only reach this after a non-empty fullText was built. */
|
||||
if (mostSpecific === undefined) return { text: '', omitted: [], truncated: [] }
|
||||
if (mostSpecific === undefined) return { text: '', omitted: [], truncated: [], represented: [] }
|
||||
const omitted = files.slice(0, -1).map(file => ({ absolutePath: file.absolutePath, displayPath: file.displayPath }))
|
||||
const originalBytes = byteLength(mostSpecific.content)
|
||||
|
||||
for (const candidateStyle of [style, { ...style, intro: COMPACT_WORKSPACE_CONTEXT_INTRO }]) {
|
||||
const truncatedFile = truncateToFit(mostSpecific, [], maxBytes, omitted, candidateStyle)
|
||||
const includedBytes = byteLength(truncatedFile.content)
|
||||
const truncated = [{
|
||||
displayPath: mostSpecific.displayPath,
|
||||
originalBytes: byteLength(mostSpecific.content),
|
||||
includedBytes: byteLength(truncatedFile.content),
|
||||
originalBytes,
|
||||
includedBytes,
|
||||
}]
|
||||
const text = buildInstructionText([truncatedFile], maxBytes, omitted, truncated, candidateStyle)
|
||||
if (byteLength(text) <= maxBytes) return { text, omitted, truncated }
|
||||
if (byteLength(text) <= maxBytes) {
|
||||
const represented = includedBytes > 0 || originalBytes === 0 ? [mostSpecific] : []
|
||||
return { text, omitted, truncated, represented }
|
||||
}
|
||||
}
|
||||
|
||||
const truncated = [{
|
||||
displayPath: mostSpecific.displayPath,
|
||||
originalBytes: byteLength(mostSpecific.content),
|
||||
originalBytes,
|
||||
includedBytes: 0,
|
||||
}]
|
||||
const compactNotice = escapeInstructionFrameBody(markerText(maxBytes, omitted, truncated))
|
||||
const compactWithHeading = escapeInstructionFrameBody(
|
||||
[compactNotice, style.section(withTruncatedContent(mostSpecific, 0))].join('\n\n'),
|
||||
)
|
||||
if (byteLength(compactWithHeading) <= maxBytes) return { text: compactWithHeading, omitted, truncated }
|
||||
if (byteLength(compactWithHeading) <= maxBytes) {
|
||||
const represented = originalBytes === 0 ? [mostSpecific] : []
|
||||
return { text: compactWithHeading, omitted, truncated, represented }
|
||||
}
|
||||
const text = byteLength(compactNotice) <= maxBytes ? compactNotice : truncateUtf8(compactNotice, maxBytes)
|
||||
return { text, omitted, truncated }
|
||||
return { text, omitted, truncated, represented: [] }
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a baseline together with the exact source files semantically represented in it.
|
||||
* @param files - loaded files ordered from broadest to most specific.
|
||||
* @param options - rendering byte budget and whether this baseline supersedes a visible predecessor.
|
||||
* @returns bounded public rendering plus files with surviving content, including genuinely empty files.
|
||||
* @internal
|
||||
*/
|
||||
export function renderWorkspaceInstructionSet(
|
||||
files: LoadedInstructionFile[],
|
||||
options: { maxBytes: number; replacePreviousBaseline?: boolean },
|
||||
): { rendered: RenderedWorkspaceContext; included: LoadedInstructionFile[] } {
|
||||
const style = baselineRenderStyle(files, options.replacePreviousBaseline)
|
||||
const { represented, ...rendered } = renderInstructionContext(files, options.maxBytes, style)
|
||||
return { rendered, included: represented }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -305,13 +357,5 @@ export function renderWorkspaceContext(
|
||||
files: LoadedInstructionFile[],
|
||||
options: { maxBytes: number; replacePreviousBaseline?: boolean },
|
||||
): RenderedWorkspaceContext {
|
||||
const style = options.replacePreviousBaseline === true
|
||||
? {
|
||||
...BASELINE_RENDER_STYLE,
|
||||
intro: files.length === 0
|
||||
? EMPTY_REPLACEMENT_WORKSPACE_CONTEXT_INTRO
|
||||
: REPLACEMENT_WORKSPACE_CONTEXT_INTRO,
|
||||
}
|
||||
: BASELINE_RENDER_STYLE
|
||||
return renderInstructionContext(files, options.maxBytes, style)
|
||||
return renderWorkspaceInstructionSet(files, options).rendered
|
||||
}
|
||||
|
||||
@@ -422,6 +422,10 @@ export async function reconcileInstructionContext(
|
||||
}
|
||||
if (items.length === 0) return undefined
|
||||
const rendered = renderInstructionChanges(items, resolved.maxBytes)
|
||||
// When no transition survived rendering (tiny budgets render notice-only
|
||||
// text), emit nothing and commit nothing — the uncommitted versions make the
|
||||
// next pass retry instead of spamming notice-only contexts.
|
||||
if (rendered.text.length === 0 || rendered.changes.length === 0) return undefined
|
||||
return {
|
||||
context: workspaceContextHook(rendered.text, rendered.changes),
|
||||
versionUpdates: retainedInstructionVersionUpdates(versionUpdates, rendered.changes),
|
||||
|
||||
@@ -40,7 +40,7 @@ import {
|
||||
type InstructionVersionCache,
|
||||
} from '../src/state.ts'
|
||||
import { resolveConfig } from '../src/config.ts'
|
||||
import { candidateScopeKey, renderInstructionChanges, USER_GLOBAL_DIRECTORY, USER_GLOBAL_FILE } from '../src/render.ts'
|
||||
import { candidateScopeKey, renderInstructionChanges, renderWorkspaceInstructionSet, USER_GLOBAL_DIRECTORY, USER_GLOBAL_FILE } from '../src/render.ts'
|
||||
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
|
||||
/** Per-candidate reconciliation scope key: directory paired with the file name. */
|
||||
@@ -859,6 +859,33 @@ describe('workspace context rendering', () => {
|
||||
expect(Buffer.byteLength(rendered.text, 'utf8')).toBe(120)
|
||||
})
|
||||
|
||||
it('represents a genuinely empty instruction when its compact heading fits', () => {
|
||||
const file = { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: '' }
|
||||
const rendered = renderWorkspaceInstructionSet([file], { maxBytes: 117 })
|
||||
|
||||
expect(rendered.rendered.text).toContain('truncated pkg/AGENTS.md from 0 to 0 bytes')
|
||||
expect(rendered.rendered.text).toContain('Instructions from: pkg/AGENTS.md')
|
||||
expect(rendered.included).toEqual([file])
|
||||
})
|
||||
|
||||
it('represents a genuinely empty instruction through the framed compact-intro path', () => {
|
||||
const file = {
|
||||
absolutePath: '/repo/pkg/AGENTS.md',
|
||||
displayPath: 'pkg/AGENTS.md',
|
||||
content: '',
|
||||
}
|
||||
|
||||
const rendered = renderWorkspaceInstructionSet([file], { maxBytes: 300 })
|
||||
|
||||
expect(rendered.rendered.text).toContain('<system-reminder>')
|
||||
expect(rendered.rendered.text).toContain('Workspace instructions were omitted or truncated')
|
||||
expect(rendered.rendered.text).toContain('Instructions from: pkg/AGENTS.md')
|
||||
expect(rendered.rendered.truncated).toEqual([
|
||||
{ displayPath: 'pkg/AGENTS.md', originalBytes: 0, includedBytes: 0 },
|
||||
])
|
||||
expect(rendered.included).toEqual([file])
|
||||
})
|
||||
|
||||
it('truncates the compact notice itself when the render budget is smaller than the notice', () => {
|
||||
const rendered = renderWorkspaceContext([
|
||||
{ absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: 'x'.repeat(1000) },
|
||||
@@ -869,6 +896,76 @@ describe('workspace context rendering', () => {
|
||||
expect(Buffer.byteLength(rendered.text, 'utf8')).toBe(20)
|
||||
})
|
||||
|
||||
it('does not commit a change when only the generic compact notice survives', () => {
|
||||
const change = {
|
||||
action: 'set' as const,
|
||||
scope: sk('pkg', 'AGENTS.md'),
|
||||
path: 'pkg/AGENTS.md',
|
||||
digest: 'digest',
|
||||
}
|
||||
const rendered = renderInstructionChanges([{
|
||||
change,
|
||||
file: { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: 'x'.repeat(1000) },
|
||||
}], 20)
|
||||
|
||||
expect(rendered.text).toBe('Workspace instructio')
|
||||
expect(rendered.changes).toEqual([])
|
||||
})
|
||||
|
||||
it('commits a change when its file-specific semantic section survives truncation', () => {
|
||||
const change = {
|
||||
action: 'replace' as const,
|
||||
scope: sk('pkg', 'AGENTS.md'),
|
||||
path: 'pkg/AGENTS.md',
|
||||
digest: 'digest',
|
||||
}
|
||||
const rendered = renderInstructionChanges([{
|
||||
change,
|
||||
file: { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: 'x'.repeat(1000) },
|
||||
}], 400)
|
||||
|
||||
expect(rendered.text).toContain('Updated instructions from: pkg/AGENTS.md')
|
||||
expect(rendered.changes).toEqual([change])
|
||||
})
|
||||
|
||||
// Each prose-derived budget is the smallest current value that retains the named heading plus a zero-byte marker.
|
||||
it.each([
|
||||
{ action: 'set' as const, maxBytes: 327, heading: 'Additional instructions from:' },
|
||||
{ action: 'replace' as const, maxBytes: 256, heading: 'Updated instructions from:' },
|
||||
])('does not commit a $action change when its heading survives with zero content bytes', ({ action, maxBytes, heading }) => {
|
||||
const change = {
|
||||
action,
|
||||
scope: sk('pkg', 'AGENTS.md'),
|
||||
path: 'pkg/AGENTS.md',
|
||||
digest: 'digest',
|
||||
}
|
||||
const rendered = renderInstructionChanges([{
|
||||
change,
|
||||
file: { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: 'x'.repeat(1000) },
|
||||
}], maxBytes)
|
||||
|
||||
expect(rendered.text).toContain(heading)
|
||||
expect(rendered.text).toContain('from 1000 to 0 bytes')
|
||||
expect(rendered.changes).toEqual([])
|
||||
})
|
||||
|
||||
it('does not commit a multibyte change when the budget cuts its first code point', () => {
|
||||
const change = {
|
||||
action: 'set' as const,
|
||||
scope: sk('pkg', 'AGENTS.md'),
|
||||
path: 'pkg/AGENTS.md',
|
||||
digest: 'digest',
|
||||
}
|
||||
const rendered = renderInstructionChanges([{
|
||||
change,
|
||||
file: { absolutePath: '/repo/pkg/AGENTS.md', displayPath: 'pkg/AGENTS.md', content: '😀'.repeat(100) },
|
||||
}], 366)
|
||||
|
||||
expect(rendered.text).not.toContain('<27>')
|
||||
expect(rendered.text).not.toContain('😀')
|
||||
expect(rendered.changes).toEqual([])
|
||||
})
|
||||
|
||||
it('keeps compact truncation notices within budget when a multibyte display path is cut', () => {
|
||||
const rendered = renderWorkspaceContext([
|
||||
{ absolutePath: '/repo/路径/AGENTS.md', displayPath: '路径/AGENTS.md', content: 'x'.repeat(1000) },
|
||||
@@ -1836,21 +1933,30 @@ describe('workspace context request injection', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('does not expose state markers when a tiny budget reduces the baseline contribution', async () => {
|
||||
it.each([10, 120])('does not expose state markers when baseline content is omitted at %i bytes', async (maxBytes) => {
|
||||
const root = await tempRepo()
|
||||
const home = await tempRepo()
|
||||
try {
|
||||
await mkdir(join(root, '.git'), { recursive: true })
|
||||
await write(join(root, 'AGENTS.md'), 'repo rule')
|
||||
await write(join(root, 'AGENTS.md'), 'x'.repeat(1000))
|
||||
const ctx = new Context()
|
||||
await mountWorkspaceContext(ctx, { dshHome: home, maxBytes: 10 })
|
||||
await mountWorkspaceContext(ctx, { dshHome: home, maxBytes })
|
||||
const agent = stubAgent(root)
|
||||
|
||||
await composeBaselinePrefix(ctx, agent)
|
||||
|
||||
expect(agent.session.events.filter(event =>
|
||||
const contexts = agent.session.events.filter(event =>
|
||||
event.type === 'user/message' && event.data.source.kind !== 'user',
|
||||
)).toHaveLength(1)
|
||||
)
|
||||
expect(contexts).toHaveLength(1)
|
||||
const source = contexts[0]?.type === 'user/message' ? contexts[0].data.source : undefined
|
||||
expect(source?.kind === 'workspace-instructions' ? source.changes : undefined).toEqual([])
|
||||
if (maxBytes === 120) {
|
||||
expect(derivedText(agent)).toContain('Instructions from: AGENTS.md')
|
||||
expect(derivedText(agent)).toContain('from 1000 to 0 bytes')
|
||||
} else {
|
||||
expect(derivedText(agent)).not.toContain('Instructions from: AGENTS.md')
|
||||
}
|
||||
expect(derivedText(agent)).not.toContain('workspace-context:')
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true })
|
||||
@@ -4171,6 +4277,47 @@ describe('dynamic nested workspace context injection', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('retries a nested instruction touch when only a truncated budget notice was rendered', async () => {
|
||||
const root = join(await tempRepo(), 'virtual-repo')
|
||||
const home = join(await tempRepo(), 'virtual-home')
|
||||
const ctx = new Context()
|
||||
try {
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(RecordingFileSystem)
|
||||
const fs = ctx.fs as RecordingFileSystem
|
||||
const instructionPath = join(root, 'pkg/AGENTS.md')
|
||||
fs.entries.set(join(root, '.git'), { type: 'directory' })
|
||||
fs.entries.set(instructionPath, { type: 'file', content: 'x'.repeat(1000) })
|
||||
fs.entries.set(join(root, 'pkg/file.txt'), { type: 'file', content: 'hello' })
|
||||
await ctx.plugin(ToolFs)
|
||||
await ctx.plugin(workspaceContext, { dshHome: home, maxBytes: 20 })
|
||||
const agent = stubAgent(root)
|
||||
|
||||
const first = await ctx.tools.execute({
|
||||
signal: testToolSignal,
|
||||
callId: CallId('read-tiny-budget-1'), name: 'read', arguments: { file_path: join('pkg', 'file.txt') }, agent,
|
||||
})
|
||||
await syncWorkspaceContext(ctx, agent)
|
||||
const second = await ctx.tools.execute({
|
||||
signal: testToolSignal,
|
||||
callId: CallId('read-tiny-budget-2'), name: 'read', arguments: { file_path: join('pkg', 'file.txt') }, agent,
|
||||
})
|
||||
await syncWorkspaceContext(ctx, agent)
|
||||
|
||||
expect(first.additionalContexts).toBeUndefined()
|
||||
expect(second.additionalContexts).toBeUndefined()
|
||||
// Nothing was emitted, and the uncommitted version made the second sync
|
||||
// probe the instruction file again — the retry.
|
||||
expect(agent.inbox.nextStep).toHaveLength(0)
|
||||
expect(fs.readTargets.filter(path => path === instructionPath)).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await rm(dirname(root), { recursive: true, force: true })
|
||||
await rm(dirname(home), { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('does not attach nested instructions after a failed file read', async () => {
|
||||
const root = await tempRepo()
|
||||
const home = await tempRepo()
|
||||
@@ -4257,7 +4404,7 @@ describe('workspace context inbox synchronization', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps a dynamic change within a one-byte positive render budget', async () => {
|
||||
it('holds back a dynamic change a one-byte positive render budget cannot represent', async () => {
|
||||
const root = await tempRepo()
|
||||
const home = await tempRepo()
|
||||
const ctx = new Context()
|
||||
@@ -4275,8 +4422,10 @@ describe('workspace context inbox synchronization', () => {
|
||||
|
||||
await syncWorkspaceContext(ctx, agent)
|
||||
|
||||
expect(agent.inbox.nextStep).toHaveLength(1)
|
||||
expect(Buffer.byteLength(blocksText(agent.inbox.nextStep[0]?.content), 'utf8')).toBeLessThanOrEqual(1)
|
||||
// One byte cannot semantically represent the transition, so nothing is
|
||||
// emitted and nothing commits — the uncommitted version retries on the
|
||||
// next touch instead of committing state the model never saw.
|
||||
expect(agent.inbox.nextStep).toHaveLength(0)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await rm(root, { recursive: true, force: true })
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md
|
||||
README.md: 4b65bc3c5a5b31a8b4f96ec86e2239c2da680897
|
||||
README.zh.md: 45d9646f89623d2de106cd8e1dab38941c912977
|
||||
README.md: d17dc0747833a0ecb85505260badc79ad739f27f
|
||||
README.zh.md: e13ab2f04b84b59fbb3846c08139e679b2d43be1
|
||||
|
||||
@@ -20,7 +20,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing.
|
||||
- **`readBytes`** — raw whole-file bytes with no decoding or binary rejection (the `read_image` tool validates content through the attachment service). The required byte cap short-circuits on the stat size before any content I/O; the subsequent stream reads at most one byte beyond the cap, so a file growing after stat still fails `FS_TOO_LARGE` without unbounded buffering.
|
||||
- **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`.
|
||||
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a regular file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`, while a non-regular path entry is preserved and rejected with `FS_NOT_REGULAR_FILE`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
|
||||
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a regular file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`, while a non-regular path entry is preserved and rejected with `FS_NOT_REGULAR_FILE`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). An overwrite returns the prior text as its contextual diff basis only when both the opened prior file and UTF-8 replacement are strictly below `config.diffBasisMaxBytes` (default 10 MiB). The descriptor read enforces that limit even if an external writer replaces or changes the file size after the initial probe. Otherwise the provider returns `before: null`, so presentation uses its whole-file fallback.
|
||||
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
|
||||
|
||||
The package-root SDK surface is the default/named `LocalFileSystem` class plus `Config`. Raw I/O lives in `src/fsio.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the thin service wiring.
|
||||
@@ -36,9 +36,9 @@ No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)).
|
||||
- **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`).
|
||||
- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard.
|
||||
- **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path.
|
||||
- **A sub-limit overwrite still buffers a contextual basis** — `writeText` may retain up to just below `config.diffBasisMaxBytes` of prior text in addition to the caller-owned replacement; the bound does not cap the returned `after` value or presentation's whole-file fallback.
|
||||
- **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits.
|
||||
- **The per-target mutation lock is in-process only** — guarded create still uses an atomic no-replace publication across processes, but replacement writers in another process are caught only when the optional version guard observes their metadata change; they are never serialized.
|
||||
- **Guarded creation requires hard-link support** — filesystems or mounts that reject hard-link publication cannot serve `createIfAbsent`; the provider preserves the missing target and reports `FS_IO_ERROR`.
|
||||
|
||||
@@ -20,7 +20,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片解码,因此超大文件无需整体保存在内存中,消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑。
|
||||
- **`readBytes`**:按原始字节读取整个文件,不做解码或二进制拒绝(`read_image` 工具通过附件服务校验内容)。必填的字节上限在任何内容 I/O 之前先按 stat 大小短路;随后的流最多多读一个字节,因此 stat 之后增长的文件仍会以 `FS_TOO_LARGE` 失败,不会无界缓冲。
|
||||
- **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。
|
||||
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的普通文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;非普通路径条目也会被保留,并以 `FS_NOT_REGULAR_FILE` 拒绝;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
|
||||
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的普通文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;非普通路径条目也会被保留,并以 `FS_NOT_REGULAR_FILE` 拒绝;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。仅当打开后的旧文件和 UTF-8 替换内容都严格低于 `config.diffBasisMaxBytes`(默认 10 MiB)时,覆写才返回旧文本作为上下文 diff 基础。即使外部写入方在初次探测后替换文件或改变文件大小,文件描述符读取仍会强制执行该上限;否则提供方返回 `before: null`,由展示层使用整文件回退。
|
||||
- **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。
|
||||
|
||||
包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。
|
||||
@@ -36,9 +36,9 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
## 已知限制与延期工作
|
||||
|
||||
- **`config.cwd` 不是沙箱**:它是解析默认值,而非约束;绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall(瀑布式事件)上的权限插件实施约束(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences))。
|
||||
- **覆盖会把整个旧文件读入内存**:只用于 UI diff;在大小阈值之上限制这次预读取的工作延期处理(`TODO(overwrite-diff-bound)`)。
|
||||
- **版本 token 依赖文件系统元数据**:它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime;如果存储层在重写时无法更新其中任何一项事实,仍可能绕过陈旧防护。
|
||||
- **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。
|
||||
- **低于上限的覆写仍会缓冲上下文基础**:`writeText` 除调用方持有的替换内容外,最多还会保留略低于 `config.diffBasisMaxBytes` 的旧文本;该上限不限制返回的 `after` 值,也不限制展示层的整文件回退。
|
||||
- **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。
|
||||
- **每目标变更锁仅限进程内**:即使跨进程,带防护的创建仍采用原子且不替换的发布方式;但只有当可选版本防护观察到元数据变化时,系统才能发现其他进程中的替换写入方,且绝不会将其串行化。
|
||||
- **带防护的创建要求支持硬链接**:拒绝硬链接发布的文件系统或挂载点无法支持 `createIfAbsent`;提供方会使目标保持缺失状态并报告 `FS_IO_ERROR`。
|
||||
|
||||
@@ -15,6 +15,8 @@ import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs'
|
||||
import { copyFileDaclWin32, replaceFileWin32 } from './win32.ts'
|
||||
|
||||
const BINARY_SAMPLE_BYTES = 8192
|
||||
// Bound one non-abortable FileHandle.read so cancellation is observed between chunks.
|
||||
const DIFF_BASIS_READ_CHUNK_BYTES = 64 * 1024
|
||||
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
|
||||
@@ -74,9 +76,8 @@ function versionOf(info: BigIntStats): FsVersion {
|
||||
}
|
||||
|
||||
/**
|
||||
* Test hook: lets specs pin the atomic-write temp names (to prove
|
||||
* exclusive-open behavior without a name race) and observe the staged temp
|
||||
* file before it is renamed over the target.
|
||||
* Test hook: lets specs pin the atomic-write temp names (to prove exclusive-open behavior without
|
||||
* a name race), override native boundaries, and observe the staged temp file before publication.
|
||||
*/
|
||||
export interface FsIoInternals {
|
||||
/** Override the host platform for native-publication unit coverage. */
|
||||
@@ -680,21 +681,67 @@ export async function readForEdit(
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort overwrite diff basis. Binary or invalid UTF-8 returns `null` so the write still
|
||||
* succeeds and presentation falls back to a whole-file diff.
|
||||
* @param absolutePath - the file to read (typically a target key); it must exist.
|
||||
* @param signal - aborts the read (`FS_ABORTED`).
|
||||
* @returns the LF-normalized text, or null for a binary or non-UTF-8 file.
|
||||
* Best-effort overwrite diff basis. Binary, invalid UTF-8, a file at/above the byte limit,
|
||||
* or a file deleted/made unreadable after the caller's preflight returns `null` so the write
|
||||
* still succeeds and presentation falls back to a whole-file diff. The bound is enforced on
|
||||
* the opened descriptor rather than a prior path stat, so concurrent external replacement or
|
||||
* size changes cannot make this helper buffer more than `maxBytes`.
|
||||
* @param absolutePath - the file to read (typically a target key).
|
||||
* @param maxBytes - exclusive upper bound for bytes held as the contextual-diff basis.
|
||||
* @param signal - aborts the read (`FS_ABORTED`); cancellation propagates, unlike I/O failure.
|
||||
* @returns the LF-normalized text, or null for a non-regular, at/above-limit, binary, non-UTF-8,
|
||||
* descriptor-size-changed, or unreadable file.
|
||||
*/
|
||||
export async function readTextForDiff(absolutePath: string, signal?: AbortSignal): Promise<string | null> {
|
||||
const buffer = await readFileAbortable(absolutePath, 'read', signal)
|
||||
if (buffer.includes(0)) return null
|
||||
export async function readTextForDiff(
|
||||
absolutePath: string,
|
||||
maxBytes: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string | null> {
|
||||
throwIfAborted(signal, 'read')
|
||||
try {
|
||||
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(buffer))
|
||||
const handle = await open(absolutePath, 'r')
|
||||
let buffer: Buffer
|
||||
let total = 0
|
||||
let openedSize = 0
|
||||
try {
|
||||
throwIfAborted(signal, 'read')
|
||||
const info = await handle.stat()
|
||||
throwIfAborted(signal, 'read')
|
||||
if (!info.isFile()) return null
|
||||
if (info.size >= maxBytes) return null
|
||||
openedSize = info.size
|
||||
// One extra byte detects growth after stat without retaining per-read backing buffers.
|
||||
buffer = Buffer.allocUnsafe(openedSize + 1)
|
||||
while (total < buffer.length) {
|
||||
throwIfAborted(signal, 'read')
|
||||
const length = Math.min(buffer.length - total, DIFF_BASIS_READ_CHUNK_BYTES)
|
||||
const { bytesRead } = await handle.read(buffer, total, length, null)
|
||||
if (bytesRead === 0) break
|
||||
total += bytesRead
|
||||
}
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
throwIfAborted(signal, 'read')
|
||||
if (total !== openedSize) return null
|
||||
const basis = buffer.subarray(0, total)
|
||||
if (basis.includes(0)) return null
|
||||
try {
|
||||
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(basis))
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes;
|
||||
* any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
return null
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
return null
|
||||
// Cancellation is the caller's intent and still propagates.
|
||||
if (error instanceof FsError) throw error
|
||||
// A descriptor-phase errno — deleted or made unreadable after the caller's
|
||||
// preflight, or a faulted read — costs only the optional basis: a committed
|
||||
// write must not fail for a presentation-only pre-read.
|
||||
if (error instanceof Error && 'code' in error) return null
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import { constants as bufferConstants } from 'node:buffer'
|
||||
import { isAbsolute, relative, resolve, sep } from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import z from 'schemastery'
|
||||
@@ -40,9 +41,19 @@ import type { FsIoInternals } from './fsio.ts'
|
||||
export interface Config {
|
||||
/** Base directory for relative paths. Defaults to `process.cwd()`. */
|
||||
cwd?: string
|
||||
/**
|
||||
* Exclusive UTF-8 byte limit on each overwrite-diff side, capped by the
|
||||
* runtime's safe allocation/decode maximum. Defaults to 10 MiB.
|
||||
*/
|
||||
diffBasisMaxBytes?: number
|
||||
}
|
||||
|
||||
type ResolvedConfig = Required<Config>
|
||||
const DEFAULT_DIFF_BASIS_MAX_BYTES = 10 * 1024 * 1024
|
||||
const MAX_DIFF_BASIS_BYTES = Math.min(
|
||||
bufferConstants.MAX_LENGTH,
|
||||
bufferConstants.MAX_STRING_LENGTH,
|
||||
)
|
||||
|
||||
/**
|
||||
* The host-filesystem backend. Reads resolve relative paths from {@link Config.cwd}
|
||||
@@ -53,11 +64,12 @@ type ResolvedConfig = Required<Config>
|
||||
export class LocalFileSystem extends FileSystem {
|
||||
static Config: z<Config> = z.object({
|
||||
cwd: z.string().default(process.cwd()),
|
||||
diffBasisMaxBytes: z.number().default(DEFAULT_DIFF_BASIS_MAX_BYTES),
|
||||
})
|
||||
|
||||
/** Validated config (schemastery applied the defaults before construction). */
|
||||
readonly config: ResolvedConfig
|
||||
/** Test hook forwarded to fsio (force streaming path, pin temp names). */
|
||||
/** Test hook forwarded to fsio for atomic-publication boundaries. */
|
||||
internals: FsIoInternals = {}
|
||||
/** Per-targetKey tail promise: serializes mutating ops so the read→guard→write
|
||||
* window can't interleave, making concurrent writes/edits deterministically
|
||||
@@ -66,7 +78,13 @@ export class LocalFileSystem extends FileSystem {
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
this.config = config as ResolvedConfig
|
||||
const resolved = config as ResolvedConfig
|
||||
if (!Number.isSafeInteger(resolved.diffBasisMaxBytes)
|
||||
|| resolved.diffBasisMaxBytes <= 0
|
||||
|| resolved.diffBasisMaxBytes > MAX_DIFF_BASIS_BYTES) {
|
||||
throw new Error(`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${MAX_DIFF_BASIS_BYTES}`)
|
||||
}
|
||||
this.config = resolved
|
||||
}
|
||||
|
||||
/** Run `op` with exclusive access to `targetKey` (FIFO per key). */
|
||||
@@ -169,9 +187,16 @@ export class LocalFileSystem extends FileSystem {
|
||||
}
|
||||
// No expectation means an unconditional but still atomic write.
|
||||
|
||||
// Preserve prior text for contextual diffs; null falls back to a whole-file diff.
|
||||
// TODO(overwrite-diff-bound): cap this UI-only pre-read for large files.
|
||||
const before = existing ? await readTextForDiff(target.targetKey, signal) : null
|
||||
// Capture an optional contextual-diff basis before the write. The bounded
|
||||
// reader checks the opened file itself, so an external replacement after
|
||||
// `probe()` cannot turn this best-effort presentation read into an
|
||||
// unbounded allocation. Either side at/above the configured limit yields
|
||||
// `before: null`; consumers retain their whole-file fallback.
|
||||
const diffable = existing !== null
|
||||
&& Buffer.byteLength(content, 'utf8') < this.config.diffBasisMaxBytes
|
||||
const before = diffable
|
||||
? await readTextForDiff(target.targetKey, this.config.diffBasisMaxBytes, signal)
|
||||
: null
|
||||
await writeFileAtomic(
|
||||
target.targetKey,
|
||||
content,
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { constants as bufferConstants } from 'node:buffer'
|
||||
import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -43,13 +44,39 @@ async function versionOf(target: FsTarget): Promise<FsVersion> {
|
||||
return info.version
|
||||
}
|
||||
|
||||
async function remountWithDiffLimit(diffBasisMaxBytes: number): Promise<void> {
|
||||
await fiber.dispose()
|
||||
fiber = await ctx.plugin(LocalFileSystem, { cwd: dir, diffBasisMaxBytes })
|
||||
fs = ctx.fs as LocalFileSystem
|
||||
}
|
||||
|
||||
describe('registration', () => {
|
||||
it('registers LocalFileSystem as ctx.fs with a default cwd', async () => {
|
||||
const bare = new Context()
|
||||
const bareFiber = await bare.plugin(LocalFileSystem)
|
||||
expect((bare.fs as LocalFileSystem).config.cwd).toBe(process.cwd())
|
||||
expect((bare.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(10 * 1024 * 1024)
|
||||
await bareFiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects non-positive, fractional, unsafe, or unallocatable diff-basis limits', async () => {
|
||||
const maxDiffBasisBytes = Math.min(
|
||||
bufferConstants.MAX_LENGTH,
|
||||
bufferConstants.MAX_STRING_LENGTH,
|
||||
)
|
||||
const valid = new Context()
|
||||
const validFiber = await valid.plugin(LocalFileSystem, { diffBasisMaxBytes: maxDiffBasisBytes })
|
||||
expect((valid.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(maxDiffBasisBytes)
|
||||
await validFiber.dispose()
|
||||
|
||||
for (const diffBasisMaxBytes of [0, -1, 1.5, maxDiffBasisBytes + 1, Number.MAX_SAFE_INTEGER + 1]) {
|
||||
const invalid = new Context()
|
||||
await expect(invalid.plugin(LocalFileSystem, { diffBasisMaxBytes })).rejects.toThrow(
|
||||
`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${maxDiffBasisBytes}`,
|
||||
)
|
||||
await invalid.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('resolve', () => {
|
||||
@@ -481,6 +508,54 @@ describe('writeText', () => {
|
||||
expect(outcome.after).toBe('now valid')
|
||||
})
|
||||
|
||||
it('an overwrite of a prior file AT the whole-file bound reports before:null (undiffable), still succeeds', async () => {
|
||||
// The configured bound keeps the fixture small; 8 bytes at a bound of 8
|
||||
// pins the exclusive edge without coupling this provider to a read tool.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'big.txt'), '12345678')
|
||||
const target = await fs.resolve('big.txt')
|
||||
const outcome = await fs.writeText(target, 'tiny')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('tiny')
|
||||
})
|
||||
|
||||
it('an overwrite whose NEW content is at the whole-file bound reports before:null (no huge contextual diff)', async () => {
|
||||
// The bound gates BOTH sides of the diff pair: a small prior file rewritten
|
||||
// with at/above-bound content yields no contextual-hunk basis either, since
|
||||
// a small-to-huge rewrite's hunk is as large as the new content — the
|
||||
// consumer must fall back to the whole-file diff card, exactly like a
|
||||
// create of the same size.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'grow.txt'), 'tiny')
|
||||
const target = await fs.resolve('grow.txt')
|
||||
const outcome = await fs.writeText(target, '12345678')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('12345678')
|
||||
})
|
||||
|
||||
it('gates the NEW content by UTF-8 byte length, not character count', async () => {
|
||||
// Three CJK characters are 9 UTF-8 bytes: below an 8-byte bound by
|
||||
// characters but at/above it by bytes, so the basis must be declined.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'cjk.txt'), 'tiny')
|
||||
const target = await fs.resolve('cjk.txt')
|
||||
const outcome = await fs.writeText(target, '你好吗')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('你好吗')
|
||||
})
|
||||
|
||||
it('an overwrite with BOTH sides below the whole-file bound keeps its contextual before basis', async () => {
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'small.txt'), '1234567')
|
||||
const target = await fs.resolve('small.txt')
|
||||
const outcome = await fs.writeText(target, 'new')
|
||||
expect(outcome.before).toBe('1234567')
|
||||
expect(outcome.after).toBe('new')
|
||||
})
|
||||
|
||||
it('releases per-target mutation locks after success and failure', async () => {
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.writeText(target, 'created', { kind: 'createIfAbsent' })
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* policy and lives in `dsh-fs-policy`, so it is not tested here.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
probe,
|
||||
probeNoFollow,
|
||||
readForEdit,
|
||||
readTextForDiff,
|
||||
readWholeText,
|
||||
resolveLocalTarget,
|
||||
restoreLineEndings,
|
||||
@@ -316,6 +317,261 @@ describe('readWholeText', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('readTextForDiff', () => {
|
||||
it('returns normalized text only when the opened file is strictly below the limit', async () => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'a\r\nb')
|
||||
expect(await readTextForDiff(file, 5)).toBe('a\nb')
|
||||
expect(await readTextForDiff(file, 4)).toBeNull()
|
||||
})
|
||||
|
||||
it('bounds the actual opened file rather than trusting an earlier path size', async () => {
|
||||
const file = join(dir, 'replaced.txt')
|
||||
await writeFile(file, 'tiny')
|
||||
const earlierSize = (await stat(file)).size
|
||||
await writeFile(file, '123456789')
|
||||
expect(earlierSize).toBeLessThan(8)
|
||||
expect(await readTextForDiff(file, 8)).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null when the opened file shrinks after descriptor stat', async () => {
|
||||
const file = join(dir, 'shrinking.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
await writeFile(file, 'abc')
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 8)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null when the opened file grows after descriptor stat', async () => {
|
||||
const file = join(dir, 'growing.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
await writeFile(file, 'abcdef-grown')
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 32)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null when the file vanishes before the basis open (deletion race)', async () => {
|
||||
expect(await readTextForDiff(join(dir, 'deleted-after-preflight.txt'), 32)).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null when the opened descriptor is no longer a regular file', async () => {
|
||||
const file = join(dir, 'swapped.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
return Object.assign(info, { isFile: () => false })
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 32)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('propagates a non-errno fault instead of masking it as a null basis', async () => {
|
||||
const file = join(dir, 'faulted.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open() {
|
||||
throw new TypeError('forged programming fault')
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
await expect(isolatedReadTextForDiff(file, 32)).rejects.toThrow('forged programming fault')
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null for binary and invalid UTF-8 without blocking the caller write', async () => {
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
expect(await readTextForDiff(join(dir, 'bin'), 8)).toBeNull()
|
||||
expect(await readTextForDiff(join(dir, 'bad'), 8)).toBeNull()
|
||||
})
|
||||
|
||||
it('honors a pre-aborted signal', async () => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'text')
|
||||
await expect(readTextForDiff(file, 8, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
|
||||
it.each(['open', 'stat'] as const)('observes cancellation immediately after %s', async (stage) => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'text')
|
||||
const reached = Promise.withResolvers<undefined>()
|
||||
const release = Promise.withResolvers<undefined>()
|
||||
let statCalls = 0
|
||||
const allocate = vi.spyOn(Buffer, 'allocUnsafe')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
if (stage === 'open') {
|
||||
reached.resolve(undefined)
|
||||
await release.promise
|
||||
}
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
statCalls += 1
|
||||
const info = await handle.stat(...statArgs)
|
||||
if (stage === 'stat') {
|
||||
reached.resolve(undefined)
|
||||
await release.promise
|
||||
}
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
const controller = new AbortController()
|
||||
const pending = isolatedReadTextForDiff(file, 8, controller.signal)
|
||||
await reached.promise
|
||||
const allocationCalls = allocate.mock.calls.length
|
||||
controller.abort()
|
||||
release.resolve(undefined)
|
||||
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
expect(statCalls).toBe(stage === 'open' ? 0 : 1)
|
||||
expect(allocate).toHaveBeenCalledTimes(allocationCalls)
|
||||
} finally {
|
||||
release.resolve(undefined)
|
||||
allocate.mockRestore()
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('bounds descriptor reads and observes cancellation before the next chunk', async () => {
|
||||
const file = join(dir, 'large-basis.txt')
|
||||
const fileBytes = 200 * 1024
|
||||
await writeFile(file, 'x'.repeat(fileBytes))
|
||||
const firstRead = Promise.withResolvers<undefined>()
|
||||
const releaseFirstRead = Promise.withResolvers<undefined>()
|
||||
const readLengths: number[] = []
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
stat: handle.stat.bind(handle),
|
||||
close: handle.close.bind(handle),
|
||||
async read(buffer: Buffer, offset: number, length: number, position: number | null) {
|
||||
readLengths.push(length)
|
||||
const result = await handle.read(buffer, offset, length, position)
|
||||
if (readLengths.length === 1) {
|
||||
firstRead.resolve(undefined)
|
||||
await releaseFirstRead.promise
|
||||
}
|
||||
return result
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
const controller = new AbortController()
|
||||
const pending = isolatedReadTextForDiff(file, fileBytes + 1, controller.signal)
|
||||
await firstRead.promise
|
||||
expect(readLengths).toEqual([64 * 1024])
|
||||
controller.abort()
|
||||
releaseFirstRead.resolve(undefined)
|
||||
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
expect(readLengths).toHaveLength(1)
|
||||
} finally {
|
||||
releaseFirstRead.resolve(undefined)
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('streamWholeText', () => {
|
||||
it('streams the whole file as decoded text', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/fs-sandbox/README.md
|
||||
README.md: c40fc7999ab85a70702f65a5675208163f5fc351
|
||||
README.zh.md: 15db5abbfc5307c0570925026ec435d8dbb51bf2
|
||||
README.md: ae1fd746c711a86e308a02e0054ba478e8d913c0
|
||||
README.zh.md: e25a3467c06fbd93de9bb75d6b364e8da5a451fe
|
||||
|
||||
@@ -4,6 +4,8 @@ English | [中文](README.zh.md)
|
||||
|
||||
`SandboxedFileSystem` extends [`LocalFileSystem`](../fs-local/README.md) and registers as `ctx.fs`. It inherits every text-storage mechanic verbatim (resolve, stat, read/stream, list, the atomic write, the read-match-write edit critical section) and adds only a per-call MODE fence on `writeText`/`editText`. Reads always pass through — every mode permits reading.
|
||||
|
||||
Its plugin config is the local backend config unchanged: `cwd` remains the relative-path resolution default, and `diffBasisMaxBytes` bounds the optional overwrite contextual-diff basis.
|
||||
|
||||
Loading it INSTEAD OF `dsh-fs-local`, together with a [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md), is the whole swap; the model-facing tools (`dsh-tool-fs`) are untouched. The tool layer resolves the calling session's mode and cwd into the SAME per-call policy bash receives, so the two families never confine to different roots.
|
||||
|
||||
## The fence
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
`SandboxedFileSystem` 扩展 [`LocalFileSystem`](../fs-local/README.md) 并注册为 `ctx.fs`。它逐字继承全部文本存储机制(解析、stat、读取/流式读取、列出、原子写入、按读取、匹配、写入顺序执行的编辑临界区),只为 `writeText`/`editText` 增加按调用的模式围栏。读取始终直接通过:所有模式都允许读取。
|
||||
|
||||
它原样复用本地后端配置:`cwd` 仍是相对路径的解析默认值,`diffBasisMaxBytes` 则限制可选的覆写上下文 diff 基础。
|
||||
|
||||
只需加载它来替代 `dsh-fs-local`,并同时加载 [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md),即可完成替换;面向模型的工具(`dsh-tool-fs`)无需改动。工具层把调用会话的模式和 cwd 解析为与 bash 相同的按调用策略,因此两个能力族绝不会约束到不同根目录。
|
||||
|
||||
## 围栏
|
||||
|
||||
@@ -41,10 +41,10 @@ import type {} from '@deepseek-ai/dsh-sandbox-policy'
|
||||
import { isPathUnder } from './containment.ts'
|
||||
|
||||
/**
|
||||
* Plugin config: the local backend's knobs, verbatim (only `cwd`, the resolve
|
||||
* base for relative paths). The sandbox default (mode + `workspace-write`
|
||||
* fallback root) is NOT here — `ctx.sandboxPolicy` resolves each calling
|
||||
* session for every enforcing capability.
|
||||
* Plugin config: the local backend's knobs verbatim (`cwd` resolution default
|
||||
* and `diffBasisMaxBytes` overwrite-presentation bound). The sandbox default
|
||||
* (mode + `workspace-write` fallback root) is NOT here — `ctx.sandboxPolicy`
|
||||
* resolves each calling session for every enforcing capability.
|
||||
*/
|
||||
export type Config = LocalConfig
|
||||
|
||||
|
||||
@@ -132,10 +132,11 @@ export interface FsWriteOutcome {
|
||||
version: FsVersion
|
||||
/**
|
||||
* The file's content BEFORE the write, or `null` when the file did not exist
|
||||
* (a create) or was undiffable (binary/non-UTF-8). LF-normalized storage text
|
||||
* (the diff basis), never a diff — a consumer computes the result-time
|
||||
* contextual diff from `before`/`after` when `before` is present, else falls
|
||||
* back to a whole-file diff.
|
||||
* (a create) or the backend declined a contextual basis (for example, a
|
||||
* binary/non-UTF-8 prior file or either overwrite side reaching its exclusive limit).
|
||||
* LF-normalized storage text (the diff basis), never a diff — a consumer
|
||||
* computes the result-time contextual diff from `before`/`after` when
|
||||
* `before` is present, else falls back to a whole-file diff.
|
||||
*/
|
||||
before: string | null
|
||||
/** The file's content AFTER the write, LF-normalized to share `before`'s diff basis. */
|
||||
|
||||
@@ -105,6 +105,8 @@ async function tightenModes(dir: string): Promise<void> {
|
||||
if (entry.isDirectory()) {
|
||||
await tightenModes(target)
|
||||
} else {
|
||||
/* v8 ignore next -- Windows mode bits cannot represent POSIX owner-execute state;
|
||||
* the Windows native gate preserves the DACL while the POSIX suite covers this branch. */
|
||||
await chmod(target, ((await stat(target)).mode & 0o100) === 0 ? 0o600 : 0o700)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ import type {
|
||||
StreamChunk,
|
||||
TokenUsage,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { LlmAdapter, LlmError, assertNever, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import { LlmAdapter, LlmError, ReasoningEffortId, assertNever, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/**
|
||||
* One recorded model call. `throw` may replay prefix chunks before failing;
|
||||
@@ -54,6 +54,18 @@ export interface ReplayModelConfig {
|
||||
contextWindow?: number
|
||||
/** Optional declared input modalities, so a scenario can exercise capability gates (e.g. image-capable `read_image`). */
|
||||
inputModalities?: readonly ModelModality[]
|
||||
/**
|
||||
* Optional per-request output cap the replay route materializes when callers
|
||||
* omit one, so replay reconstructs the request header a live catalog produced.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/** Optional reasoning-effort ids the replay route accepts, in display order. */
|
||||
reasoningEfforts?: string[]
|
||||
/**
|
||||
* Optional effort materialized when callers omit one; must appear in
|
||||
* {@link reasoningEfforts} or call resolution rejects the route.
|
||||
*/
|
||||
defaultReasoningEffort?: string
|
||||
}
|
||||
|
||||
/** One provider route exposed by the replay adapter. */
|
||||
@@ -592,6 +604,19 @@ class ReplayAdapter extends LlmAdapter {
|
||||
...configuredModel?.contextWindow === undefined
|
||||
? {}
|
||||
: { context: { contextWindow: configuredModel.contextWindow } },
|
||||
...configuredModel?.defaultMaxTokens === undefined
|
||||
? {}
|
||||
: { defaultMaxTokens: configuredModel.defaultMaxTokens },
|
||||
...configuredModel?.reasoningEfforts === undefined
|
||||
? {}
|
||||
: {
|
||||
reasoning: {
|
||||
efforts: configuredModel.reasoningEfforts.map(id => ({ id: ReasoningEffortId(id), name: id })),
|
||||
...configuredModel.defaultReasoningEffort === undefined
|
||||
? {}
|
||||
: { defaultEffort: ReasoningEffortId(configuredModel.defaultReasoningEffort) },
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -593,8 +593,15 @@ describe('installLlmReplay (through the real LlmService)', () => {
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
},
|
||||
models: [
|
||||
{ id: 'flash', contextWindow: 128_000, inputModalities: ['text', 'image'] },
|
||||
{ id: 'pro', name: 'Pro', description: 'Larger model' },
|
||||
{
|
||||
id: 'flash',
|
||||
contextWindow: 128_000,
|
||||
inputModalities: ['text', 'image'],
|
||||
defaultMaxTokens: 64_000,
|
||||
reasoningEfforts: ['off', 'max'],
|
||||
defaultReasoningEffort: 'max',
|
||||
},
|
||||
{ id: 'pro', name: 'Pro', description: 'Larger model', reasoningEfforts: ['high'] },
|
||||
],
|
||||
},
|
||||
{ id: 'empty' },
|
||||
@@ -613,9 +620,19 @@ describe('installLlmReplay (through the real LlmService)', () => {
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'flash')).resolves.toMatchObject({
|
||||
context: { contextWindow: 128_000 },
|
||||
inputModalities: ['text', 'image'],
|
||||
defaultMaxTokens: 64_000,
|
||||
reasoning: {
|
||||
efforts: [{ id: 'off', name: 'off' }, { id: 'max', name: 'max' }],
|
||||
defaultEffort: 'max',
|
||||
},
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'pro')).resolves.not.toHaveProperty('inputModalities')
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'pro')).resolves.not.toHaveProperty('context')
|
||||
// Efforts without a configured default preserve the provider's own default.
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'pro')).resolves.toMatchObject({
|
||||
reasoning: { efforts: [{ id: 'high', name: 'high' }] },
|
||||
})
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'pro')).resolves.not.toHaveProperty('defaultMaxTokens')
|
||||
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted')).resolves.not.toHaveProperty('context')
|
||||
await expect(ctx.llm.resolveModelInfo('empty', 'unlisted')).resolves.not.toHaveProperty('context')
|
||||
expect(ctx.llm.providerRetryPolicy('deepseek')).toMatchObject({
|
||||
|
||||
Reference in New Issue
Block a user