fix(apiproxy): cancel attachment reads during export
Response-consumer cancellation already stopped lineage reads, persistence reads, and ZIP production, but the final attachment phase called readImage without the producer signal. A slow or stalled attachment backend could therefore keep working after the browser abandoned the download and prevent the producer from settling.\n\nExtend the attachment read seam with optional cancellation, forward it through the local backend into Node's filesystem read, and preserve the abort reason rather than wrapping it as a storage failure. The exporter now passes its combined request/consumer signal to every attachment read.\n\nCover both ownership boundaries: the local-store test proves filesystem forwarding and cancellation identity, while the assembled export test cancels a reader during a pending attachment provider call. Regenerate the Cordis API catalog and paired documentation so implementers can rely on the new contract.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/attachment/attachment-local/README.md
|
||||
README.md: 80001b29b392fe1c8b663f46d47f1ec0726e6d0f
|
||||
README.zh.md: c3b95ace06b9f5ada156f20f33a1740a235400aa
|
||||
README.md: ba0b9efb2cf51bfef671020bed4a2c16f6ee0119
|
||||
README.zh.md: 8e2474357a0dbb5e8834a3b25de7a977827a29e3
|
||||
|
||||
@@ -4,7 +4,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachment). Objects land at `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>` and are addressed by an opaque `sha256:` id. Each process proves a home durable once by syncing every ancestor entry to the filesystem root, so a directory another process created but has not yet synced is never mistaken for a safe boundary. Writes then use a private staging directory, owner-only files, a synced temporary file, an atomic exclusive hard-link publish, and directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) so the reported reference survives a crash. Write admission and reads fully decode the raster before accepting its format and dimensions; reads also re-check the digest and logged metadata. Byte and pixel limits are write-time admission policy, so a later policy reduction does not make already-admitted history unreadable.
|
||||
|
||||
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata, never this host path.
|
||||
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata, never this host path. `readImage` forwards optional cancellation into the filesystem read, observes it around verification, and preserves it instead of wrapping it as `ATTACHMENT_READ_FAILED`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
这是 [`@deepseek-ai/dsh-attachment`](../attachment) 的私有本地实现。对象存放在 `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>`,并通过不透明的 `sha256:` 标识符寻址。每个进程都会通过将每个祖先目录项逐级同步到文件系统根目录,为某个 home 一次性证明其持久性,因此绝不会把另一个进程已经创建但尚未同步的目录误认为安全边界。随后,写入过程使用私有暂存目录、仅所有者可访问的文件、经过同步的临时文件、原子且排他的硬链接发布,并对发布路径执行目录同步(适用于 POSIX;Windows 依赖文件系统元数据日志),确保已报告的引用能够在崩溃后继续存在。写入准入与读取都会完整解码光栅图片,之后才接受其格式和尺寸;读取还会重新校验摘要和已记录的元数据。字节和像素限制属于写入时的准入策略,因此后续收紧限制不会导致已经接纳的历史记录变得不可读。
|
||||
|
||||
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据,绝不包含这个宿主路径。
|
||||
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据,绝不包含这个宿主路径。`readImage` 会把可选取消信号传入文件系统读取、在校验前后观察该信号,并保留取消语义,而不会将其包装成 `ATTACHMENT_READ_FAILED`。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -68,8 +68,8 @@ export class LocalAttachmentStore extends AttachmentStore {
|
||||
return saveImageFile(this.root, input, this.imageLimits)
|
||||
}
|
||||
|
||||
async readImage(ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
|
||||
return readImageFile(this.root, ref)
|
||||
async readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment> {
|
||||
return readImageFile(this.root, ref, signal)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -197,22 +197,32 @@ export async function saveImageFile(root: string, input: SaveImageAttachment, li
|
||||
* Read and verify one content-addressed image.
|
||||
* @param root - absolute `DSH_HOME/attachments/v1` root.
|
||||
* @param ref - reference recorded in the session log.
|
||||
* @param signal - optional cancellation for filesystem and verification work.
|
||||
* @returns verified bytes and reference.
|
||||
* @throws the signal reason when aborted, or an AttachmentError when verification fails.
|
||||
*/
|
||||
export async function readImageFile(root: string, ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
|
||||
export async function readImageFile(
|
||||
root: string,
|
||||
ref: ImageAttachmentRef,
|
||||
signal?: AbortSignal,
|
||||
): Promise<StoredImageAttachment> {
|
||||
signal?.throwIfAborted()
|
||||
const sha256 = ensureReference(ref)
|
||||
let data: Uint8Array
|
||||
try {
|
||||
data = new Uint8Array(await readFile(objectPath(root, sha256)))
|
||||
data = new Uint8Array(await readFile(objectPath(root, sha256), { signal }))
|
||||
} catch (error) {
|
||||
signal?.throwIfAborted()
|
||||
if (error instanceof Error && 'code' in error && error.code === 'ENOENT') throw new AttachmentError('Attachment object is missing.', 'ATTACHMENT_NOT_FOUND')
|
||||
throw new AttachmentError('Unable to read image attachment.', 'ATTACHMENT_READ_FAILED', { cause: error })
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
if (digest(data) !== sha256) throw new AttachmentError('Stored attachment failed integrity verification.', 'ATTACHMENT_CORRUPT')
|
||||
// The digest proves these are the exact bytes admission fully decoded, so
|
||||
// the read path only re-derives the header fields (no raster decode, no
|
||||
// per-request pixel amplification on history replay).
|
||||
const metadata = await probeImage(data)
|
||||
signal?.throwIfAborted()
|
||||
if (metadata.mediaType !== ref.mediaType || data.byteLength !== ref.bytes
|
||||
|| metadata.width !== ref.width || metadata.height !== ref.height) {
|
||||
throw new AttachmentError('Stored attachment metadata does not match its reference.', 'ATTACHMENT_CORRUPT')
|
||||
|
||||
@@ -9,12 +9,23 @@ import sharp from 'sharp'
|
||||
import type { ImageAttachmentLimits } from '@deepseek-ai/dsh-attachment'
|
||||
import { readImageFile, saveImageFile } from '../src/store.ts'
|
||||
|
||||
const fsControl = vi.hoisted(() => ({ syncedDirectories: [] as string[] }))
|
||||
const fsControl = vi.hoisted(() => ({
|
||||
readSignals: [] as AbortSignal[],
|
||||
syncedDirectories: [] as string[],
|
||||
}))
|
||||
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
readFile(...args: Parameters<typeof actual.readFile>): ReturnType<typeof actual.readFile> {
|
||||
const options = args[1]
|
||||
if (typeof options === 'object' && options !== null) {
|
||||
const signal = (options as { signal?: AbortSignal }).signal
|
||||
if (signal !== undefined) fsControl.readSignals.push(signal)
|
||||
}
|
||||
return actual.readFile(...args)
|
||||
},
|
||||
async open(...args: Parameters<typeof actual.open>): ReturnType<typeof actual.open> {
|
||||
if (args[1] === constants.O_RDONLY) fsControl.syncedDirectories.push(String(args[0]))
|
||||
return actual.open(...args)
|
||||
@@ -130,6 +141,20 @@ describe('local attachment store', () => {
|
||||
await expect(readImageFile(storageRoot, ref)).resolves.toEqual({ ref, data: PNG })
|
||||
})
|
||||
|
||||
it('forwards read cancellation to the filesystem and preserves its reason', async () => {
|
||||
const storageRoot = await root()
|
||||
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
|
||||
const controller = new AbortController()
|
||||
fsControl.readSignals.length = 0
|
||||
|
||||
await expect(readImageFile(storageRoot, ref, controller.signal)).resolves.toEqual({ ref, data: PNG })
|
||||
expect(fsControl.readSignals).toEqual([controller.signal])
|
||||
|
||||
const cancellation = new Error('attachment read cancelled')
|
||||
controller.abort(cancellation)
|
||||
await expect(readImageFile(storageRoot, ref, controller.signal)).rejects.toBe(cancellation)
|
||||
})
|
||||
|
||||
it('rejects malformed bytes, mismatched declarations, byte limits, and decoded-pixel limits', async () => {
|
||||
const storageRoot = await root()
|
||||
await expect(saveImageFile(storageRoot, {
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
|
||||
README.md: 4f450316294e554396adb9a8454051a08d9befd3
|
||||
README.zh.md: fe51b0003cdf1659c7c56106b97c6f3139ebe890
|
||||
README.md: baeeca0cf939f1a3d4608769b362d532507b90f5
|
||||
README.zh.md: 238b90794c510e71fffe34d62b044a5c2ece8a6e
|
||||
|
||||
@@ -4,7 +4,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
The durable attachment seam. `ctx.attachments` validates and atomically commits immutable image bytes, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, or base64 in session events.
|
||||
|
||||
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the same admission policy without persisting; batch writers validate every member first so a malformed member cannot strand earlier members as unreferenced objects. `saveImage` commits each accepted image before any model-visible session event is published, and `readImage` verifies the content-addressed object against its logged metadata.
|
||||
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the same admission policy without persisting; batch writers validate every member first so a malformed member cannot strand earlier members as unreferenced objects. `saveImage` commits each accepted image before any model-visible session event is published, and `readImage` verifies the content-addressed object against its logged metadata. Callers may cancel `readImage`; implementations observe cancellation around backend and verification work and preserve it instead of translating it into a storage failure.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
持久附件服务边界。`ctx.attachments` 校验并以原子方式提交不可变图片字节,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL 或 base64。
|
||||
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行相同的准入策略,但不执行持久化;批量写入方会先校验每个成员,避免某个格式错误的成员使较早的成员成为无引用对象。`saveImage` 会在发布任何模型可见的会话事件前提交每张已接受的图片,`readImage` 则根据已记录的元数据校验内容寻址对象。
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行相同的准入策略,但不执行持久化;批量写入方会先校验每个成员,避免某个格式错误的成员使较早的成员成为无引用对象。`saveImage` 会在发布任何模型可见的会话事件前提交每张已接受的图片,`readImage` 则根据已记录的元数据校验内容寻址对象。调用方可以取消 `readImage`;实现会在后端读取与校验工作的边界观察取消,并保留取消语义,而不会将其转换为存储失败。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -52,9 +52,11 @@ export abstract class AttachmentStore extends Service {
|
||||
/**
|
||||
* Read one image and verify that bytes still match the recorded reference.
|
||||
* @param ref - durable reference from the session log.
|
||||
* @param signal - optional cancellation for backend read and verification work.
|
||||
* @returns the verified bytes and canonical reference.
|
||||
* @throws the signal reason when aborted, or a storage error when verification fails.
|
||||
*/
|
||||
abstract readImage(ref: ImageAttachmentRef): Promise<StoredImageAttachment>
|
||||
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
|
||||
}
|
||||
|
||||
export default AttachmentStore
|
||||
|
||||
Reference in New Issue
Block a user