Merge remote-tracking branch 'origin/master' into feat/md-incre-2

Port the two master-side markdown fixes onto the mdast renderer:
cjkFriendlyStrong joins both parse.ts grammars as a micromark extension
(ex remark plugin), and inline-code HTTP(S) URLs gain the safe anchor in
render.tsx without normalizeUri. Re-record the two new web-e2e goldens
whose footer clock spacing diverged from the nine established goldens,
and drop the timing-flaky LLM duration segment one of them captured.
This commit is contained in:
07akioni
2026-08-06 14:37:56 +08:00
430 changed files with 11455 additions and 2427 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/client/ui-primitives/README.md
README.md: 04fb50eedb59f1028145b4985a0cb7560d388492
README.zh.md: 753905915f6ac501ade9e8473ad747c1356e1556
README.md: 385730c94831d2fd4af83f9eca0f55941551c796
README.zh.md: b8a75dbffc6549f6294dfda5988c67d6569386c9

View File

@@ -10,7 +10,7 @@ Pure React atoms (zero cordis): StateDot, ic_ds_* icons, Button/Pill/Menu/Modal/
## Markdown rendering
`MarkdownText` renders GFM and `$…$`, `$$…$$`, `\(…\)`, and `\[…\]` TeX math from untrusted assistant output through React elements, with math typeset by KaTeX and trusted commands disabled; block-level same-line `$$…$$` is display math, including `\tag{}`. It omits raw HTML, neutralizes relative and non-HTTP(S)/mailto links, opens HTTP(S) links with safe external-link attributes, and renders absolute HTTP(S) images without a referrer; relative paths, absolute local paths, `file:` URLs, and unsupported schemes retain their alt text. While a reply streams, `MarkdownText` parses incrementally: all but the trailing two blocks freeze as cached React elements and only the source tail behind them re-parses per chunk, so per-chunk work tracks the tail instead of the whole reply ([mechanism and DOM-parity contract](../../../.agents/notes/implemented/architecture/2026-08-06-web-markdown-incremental-ast-renderer.md)). `MessageText` remains the literal-text primitive for user-authored content. `extractMarkdownPlainText` removes Markdown presentation markup for compact labels while preserving raw HTML as literal text. Element spacing, responsive images, tables, links, and inline code use the same `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` tokens as deepsuite `@deepseek/md`. Fenced blocks render through `CodeBlock` (language banner, copy control, shiki for the registered grammars).
`MarkdownText` renders GFM and `$…$`, `$$…$$`, `\(…\)`, and `\[…\]` TeX math from untrusted assistant output through React elements, with math typeset by KaTeX and trusted commands disabled; block-level same-line `$$…$$` is display math, including `\tag{}`. A narrow micromark extension lets asterisk strong emphasis ending in punctuation close before adjacent CJK text, where prose normally omits the whitespace CommonMark requires; single-asterisk emphasis, non-CJK adjacency, escapes, code, and math retain upstream parsing. It omits raw HTML, neutralizes relative and non-HTTP(S)/mailto links, opens HTTP(S) links with safe external-link attributes, and renders absolute HTTP(S) images without a referrer; relative paths, absolute local paths, `file:` URLs, and unsupported schemes retain their alt text. Inline code whose complete value is an absolute HTTP(S) URL keeps its code styling and gains the same safe external anchor; commands, partial URLs, other schemes, and fenced code remain inert. While a reply streams, `MarkdownText` parses incrementally: all but the trailing two blocks freeze as cached React elements and only the source tail behind them re-parses per chunk, so per-chunk work tracks the tail instead of the whole reply ([mechanism and DOM-parity contract](../../../.agents/notes/implemented/architecture/2026-08-06-web-markdown-incremental-ast-renderer.md)). `MessageText` remains the literal-text primitive for user-authored content. `extractMarkdownPlainText` removes Markdown presentation markup for compact labels while preserving raw HTML as literal text. Element spacing, responsive images, tables, links, and inline code use the same `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` tokens as deepsuite `@deepseek/md`. Fenced blocks render through `CodeBlock` (language banner, copy control, shiki for the registered grammars).
## Terminal output

View File

@@ -10,7 +10,7 @@
## Markdown 渲染
`MarkdownText` 通过 React 元素渲染来自不受信任 assistant 输出的 GFM 与 `$…$`、`$$…$$`、`\(…\)` 和 `\[…\]` TeX 公式,公式由 KaTeX 排版并禁用受信任命令;块级同一行 `$$…$$` 是显示公式并支持 `\tag{}`。它会省略原始 HTML,使相对链接及非 HTTP(S)/mailto 链接失效,以安全的外部链接属性打开 HTTP(S) 链接,并在不发送 referrer 的情况下渲染采用绝对 HTTP(S) URL 的图片;相对路径、绝对本地路径、`file:` URL 与不受支持的 scheme 会保留其 alt 文本。回复流式输出期间,`MarkdownText` 增量解析:除末尾两个块外全部冻结为缓存的 React 元素,每个分片只重新解析其后的源文本尾部,因此每分片的工作量跟随尾部而非整个回复([机制与 DOM 一致性契约](../../../.agents/notes/implemented/architecture/2026-08-06-web-markdown-incremental-ast-renderer.md))。`MessageText` 仍是用户创作内容使用的字面文本原语。`extractMarkdownPlainText` 会移除 Markdown 呈现标记以用于紧凑标签,同时将原始 HTML 保留为字面文本。元素间距、响应式图片、表格、链接与行内代码使用与 deepsuite `@deepseek/md` 相同的 `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` token。围栏代码块通过 `CodeBlock` 渲染(语言横幅、复制控件,以及对已注册语法使用 shiki)。
`MarkdownText` 通过 React 元素渲染来自不受信任 assistant 输出的 GFM 与 `$…$`、`$$…$$`、`\(…\)` 和 `\[…\]` TeX 公式,公式由 KaTeX 排版并禁用受信任命令;块级同一行 `$$…$$` 是显示公式并支持 `\tag{}`。一个小范围的 micromark 扩展允许由星号标记、以标点结尾的粗体在紧邻的 CJK 文本前闭合,以适应 CJK 文本通常省略 CommonMark 所要求空格的写法;单星号强调、紧邻非 CJK 文本的情况、转义、代码与数学公式仍沿用上游解析行为。它会省略原始 HTML,使相对链接及非 HTTP(S)/mailto 链接失效,以安全的外部链接属性打开 HTTP(S) 链接,并在不发送 referrer 的情况下渲染采用绝对 HTTP(S) URL 的图片;相对路径、绝对本地路径、`file:` URL 与不受支持的 scheme 会保留其 alt 文本。完整内容为绝对 HTTP(S) URL 的行内代码会保留代码样式,并获得同样安全的外部链接;命令、非完整 URL、其他 scheme 与围栏代码仍不会成为链接。回复流式输出期间,`MarkdownText` 增量解析:除末尾两个块外全部冻结为缓存的 React 元素,每个分片只重新解析其后的源文本尾部,因此每分片的工作量跟随尾部而非整个回复([机制与 DOM 一致性契约](../../../.agents/notes/implemented/architecture/2026-08-06-web-markdown-incremental-ast-renderer.md))。`MessageText` 仍是用户创作内容使用的字面文本原语。`extractMarkdownPlainText` 会移除 Markdown 呈现标记以用于紧凑标签,同时将原始 HTML 保留为字面文本。元素间距、响应式图片、表格、链接与行内代码使用与 deepsuite `@deepseek/md` 相同的 `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` token。围栏代码块通过 `CodeBlock` 渲染(语言横幅、复制控件,以及对已注册语法使用 shiki)。
## 终端输出

View File

@@ -28,10 +28,12 @@
"mdast-util-from-markdown": "^2.0.3",
"mdast-util-gfm": "^3.1.0",
"mdast-util-math": "^3.0.0",
"micromark-core-commonmark": "^2.0.3",
"micromark-extension-gfm": "^3.0.0",
"micromark-extension-math": "^3.1.0",
"micromark-factory-space": "^2.0.1",
"micromark-util-character": "^2.1.1",
"micromark-util-classify-character": "^2.0.1",
"micromark-util-sanitize-uri": "^2.0.1",
"micromark-util-symbol": "^2.0.1",
"micromark-util-types": "^2.0.2",

View File

@@ -0,0 +1,83 @@
/** Let asterisk strong emphasis close after punctuation when CJK prose continues without whitespace. */
import { attention } from 'micromark-core-commonmark'
import { unicodePunctuation } from 'micromark-util-character'
import { classifyCharacter } from 'micromark-util-classify-character'
import { codes, constants } from 'micromark-util-symbol'
import type { Construct, Extension, State, Tokenizer } from 'micromark-util-types'
const cjkCharacter = new RegExp([
'\\p{Script_Extensions=Han}',
'\\p{Script_Extensions=Hiragana}',
'\\p{Script_Extensions=Katakana}',
'\\p{Script_Extensions=Hangul}',
'\\p{Script_Extensions=Bopomofo}',
].join('|'), 'u')
function isCjkCharacter(code: number | null): boolean {
return code !== null && code >= 0 && cjkCharacter.test(String.fromCodePoint(code))
}
const tokenizeCjkFriendlyAttention: Tokenizer = function (effects, ok, nok) {
const configuredAttentionMarkers = this.parser.constructs.attentionMarkers.null
if (configuredAttentionMarkers === undefined) {
throw new Error('micromark CommonMark attention markers are unavailable')
}
const attentionMarkers = configuredAttentionMarkers
const previous = this.previous
const before = classifyCharacter(previous)
let marker: number | null = codes.eof
return start
function start(code: number | null): State | undefined {
/* v8 ignore next -- this text construct is dispatched only for an asterisk. */
if (code !== codes.asterisk) return nok(code)
marker = code
effects.enter('attentionSequence')
return inside(code)
}
function inside(code: number | null): State | undefined {
if (code === marker) {
effects.consume(code)
return inside
}
const token = effects.exit('attentionSequence')
const after = classifyCharacter(code)
const open = !after || (after === constants.characterGroupPunctuation && Boolean(before))
|| attentionMarkers.includes(code)
const commonMarkClose = !before
|| (before === constants.characterGroupPunctuation && Boolean(after))
|| attentionMarkers.includes(previous)
const markerCount = token.end.offset - token.start.offset
const cjkStrongClose = markerCount >= 2
&& unicodePunctuation(previous)
&& isCjkCharacter(code)
const close = commonMarkClose || cjkStrongClose
token._open = open
token._close = close
return ok(code)
}
}
const cjkFriendlyAttention: Construct = {
name: 'cjkFriendlyAttention',
resolveAll: attention.resolveAll,
tokenize: tokenizeCjkFriendlyAttention,
}
const cjkFriendlyStrongExtension: Extension = {
text: { [codes.asterisk]: cjkFriendlyAttention },
}
/**
* Extend CommonMark asterisk strong emphasis for punctuation-delimited CJK
* prose, as a micromark syntax extension for `fromMarkdown`.
* @returns The micromark syntax extension.
*/
export function cjkFriendlyStrong(): Extension {
return cjkFriendlyStrongExtension
}

View File

@@ -12,6 +12,7 @@ import { gfmFromMarkdown } from 'mdast-util-gfm'
import { mathFromMarkdown } from 'mdast-util-math'
import { gfm } from 'micromark-extension-gfm'
import { math } from 'micromark-extension-math'
import { cjkFriendlyStrong } from './cjkFriendlyStrong.ts'
import { mathCompatibility } from './mathCompatibility.ts'
/**
@@ -22,7 +23,7 @@ import { mathCompatibility } from './mathCompatibility.ts'
*/
export function parseGfm(text: string): Root {
return fromMarkdown(text, {
extensions: [gfm()],
extensions: [gfm(), cjkFriendlyStrong()],
mdastExtensions: [gfmFromMarkdown()],
})
}
@@ -35,7 +36,7 @@ export function parseGfm(text: string): Root {
*/
export function parseGfmWithMath(text: string): Root {
return fromMarkdown(text, {
extensions: [gfm(), mathCompatibility(), math()],
extensions: [gfm(), cjkFriendlyStrong(), mathCompatibility(), math()],
mdastExtensions: [gfmFromMarkdown(), mathFromMarkdown()],
})
}

View File

@@ -208,9 +208,17 @@ function renderNode(node: Md.RootContent, key: Key, context: MarkdownRenderConte
return <em key={key}>{renderChildren(node.children, context)}</em>
case 'delete':
return <del key={key}>{renderChildren(node.children, context)}</del>
case 'inlineCode':
case 'inlineCode': {
// Parity with mdast-util-to-hast: inline code renders line endings as spaces.
return <code key={key}>{node.value.replace(/\r?\n|\r/g, ' ')}</code>
const value = node.value.replace(/\r?\n|\r/g, ' ')
// An inline-code token that is entirely an absolute HTTP(S) URL keeps
// its code chrome and gains the same safe external anchor as a link;
// commands, partial URLs, and other schemes stay inert. The value is
// authored text, not a parsed destination, so no normalizeUri: port,
// path, and query render unchanged.
const href = inlineCodeHttpUrl(value)
return <code key={key}>{href === undefined ? value : renderSafeLink(href, [value], 'link')}</code>
}
case 'html':
// No HTML parser enters the pipeline: raw HTML stays literal text.
return node.value
@@ -382,8 +390,9 @@ function renderTableRow(
return <tr key={key}>{cells}</tr>
}
function renderAnchor(url: string, children: ReactNode[], key: Key): ReactNode {
const safeHref = sanitizeUrl(normalizeUri(url))
/** Anchor over an already-authored href: allowlisted or unwrapped, external links get the safe attributes. */
function renderSafeLink(href: string, children: ReactNode[], key: Key): ReactNode {
const safeHref = sanitizeUrl(href)
if (safeHref === '') return <Fragment key={key}>{children}</Fragment>
const external = ['http:', 'https:'].includes(new URL(safeHref).protocol)
return (
@@ -397,6 +406,26 @@ function renderAnchor(url: string, children: ReactNode[], key: Key): ReactNode {
)
}
/** Anchor over a parsed markdown destination, which hast normalized before the allowlist saw it. */
function renderAnchor(url: string, children: ReactNode[], key: Key): ReactNode {
return renderSafeLink(normalizeUri(url), children, key)
}
/**
* The complete inline-code value when it is exactly an absolute HTTP(S) URL
* (no surrounding whitespace); anything else stays inert code.
*/
function inlineCodeHttpUrl(value: string): string | undefined {
if (value.trim() !== value) return undefined
try {
const protocol = new URL(value).protocol
return protocol === 'http:' || protocol === 'https:' ? value : undefined
} catch {
// Not an absolute URL at all — the only way new URL() rejects a string.
return undefined
}
}
function renderImage(url: string, alt: string, key: Key): ReactNode {
const imageSrc = remoteImageUrl(sanitizeUrl(normalizeUri(url)))
if (imageSrc === undefined) {

View File

@@ -0,0 +1,20 @@
<div class="_markdown_404681">
<p>
<strong>
#text "注意:"
#text "内容在标点后直接闭合。"
<p>
#text "**Notice:**text keeps upstream parsing."
<p>
#text "*提醒!*单星号也保持上游行为。"
<p>
<code>
<a href="https://example.com/preview?q=one%20two#result" rel="noopener noreferrer" target="_blank">
#text "https://example.com/preview?q=one%20two#result"
#text " 与 "
<code>
#text "curl http://127.0.0.1:3199/"
#text " 以及 "
<code>
#text "javascript:alert(1)"
#text "。"

View File

@@ -0,0 +1,20 @@
<div class="_markdown_404681">
<p>
<strong>
#text "注意:"
#text "内容在标点后直接闭合。"
<p>
#text "**Notice:**text keeps upstream parsing."
<p>
#text "*提醒!*单星号也保持上游行为。"
<p>
<code>
<a href="https://example.com/preview?q=one%20two#result" rel="noopener noreferrer" target="_blank">
#text "https://example.com/preview?q=one%20two#result"
#text " 与 "
<code>
#text "curl http://127.0.0.1:3199/"
#text " 以及 "
<code>
#text "javascript:alert(1)"
#text "。"

View File

@@ -216,6 +216,15 @@ const CORPUS: Record<string, string> = {
'```',
].join('\n'),
'gfm-strikethrough-and-literals': 'Mixed ~~gone~~ text with www.example.com literal and user@example.com email.',
'cjk-strong-and-inline-code-url': [
'**注意:**内容在标点后直接闭合。',
'',
'**Notice:**text keeps upstream parsing.',
'',
'*提醒!*单星号也保持上游行为。',
'',
'`https://example.com/preview?q=one%20two#result` 与 `curl http://127.0.0.1:3199/` 以及 `javascript:alert(1)`。',
].join('\n'),
'definition-only': '[unused]: https://example.com/unused',
'streaming-typical-partial': '## Streaming\n\n- first\n- **unfinished',
}

View File

@@ -2,6 +2,7 @@
import { cleanup, fireEvent, render, screen } from '@testing-library/react'
import { afterEach, describe, expect, it } from 'vitest'
import { JsonBlock, MarkdownText, MessageText } from '@deepseek-ai/dsh-client-ui-primitives'
import { cjkFriendlyStrong } from '../src/markdown/cjkFriendlyStrong.ts'
import { mathCompatibility } from '../src/markdown/mathCompatibility.ts'
afterEach(cleanup)
@@ -67,6 +68,100 @@ describe('MarkdownText', () => {
expect(screen.getByRole('link', { name: 'https://deepseek.com' })).toBeTruthy()
})
it('closes punctuation-terminated strong emphasis before adjacent CJK text', () => {
const cases = [
['**注意:**内容', '注意:'],
['**Notice:**内容', 'Notice:'],
['**事件中间件(waterfall)**实现', '事件中间件(waterfall)'],
['**事件中间件(waterfall)**实现', '事件中间件(waterfall)'],
['**句号。**后续', '句号。'],
['**Period.**后续', 'Period.'],
['**提醒!**继续', '提醒!'],
['**Warning!**继续', 'Warning!'],
] as const
const source = cases.map(([markdown]) => markdown).join('\n\n')
for (const streaming of [false, true]) {
const rendered = render(<MarkdownText text={source} streaming={streaming} />)
expect([...rendered.container.querySelectorAll('strong')].map(node => node.textContent))
.toEqual(cases.map(([, strong]) => strong))
rendered.unmount()
}
})
it('keeps the CJK strong extension out of escaped, code, math, and ASCII contexts', () => {
const source = [
String.raw`\**注意:**内容`,
'`**注意:**内容`',
'**Notice:**text',
'*提醒!*继续',
'$**注意:**内容$',
'```md',
'**注意:**内容',
'```',
'**普通**内容',
'*普通*内容',
].join('\n\n')
const { container } = render(<MarkdownText text={source} />)
expect([...container.querySelectorAll('strong')].map(node => node.textContent)).toEqual(['普通'])
expect([...container.querySelectorAll('em')].map(node => node.textContent)).toEqual(['普通'])
expect(container.querySelector('code')?.textContent).toBe('**注意:**内容')
expect(container.querySelector('.katex annotation')?.textContent).toBe('**注意:**内容')
expect(container.querySelector('pre code')?.textContent).toContain('**注意:**内容')
expect(container.textContent).toContain('**Notice:**text')
expect(container.textContent).toContain('*提醒!*继续')
expect(container.textContent).toContain('**注意:**内容')
})
it('links complete HTTP(S) inline code without promoting commands, unsafe schemes, or fences', () => {
const localUrl = 'http://127.0.0.1:3199/?demo=1'
const remoteUrl = 'https://example.com/preview?q=one%20two#result'
const source = [
`\`${localUrl}\``,
`\`${remoteUrl}\``,
'`curl http://127.0.0.1:3199/?demo=1`',
'`javascript:alert(1)`',
'`mailto:dev@example.com`',
`\` ${localUrl} \``,
'```',
localUrl,
'```',
].join('\n\n')
const { container } = render(<MarkdownText text={source} />)
const links = screen.getAllByRole('link')
expect(links.map(link => link.getAttribute('href'))).toEqual([localUrl, remoteUrl])
for (const link of links) {
expect(link.closest('code')).not.toBeNull()
expect(link.getAttribute('target')).toBe('_blank')
expect(link.getAttribute('rel')).toBe('noopener noreferrer')
}
links[0]?.focus()
expect(document.activeElement).toBe(links[0])
expect(screen.getByText('curl http://127.0.0.1:3199/?demo=1').closest('a')).toBeNull()
expect(screen.getByText('javascript:alert(1)').closest('a')).toBeNull()
expect(screen.getByText('mailto:dev@example.com').closest('a')).toBeNull()
const paddedCode = [...container.querySelectorAll('code')]
.find(code => code.textContent === ` ${localUrl} `)
expect(paddedCode?.querySelector('a')).toBeNull()
expect(container.querySelector('pre code a')).toBeNull()
})
it('exposes the CJK strong syntax as a micromark extension needing CommonMark attention markers', () => {
const extension = cjkFriendlyStrong()
expect(cjkFriendlyStrong()).toBe(extension)
const construct = extension.text?.[42]
const tokenizer = Array.isArray(construct) ? construct[0]?.tokenize : construct?.tokenize
expect(tokenizer).toBeTypeOf('function')
expect(() => tokenizer?.call({
parser: { constructs: { attentionMarkers: {} } },
previous: null,
} as never, {} as never, () => undefined, () => undefined)).toThrow(
'micromark CommonMark attention markers are unavailable',
)
})
it('a fence labeled with an inherited object key renders plain, never crashing shiki', () => {
for (const label of ['constructor', '__proto__', 'toString', 'hasOwnProperty']) {
const { container, unmount } = render(<MarkdownText text={'```' + label + '\ncode body\n```'} />)