fix(web): keep the approval scenario's goldens platform-neutral
The answered-transcript golden captured the OS's own refusal of the denied first attempt — "bash: notes.txt: Operation not permitted" on macOS against "bash: line 1: notes.txt: Read-only file system" on Linux — so it passed locally and failed the Linux snapshot lane. The scenario now keeps one golden (the waiting panel, platform-neutral) and asserts the answered state on the world instead: the decided outcome, the file the escalated command actually wrote, DONE, the panel gone, and the composer re-enabled. The file assertion is stronger evidence than the transcript dump it replaces — it proves the grant reached the executor.
This commit is contained in:
@@ -29,10 +29,9 @@ import { connectFreshWorkspace, newEnglishPage, saveFailureShot } from './suppor
|
||||
|
||||
const SNAPSHOT_DIR = fileURLToPath(new URL('./snapshots/approval-composer', import.meta.url))
|
||||
const FIXTURE = join(SNAPSHOT_DIR, 'session.jsonl')
|
||||
// The scenario's one golden: the waiting panel. Everything the answered state
|
||||
// proves is asserted directly — see the world-state block at the end.
|
||||
const UI_EXPECTED = join(SNAPSHOT_DIR, 'ui.expected.md')
|
||||
// Second golden: the answered transcript — the granted escalation ran and the
|
||||
// turn finished, the state the waiting golden cannot see.
|
||||
const ANSWERED_EXPECTED = join(SNAPSHOT_DIR, 'answered.expected.md')
|
||||
const MODE = webSnapshotMode()
|
||||
|
||||
// Irreducible payload: the command has to be long enough to pass the card's
|
||||
@@ -160,19 +159,23 @@ describe('web e2e: approval takeover keeps its actions reachable', () => {
|
||||
return
|
||||
}
|
||||
// World state: the granted escalation is what let the command run, and the
|
||||
// panel leaves with the regular composer restored.
|
||||
// panel leaves with the regular composer restored. Asserted on the world
|
||||
// and the DOM rather than through a transcript golden — the denied first
|
||||
// attempt renders the OS's own refusal ("Operation not permitted" on
|
||||
// macOS, "Read-only file system" on Linux), so the answered transcript is
|
||||
// not a platform-neutral golden surface.
|
||||
expect(JSON.stringify(sessionEvents.filter(e => e.type === 'approval/decided').at(-1)))
|
||||
.toContain('allowed-once')
|
||||
const written = await readFile(join(scaffold.workspaceCwd, 'workspace', 'notes.txt'), 'utf8')
|
||||
expect(written).toContain(TOKENS.slice(0, 64))
|
||||
await expect.poll(() => page.getByText('DONE', { exact: true }).count(), { timeout: 20_000 }).toBeGreaterThanOrEqual(1)
|
||||
expect(await page.locator('[data-approval-key]').count()).toBe(0)
|
||||
await expect.poll(() => page.locator('textarea').first().isEnabled(), { timeout: 10_000 }).toBe(true)
|
||||
const answered = await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd)
|
||||
await compareOrRefreshGolden(ANSWERED_EXPECTED, answered, MODE)
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
}, 300_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('keeps the fixture inventory closed', async () => {
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, ['session.jsonl', 'ui.expected.md', 'answered.expected.md'])
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, ['session.jsonl', 'ui.expected.md'])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
- banner:
|
||||
- navigation "Session hierarchy":
|
||||
- button "Write a file named notes.txt" [disabled]
|
||||
- tablist:
|
||||
- tab "Chat" [selected]
|
||||
- tab "Trajectory"
|
||||
- img
|
||||
- text: "/permission read-only Permission preset: read-only. Write a file named notes.txt in the workspace containing exactly this text on one line: tok63z tokc7y tokibx tokofw tokujv tok10nu tok16rt tok1cvs tok1izr tok1p3q tok1v7p tok21bo tok2a4 tok8e3 tokei2 tokkm1 tokqq0 tokwtz tok12xy tok191x tok1f5w tok1l9v tok1rdu tok1xht tok23ls tok4k8 tokao7 tokgs6 tokmw5 tokt04 tokz43 tok1582 tok1bc1 tok1hg0 tok1njz tok1tny tok1zrx tokqd tok6uc tokcyb tokj2a tokp69 tokva8 tok11e7 tok17i6 tok1dm5 tok1jq4 tok1pu3 tok1vy2 tok2221 tok30h tok94g tokf8f toklce tokrgd tokxkc tok13ob tok19sa tok1fw9 tok1m08 tok1s47 tok1y86 tok24c5 tok5al tokbek tokhij toknmi toktqh tokzug tok15yf tok1c2e tok1i6d tok1oac tok1ueb tok20ia tok1gq tok7kp tokdoo tokjsn tokpwm tokw0l tok124k tok188j tok1eci tok1kgh tok1qkg tok1wof tok22se tok3qu tok9ut tokfys tokm2r toks6q tokyap tok14eo tok1ain tok1gmm tok1mql tok1suk tok1yyj tok252i tok60y tokc4x toki8w tokocv tokugu tok10kt tok16os tok1csr tok1iwq tok1p0p tok1v4o tok218n tok273 tok8b2 tokef1 tokkj0 tokqmz tokwqy tok12ux tok18yw tok1f2v tok1l6u tok1rat tok1xes tok23ir tok4h7 tokal6 tokgp5 tokmt4 toksx3 tokz12 tok1551 tok1b90 tok1hcz tok1ngy tok1tkx tok1zow toknc tok6rb tokcva tokiz9 tokp38 tokv77 tok11b6 tok17f5 tok1dj4 tok1jn3 tok1pr2 tok1vv1 tok21z0 tok2xg tok91f tokf5e tokl9d tokrdc tokxhb tok13la tok19p9 tok1ft8 tok1lx7 tok1s16 tok1y55 tok2494 tok57k tokbbj tokhfi toknjh toktng tokzrf tok15ve tok1bzd tok1i3c tok1o7b tok1uba tok20f9 tok1dp tok7ho tokdln tokjpm tokptl tokvxk tok121j tok185i tok1e9h tok1kdg tok1qhf tok1wle tok22pd tok3nt tok9rs tokfvr toklzq toks3p toky7o tok14bn tok1afm tok1gjl tok1mnk tok1srj tok1yvi tok24zh tok5xx tokc1w toki5v toko9u tokudt tok10hs tok16lr tok1cpq tok1itp tok1oxo tok1v1n tok215m tok242 tok881 tokec0 tokkfz tokqjy tokwnx. Use one bash command with the literal text inline. Then reply with the single word DONE and stop. {{clock}}"
|
||||
- button "复制":
|
||||
- img
|
||||
- button "在新对话中分支":
|
||||
- img
|
||||
- button "编辑":
|
||||
- img
|
||||
- button "Think The user wants me to write a file named notes.txt with a specific line of text. Let me do this with a single bash command using echo.":
|
||||
- img
|
||||
- img
|
||||
- text: Think The user wants me to write a file named notes.txt with a specific line of text. Let me do this with a single bash command using echo.
|
||||
- img
|
||||
- text: Bash Write notes.txt with the specified text 失败 workspace echo 'tok63z tokc7y tokibx tokofw tokujv tok10nu tok16rt tok1cvs tok1izr tok1p3q tok1v7p tok21bo tok2a4 tok8e3 tokei2 tokkm1 tokqq0 tokwtz tok12xy tok191x tok1f5w tok1l9v tok1rdu tok1xht tok23ls tok4k8 tokao7 tokgs6 tokmw5 tokt04 tokz43 tok1582 tok1bc1 tok1hg0 tok1njz tok1tny tok1zrx tokqd tok6uc tokcyb tokj2a tokp69 tokva8 tok11e7 tok17i6 tok1dm5 tok1jq4 tok1pu3 tok1vy2 tok2221 tok30h tok94g tokf8f toklce tokrgd tokxkc tok13ob tok19sa tok1fw9 tok1m08 tok1s47 tok1y86 tok24c5 tok5al tokbek tokhij toknmi toktqh tokzug tok15yf tok1c2e tok1i6d tok1oac tok1ueb tok20ia tok1gq tok7kp tokdoo tokjsn tokpwm tokw0l tok124k tok188j tok1eci tok1kgh tok1qkg tok1wof tok22se tok3qu tok9ut tokfys tokm2r toks6q tokyap tok14eo tok1ain tok1gmm tok1mql tok1suk tok1yyj tok252i tok60y tokc4x toki8w tokocv tokugu tok10kt tok16os tok1csr tok1iwq tok1p0p tok1v4o tok218n tok273 tok8b2 tokef1 tokkj0 tokqmz tokwqy tok12ux tok18yw tok1f2v tok1l6u tok1rat tok1xes tok23ir tok4h7 tokal6 tokgp5 tokmt4 toksx3 tokz12 tok1551 tok1b90 tok1hcz tok1ngy tok1tkx tok1zow toknc tok6rb tokcva tokiz9 tokp38 tokv77 tok11b6 tok17f5 tok1dj4 tok1jn3 tok1pr2 tok1vv1 tok21z0 tok2xg tok91f tokf5e tokl9d tokrdc tokxhb tok13la tok19p9 tok1ft8 tok1lx7 tok1s16 tok1y55 tok2494 tok57k tokbbj tokhfi toknjh tokktng tokzrf tok15ve tok1bzd tok1i3c tok1o7b tok1uba tok20f9 tok1dp tok7ho tokdln tokjpm tokptl tokvxk tok121j tok185i tok1e9h tok1kdg tok1qhf tok1wle tok22pd tok3nt tok9rs tokfvr toklzq toks3p toky7o tok14bn tok1afm tok1gjl tok1mnk tok1srj tok1yvi tok24zh tok5xx tokc1w toki5v toko9u tokudt tok10hs tok16lr tok1cpq tok1itp tok1oxo tok1v1n tok215m tok242 tok881 tokec0 tokfz tokqjy tokwnx' > notes.txt 退出码 1
|
||||
- button "复制"
|
||||
- text: "[stderr] bash: notes.txt: Operation not permitted [sandbox: file access denied under read-only mode] [sandbox: escalation available — retry this exact command once with sandbox_permissions (the narrowest wider mode that suffices) + justification; the approval prompt asks the user]"
|
||||
- button "Think The sandbox denied the file write. I need to retry with sandbox_permissions set to \"workspace-write\" (the narrowest wider mode) and provide a justification.":
|
||||
- img
|
||||
- img
|
||||
- text: Think The sandbox denied the file write. I need to retry with sandbox_permissions set to "workspace-write" (the narrowest wider mode) and provide a justification.
|
||||
- img
|
||||
- text: Bash Write notes.txt with the specified text 已完成 workspace echo 'tok63z tokc7y tokibx tokofw tokujv tok10nu tok16rt tok1cvs tok1izr tok1p3q tok1v7p tok21bo tok2a4 tok8e3 tokei2 tokkm1 tokqq0 tokwtz tok12xy tok191x tok1f5w tok1l9v tok1rdu tok1xht tok23ls tok4k8 tokao7 tokgs6 tokmw5 tokt04 tokz43 tok1582 tok1bc1 tok1hg0 tok1njz tok1tny tok1zrx tokqd tok6uc tokcyb tokj2a tokp69 tokva8 tok11e7 tok17i6 tok1dm5 tok1jq4 tok1pu3 tok1vy2 tok2221 tok30h tok94g tokf8f toklce tokrgd tokxkc tok13ob tok19sa tok1fw9 tok1m08 tok1s47 tok1y86 tok24c5 tok5al tokbek tokhij toknmi toktqh tokzug tok15yf tok1c2e tok1i6d tok1oac tok1ueb tok20ia tok1gq tok7kp tokdoo tokjsn tokpwm tokw0l tok124k tok188j tok1eci tok1kgh tok1qkg tok1wof tok22se tok3qu tok9ut tokfys tokm2r toks6q tokyap tok14eo tok1ain tok1gmm tok1mql tok1suk tok1yyj tok252i tok60y tokc4x toki8w tokocv tokugu tok10kt tok16os tok1csr tok1iwq tok1p0p tok1v4o tok218n tok273 tok8b2 tokef1 tokkj0 tokqmz tokwqy tok12ux tok18yw tok1f2v tok1l6u tok1rat tok1xes tok23ir tok4h7 tokal6 tokgp5 tokmt4 toksx3 tokz12 tok1551 tok1b90 tok1hcz tok1ngy tok1tkx tok1zow toknc tok6rb tokcva tokiz9 tokp38 tokv77 tok11b6 tok17f5 tok1dj4 tok1jn3 tok1pr2 tok1vv1 tok21z0 tok2xg tok91f tokf5e tokl9d tokrdc tokxhb tok13la tok19p9 tok1ft8 tok1lx7 tok1s16 tok1y55 tok2494 tok57k tokbbj tokhfi toknjh tokktng tokzrf tok15ve tok1bzd tok1i3c tok1o7b tok1uba tok20f9 tok1dp tok7ho tokdln tokjpm tokptl tokvxk tok121j tok185i tok1e9h tok1kdg tok1qhf tok1wle tok22pd tok3nt tok9rs tokfvr toklzq toks3p toky7o tok14bn tok1afm tok1gjl tok1mnk tok1srj tok1yvi tok24zh tok5xx tokc1w toki5v toko9u tokudt tok10hs tok16lr tok1cpq tok1itp tok1oxo tok1v1n tok215m tok242 tok881 tokec0 tokfz tokqjy tokwnx' > notes.txt
|
||||
- button "复制"
|
||||
- text: (no output)
|
||||
- button "Think The file was written successfully. Let me verify it was created correctly.":
|
||||
- img
|
||||
- img
|
||||
- text: Think The file was written successfully. Let me verify it was created correctly.
|
||||
- img
|
||||
- text: Read
|
||||
- button "notes.txt"
|
||||
- button "Think The file was created successfully with the exact text on one line as requested. Now I need to reply with the single word DONE.":
|
||||
- img
|
||||
- img
|
||||
- text: Think The file was created successfully with the exact text on one line as requested. Now I need to reply with the single word DONE.
|
||||
- paragraph: DONE
|
||||
- button "复制":
|
||||
- img
|
||||
- button "在新对话中分支":
|
||||
- img
|
||||
- text: {{clock}}
|
||||
- textbox "Message the agent"
|
||||
- button "Add attachment":
|
||||
- img
|
||||
- 'button "Access mode, current: Read Only"': Read Only
|
||||
- button "Plan mode off, press to turn on": Plan off
|
||||
- button "Select model, current DeepSeek-V4-Flash":
|
||||
- text: DeepSeek-V4-Flash
|
||||
- img
|
||||
- button "Send message" [disabled]
|
||||
- text: 1 turns · 4 steps Tool call {{duration}} Cache hit 96% Input 27.4K tok · Output 1.9K tok
|
||||
Reference in New Issue
Block a user