build(web): replace Playwright MCP with project-scoped Playwright CLI
- feat: add `.agents/skills/playwright-cli/` with complete Skill markdown, agent interface, and 9 reference files covering session management, spec-driven testing, video recording, tracing, storage, request mocking, element attributes, test debugging, and running custom code - feat: add `.playwright/cli.config.json` to configure the Playwright CLI - feat: add `web/package.json` `playwright:cli` script that resolves `playwright cli` from the repo root - build: remove `@playwright/mcp` dependency from `web/package.json` - build: set `XDG_CACHE_HOME` in `mise.toml` - build: delete `.codex/config.toml` (MCP server config) - docs: update `README.md`, `docs/web/roadmap.md`, `docs/web/decisions.md`, and `web/README.md` to reflect the new Playwright CLI approach
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# Inspecting Element Attributes
|
||||
|
||||
When the snapshot doesn't show an element's `id`, `class`, `data-*` attributes, or other DOM properties, use `eval` to inspect them.
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
# snapshot shows a button as e7 but doesn't reveal its id or data attributes
|
||||
|
||||
# get the element's id
|
||||
mise exec -- npm run playwright:cli -- eval "el => el.id" e7
|
||||
|
||||
# get all CSS classes
|
||||
mise exec -- npm run playwright:cli -- eval "el => el.className" e7
|
||||
|
||||
# get a specific attribute
|
||||
mise exec -- npm run playwright:cli -- eval "el => el.getAttribute('data-testid')" e7
|
||||
mise exec -- npm run playwright:cli -- eval "el => el.getAttribute('aria-label')" e7
|
||||
|
||||
# get a computed style property
|
||||
mise exec -- npm run playwright:cli -- eval "el => getComputedStyle(el).display" e7
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
# Running Playwright Tests
|
||||
|
||||
Run Playwright tests from `web/` through the repository script. To avoid opening the interactive HTML report, use the `PLAYWRIGHT_HTML_OPEN=never` environment variable.
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e --
|
||||
```
|
||||
|
||||
# Debugging Playwright Tests
|
||||
|
||||
To debug a failing Playwright test, run it with `--debug=cli` option. This command will pause the test at the start and print the debugging instructions.
|
||||
|
||||
**IMPORTANT**: run the command in the background and check the output until "Debugging Instructions" is printed. Make sure to stop the command after you have finished.
|
||||
|
||||
Once instructions containing a session name are printed, use `mise exec -- npm run playwright:cli --` to attach the session and explore the page.
|
||||
|
||||
```bash
|
||||
# Run the test
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e -- --debug=cli
|
||||
# ...
|
||||
# ... debugging instructions for "tw-abcdef" session ...
|
||||
# ...
|
||||
|
||||
# Attach to the test
|
||||
mise exec -- npm run playwright:cli -- attach tw-abcdef
|
||||
```
|
||||
|
||||
Keep the test running in the background while you explore and look for a fix.
|
||||
The test is paused at the start, so you should step over or pause at a particular location
|
||||
where the problem is most likely to be.
|
||||
|
||||
Every action you perform with `mise exec -- npm run playwright:cli --` generates corresponding Playwright TypeScript code.
|
||||
This code appears in the output and can be copied directly into the test. Most of the time, a specific locator or an expectation should be updated, but it could also be a bug in the app. Use your judgement.
|
||||
|
||||
After fixing the test, stop the background test run. Rerun to check that test passes.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Request Mocking
|
||||
|
||||
Intercept, mock, modify, and block network requests.
|
||||
|
||||
## CLI Route Commands
|
||||
|
||||
```bash
|
||||
# Mock with custom status
|
||||
mise exec -- npm run playwright:cli -- route "**/*.jpg" --status=404
|
||||
|
||||
# Mock with JSON body
|
||||
mise exec -- npm run playwright:cli -- route "**/api/users" --body='[{"id":1,"name":"Alice"}]' --content-type=application/json
|
||||
|
||||
# Mock with custom headers
|
||||
mise exec -- npm run playwright:cli -- route "**/api/data" --body='{"ok":true}' --header="X-Custom: value"
|
||||
|
||||
# Remove headers from requests
|
||||
mise exec -- npm run playwright:cli -- route "**/*" --remove-header=cookie,authorization
|
||||
|
||||
# List active routes
|
||||
mise exec -- npm run playwright:cli -- route-list
|
||||
|
||||
# Remove a route or all routes
|
||||
mise exec -- npm run playwright:cli -- unroute "**/*.jpg"
|
||||
mise exec -- npm run playwright:cli -- unroute
|
||||
```
|
||||
|
||||
## URL Patterns
|
||||
|
||||
```
|
||||
**/api/users - Exact path match
|
||||
**/api/*/details - Wildcard in path
|
||||
**/*.{png,jpg,jpeg} - Match file extensions
|
||||
**/search?q=* - Match query parameters
|
||||
```
|
||||
|
||||
## Advanced Mocking with run-code
|
||||
|
||||
For conditional responses, request body inspection, response modification, or delays:
|
||||
|
||||
### Conditional Response Based on Request
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.route('**/api/login', route => {
|
||||
const body = route.request().postDataJSON();
|
||||
if (body.username === 'admin') {
|
||||
route.fulfill({ body: JSON.stringify({ token: 'mock-token' }) });
|
||||
} else {
|
||||
route.fulfill({ status: 401, body: JSON.stringify({ error: 'Invalid' }) });
|
||||
}
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
### Modify Real Response
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.route('**/api/user', async route => {
|
||||
const response = await route.fetch();
|
||||
const json = await response.json();
|
||||
json.isPremium = true;
|
||||
await route.fulfill({ response, json });
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
### Simulate Network Failures
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.route('**/api/offline', route => route.abort('internetdisconnected'));
|
||||
}"
|
||||
# Options: connectionrefused, timedout, connectionreset, internetdisconnected
|
||||
```
|
||||
|
||||
### Delayed Response
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.route('**/api/slow', async route => {
|
||||
await new Promise(r => setTimeout(r, 3000));
|
||||
route.fulfill({ body: JSON.stringify({ data: 'loaded' }) });
|
||||
});
|
||||
}"
|
||||
```
|
||||
@@ -0,0 +1,241 @@
|
||||
# Running Custom Playwright Code
|
||||
|
||||
Use `run-code` to execute arbitrary Playwright code for advanced scenarios not covered by CLI commands.
|
||||
|
||||
## Syntax
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
// Your Playwright code here
|
||||
// Access page.context() for browser context operations
|
||||
}"
|
||||
```
|
||||
|
||||
You can also load the function from a file:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code --filename=./my-script.js
|
||||
```
|
||||
|
||||
|
||||
The code must be a single function expression, it is wrapped in `(...)` and evaluated.
|
||||
import/export/require syntax is not supported.
|
||||
|
||||
## Geolocation
|
||||
|
||||
```bash
|
||||
# Grant geolocation permission and set location
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().grantPermissions(['geolocation']);
|
||||
await page.context().setGeolocation({ latitude: 37.7749, longitude: -122.4194 });
|
||||
}"
|
||||
|
||||
# Set location to London
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().grantPermissions(['geolocation']);
|
||||
await page.context().setGeolocation({ latitude: 51.5074, longitude: -0.1278 });
|
||||
}"
|
||||
|
||||
# Clear geolocation override
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().clearPermissions();
|
||||
}"
|
||||
```
|
||||
|
||||
## Permissions
|
||||
|
||||
```bash
|
||||
# Grant multiple permissions
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().grantPermissions([
|
||||
'geolocation',
|
||||
'notifications',
|
||||
'camera',
|
||||
'microphone'
|
||||
]);
|
||||
}"
|
||||
|
||||
# Grant permissions for specific origin
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().grantPermissions(['clipboard-read'], {
|
||||
origin: 'https://example.com'
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
## Media Emulation
|
||||
|
||||
```bash
|
||||
# Emulate dark color scheme
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.emulateMedia({ colorScheme: 'dark' });
|
||||
}"
|
||||
|
||||
# Emulate light color scheme
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.emulateMedia({ colorScheme: 'light' });
|
||||
}"
|
||||
|
||||
# Emulate reduced motion
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.emulateMedia({ reducedMotion: 'reduce' });
|
||||
}"
|
||||
|
||||
# Emulate print media
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.emulateMedia({ media: 'print' });
|
||||
}"
|
||||
```
|
||||
|
||||
## Wait Strategies
|
||||
|
||||
```bash
|
||||
# Wait for network idle
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.waitForLoadState('networkidle');
|
||||
}"
|
||||
|
||||
# Wait for specific element
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.locator('.loading').waitFor({ state: 'hidden' });
|
||||
}"
|
||||
|
||||
# Wait for function to return true
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.waitForFunction(() => window.appReady === true);
|
||||
}"
|
||||
|
||||
# Wait with timeout
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.locator('.result').waitFor({ timeout: 10000 });
|
||||
}"
|
||||
```
|
||||
|
||||
## Frames and Iframes
|
||||
|
||||
```bash
|
||||
# Work with iframe
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
const frame = page.locator('iframe#my-iframe').contentFrame();
|
||||
await frame.locator('button').click();
|
||||
}"
|
||||
|
||||
# Get all frames
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
const frames = page.frames();
|
||||
return frames.map(f => f.url());
|
||||
}"
|
||||
```
|
||||
|
||||
## File Downloads
|
||||
|
||||
```bash
|
||||
# Handle file download
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
const downloadPromise = page.waitForEvent('download');
|
||||
await page.getByRole('link', { name: 'Download' }).click();
|
||||
const download = await downloadPromise;
|
||||
await download.saveAs('./downloaded-file.pdf');
|
||||
return download.suggestedFilename();
|
||||
}"
|
||||
```
|
||||
|
||||
## Clipboard
|
||||
|
||||
```bash
|
||||
# Read clipboard (requires permission)
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().grantPermissions(['clipboard-read']);
|
||||
return await page.evaluate(() => navigator.clipboard.readText());
|
||||
}"
|
||||
|
||||
# Write to clipboard
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.evaluate(text => navigator.clipboard.writeText(text), 'Hello clipboard!');
|
||||
}"
|
||||
```
|
||||
|
||||
## Page Information
|
||||
|
||||
```bash
|
||||
# Get page title
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return await page.title();
|
||||
}"
|
||||
|
||||
# Get current URL
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return page.url();
|
||||
}"
|
||||
|
||||
# Get page content
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return await page.content();
|
||||
}"
|
||||
|
||||
# Get viewport size
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return page.viewportSize();
|
||||
}"
|
||||
```
|
||||
|
||||
## JavaScript Execution
|
||||
|
||||
```bash
|
||||
# Execute JavaScript and return result
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return await page.evaluate(() => {
|
||||
return {
|
||||
userAgent: navigator.userAgent,
|
||||
language: navigator.language,
|
||||
cookiesEnabled: navigator.cookieEnabled
|
||||
};
|
||||
});
|
||||
}"
|
||||
|
||||
# Pass arguments to evaluate
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
const multiplier = 5;
|
||||
return await page.evaluate(m => document.querySelectorAll('li').length * m, multiplier);
|
||||
}"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```bash
|
||||
# Try-catch in run-code
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
try {
|
||||
await page.getByRole('button', { name: 'Submit' }).click({ timeout: 1000 });
|
||||
return 'clicked';
|
||||
} catch (e) {
|
||||
return 'element not found';
|
||||
}
|
||||
}"
|
||||
```
|
||||
|
||||
## Complex Workflows
|
||||
|
||||
```bash
|
||||
# Login and save state
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.goto('https://example.com/login');
|
||||
await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
|
||||
await page.getByRole('textbox', { name: 'Password' }).fill('secret');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
await page.waitForURL('**/dashboard');
|
||||
await page.context().storageState({ path: 'auth.json' });
|
||||
return 'Login successful';
|
||||
}"
|
||||
|
||||
# Scrape data from multiple pages
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
const results = [];
|
||||
for (let i = 1; i <= 3; i++) {
|
||||
await page.goto(\`https://example.com/page/\${i}\`);
|
||||
const items = await page.locator('.item').allTextContents();
|
||||
results.push(...items);
|
||||
}
|
||||
return results;
|
||||
}"
|
||||
```
|
||||
@@ -0,0 +1,225 @@
|
||||
# Browser Session Management
|
||||
|
||||
Run multiple isolated browser sessions concurrently with state persistence.
|
||||
|
||||
## Named Browser Sessions
|
||||
|
||||
Use `-s` flag to isolate browser contexts:
|
||||
|
||||
```bash
|
||||
# Browser 1: Authentication flow
|
||||
mise exec -- npm run playwright:cli -- -s=auth open https://app.example.com/login
|
||||
|
||||
# Browser 2: Public browsing (separate cookies, storage)
|
||||
mise exec -- npm run playwright:cli -- -s=public open https://example.com
|
||||
|
||||
# Commands are isolated by browser session
|
||||
mise exec -- npm run playwright:cli -- -s=auth fill e1 "user@example.com"
|
||||
mise exec -- npm run playwright:cli -- -s=public snapshot
|
||||
```
|
||||
|
||||
## Browser Session Isolation Properties
|
||||
|
||||
Each browser session has independent:
|
||||
- Cookies
|
||||
- LocalStorage / SessionStorage
|
||||
- IndexedDB
|
||||
- Cache
|
||||
- Browsing history
|
||||
- Open tabs
|
||||
|
||||
## Browser Session Commands
|
||||
|
||||
```bash
|
||||
# List all browser sessions
|
||||
mise exec -- npm run playwright:cli -- list
|
||||
|
||||
# Stop a browser session (close the browser)
|
||||
mise exec -- npm run playwright:cli -- close # stop the default browser
|
||||
mise exec -- npm run playwright:cli -- -s=mysession close # stop a named browser
|
||||
|
||||
# Stop all browser sessions
|
||||
mise exec -- npm run playwright:cli -- close-all
|
||||
|
||||
# Forcefully kill all daemon processes (for stale/zombie processes)
|
||||
mise exec -- npm run playwright:cli -- kill-all
|
||||
|
||||
# Delete browser session user data (profile directory)
|
||||
mise exec -- npm run playwright:cli -- delete-data # delete default browser data
|
||||
mise exec -- npm run playwright:cli -- -s=mysession delete-data # delete named browser data
|
||||
```
|
||||
|
||||
## Environment Variable
|
||||
|
||||
Set a default browser session name via environment variable:
|
||||
|
||||
```bash
|
||||
export PLAYWRIGHT_CLI_SESSION="mysession"
|
||||
mise exec -- npm run playwright:cli -- open example.com # Uses "mysession" automatically
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Concurrent Scraping
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Scrape multiple sites concurrently
|
||||
|
||||
# Start all browsers
|
||||
mise exec -- npm run playwright:cli -- -s=site1 open https://site1.com &
|
||||
mise exec -- npm run playwright:cli -- -s=site2 open https://site2.com &
|
||||
mise exec -- npm run playwright:cli -- -s=site3 open https://site3.com &
|
||||
wait
|
||||
|
||||
# Take snapshots from each
|
||||
mise exec -- npm run playwright:cli -- -s=site1 snapshot
|
||||
mise exec -- npm run playwright:cli -- -s=site2 snapshot
|
||||
mise exec -- npm run playwright:cli -- -s=site3 snapshot
|
||||
|
||||
# Cleanup
|
||||
mise exec -- npm run playwright:cli -- close-all
|
||||
```
|
||||
|
||||
### A/B Testing Sessions
|
||||
|
||||
```bash
|
||||
# Test different user experiences
|
||||
mise exec -- npm run playwright:cli -- -s=variant-a open "https://app.com?variant=a"
|
||||
mise exec -- npm run playwright:cli -- -s=variant-b open "https://app.com?variant=b"
|
||||
|
||||
# Compare
|
||||
mise exec -- npm run playwright:cli -- -s=variant-a screenshot
|
||||
mise exec -- npm run playwright:cli -- -s=variant-b screenshot
|
||||
```
|
||||
|
||||
### Persistent Profile
|
||||
|
||||
By default, browser profile is kept in memory only. Use `--persistent` flag on `open` to persist the browser profile to disk:
|
||||
|
||||
```bash
|
||||
# Use persistent profile (auto-generated location)
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --persistent
|
||||
|
||||
# Use persistent profile with custom directory
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --profile=/path/to/profile
|
||||
```
|
||||
|
||||
## Attaching to a Running Browser
|
||||
|
||||
Use `attach` to connect to a browser that is already running, instead of launching a new one.
|
||||
|
||||
### Attach by channel name
|
||||
|
||||
Connect to a running Chrome or Edge instance by its channel name. The browser must have remote debugging enabled — navigate to `chrome://inspect/#remote-debugging` in the target browser and check "Allow remote debugging for this browser instance".
|
||||
|
||||
```bash
|
||||
# Attach to Chrome
|
||||
mise exec -- npm run playwright:cli -- attach --cdp=chrome
|
||||
|
||||
# Attach to Chrome Canary
|
||||
mise exec -- npm run playwright:cli -- attach --cdp=chrome-canary
|
||||
|
||||
# Attach to Microsoft Edge
|
||||
mise exec -- npm run playwright:cli -- attach --cdp=msedge
|
||||
|
||||
# Attach to Edge Dev
|
||||
mise exec -- npm run playwright:cli -- attach --cdp=msedge-dev
|
||||
```
|
||||
|
||||
Supported channels: `chrome`, `chrome-beta`, `chrome-dev`, `chrome-canary`, `msedge`, `msedge-beta`, `msedge-dev`, `msedge-canary`.
|
||||
|
||||
When `--session` is not provided, the session is named after the channel (e.g. `--cdp=msedge` creates a session called `msedge`), so parallel attaches to Chrome and Edge don't collide on `default`. Pass `--session=<name>` to override.
|
||||
|
||||
### Attach via CDP endpoint
|
||||
|
||||
Connect to a browser that exposes a Chrome DevTools Protocol endpoint:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- attach --cdp=http://localhost:9222
|
||||
```
|
||||
|
||||
### Attach via browser extension
|
||||
|
||||
Connect to a browser with the Playwright extension installed:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- attach --extension
|
||||
```
|
||||
|
||||
### Detach
|
||||
|
||||
Tear down an attached session without affecting the external browser:
|
||||
|
||||
```bash
|
||||
# Detach the default attached session
|
||||
mise exec -- npm run playwright:cli -- detach
|
||||
|
||||
# Detach a specific attached session
|
||||
mise exec -- npm run playwright:cli -- -s=msedge detach
|
||||
```
|
||||
|
||||
`detach` only works on sessions created via `attach`. For sessions created via `open`, use `close`.
|
||||
|
||||
## Default Browser Session
|
||||
|
||||
When `-s` is omitted, commands use the default browser session:
|
||||
|
||||
```bash
|
||||
# These use the same default browser session
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
mise exec -- npm run playwright:cli -- close # Stops default browser
|
||||
```
|
||||
|
||||
## Browser Session Configuration
|
||||
|
||||
Configure a browser session with specific settings when opening:
|
||||
|
||||
```bash
|
||||
# Open with config file
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --config=.playwright/my-cli.json
|
||||
|
||||
# Open with specific browser
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --browser=firefox
|
||||
|
||||
# Open in headed mode
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --headed
|
||||
|
||||
# Open with persistent profile
|
||||
mise exec -- npm run playwright:cli -- open https://example.com --persistent
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Name Browser Sessions Semantically
|
||||
|
||||
```bash
|
||||
# GOOD: Clear purpose
|
||||
mise exec -- npm run playwright:cli -- -s=github-auth open https://github.com
|
||||
mise exec -- npm run playwright:cli -- -s=docs-scrape open https://docs.example.com
|
||||
|
||||
# AVOID: Generic names
|
||||
mise exec -- npm run playwright:cli -- -s=s1 open https://github.com
|
||||
```
|
||||
|
||||
### 2. Always Clean Up
|
||||
|
||||
```bash
|
||||
# Stop browsers when done
|
||||
mise exec -- npm run playwright:cli -- -s=auth close
|
||||
mise exec -- npm run playwright:cli -- -s=scrape close
|
||||
|
||||
# Or stop all at once
|
||||
mise exec -- npm run playwright:cli -- close-all
|
||||
|
||||
# If browsers become unresponsive or zombie processes remain
|
||||
mise exec -- npm run playwright:cli -- kill-all
|
||||
```
|
||||
|
||||
### 3. Delete Stale Browser Data
|
||||
|
||||
```bash
|
||||
# Remove old browser data to free disk space
|
||||
mise exec -- npm run playwright:cli -- -s=oldsession delete-data
|
||||
```
|
||||
@@ -0,0 +1,305 @@
|
||||
# Spec-driven testing (plan → generate → heal)
|
||||
|
||||
End-to-end workflow for authoring and maintaining Playwright tests using `mise exec -- npm run playwright:cli --`. The three sections below can be used independently:
|
||||
|
||||
- **Planning** — explore the app, produce a spec file describing what to test.
|
||||
- **Generate** — turn a spec into Playwright test files. Update the spec if it's vague or stale.
|
||||
- **Heal** — diagnose failing tests, fix the code, reconcile the spec with reality.
|
||||
|
||||
All three lean on the same mechanic: run `mise exec -- npm run test:e2e -- --debug=cli` in the background, then `mise exec -- npm run playwright:cli -- attach tw-XXXX` to drive the paused page interactively. See [playwright-tests.md](playwright-tests.md) for the debug/attach mechanics and [test-generation.md](test-generation.md) for how every `mise exec -- npm run playwright:cli --` action emits Playwright TypeScript.
|
||||
|
||||
---
|
||||
|
||||
## 1. Planning
|
||||
|
||||
Goal: produce a spec file (e.g. `specs/<feature>.plan.md`) that enumerates the scenarios to test. **Always** write the spec to a file.
|
||||
|
||||
### 1.1 Prerequisite: workspace
|
||||
|
||||
Check the workspace has Playwright installed before anything else:
|
||||
|
||||
```bash
|
||||
# These confirm the MyGO Web workspace and locked CLI:
|
||||
test -f playwright.config.ts
|
||||
mise exec -- npm run playwright:cli -- --version
|
||||
```
|
||||
|
||||
If dependencies are missing, restore the repository lockfile instead of bootstrapping a new Playwright project:
|
||||
|
||||
```bash
|
||||
mise exec -- npm ci
|
||||
```
|
||||
|
||||
### 1.2 Prerequisite: seed test
|
||||
|
||||
A **seed test** is a minimal test that lands the page in the state every scenario starts from: navigation to the app, any required login, feature flags, etc. Scenarios assume a fresh start *after* the seed. `--debug=cli` pauses *inside* this test, so the seed is where every planning and generation session begins.
|
||||
|
||||
Minimum viable seed:
|
||||
|
||||
```ts
|
||||
// e2e/seed.spec.ts
|
||||
import { test } from '@playwright/test';
|
||||
|
||||
test('seed', async ({ page }) => {
|
||||
await page.goto('https://example.com/');
|
||||
});
|
||||
```
|
||||
|
||||
Preferred — push navigation into a fixture so scenario tests reuse it:
|
||||
|
||||
```ts
|
||||
// e2e/fixtures.ts
|
||||
import { test as baseTest } from '@playwright/test';
|
||||
export { expect } from '@playwright/test';
|
||||
|
||||
export const test = baseTest.extend({
|
||||
page: async ({ page }, use) => {
|
||||
await page.goto('https://example.com/');
|
||||
await use(page);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
```ts
|
||||
// e2e/seed.spec.ts
|
||||
import { test } from './fixtures';
|
||||
|
||||
test('seed', async ({ page }) => {
|
||||
// Fixture already navigates. This empty body tells agents where to start.
|
||||
});
|
||||
```
|
||||
|
||||
If no seed exists, create one that at least navigates to the app.
|
||||
|
||||
### 1.3 Explore the app
|
||||
|
||||
Launch the app via the seed in the background and attach:
|
||||
|
||||
```bash
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e -- e2e/seed.spec.ts --debug=cli
|
||||
# wait for "Debugging Instructions" and the session name tw-XXXX
|
||||
mise exec -- npm run playwright:cli -- attach tw-XXXX
|
||||
```
|
||||
|
||||
Resume so the seed runs, then probe the app:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- resume # resume so that seed test runs fully
|
||||
mise exec -- npm run playwright:cli -- snapshot # inventory of interactive elements
|
||||
mise exec -- npm run playwright:cli -- click e5 # follow a flow
|
||||
mise exec -- npm run playwright:cli -- eval "location.href" # read URL / state
|
||||
mise exec -- npm run playwright:cli -- show --annotate # ask the user to point at something
|
||||
```
|
||||
|
||||
Map out:
|
||||
|
||||
- Interactive surfaces (forms, buttons, lists, filters, modals).
|
||||
- Primary user journeys end-to-end.
|
||||
- Edge cases: empty states, validation errors, very long input, boundary values.
|
||||
- Persistence: reload, local/session storage, URL fragments.
|
||||
- Navigation: which controls change the URL, back/forward behaviour.
|
||||
|
||||
**Important**: Do not just open the app url with mise exec -- npm run playwright:cli --, always go through the test to capture any custom setup done there.
|
||||
**Important**: Stop the background test when done exploring.
|
||||
|
||||
### 1.4 Write the spec file
|
||||
|
||||
Save under `specs/<feature>.plan.md`. Use this structure:
|
||||
|
||||
```markdown
|
||||
# <Feature> Test Plan
|
||||
|
||||
## Application Overview
|
||||
|
||||
<One paragraph describing what the feature does and why it matters.>
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### 1. <Group Name>
|
||||
|
||||
**Seed:** `e2e/seed.spec.ts`
|
||||
|
||||
#### 1.1. <kebab-case-scenario-name>
|
||||
|
||||
**File:** `e2e/<group>/<kebab-case-scenario-name>.spec.ts`
|
||||
|
||||
**Steps:**
|
||||
1. <Concrete user step>
|
||||
- expect: <observable outcome>
|
||||
- expect: <another observable outcome>
|
||||
2. <Next step>
|
||||
- expect: <outcome>
|
||||
|
||||
#### 1.2. <next-scenario>
|
||||
...
|
||||
|
||||
### 2. <Next Group>
|
||||
|
||||
**Seed:** `e2e/seed.spec.ts`
|
||||
...
|
||||
```
|
||||
|
||||
Guidelines:
|
||||
|
||||
- Each scenario is independent and starts from the seed's fresh state — never chain scenarios.
|
||||
- Scenario names are kebab-case and match the test file name (`should-add-single-todo` → `should-add-single-todo.spec.ts`).
|
||||
- Cover happy path, edge cases, validation, negative flows, persistence.
|
||||
- Write steps at the user level ("Type 'Buy milk' into the input"), not the API level ("call `fill`").
|
||||
- Put observable outcomes in `- expect:` bullets; each becomes an assertion during generation.
|
||||
|
||||
---
|
||||
|
||||
## 2. Generate
|
||||
|
||||
Goal: take a spec file and produce Playwright test files. Optionally update the spec if it has drifted.
|
||||
|
||||
### 2.1 Inputs
|
||||
|
||||
- **Spec file**, e.g. `specs/basic-operations.plan.md`.
|
||||
- **Target**: either a single scenario (e.g. `1.2`), a whole group (`1`), or all.
|
||||
- **Seed file**, read from the `**Seed:**` line of the scenario's group.
|
||||
|
||||
### 2.2 Generate one scenario
|
||||
|
||||
For each target scenario, in sequence (never in parallel — scenarios share the seed session):
|
||||
|
||||
```bash
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e -- <seed-file> --debug=cli # background
|
||||
mise exec -- npm run playwright:cli -- attach tw-XXXX
|
||||
# resume
|
||||
```
|
||||
|
||||
**Do not** just open the app url with mise exec -- npm run playwright:cli --, always go through the test to capture any custom setup done there.
|
||||
|
||||
Walk the scenario's `Steps:` one by one with `mise exec -- npm run playwright:cli --`, treating the spec as the plan and the live app as the source of truth. If a step is vague ("click the button" — which button?), references an element that no longer exists, or contradicts the app's actual behaviour, use your judgement: update the spec to match what the app really does, then keep going. Editing the spec mid-generation is expected.
|
||||
|
||||
Every action prints the equivalent Playwright TypeScript (see [test-generation.md](test-generation.md)):
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- snapshot # find refs
|
||||
mise exec -- npm run playwright:cli -- fill e3 "John Doe" # -> page.getByRole('textbox', {...}).fill(...)
|
||||
mise exec -- npm run playwright:cli -- press Enter
|
||||
mise exec -- npm run playwright:cli -- click e7
|
||||
```
|
||||
|
||||
For each `- expect:` bullet, add an explicit assertion. See [test-generation.md](test-generation.md) for details.
|
||||
|
||||
Collect the generated code and write the test file at the path given in the spec:
|
||||
|
||||
```ts
|
||||
// spec: specs/basic-operations.plan.md
|
||||
// seed: e2e/seed.spec.ts
|
||||
import { test, expect } from './fixtures'; // or '@playwright/test' if no fixtures file
|
||||
|
||||
test.describe('Signing in and out', () => {
|
||||
test('should sign in', async ({ page }) => {
|
||||
// 1. Navigate to the application
|
||||
// (handled by the seed fixture)
|
||||
|
||||
// 2. Type 'John Doe' into the username field
|
||||
await page.getByRole('textbox', { name: 'username' }).fill('John Doe');
|
||||
|
||||
// 3. Type password
|
||||
await page.getByRole('textbox', { name: 'password' }).fill('TestPassword');
|
||||
|
||||
// 4. Press Enter to submit
|
||||
await page.getByRole('textbox', { name: 'password' }).press('Enter');
|
||||
|
||||
await expect(page.getByRole('heading')).toContainText('Welcome, John Doe!');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- **One test per file.** File path, describe name, and test name come verbatim from the spec (minus the ordinal).
|
||||
- Prefix each numbered step with a `// N. <step text>` comment before its actions.
|
||||
- Use the describe group name verbatim from the spec (no `1.` ordinal).
|
||||
- Import from `./fixtures` if the project has one; otherwise `@playwright/test`.
|
||||
- **Important**: close the CLI session and stop the background test before moving to the next scenario.
|
||||
|
||||
### 2.3 Generate multiple scenarios
|
||||
|
||||
Loop 2.2 over the targeted scenarios one at a time, restarting the seed between each so every test starts from a clean page. This is safe to parallelise due to unique generated session names - just make sure each test run is stopped.
|
||||
|
||||
### 2.4 Run generated tests
|
||||
|
||||
After generation, run the new tests once:
|
||||
|
||||
```bash
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e -- e2e/<group>/<scenario>.spec.ts
|
||||
```
|
||||
|
||||
Any failure goes to Section 3.
|
||||
|
||||
---
|
||||
|
||||
## 3. Heal
|
||||
|
||||
Goal: fix failing tests, and update the spec if the app's intended behaviour changed.
|
||||
|
||||
### 3.1 Find failing tests
|
||||
|
||||
```bash
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e --
|
||||
```
|
||||
|
||||
Record the list of failing `<file>:<line>` entries and process them one at a time. Do not attempt parallel fixes — shared state and the single CLI session make that fragile.
|
||||
|
||||
### 3.2 Debug one failure
|
||||
|
||||
Run the single failing test in debug mode in the background, then attach:
|
||||
|
||||
```bash
|
||||
PLAYWRIGHT_HTML_OPEN=never mise exec -- npm run test:e2e -- e2e/<group>/<scenario>.spec.ts:<line> --debug=cli
|
||||
# wait for "Debugging Instructions" and the tw-XXXX session name
|
||||
mise exec -- npm run playwright:cli -- attach tw-XXXX
|
||||
```
|
||||
|
||||
The test is paused at the start. Step forward or run to until just before the failing action or assertion, then diagnose:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- snapshot # did the element change / move / rename?
|
||||
mise exec -- npm run playwright:cli -- console # app-side errors?
|
||||
mise exec -- npm run playwright:cli -- requests # failed request? wrong payload?
|
||||
mise exec -- npm run playwright:cli -- show --annotate # ask the user to point somewhere
|
||||
```
|
||||
|
||||
Common causes: selector drift, new wrapper element, label/ARIA rename, timing (transition, async load), assertion text updated in the app, test data leaking between runs.
|
||||
|
||||
Rehearse the corrected interaction with `mise exec -- npm run playwright:cli --` — the generated code in the output is what you paste back into the test.
|
||||
|
||||
### 3.3 Apply the fix
|
||||
|
||||
Edit the test file: update the locator, assertion, step order, or inputs to match the corrected behaviour. Stop the background debug run. Rerun the single test to confirm green.
|
||||
|
||||
Never skip hooks or add sleeps as a fix. Never use `networkidle`.
|
||||
|
||||
### 3.4 Reconcile with the spec
|
||||
|
||||
Open the spec referenced by the `// spec:` header in the test file and locate the scenario that matches the test.
|
||||
|
||||
- **Fix was purely technical** (locator drift, better assertion shape) and the spec's user-level behaviour still matches the app → leave the spec alone.
|
||||
- **Fix changed user-visible steps, inputs, order, or expected outcomes** that the spec describes → update the spec to match reality. Keep the scenario id and file path stable; only the step / expect lines change.
|
||||
- **Unclear whether the app change is intentional** (spec is stale) **or a regression** (test was right, app is wrong) → **stop and ask the user**. Provide:
|
||||
- the scenario id (e.g. `2.3`),
|
||||
- the spec lines that no longer match,
|
||||
- the observed app behaviour (quote a snapshot excerpt or a concrete outcome).
|
||||
|
||||
Only after the user answers, either update the spec (intentional change) or file/flag the test as covering a bug (regression).
|
||||
|
||||
### 3.5 Iteration and giving up
|
||||
|
||||
- Fix failures one at a time; rerun after each.
|
||||
- If after thorough investigation you are confident the test is correct but the app is wrong *and* the user has confirmed it's a bug: mark the test `test.fixme(...)` with a comment pointing at the user's decision or issue link. Never silently skip.
|
||||
|
||||
---
|
||||
|
||||
## Cross-references
|
||||
|
||||
| For... | See |
|
||||
|---|---|
|
||||
| `--debug=cli` / attach mechanics | [playwright-tests.md](playwright-tests.md) |
|
||||
| How `mise exec -- npm run playwright:cli --` actions become TS | [test-generation.md](test-generation.md) |
|
||||
| Mocking requests during exploration/generation | [request-mocking.md](request-mocking.md) |
|
||||
| Managing the CLI browser session | [session-management.md](session-management.md) |
|
||||
@@ -0,0 +1,275 @@
|
||||
# Storage Management
|
||||
|
||||
Manage cookies, localStorage, sessionStorage, and browser storage state.
|
||||
|
||||
## Storage State
|
||||
|
||||
Save and restore complete browser state including cookies and storage.
|
||||
|
||||
### Save Storage State
|
||||
|
||||
```bash
|
||||
# Save to auto-generated filename (storage-state-{timestamp}.json)
|
||||
mise exec -- npm run playwright:cli -- state-save
|
||||
|
||||
# Save to specific filename
|
||||
mise exec -- npm run playwright:cli -- state-save .artifacts/playwright-cli/my-auth-state.json
|
||||
```
|
||||
|
||||
### Restore Storage State
|
||||
|
||||
```bash
|
||||
# Load storage state from file
|
||||
mise exec -- npm run playwright:cli -- state-load .artifacts/playwright-cli/my-auth-state.json
|
||||
|
||||
# Reload page to apply cookies
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
```
|
||||
|
||||
### Storage State File Format
|
||||
|
||||
The saved file contains:
|
||||
|
||||
```json
|
||||
{
|
||||
"cookies": [
|
||||
{
|
||||
"name": "session_id",
|
||||
"value": "abc123",
|
||||
"domain": "example.com",
|
||||
"path": "/",
|
||||
"expires": 1893456000,
|
||||
"httpOnly": true,
|
||||
"secure": true,
|
||||
"sameSite": "Lax"
|
||||
}
|
||||
],
|
||||
"origins": [
|
||||
{
|
||||
"origin": "https://example.com",
|
||||
"localStorage": [
|
||||
{ "name": "theme", "value": "dark" },
|
||||
{ "name": "user_id", "value": "12345" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Cookies
|
||||
|
||||
### List All Cookies
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-list
|
||||
```
|
||||
|
||||
### Filter Cookies by Domain
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-list --domain=example.com
|
||||
```
|
||||
|
||||
### Filter Cookies by Path
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-list --path=/api
|
||||
```
|
||||
|
||||
### Get Specific Cookie
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-get session_id
|
||||
```
|
||||
|
||||
### Set a Cookie
|
||||
|
||||
```bash
|
||||
# Basic cookie
|
||||
mise exec -- npm run playwright:cli -- cookie-set session abc123
|
||||
|
||||
# Cookie with options
|
||||
mise exec -- npm run playwright:cli -- cookie-set session abc123 --domain=example.com --path=/ --httpOnly --secure --sameSite=Lax
|
||||
|
||||
# Cookie with expiration (Unix timestamp)
|
||||
mise exec -- npm run playwright:cli -- cookie-set remember_me token123 --expires=1893456000
|
||||
```
|
||||
|
||||
### Delete a Cookie
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-delete session_id
|
||||
```
|
||||
|
||||
### Clear All Cookies
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- cookie-clear
|
||||
```
|
||||
|
||||
### Advanced: Multiple Cookies or Custom Options
|
||||
|
||||
For complex scenarios like adding multiple cookies at once, use `run-code`:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.context().addCookies([
|
||||
{ name: 'session_id', value: 'sess_abc123', domain: 'example.com', path: '/', httpOnly: true },
|
||||
{ name: 'preferences', value: JSON.stringify({ theme: 'dark' }), domain: 'example.com', path: '/' }
|
||||
]);
|
||||
}"
|
||||
```
|
||||
|
||||
## Local Storage
|
||||
|
||||
### List All localStorage Items
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-list
|
||||
```
|
||||
|
||||
### Get Single Value
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-get token
|
||||
```
|
||||
|
||||
### Set Value
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-set theme dark
|
||||
```
|
||||
|
||||
### Set JSON Value
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-set user_settings '{"theme":"dark","language":"en"}'
|
||||
```
|
||||
|
||||
### Delete Single Item
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-delete token
|
||||
```
|
||||
|
||||
### Clear All localStorage
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- localstorage-clear
|
||||
```
|
||||
|
||||
### Advanced: Multiple Operations
|
||||
|
||||
For complex scenarios like setting multiple values at once, use `run-code`:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.evaluate(() => {
|
||||
localStorage.setItem('token', 'jwt_abc123');
|
||||
localStorage.setItem('user_id', '12345');
|
||||
localStorage.setItem('expires_at', Date.now() + 3600000);
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
## Session Storage
|
||||
|
||||
### List All sessionStorage Items
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- sessionstorage-list
|
||||
```
|
||||
|
||||
### Get Single Value
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- sessionstorage-get form_data
|
||||
```
|
||||
|
||||
### Set Value
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- sessionstorage-set step 3
|
||||
```
|
||||
|
||||
### Delete Single Item
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- sessionstorage-delete step
|
||||
```
|
||||
|
||||
### Clear sessionStorage
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- sessionstorage-clear
|
||||
```
|
||||
|
||||
## IndexedDB
|
||||
|
||||
### List Databases
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
return await page.evaluate(async () => {
|
||||
const databases = await indexedDB.databases();
|
||||
return databases;
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
### Delete Database
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- run-code "async page => {
|
||||
await page.evaluate(() => {
|
||||
indexedDB.deleteDatabase('myDatabase');
|
||||
});
|
||||
}"
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Authentication State Reuse
|
||||
|
||||
```bash
|
||||
# Step 1: Login and save state
|
||||
mise exec -- npm run playwright:cli -- open https://app.example.com/login
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
mise exec -- npm run playwright:cli -- fill e1 "user@example.com"
|
||||
mise exec -- npm run playwright:cli -- fill e2 "password123"
|
||||
mise exec -- npm run playwright:cli -- click e3
|
||||
|
||||
# Save the authenticated state
|
||||
mise exec -- npm run playwright:cli -- state-save .artifacts/playwright-cli/auth.json
|
||||
|
||||
# Step 2: Later, restore state and skip login
|
||||
mise exec -- npm run playwright:cli -- state-load .artifacts/playwright-cli/auth.json
|
||||
mise exec -- npm run playwright:cli -- open https://app.example.com/dashboard
|
||||
# Already logged in!
|
||||
```
|
||||
|
||||
### Save and Restore Roundtrip
|
||||
|
||||
```bash
|
||||
# Set up authentication state
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
mise exec -- npm run playwright:cli -- eval "() => { document.cookie = 'session=abc123'; localStorage.setItem('user', 'john'); }"
|
||||
|
||||
# Save state to file
|
||||
mise exec -- npm run playwright:cli -- state-save .artifacts/playwright-cli/my-session.json
|
||||
|
||||
# ... later, in a new session ...
|
||||
|
||||
# Restore state
|
||||
mise exec -- npm run playwright:cli -- state-load .artifacts/playwright-cli/my-session.json
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
# Cookies and localStorage are restored!
|
||||
```
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Never commit storage state files containing auth tokens
|
||||
- Add `*.auth-state.json` to `.gitignore`
|
||||
- Delete state files after automation completes
|
||||
- Use environment variables for sensitive data
|
||||
- By default, sessions run in-memory mode which is safer for sensitive operations
|
||||
@@ -0,0 +1,134 @@
|
||||
# Test Generation
|
||||
|
||||
Generate Playwright test code automatically as you interact with the browser.
|
||||
|
||||
## How It Works
|
||||
|
||||
Every action you perform with `mise exec -- npm run playwright:cli --` generates corresponding Playwright TypeScript code.
|
||||
This code appears in the output and can be copied directly into your test files.
|
||||
|
||||
## Example Workflow
|
||||
|
||||
```bash
|
||||
# Start a session
|
||||
mise exec -- npm run playwright:cli -- open https://example.com/login
|
||||
|
||||
# Take a snapshot to see elements
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
# Output shows: e1 [textbox "Email"], e2 [textbox "Password"], e3 [button "Sign In"]
|
||||
|
||||
# Fill form fields - generates code automatically
|
||||
mise exec -- npm run playwright:cli -- fill e1 "user@example.com"
|
||||
# Ran Playwright code:
|
||||
# await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
|
||||
|
||||
mise exec -- npm run playwright:cli -- fill e2 "password123"
|
||||
# Ran Playwright code:
|
||||
# await page.getByRole('textbox', { name: 'Password' }).fill('password123');
|
||||
|
||||
mise exec -- npm run playwright:cli -- click e3
|
||||
# Ran Playwright code:
|
||||
# await page.getByRole('button', { name: 'Sign In' }).click();
|
||||
```
|
||||
|
||||
## Building a Test File
|
||||
|
||||
Collect the generated code into a Playwright test:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('login flow', async ({ page }) => {
|
||||
// Generated code from mise exec -- npm run playwright:cli -- session:
|
||||
await page.goto('https://example.com/login');
|
||||
await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
|
||||
await page.getByRole('textbox', { name: 'Password' }).fill('password123');
|
||||
await page.getByRole('button', { name: 'Sign In' }).click();
|
||||
|
||||
// Add assertions
|
||||
await expect(page).toHaveURL(/.*dashboard/);
|
||||
});
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Use Semantic Locators
|
||||
|
||||
The generated code uses role-based locators when possible, which are more resilient:
|
||||
|
||||
```typescript
|
||||
// Generated (good - semantic)
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Avoid (fragile - CSS selectors)
|
||||
await page.locator('#submit-btn').click();
|
||||
```
|
||||
|
||||
### 2. Explore Before Recording
|
||||
|
||||
Take snapshots to understand the page structure before recording actions:
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
# Review the element structure
|
||||
mise exec -- npm run playwright:cli -- click e5
|
||||
```
|
||||
|
||||
### 3. Add Assertions Manually
|
||||
|
||||
Generated code captures actions but not assertions. Add expectations in your test using one of the recommended matchers:
|
||||
|
||||
- `toBeVisible()` — element is rendered and visible
|
||||
- `toHaveText(text)` — element text content matches
|
||||
- `toHaveValue(value) / toBeEmpty()` — input/select value matches
|
||||
- `toBeChecked() / toBeUnchecked()` — checkbox state matches
|
||||
- `toMatchAriaSnapshot(snapshot)` — page (or locator) matches a partial accessibility snapshot
|
||||
|
||||
Use `mise exec -- npm run playwright:cli -- generate-locator <target>` to produce the locator expression for the assertion, and the snapshot/eval commands to capture the expected value.
|
||||
|
||||
When asserting text content, make sure that generated locator does not contain text from the element itself. `getByTestId()` or `getByLabel()` usually work well with asserting text. When locator is text-based, prefer `toBeVisible()` instead.
|
||||
|
||||
Snapshot to be matched does not have to contain all the information - only capture what's necessary for the assertion. You can use regular expressions for unstable values.
|
||||
|
||||
```bash
|
||||
# Get a stable locator for an element ref to use in the assertion
|
||||
mise exec -- npm run playwright:cli -- --raw generate-locator e5
|
||||
# getByRole('button', { name: 'Submit' })
|
||||
|
||||
# Capture expected text content for toHaveText
|
||||
mise exec -- npm run playwright:cli -- --raw eval "el => el.textContent" e5
|
||||
|
||||
# Capture expected input value for toHaveValue/toBeEmpty
|
||||
mise exec -- npm run playwright:cli -- --raw eval "el => el.value" e5
|
||||
|
||||
# Capture expected aria snapshot for toMatchAriaSnapshot/toBeChecked
|
||||
# (whole page, or use a ref to scope to a region)
|
||||
mise exec -- npm run playwright:cli -- --raw snapshot
|
||||
mise exec -- npm run playwright:cli -- --raw snapshot e5
|
||||
```
|
||||
|
||||
```typescript
|
||||
// Generated action
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Manual assertions using the outputs above:
|
||||
await expect(page.getByRole('alert', { name: 'Success' })).toBeVisible();
|
||||
await expect(page.getByTestId('main-header')).toHaveText('Welcome, user');
|
||||
await expect(page.getByRole('textbox', { name: 'Email' })).toHaveValue('user@example.com');
|
||||
await expect(page.getByRole('checkbox', { name: 'Enable notifications' })).toBeChecked();
|
||||
|
||||
// toMatchAriaSnapshot on the whole page, finds a matching region
|
||||
await expect(page).toMatchAriaSnapshot(`
|
||||
- heading "Welcome, user"
|
||||
- link /\\d+ new messages?/
|
||||
- button "Sign out"
|
||||
`);
|
||||
|
||||
// toMatchAriaSnapshot scoped to a region
|
||||
await expect(page.getByRole('navigation')).toMatchAriaSnapshot(`
|
||||
- link "Home"
|
||||
- link /\\d+ new messages?/
|
||||
- link "Profile"
|
||||
`);
|
||||
```
|
||||
@@ -0,0 +1,139 @@
|
||||
# Tracing
|
||||
|
||||
Capture detailed execution traces for debugging and analysis. Traces include DOM snapshots, screenshots, network activity, and console logs.
|
||||
|
||||
## Basic Usage
|
||||
|
||||
```bash
|
||||
# Start trace recording
|
||||
mise exec -- npm run playwright:cli -- tracing-start
|
||||
|
||||
# Perform actions
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
mise exec -- npm run playwright:cli -- click e1
|
||||
mise exec -- npm run playwright:cli -- fill e2 "test"
|
||||
|
||||
# Stop trace recording
|
||||
mise exec -- npm run playwright:cli -- tracing-stop
|
||||
```
|
||||
|
||||
## Trace Output Files
|
||||
|
||||
When you start tracing, Playwright creates a `traces/` directory with several files:
|
||||
|
||||
### `trace-{timestamp}.trace`
|
||||
|
||||
**Action log** - The main trace file containing:
|
||||
- Every action performed (clicks, fills, navigations)
|
||||
- DOM snapshots before and after each action
|
||||
- Screenshots at each step
|
||||
- Timing information
|
||||
- Console messages
|
||||
- Source locations
|
||||
|
||||
### `trace-{timestamp}.network`
|
||||
|
||||
**Network log** - Complete network activity:
|
||||
- All HTTP requests and responses
|
||||
- Request headers and bodies
|
||||
- Response headers and bodies
|
||||
- Timing (DNS, connect, TLS, TTFB, download)
|
||||
- Resource sizes
|
||||
- Failed requests and errors
|
||||
|
||||
### `resources/`
|
||||
|
||||
**Resources directory** - Cached resources:
|
||||
- Images, fonts, stylesheets, scripts
|
||||
- Response bodies for replay
|
||||
- Assets needed to reconstruct page state
|
||||
|
||||
## What Traces Capture
|
||||
|
||||
| Category | Details |
|
||||
|----------|---------|
|
||||
| **Actions** | Clicks, fills, hovers, keyboard input, navigations |
|
||||
| **DOM** | Full DOM snapshot before/after each action |
|
||||
| **Screenshots** | Visual state at each step |
|
||||
| **Network** | All requests, responses, headers, bodies, timing |
|
||||
| **Console** | All console.log, warn, error messages |
|
||||
| **Timing** | Precise timing for each operation |
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Debugging Failed Actions
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- tracing-start
|
||||
mise exec -- npm run playwright:cli -- open https://app.example.com
|
||||
|
||||
# This click fails - why?
|
||||
mise exec -- npm run playwright:cli -- click e5
|
||||
|
||||
mise exec -- npm run playwright:cli -- tracing-stop
|
||||
# Open trace to see DOM state when click was attempted
|
||||
```
|
||||
|
||||
### Analyzing Performance
|
||||
|
||||
```bash
|
||||
mise exec -- npm run playwright:cli -- tracing-start
|
||||
mise exec -- npm run playwright:cli -- open https://slow-site.com
|
||||
mise exec -- npm run playwright:cli -- tracing-stop
|
||||
|
||||
# View network waterfall to identify slow resources
|
||||
```
|
||||
|
||||
### Capturing Evidence
|
||||
|
||||
```bash
|
||||
# Record a complete user flow for documentation
|
||||
mise exec -- npm run playwright:cli -- tracing-start
|
||||
|
||||
mise exec -- npm run playwright:cli -- open https://app.example.com/checkout
|
||||
mise exec -- npm run playwright:cli -- fill e1 "4111111111111111"
|
||||
mise exec -- npm run playwright:cli -- fill e2 "12/25"
|
||||
mise exec -- npm run playwright:cli -- fill e3 "123"
|
||||
mise exec -- npm run playwright:cli -- click e4
|
||||
|
||||
mise exec -- npm run playwright:cli -- tracing-stop
|
||||
# Trace shows exact sequence of events
|
||||
```
|
||||
|
||||
## Trace vs Video vs Screenshot
|
||||
|
||||
| Feature | Trace | Video | Screenshot |
|
||||
|---------|-------|-------|------------|
|
||||
| **Format** | .trace file | .webm video | .png/.jpeg image |
|
||||
| **DOM inspection** | Yes | No | No |
|
||||
| **Network details** | Yes | No | No |
|
||||
| **Step-by-step replay** | Yes | Continuous | Single frame |
|
||||
| **File size** | Medium | Large | Small |
|
||||
| **Best for** | Debugging | Demos | Quick capture |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Start Tracing Before the Problem
|
||||
|
||||
```bash
|
||||
# Trace the entire flow, not just the failing step
|
||||
mise exec -- npm run playwright:cli -- tracing-start
|
||||
mise exec -- npm run playwright:cli -- open https://example.com
|
||||
# ... all steps leading to the issue ...
|
||||
mise exec -- npm run playwright:cli -- tracing-stop
|
||||
```
|
||||
|
||||
### 2. Clean Up Old Traces
|
||||
|
||||
Traces can consume significant disk space:
|
||||
|
||||
```bash
|
||||
# Remove traces older than 7 days
|
||||
find ../.artifacts/playwright-cli -type f -mtime +7 -delete
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Traces add overhead to automation
|
||||
- Large traces can consume significant disk space
|
||||
- Some dynamic content may not replay perfectly
|
||||
@@ -0,0 +1,143 @@
|
||||
# Video Recording
|
||||
|
||||
Capture browser automation sessions as video for debugging, documentation, or verification. Produces WebM (VP8/VP9 codec).
|
||||
|
||||
## Basic Recording
|
||||
|
||||
```bash
|
||||
# Open browser first
|
||||
mise exec -- npm run playwright:cli -- open
|
||||
|
||||
# Start recording
|
||||
mise exec -- npm run playwright:cli -- video-start .artifacts/playwright-cli/demo.webm
|
||||
|
||||
# Add a chapter marker for section transitions
|
||||
mise exec -- npm run playwright:cli -- video-chapter "Getting Started" --description="Opening the homepage" --duration=2000
|
||||
|
||||
# Navigate and perform actions
|
||||
mise exec -- npm run playwright:cli -- goto https://example.com
|
||||
mise exec -- npm run playwright:cli -- snapshot
|
||||
mise exec -- npm run playwright:cli -- click e1
|
||||
|
||||
# Add another chapter
|
||||
mise exec -- npm run playwright:cli -- video-chapter "Filling Form" --description="Entering test data" --duration=2000
|
||||
mise exec -- npm run playwright:cli -- fill e2 "test input"
|
||||
|
||||
# Stop and save
|
||||
mise exec -- npm run playwright:cli -- video-stop
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Use Descriptive Filenames
|
||||
|
||||
```bash
|
||||
# Include context in filename
|
||||
mise exec -- npm run playwright:cli -- video-start .artifacts/playwright-cli/recordings/login-flow-2024-01-15.webm
|
||||
mise exec -- npm run playwright:cli -- video-start .artifacts/playwright-cli/recordings/checkout-test-run-42.webm
|
||||
```
|
||||
|
||||
### 2. Record entire hero scripts.
|
||||
|
||||
When recording a video for the user or as a proof of work, it is best to create a code snippet and execute it with run-code.
|
||||
It allows inserting appropriate pauses between the actions and annotating the video. There are new Playwright APIs for that.
|
||||
|
||||
1) Perform scenario using CLI and take note of all locators and actions. You'll need those locators to request their bounding boxes for highlight.
|
||||
2) Create a file with the intended script for video (below). Use pressSequentially w/ delay for nice typing, make reasonable pauses.
|
||||
3) Use mise exec -- npm run playwright:cli -- run-code --filename your-script.js
|
||||
|
||||
**Important**: Overlays are `pointer-events: none` — they do not interfere with page interactions. You can safely keep sticky overlays visible while clicking, filling, or performing any actions on the page.
|
||||
|
||||
```js
|
||||
async page => {
|
||||
await page.screencast.start({ path: 'video.webm', size: { width: 1280, height: 800 } });
|
||||
await page.goto('https://demo.playwright.dev/todomvc');
|
||||
|
||||
// Show a chapter card — blurs the page and shows a dialog.
|
||||
// Blocks until duration expires, then auto-removes.
|
||||
// Use this for simple use cases, but always feel free to hand-craft your own beautiful
|
||||
// overlay via await page.screencast.showOverlay().
|
||||
await page.screencast.showChapter('Adding Todo Items', {
|
||||
description: 'We will add several items to the todo list.',
|
||||
duration: 2000,
|
||||
});
|
||||
|
||||
// Perform action
|
||||
await page.getByRole('textbox', { name: 'What needs to be done?' }).pressSequentially('Walk the dog', { delay: 60 });
|
||||
await page.getByRole('textbox', { name: 'What needs to be done?' }).press('Enter');
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
// Show next chapter
|
||||
await page.screencast.showChapter('Verifying Results', {
|
||||
description: 'Checking the item appeared in the list.',
|
||||
duration: 2000,
|
||||
});
|
||||
|
||||
// Add a sticky annotation that stays while you perform actions.
|
||||
// Overlays are pointer-events: none, so they won't block clicks.
|
||||
const annotation = await page.screencast.showOverlay(`
|
||||
<div style="position: absolute; top: 8px; right: 8px;
|
||||
padding: 6px 12px; background: rgba(0,0,0,0.7);
|
||||
border-radius: 8px; font-size: 13px; color: white;">
|
||||
✓ Item added successfully
|
||||
</div>
|
||||
`);
|
||||
|
||||
// Perform more actions while the annotation is visible
|
||||
await page.getByRole('textbox', { name: 'What needs to be done?' }).pressSequentially('Buy groceries', { delay: 60 });
|
||||
await page.getByRole('textbox', { name: 'What needs to be done?' }).press('Enter');
|
||||
await page.waitForTimeout(1500);
|
||||
|
||||
// Remove the annotation when done
|
||||
await annotation.dispose();
|
||||
|
||||
// You can also highlight relevant locators and provide contextual annotations.
|
||||
const bounds = await page.getByText('Walk the dog').boundingBox();
|
||||
await page.screencast.showOverlay(`
|
||||
<div style="position: absolute;
|
||||
top: ${bounds.y}px;
|
||||
left: ${bounds.x}px;
|
||||
width: ${bounds.width}px;
|
||||
height: ${bounds.height}px;
|
||||
border: 1px solid red;">
|
||||
</div>
|
||||
<div style="position: absolute;
|
||||
top: ${bounds.y + bounds.height + 5}px;
|
||||
left: ${bounds.x + bounds.width / 2}px;
|
||||
transform: translateX(-50%);
|
||||
padding: 6px;
|
||||
background: #808080;
|
||||
border-radius: 10px;
|
||||
font-size: 14px;
|
||||
color: white;">Check it out, it is right above this text
|
||||
</div>
|
||||
`, { duration: 2000 });
|
||||
|
||||
await page.screencast.stop();
|
||||
}
|
||||
```
|
||||
|
||||
Embrace creativity, overlays are powerful.
|
||||
|
||||
### Overlay API Summary
|
||||
|
||||
| Method | Use Case |
|
||||
|--------|----------|
|
||||
| `page.screencast.showChapter(title, { description?, duration?, styleSheet? })` | Full-screen chapter card with blurred backdrop — ideal for section transitions |
|
||||
| `page.screencast.showOverlay(html, { duration? })` | Custom HTML overlay — use for callouts, labels, highlights |
|
||||
| `disposable.dispose()` | Remove a sticky overlay added without duration |
|
||||
| `page.screencast.hideOverlays()` / `page.screencast.showOverlays()` | Temporarily hide/show all overlays |
|
||||
|
||||
## Tracing vs Video
|
||||
|
||||
| Feature | Video | Tracing |
|
||||
|---------|-------|---------|
|
||||
| Output | WebM file | Trace file (viewable in Trace Viewer) |
|
||||
| Shows | Visual recording | DOM snapshots, network, console, actions |
|
||||
| Use case | Demos, documentation | Debugging, analysis |
|
||||
| Size | Larger | Smaller |
|
||||
|
||||
## Limitations
|
||||
|
||||
- Recording adds slight overhead to automation
|
||||
- Large recordings can consume significant disk space
|
||||
Reference in New Issue
Block a user