52dd56ff06
- feat: add `.agents/skills/playwright-cli/` with complete Skill markdown, agent interface, and 9 reference files covering session management, spec-driven testing, video recording, tracing, storage, request mocking, element attributes, test debugging, and running custom code - feat: add `.playwright/cli.config.json` to configure the Playwright CLI - feat: add `web/package.json` `playwright:cli` script that resolves `playwright cli` from the repo root - build: remove `@playwright/mcp` dependency from `web/package.json` - build: set `XDG_CACHE_HOME` in `mise.toml` - build: delete `.codex/config.toml` (MCP server config) - docs: update `README.md`, `docs/web/roadmap.md`, `docs/web/decisions.md`, and `web/README.md` to reflect the new Playwright CLI approach
135 lines
4.9 KiB
Markdown
135 lines
4.9 KiB
Markdown
# Test Generation
|
|
|
|
Generate Playwright test code automatically as you interact with the browser.
|
|
|
|
## How It Works
|
|
|
|
Every action you perform with `mise exec -- npm run playwright:cli --` generates corresponding Playwright TypeScript code.
|
|
This code appears in the output and can be copied directly into your test files.
|
|
|
|
## Example Workflow
|
|
|
|
```bash
|
|
# Start a session
|
|
mise exec -- npm run playwright:cli -- open https://example.com/login
|
|
|
|
# Take a snapshot to see elements
|
|
mise exec -- npm run playwright:cli -- snapshot
|
|
# Output shows: e1 [textbox "Email"], e2 [textbox "Password"], e3 [button "Sign In"]
|
|
|
|
# Fill form fields - generates code automatically
|
|
mise exec -- npm run playwright:cli -- fill e1 "user@example.com"
|
|
# Ran Playwright code:
|
|
# await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
|
|
|
|
mise exec -- npm run playwright:cli -- fill e2 "password123"
|
|
# Ran Playwright code:
|
|
# await page.getByRole('textbox', { name: 'Password' }).fill('password123');
|
|
|
|
mise exec -- npm run playwright:cli -- click e3
|
|
# Ran Playwright code:
|
|
# await page.getByRole('button', { name: 'Sign In' }).click();
|
|
```
|
|
|
|
## Building a Test File
|
|
|
|
Collect the generated code into a Playwright test:
|
|
|
|
```typescript
|
|
import { test, expect } from '@playwright/test';
|
|
|
|
test('login flow', async ({ page }) => {
|
|
// Generated code from mise exec -- npm run playwright:cli -- session:
|
|
await page.goto('https://example.com/login');
|
|
await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
|
|
await page.getByRole('textbox', { name: 'Password' }).fill('password123');
|
|
await page.getByRole('button', { name: 'Sign In' }).click();
|
|
|
|
// Add assertions
|
|
await expect(page).toHaveURL(/.*dashboard/);
|
|
});
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### 1. Use Semantic Locators
|
|
|
|
The generated code uses role-based locators when possible, which are more resilient:
|
|
|
|
```typescript
|
|
// Generated (good - semantic)
|
|
await page.getByRole('button', { name: 'Submit' }).click();
|
|
|
|
// Avoid (fragile - CSS selectors)
|
|
await page.locator('#submit-btn').click();
|
|
```
|
|
|
|
### 2. Explore Before Recording
|
|
|
|
Take snapshots to understand the page structure before recording actions:
|
|
|
|
```bash
|
|
mise exec -- npm run playwright:cli -- open https://example.com
|
|
mise exec -- npm run playwright:cli -- snapshot
|
|
# Review the element structure
|
|
mise exec -- npm run playwright:cli -- click e5
|
|
```
|
|
|
|
### 3. Add Assertions Manually
|
|
|
|
Generated code captures actions but not assertions. Add expectations in your test using one of the recommended matchers:
|
|
|
|
- `toBeVisible()` — element is rendered and visible
|
|
- `toHaveText(text)` — element text content matches
|
|
- `toHaveValue(value) / toBeEmpty()` — input/select value matches
|
|
- `toBeChecked() / toBeUnchecked()` — checkbox state matches
|
|
- `toMatchAriaSnapshot(snapshot)` — page (or locator) matches a partial accessibility snapshot
|
|
|
|
Use `mise exec -- npm run playwright:cli -- generate-locator <target>` to produce the locator expression for the assertion, and the snapshot/eval commands to capture the expected value.
|
|
|
|
When asserting text content, make sure that generated locator does not contain text from the element itself. `getByTestId()` or `getByLabel()` usually work well with asserting text. When locator is text-based, prefer `toBeVisible()` instead.
|
|
|
|
Snapshot to be matched does not have to contain all the information - only capture what's necessary for the assertion. You can use regular expressions for unstable values.
|
|
|
|
```bash
|
|
# Get a stable locator for an element ref to use in the assertion
|
|
mise exec -- npm run playwright:cli -- --raw generate-locator e5
|
|
# getByRole('button', { name: 'Submit' })
|
|
|
|
# Capture expected text content for toHaveText
|
|
mise exec -- npm run playwright:cli -- --raw eval "el => el.textContent" e5
|
|
|
|
# Capture expected input value for toHaveValue/toBeEmpty
|
|
mise exec -- npm run playwright:cli -- --raw eval "el => el.value" e5
|
|
|
|
# Capture expected aria snapshot for toMatchAriaSnapshot/toBeChecked
|
|
# (whole page, or use a ref to scope to a region)
|
|
mise exec -- npm run playwright:cli -- --raw snapshot
|
|
mise exec -- npm run playwright:cli -- --raw snapshot e5
|
|
```
|
|
|
|
```typescript
|
|
// Generated action
|
|
await page.getByRole('button', { name: 'Submit' }).click();
|
|
|
|
// Manual assertions using the outputs above:
|
|
await expect(page.getByRole('alert', { name: 'Success' })).toBeVisible();
|
|
await expect(page.getByTestId('main-header')).toHaveText('Welcome, user');
|
|
await expect(page.getByRole('textbox', { name: 'Email' })).toHaveValue('user@example.com');
|
|
await expect(page.getByRole('checkbox', { name: 'Enable notifications' })).toBeChecked();
|
|
|
|
// toMatchAriaSnapshot on the whole page, finds a matching region
|
|
await expect(page).toMatchAriaSnapshot(`
|
|
- heading "Welcome, user"
|
|
- link /\\d+ new messages?/
|
|
- button "Sign out"
|
|
`);
|
|
|
|
// toMatchAriaSnapshot scoped to a region
|
|
await expect(page.getByRole('navigation')).toMatchAriaSnapshot(`
|
|
- link "Home"
|
|
- link /\\d+ new messages?/
|
|
- link "Profile"
|
|
`);
|
|
```
|