# End-to-end testing best practices

> Fast, reliable end-to-end tests: give every test a fresh tenant instead of cleaning the database, set up state through the API, use Playwright locators and auto-waiting.

End-to-end tests drive the real application: a browser or an HTTP client against a running server, a real database, real background workers. They catch what unit tests can't (the pieces don't fit together, the deploy is broken), and they're the slowest and flakiest tests you have. Most of the pain comes from one decision: how each test gets a clean starting point.

## Don't clean the database: give every test its own tenant

The usual approach is to reset state between tests: truncate the tables, delete what the test created, or restore a shared environment to a known snapshot. It looks tidy and is usually the wrong trade:

- **It's expensive.** Truncating every table after every test grows with the schema, and it's the same cost that makes Django's `TransactionTestCase` and pytest-django's `live_server` slow ([TestCase vs TransactionTestCase](https://flakytest.dev/guides/testcase-vs-transactiontestcase/)).
- **It's fragile.** "Delete what the test created" misses rows created by background jobs, webhooks and triggers, and a test that crashes halfway never cleans up at all.
- **It forces serial runs.** If one test wipes the database, no other test can run against it at the same time.
- **It doesn't work against shared environments.** You can't truncate staging while people are using it.
- **It destroys the evidence.** When a test fails, the data that would explain the failure is gone.

The better approach: every test creates its own tenant (a fresh organization, account or workspace with its own user), with a unique name, and only ever looks at data inside it. In a multi-tenant application the tenant is already the isolation boundary the product enforces, so the tests get isolation for free.

```python
# tests/e2e/conftest.py
import os
import uuid

import httpx
import pytest


@pytest.fixture(scope="session")
def api(base_url):
    with httpx.Client(
        base_url=base_url,
        headers={"Authorization": f"Bearer {os.environ['E2E_SETUP_TOKEN']}"},
        timeout=30,
    ) as client:
        yield client


@pytest.fixture
def tenant(api):
    suffix = uuid.uuid4().hex[:12]
    response = api.post(
        "/api/test-support/tenants",
        json={
            "name": f"e2e-{suffix}",
            "owner_email": f"e2e-{suffix}@example.com",
        },
    )
    response.raise_for_status()
    return response.json()  # org id, owner email, a password or a login token
```

`base_url` comes from pytest-base-url, which pytest-playwright installs: run `pytest tests/e2e --base-url https://staging.example.com` and the same tests run against any environment. There's no teardown. The tenant stays where it is.

### Create tenants through the API or a fast backdoor

Create the tenant through your public signup API if it's fast and doesn't send real email or need a CAPTCHA. Otherwise add a test-support endpoint that creates an organization, a user and a session in one call. Protect it: enable it only in test and staging environments, require a secret token, and mark what it creates (an `is_test` flag on the organization) so it can be found later. Never rely on name prefixes alone to identify test data.

### What you get

- **No teardown cost.** Creating one organization is a few inserts; cleaning a database is many.
- **Parallel runs against one environment.** Tests can't see each other's data, so `pytest -n 8` against a single server works.
- **It works against staging and other shared environments**, including smoke tests right after a deploy.
- **Failures leave the data for inspection.** Log the tenant name in the test output and you can log in as that tenant and look at exactly what the test saw.
- **No ordering dependence.** No test relies on what an earlier test left behind or on it being cleaned up.

### What it costs

- **Data accumulates.** Clean up old test tenants with a periodic job (nightly, delete organizations marked `is_test` and older than a few days), not per test. The job runs once and can be slow; the tests stay fast.
- **Tests must never assert on global state.** Assert that *this* organization has one project, never that the admin dashboard shows 1,234 projects or that the newest item in a global feed is the one you just created. Any query that crosses tenants (global counts, "latest signups", search across all accounts) needs to be scoped to the test's tenant, or tested in a lower-level test with its own database.
- **The test tenants are realistic, not empty.** That's mostly a benefit: the database has realistic size and other tenants' data, which flushes out queries that forget to filter by tenant.

Cal Paterson makes a related argument in [The argument against clearing the database between tests](https://calpaterson.com/against-database-teardown.html) (2020): teardown costs time, and tests should only assert on the data they created.

## Keep end-to-end tests few and focused

End-to-end tests are the most expensive way to check a behaviour. Use them for the critical user journeys (sign up, log in, the checkout, the core workflow your customers pay for) and test the logic behind each step with fast unit and integration tests. When an end-to-end test fails, it should mean "this journey is broken", not "a validation message changed". A few dozen good journeys catch more than hundreds of tests that each check a form field.

## Set up state through the API, not the UI

Only the thing the test is about should go through the browser. Logging in, creating the project the test edits and inviting the second user can all happen through the API, in milliseconds. With Playwright, `page.request` shares cookies with the browser context, so an API login logs the page in too:

```python
from playwright.sync_api import Page, expect


def test_owner_can_invite_a_teammate(page: Page, base_url, tenant):
    page.request.post(
        f"{base_url}/api/login",
        data={"email": tenant["owner_email"], "password": tenant["password"]},
    )
    teammate = f"teammate-{tenant['id']}@example.com"

    page.goto("/settings/team")
    page.get_by_role("button", name="Invite teammate").click()
    page.get_by_label("Email").fill(teammate)
    page.get_by_role("button", name="Send invite").click()

    expect(page.get_by_role("row", name=teammate)).to_be_visible()
```

Keep one test that logs in through the real form, so the form itself is covered.

## Locate elements like users do

Find elements by what the user sees: their role, accessible name, label or text. Playwright's `get_by_role()`, `get_by_label()` and `get_by_text()` do this, and they fail when the page stops being usable, not when someone renames a CSS class. Avoid CSS and XPath selectors, and treat test ids as a last resort for elements with no accessible name. For Selenium, [Selenium Testing Library](https://blog.pecar.me/selenium-testing-library/) brings the same queries (by role, by label) to Selenium tests.

## Wait for conditions, never sleep

`time.sleep(2)` is either too long (slow) or too short (flaky), often both on different machines. Playwright waits for you: actions like `click()` and `fill()` wait until the element is visible, enabled and stable, and `expect(...)` assertions retry until the condition holds or the timeout expires:

```python
expect(page.get_by_text("Payment received")).to_be_visible(timeout=10_000)
expect(page.get_by_role("heading", name="Your orders")).to_be_visible()
```

For background work outside the browser (an email arriving, a job finishing), poll the API for the condition with a deadline. [Tests are sleeping](https://flakytest.dev/guides/sleep-in-tests/) shows the pattern.

## Share the browser, isolate the context

Launching a browser per test costs seconds. Launch it once per session and give each test a fresh context, which has its own cookies and storage. pytest-playwright does this by default: `browser` is session-scoped, `context` and `page` are per test. With Selenium, use a session-scoped driver and delete cookies between tests. See [share the browser between tests](https://flakytest.dev/guides/share-browser-between-tests/).

## Run them in their own CI job

Run end-to-end tests in a separate CI job, in parallel with the unit tests, so a slow browser suite doesn't hold up feedback from fast tests and its failures are easy to tell apart. With tenant-per-test isolation they can also run in parallel with each other (`pytest -n auto`). [Speed up tests in CI](https://flakytest.dev/guides/speed-up-tests-in-ci/) has more on splitting jobs.

## Record what happened on failure

An end-to-end failure in CI is hard to debug from a stack trace. pytest-playwright can keep a trace, a screenshot and a video of failing tests:

```console
$ pytest tests/e2e --tracing retain-on-failure --screenshot only-on-failure --video retain-on-failure
```

They go to `test-results/` (change it with `--output`); upload that directory as a CI artifact and open traces with `playwright show-trace`. Log the tenant's name too, so you can look at its data.

## Mute flaky tests instead of retrying them

End-to-end tests are where flakiness concentrates: timing, network, shared infrastructure. Retrying them until they pass hides the failure and makes the suite slower ([retrying flaky tests isn't a fix](https://flakytest.dev/guides/retrying-flaky-tests/)). Mute the flaky test instead, and fix it: on flakytest.dev you mute a test from its page, for a week by default, with no code change. It still runs and is still reported, but its failure doesn't fail the build. See [mute flaky tests](https://flakytest.dev/guides/mute-flaky-tests/) and [common causes of flaky tests](https://flakytest.dev/guides/common-causes-of-flaky-tests/).

## Further reading

- [Cal Paterson: The argument against clearing the database between tests](https://calpaterson.com/against-database-teardown.html)
- [Playwright for Python: locators](https://playwright.dev/python/docs/locators)
- [Playwright for Python: auto-waiting](https://playwright.dev/python/docs/actionability)
- [Playwright for Python: pytest plugin](https://playwright.dev/python/docs/test-runners)
- [Playwright for Python: trace viewer](https://playwright.dev/python/docs/trace-viewer)