Speed up your test suite

Run tests in parallel with pytest-xdist

Run pytest in parallel on every CPU with pytest-xdist and -n auto. Per-worker databases with pytest-django, what breaks in parallel, pytest-randomly, and the --dist modes.

pytest runs one test at a time, in one process, on one CPU core. On an 8-core laptop or CI runner that leaves most of the machine idle. pytest-xdist starts several worker processes and spreads the tests across them, so a suite that's slow because it has many tests (not because of one slow test) finishes several times faster.

It's usually the biggest single speed-up a suite can get, and the one that needs the most care: tests that quietly depend on running alone start failing, sometimes only now and then.

Run tests in parallel

$ pip install pytest-xdist
$ pytest -n auto

-n auto starts one worker per physical CPU core. -n logical uses logical cores (needs psutil), -n 8 an exact number, and -n 0 turns xdist off. To make it the default, add it to your pytest config:

# pyproject.toml
[tool.pytest.ini_options]
addopts = "-n auto"

Running a single test then still starts all the workers; pass -n 0 for quick runs, or only add -n auto in CI.

Each worker is a separate Python process that imports your code and collects the whole suite on its own, then runs the tests the controller sends it. Session-scoped fixtures run once per worker. So startup and session setup are paid N times, which is why xdist pays off for suites that take more than a few seconds, and why a suite dominated by one 30-second test won't get faster than 30 seconds.

Databases: one per worker

pytest-django handles this for you: each worker gets its own test database, with the worker id as a suffix (test_myapp_gw0, test_myapp_gw1, ...). Tests on different workers never see each other's rows. --reuse-db works per worker, so each keeps its own database between runs.

Building each of those databases runs every migration once per worker. On a project with many migrations that adds up; see run migrations once for building one database and copying it.

Outside Django, do the same yourself with the worker_id fixture ("gw0", "gw1", ..., or "master" without xdist):

import pytest
from sqlalchemy import create_engine
from sqlalchemy_utils import create_database, database_exists


@pytest.fixture(scope="session")
def database_url(worker_id):
    name = "test_myapp" if worker_id == "master" else f"test_myapp_{worker_id}"
    url = f"postgresql+psycopg://postgres:postgres@localhost/{name}"
    if not database_exists(url):
        create_database(url)
    return url


@pytest.fixture(scope="session")
def engine(database_url):
    engine = create_engine(database_url)
    yield engine
    engine.dispose()

The worker name is also in the PYTEST_XDIST_WORKER environment variable, and the worker count in PYTEST_XDIST_WORKER_COUNT.

What breaks in parallel

Workers are separate processes, so Python globals aren't shared. What's shared is everything outside the process:

• Files in fixed paths. Two workers writing /tmp/export.csv or tests/output/ overwrite each other. Use tmp_path (per test) or tmp_path_factory (per session), which are unique per worker.
• Ports. A test server on port 8000 can only be started by one worker. Bind to port 0 and read the port the OS picked.
• Shared external state. One Redis database, one S3 bucket, one search index or a shared database without per-worker names. Namespace them by worker_id (a key prefix, a Redis database number, a bucket per worker) or use in-memory fakes.
• Order dependence. A test that only passes after another one ran (it relies on a row, a cache entry or a module global the other left behind) fails when xdist sends them to different workers.
• Once-per-run setup. A session fixture that downloads a file or seeds a shared resource runs on every worker, at the same time. Guard it with a file lock:

import json

import pytest
from filelock import FileLock


@pytest.fixture(scope="session")
def reference_data(tmp_path_factory, worker_id):
    if worker_id == "master":
        return build_reference_data()

    shared = tmp_path_factory.getbasetemp().parent / "reference_data.json"
    with FileLock(f"{shared}.lock"):
        if shared.is_file():
            return json.loads(shared.read_text())
        data = build_reference_data()
        shared.write_text(json.dumps(data))
        return data

Shake out order dependence first: pytest-randomly

Before switching to -n auto, run the suite in random order a few times. Order dependencies show up as failures you can reproduce, instead of flaky failures under xdist:

$ pip install pytest-randomly
$ pytest -p randomly
Using --randomly-seed=2817364581
...
$ pytest -p randomly --randomly-seed=last   # rerun with the same order

pytest-randomly shuffles tests by module, class and function, and resets random's seed (and Faker's, factory_boy's and model_bakery's) before each test. Once installed it's on by default; -p no:randomly turns it off. It works together with xdist. See keep tests independent for fixing what it finds.

The --dist modes

Use loadscope or loadfile when class- or module-scoped fixtures are expensive and you want them built once rather than on every worker. Use loadgroup for tests that must not run at the same time as each other. If workers finish at very different times, try worksteal: see pytest-xdist workers finish at different times.

Keep the tests correct

• Tests must not share state: files in fixed paths, ports, a shared database without per-worker names, or module globals that a test relies on another test having set. Run with -p randomly first to shake out order dependencies.
• A failure that only happens under xdist is a real isolation bug, not noise. Reproduce it with -n 0 -p randomly --randomly-seed=<seed>, or by running the two tests involved together.
• -s and print() output from workers isn't shown live; use -n 0 while debugging a single test.

How flakytest finds it

When a run used one process on a machine with two or more CPUs and test time was at least 20 seconds, flakytest estimates the parallel run time as test time divided by the CPU count, but never less than the slowest test. If that saves at least 10 seconds, it suggests Run tests in parallel on all N CPUs with the estimate, and tells you to install pytest-xdist if it isn't. Under xdist flakytest profiles every worker and records which worker ran each test, so one report covers the whole session, and it tells you when workers finish far apart.

Further reading

• pytest-xdist: running tests across multiple CPUs
• pytest-xdist: identifying the worker process
• pytest-xdist: making session-scoped fixtures execute only once
• pytest-randomly

Related guides

• pytest-xdist workers finish at different times (--dist worksteal): When some pytest-xdist workers sit idle while one still runs tests, the run takes as long as the busiest worker. Use --dist worksteal and split oversized modules or classes.
• Run migrations once with pytest-xdist and CI shards: With pytest-xdist every worker builds its own Django test database, and every CI shard does too. Migrate once and copy it with PostgreSQL template databases or pg_dump.
• Keep tests independent of each other: Tests should pass alone, in any order and in parallel. Find order-dependent pytest tests with pytest-randomly and pytest-xdist, and fix the shared state behind them.
• Speed up pytest in CI: Speed up slow CI test jobs: cache dependencies with uv or pip, use pytest-xdist on every core, shard with pytest-split, migrate once, and see why CI is slower than local.

Updated October 18, 2026 · Suggest a change