# Keep tests independent of each other

> Tests should pass alone, in any order and in parallel. Find order-dependent pytest tests with pytest-randomly and pytest-xdist, and fix the shared state behind them.

An isolated test passes when you run it alone, when you run it after any other test, and when other tests run at the same time. A test that only passes in the order your suite happens to run today is a flaky test waiting for a trigger: a new test file, a renamed module, `pytest -n auto`, or a teammate running one file to debug it.

Order-dependent failures are miserable to debug because the broken test isn't the one that fails. Test A leaves something behind, test B trips over it, and B's traceback says nothing about A.

## Three rules

1. **Every test passes alone.** `pytest tests/test_orders.py::test_refund` must work without running anything else first.
2. **Every test passes in any order.** No test relies on state another test created or cleaned up.
3. **Every test passes in parallel.** Two tests running at the same time on different workers don't share anything they both write to.

## Where coupling hides

Anything that outlives a test can couple it to the next one.

| Shared thing | How it leaks | Fix |
|---|---|---|
| Database rows | Data created outside the per-test transaction (at module import, in `setUpClass`, in session fixtures that commit, e.g. through pytest-django's `django_db_blocker`) | Create data inside the test's transaction: `TestCase` + `setUpTestData`, or a rollback fixture ([SQLAlchemy](https://flakytest.dev/guides/sqlalchemy-testing-best-practices/)) |
| Hardcoded IDs | `get(pk=1)` only works if this test's row happened to get id 1 | Use the object's `.pk`; never assume sequence values |
| Module globals | A test sets `config.FEATURE = True` and never resets it | `monkeypatch.setattr` or `override_settings` |
| Caches | `functools.lru_cache`, Django's cache, a memoized client hold values from an earlier test | `cache_clear()` / `cache.clear()` in an autouse fixture |
| Singletons and registries | Signal receivers, plugin registries, a lazily created client configured by the first test that uses it | Undo registrations in teardown; build the object per test or reset it |
| Environment variables | `os.environ["KEY"] = ...` stays for the rest of the run | `monkeypatch.setenv` / `delenv` |
| Files | Writing to the working directory or a fixed `/tmp` path | `tmp_path` |
| Time | Code compares against the real clock; the test passes until midnight or month end | time-machine with a fixed time ([freezegun to time-machine](https://flakytest.dev/guides/freezegun-to-time-machine/)) |
| Random data | The test passes for most generated values, or depends on the sequence `random` produced in earlier tests | Fix the seed per test (pytest-randomly does) and make edge cases explicit |
| Patches without undo | `mock.patch(...).start()` without `stop()`, or assigning a fake to a module attribute | `with mock.patch(...)`, the decorator form, or `monkeypatch` |
| Threads and tasks | A background thread or asyncio task keeps running into the next test | Stop and join them in teardown |

The common thread: every change to shared state needs an undo that runs even when the test fails. pytest's `monkeypatch` and `tmp_path`, Django's `override_settings` and `TestCase`, and context managers all give you that. [pytest fixtures best practices](https://flakytest.dev/guides/pytest-fixtures-best-practices/) covers fixtures that clean up after themselves.

## Shuffle the order with pytest-randomly

pytest runs tests in file order, so the same accidental dependencies keep working. pytest-randomly shuffles the order on every run (modules, then classes, then the tests within them) and resets `random.seed()` before each test, so hidden dependencies surface:

```console
$ uv add --dev pytest-randomly
$ pytest
Using --randomly-seed=1553614239
...
```

It also reseeds factory_boy, Faker, model_bakery and NumPy's legacy global random state at the start of each test, so generated data is reproducible from the seed. When a run fails, rerun with the same seed to get the same order and the same data:

```console
$ pytest --randomly-seed=1553614239
$ pytest --randomly-seed=last        # the previous run's seed (needs pytest's cache)
```

Other switches:

- `-p no:randomly` turns the plugin off for one run, for example to check that a failure is order-related.
- `--randomly-dont-reorganize` keeps the order but still resets the seed per test.
- `--randomly-dont-reset-seed` shuffles without touching `random`.

If you run pytest with plugin autoloading disabled (`PYTEST_DISABLE_PLUGIN_AUTOLOAD=1`), load it explicitly with `-p pytest_randomly`.

Django's own runner has the same idea built in: `manage.py test --shuffle` (with an optional seed to repeat an order) and `--reverse`, which runs tests in the opposite order.

## Use pytest-xdist as an isolation check

Running in parallel catches a second class of problems: tests that pass in any order but break when they run at the same time.

```console
$ pytest -n 4
```

With pytest-xdist each worker is a separate process with its own globals, and tests are spread across workers differently on each run, so a test that relied on another test running earlier in the same process starts failing. Tests that share something outside the process fail too: a fixed file path, a fixed port, a Redis database, a row with a fixed unique name in a shared database. pytest-django gives each worker its own test database; for everything else, namespace by worker with the `worker_id` fixture (or the `PYTEST_XDIST_WORKER` environment variable) or use `tmp_path` and port 0. [Run tests in parallel with pytest-xdist](https://flakytest.dev/guides/pytest-xdist/) has the setup.

## Run one test alone

The cheapest check is the one people forget. A test that only passes after the rest of its file has run depends on something an earlier test (or a module-level side effect) set up:

```console
$ pytest "tests/test_orders.py::TestRefunds::test_partial_refund"
```

When a test fails in the full run but passes alone, something earlier polluted it. When it passes in the full run but fails alone, it depends on something earlier. Either way the next step is to find the other test.

## Find the test that pollutes another

Bisecting by hand is slow. detect-test-pollution automates it: give it the failing test and the suite, and it repeatedly runs halves of the tests before it until it finds the one that causes the failure:

```console
$ uv add --dev detect-test-pollution
$ detect-test-pollution --failing-test tests/test_orders.py::test_refund --tests tests/
```

A pytest-randomly seed that reproduces the failure gives you a fixed order to work with: rerun with `--randomly-seed=<seed>` and the polluter is one of the tests that ran before the failing one in the same process.

## Order dependence looks like flakiness

A test that depends on order often looks flaky: it passes on one CI run and fails on the next, on the same commit, because something else changed the order (pytest-randomly, xdist distribution, a new test). On flakytest.dev a failure streak where a test both failed and passed on the same commit is classified as flaky, so order-dependent tests show up there alongside timing problems. [Find flaky tests](https://flakytest.dev/guides/find-flaky-tests/) and [common causes of flaky tests](https://flakytest.dev/guides/common-causes-of-flaky-tests/) cover the other causes and how to tell them apart.

## Isolation and speed pull in different directions

Wider fixture scopes, `setUpTestData` and a shared browser all make tests faster by sharing things, and every shared thing is a chance to leak. That's not a reason to avoid them, but share read-only things, give each test a cheap reset (a rollback, a fresh browser context, `flushdb()`), and keep pytest-randomly on so leaks show up the day they're introduced. [Expensive function-scoped pytest fixtures](https://flakytest.dev/guides/pytest-fixture-scope/) shows how to widen a scope without sharing mutable state.

## Further reading

- [pytest-randomly](https://github.com/pytest-dev/pytest-randomly)
- [pytest-xdist](https://pytest-xdist.readthedocs.io/en/stable/)
- [detect-test-pollution](https://github.com/asottile/detect-test-pollution)
- [Django: `test --shuffle` and `--reverse`](https://docs.djangoproject.com/en/stable/ref/django-admin/#cmdoption-test-shuffle)
- [pytest: how to monkeypatch](https://docs.pytest.org/en/stable/how-to/monkeypatch.html)