pytest-xdist workers finish at different times (--dist worksteal)

When some pytest-xdist workers sit idle while one still runs tests, the run takes as long as the busiest worker. Use --dist worksteal and split oversized modules or classes.

A parallel test run is only as fast as its busiest worker. If you run with -n 8 and seven workers are done after two minutes while the eighth keeps going for another three, the run takes five minutes and most of the machine was idle for the last three.

That happens when tests have very different durations and the scheduler hands them out without knowing which ones are slow. It gets worse with --dist loadscope or --dist loadfile, where a whole class or file has to run on a single worker.

Why workers finish at different times

pytest-xdist doesn't know how long a test will take until it has run it. With the default --dist load, the controller gives each worker a batch of tests up front and sends more as workers report back. Near the end of the run, a worker that got a few slow tests in its last batch keeps running them while the other workers have nothing left to do.

With grouped modes the units are bigger:

--dist loadscope keeps every test of a class (or a module of test functions) on one worker.
--dist loadfile keeps every test of a file on one worker.
--dist loadgroup keeps every test with the same xdist_group mark on one worker.

One test file that takes 90 seconds means at least one worker runs for 90 seconds, however many workers you have and however short the rest of the suite is.

The fix: --dist worksteal

pytest-xdist 3.2 added a work-stealing scheduler:

$ pytest -n auto --dist worksteal

# pyproject.toml
[tool.pytest.ini_options]
addopts = "-n auto --dist worksteal"

Tests are first split evenly between the workers. When a worker is about to run out of tests, it takes ("steals") part of the remaining queue of another worker. Workers stay busy until the whole queue is empty, so they finish much closer together. The pytest-xdist docs describe it as handling tests with very different durations better than load, while giving similar or better reuse of fixtures.

Split the units that are too big

worksteal distributes individual tests, like load. If you use loadscope, loadfile or loadgroup because tests in a group share an expensive class- or module-scoped fixture, switching modes means that fixture is built on several workers. Two ways out:

Split the oversized group. Move half the tests of the slowest file or class into a second one. Each part can then run on its own worker.
Make the shared fixture cheap enough to build per worker, for example session-scoped (once per worker) instead of tying tests together with loadscope. See expensive function-scoped fixtures.

Check which groups are the long ones first: pytest --durations=30 lists the slowest tests, and a flakytest report groups test time by file and class.

When scheduling can't help

One test is longer than the rest of the run divided by the workers. The run can't finish before that test does. Make the test itself faster (see find your slowest tests), or split it.
Per-worker setup. Each worker imports your code, collects the suite and runs its session fixtures (with pytest-django, it builds its own test database). Every worker pays for that, so no scheduler can parallelize it away; make the setup itself cheaper (see run migrations once).
More workers than the machine has cores. Workers then compete for the CPU and every test gets slower. Use -n auto rather than a fixed number that's too high for CI runners.

Keep the tests correct

Changing the distribution mode changes which tests run together on a worker. Tests that only pass in a particular order or next to particular other tests will show it; that's an isolation bug to fix, not a reason to keep the old mode. Run with pytest-randomly to find them (see keep tests independent).
Tests that need to run in the same process (they share a resource that can't be used concurrently) need loadgroup or a fixture-level lock, not worksteal.

How flakytest finds it

Under pytest-xdist flakytest records which worker ran each test and adds up each worker's test time. When the busiest worker ran at least 5 seconds more than the least busy one, and at least 30% more, it suggests pytest-xdist workers finish at very different times, with both totals. It estimates the saving as half the difference, since a better distribution moves work from the busiest worker to idle ones. After switching to --dist worksteal, run flakytest run again and flakytest compare the two runs.

Related guides

Run tests in parallel with pytest-xdist: Run pytest in parallel on every CPU with pytest-xdist and -n auto. Per-worker databases with pytest-django, what breaks in parallel, pytest-randomly, and the --dist modes.
Find your slowest pytest tests: List the slowest pytest tests with --durations and --durations-min, read setup vs call vs teardown times, then fix, split or mark slow tests so they stop holding up the suite.
Expensive function-scoped pytest fixtures: A pytest fixture without a scope runs again for every test that uses it. Give expensive fixtures module or session scope, or split them into a shared part and a cheap reset.
Speed up pytest in CI: Speed up slow CI test jobs: cache dependencies with uv or pip, use pytest-xdist on every core, shard with pytest-split, migrate once, and see why CI is slower than local.

Updated October 18, 2026 · Suggest a change