All environments · last 90 days
Test suite performance
Suite wall time
45.48 s
avg over 4 profiled runs
Total test time
2m 11s
sum of test durations per run
Tests
301
in 1 environment
Failure rate
0.1%
1 of 1,505 results
Suite wall time per run
Click a point to open the runWhere the time goes
Avg test time per run
CI
github-actions:test
2m 11s
Database
Network
Password hashing
App code
Factories
Migrations
Other (Libraries, DB schema, Mocking, Imports, ...)
Slowest tests
All tests →Most memory
All tests by memory →Fix next
From the latest run per environment, by time saved
1
Use a fast password hasher in tests
21.2s (32% of test time) is spent hashing passwords. Slow hashing protects real passwords; in tests it only burns CPU.
High
Password hashing
ci:github-actions:test
20.15 s
per run
Fix: In your test settings set PASSWORD_HASHERS = ["django.contrib.auth.hashers.MD5PasswordHasher"] (currently PBKDF2PasswordHasher).
Details
Only change the hasher in test settings. Tests that assert on a specific hash format need updating.
Where:
- confs/tests.py:1044 TestCreateConference.setUp · 3.93 s
- confs/tests.py:1175 TestEditConference.setUp · 2.73 s
- confs/tests.py:1176 TestEditConference.setUp · 2.63 s
- confs/tests.py:1177 TestEditConference.setUp · 2.60 s
- confs/tests.py:1372 TestApproveConference.setUp · 2.13 s
- confs/tests.py:1371 TestApproveConference.setUp · 2.11 s
- confs/tests.py:1316 TestConferenceApprovalVisibility.setUp · 1.66 s
- confs/tests.py:1317 TestConferenceApprovalVisibility.setUp · 1.65 s
- confs/tests.py:1295 TestUnapproveConference.setUp · 488 ms
- confs/tests.py:1296 TestUnapproveConference.setUp · 485 ms
Most affected tests:
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_owner_can_edit
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestConferenceApprovalVisibility::test_owner_sees_own_pending
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- fedidevs/tests.py::TestAnonymousCacheAndCsrf::test_authenticated_html_is_private_and_keeps_csrf_header
2
Use plain static file storage in tests
Static file storage/manifest handling costs 15.7s.
High
App code
ci:github-actions:test
15.74 s
per run
Fix: In test settings use STORAGES["staticfiles"] = {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"} so templates don't need a manifest.
Details
Where:
- confs/views.py:296 _conference_detail · 669 ms
- starter_packs/tests.py:841 TestShareStarterPack.test_api_formats · 653 ms
- confs/tests.py:152 TestMigratedLegacyConferences._assert_no_naive_warning · 457 ms
- accounts/views.py:425 index · 413 ms
- starter_packs/views.py:862 share_starter_pack · 333 ms
- starter_packs/tests.py:1248 TestRemoteFollowStarterPack.test_share_page_hides_remote_follow_form_when_logged_in · 303 ms
- starter_packs/tests.py:781 TestShareStarterPack.test_order_by · 260 ms
- accounts/tests.py:550 TestAccountIndexRobots.assert_noindex · 252 ms
- confs/tests.py:1132 TestCreateConference.test_mastodon_url_is_kept · 246 ms
- starter_packs/tests.py:311 TestAddAccountsSearch.test_username_search_shows_unchecked_for_new_account · 242 ms
Most affected tests:
- confs/tests.py::TestConferencePage::test_conference_with_pacific_with_date
- starter_packs/tests.py::TestShareStarterPack::test_api_formats_4___application_json____application_json__
- starter_packs/tests.py::TestRemoteFollowStarterPack::test_share_page_hides_remote_follow_form_when_logged_in
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconafrica_page
- starter_packs/tests.py::TestStarterPacks::test_index_page_language_filter
- starter_packs/tests.py::TestAddAccountsSearch::test_username_search_shows_unchecked_for_new_account
- accounts/tests.py::TestStaticPages::test_developers_on_mastodon
- confs/tests.py::TestCreateConference::test_mastodon_url_is_kept
3
Stop re-running migrations on every test run
Migrations take 17.2s per run, before the first test can start.
High
Migrations
ci:github-actions:test
15.51 s
per run
Fix: Run with --reuse-db (add it to addopts in pytest.ini/pyproject) so the test database survives between runs; pass --create-db when migrations change. If no test depends on data migrations, --nomigrations builds the schema straight from the models, which is faster still.
Details
CI should still build the database from scratch (no --reuse-db) so broken migrations are caught.
Where:
- confs/tests.py:432 TestLegacyConferenceDataMigration._migrate · 1.07 s
- confs/migrations/0031_seed_conference_tags.py:14 seed_conference_tags · 471 ms
- confs/tests.py:412 TestLegacyConferenceDataMigration._leaf_targets · 120 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:388 _get_or_create_conference · 71 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:214 _pk_batches · 63 ms
- confs/tests.py:431 TestLegacyConferenceDataMigration._migrate · 56 ms
- confs/tests.py:430 TestLegacyConferenceDataMigration._migrate · 51 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:398 migrate_legacy_pair · 37 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:325 _refresh_account_counts · 33 ms
- confs/tests.py:486 TestLegacyConferenceDataMigration._seed_overlapping_legacy_data · 23 ms
Most affected tests:
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestLegacyConferenceDataMigration::test_copies_overlapping_legacy_rows_without_clobbering_existing_data
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
4
Create shared test data in setUpTestData instead of setUp
setUp() runs before every test and takes 20.6s in total (password hashing 20.4s, db 158ms, factories 12ms) in 7 classes: TestPostAnnouncements (6 tests), TestFollowStarterPack (3 tests), TestConferenceApprovalVisibility (5 tests), TestApproveConference (6 tests).
High
Database
ci:github-actions:test
14.42 s
per run
Fix: Move object creation that tests only read from setUp() into `@classmethod def setUpTestData(cls)`: it runs once per class and each test still sees the original rows (TestCase rolls back to a savepoint).
Details
Since Django 3.2 attributes assigned in setUpTestData are deep-copied per test, so in-memory changes don't leak between tests. Keep objects a test modifies in the database (or re-fetches) as before.
Where:
- confs/tests.py:1044 TestCreateConference.setUp · 3.94 s
- confs/tests.py:1175 TestEditConference.setUp · 2.74 s
- confs/tests.py:1176 TestEditConference.setUp · 2.64 s
- confs/tests.py:1177 TestEditConference.setUp · 2.61 s
- confs/tests.py:1372 TestApproveConference.setUp · 2.13 s
- confs/tests.py:1371 TestApproveConference.setUp · 2.12 s
- confs/tests.py:1316 TestConferenceApprovalVisibility.setUp · 1.68 s
- confs/tests.py:1317 TestConferenceApprovalVisibility.setUp · 1.66 s
- confs/tests.py:1295 TestUnapproveConference.setUp · 491 ms
- confs/tests.py:1296 TestUnapproveConference.setUp · 487 ms
Most affected tests:
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_owner_can_edit
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestConferenceApprovalVisibility::test_owner_sees_own_pending
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- confs/tests.py::TestConferenceApprovalVisibility::test_other_user_does_not_see_pending
CI vs local
Tests that are much slower in CI, and whyUpload profiles from CI and from a developer machine to compare them.
Locally: FLAKYTEST_SECRET_TOKEN=… pytest --flakytest-profile
Failing and flaky tests
1 regression caught
| Test | Verdict | CI fails | Local fails |
|---|---|---|---|
| starter_packs/tests.py::TestShareStarterPack::test_api_plain_json | Regression caught | 1/5 | 0/0 |
Flaky: failed and passed on the same commit. Regression caught: fixed by a change outside the test file. Test changed: fixed by editing the test. All failing tests →
Environments
Every CI job and developer machine that reported in the last 90 days| Environment | Profiled runs | Avg wall time |
|---|---|---|
| CI github-actions:test | 4 | 45.48 s |
| CI github-actions | 0 | – |