Passed Run of Oct 18, 16:10
Where the time goes
Test time by category- Password hashing21.22 s
- Migrations17.23 s
- Libraries11.18 s
- Database7.18 s
- DB schema4.23 s
- Templates1.59 s
- Imports1.18 s
- Mocking1.14 s
- pytest374 ms
- Factories290 ms
- Unsampled186 ms
- Cache146 ms
- App code66 ms
- Deep copies46 ms
- Waiting15 ms
- Test code13 ms
- Network12 ms
- Serialization9 ms
- Logging5 ms
Session timeline
Fix next
Ranked by the time each fix saves per runUse a fast password hasher in tests
21.2s (32% of test time) is spent hashing passwords. Slow hashing protects real passwords; in tests it only burns CPU.
Details
Only change the hasher in test settings. Tests that assert on a specific hash format need updating.
Where:
- confs/tests.py:1044 TestCreateConference.setUp · 3.93 s
- confs/tests.py:1175 TestEditConference.setUp · 2.73 s
- confs/tests.py:1176 TestEditConference.setUp · 2.63 s
- confs/tests.py:1177 TestEditConference.setUp · 2.60 s
- confs/tests.py:1372 TestApproveConference.setUp · 2.13 s
- confs/tests.py:1371 TestApproveConference.setUp · 2.11 s
- confs/tests.py:1316 TestConferenceApprovalVisibility.setUp · 1.66 s
- confs/tests.py:1317 TestConferenceApprovalVisibility.setUp · 1.65 s
- confs/tests.py:1295 TestUnapproveConference.setUp · 488 ms
- confs/tests.py:1296 TestUnapproveConference.setUp · 485 ms
Most affected tests:
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_owner_can_edit
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestConferenceApprovalVisibility::test_owner_sees_own_pending
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- fedidevs/tests.py::TestAnonymousCacheAndCsrf::test_authenticated_html_is_private_and_keeps_csrf_header
Use plain static file storage in tests
Static file storage/manifest handling costs 15.7s.
Details
Where:
- confs/views.py:296 _conference_detail · 669 ms
- starter_packs/tests.py:841 TestShareStarterPack.test_api_formats · 653 ms
- confs/tests.py:152 TestMigratedLegacyConferences._assert_no_naive_warning · 457 ms
- accounts/views.py:425 index · 413 ms
- starter_packs/views.py:862 share_starter_pack · 333 ms
- starter_packs/tests.py:1248 TestRemoteFollowStarterPack.test_share_page_hides_remote_follow_form_when_logged_in · 303 ms
- starter_packs/tests.py:781 TestShareStarterPack.test_order_by · 260 ms
- accounts/tests.py:550 TestAccountIndexRobots.assert_noindex · 252 ms
- confs/tests.py:1132 TestCreateConference.test_mastodon_url_is_kept · 246 ms
- starter_packs/tests.py:311 TestAddAccountsSearch.test_username_search_shows_unchecked_for_new_account · 242 ms
Most affected tests:
- confs/tests.py::TestConferencePage::test_conference_with_pacific_with_date
- starter_packs/tests.py::TestShareStarterPack::test_api_formats_4___application_json____application_json__
- starter_packs/tests.py::TestRemoteFollowStarterPack::test_share_page_hides_remote_follow_form_when_logged_in
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconafrica_page
- starter_packs/tests.py::TestStarterPacks::test_index_page_language_filter
- starter_packs/tests.py::TestAddAccountsSearch::test_username_search_shows_unchecked_for_new_account
- accounts/tests.py::TestStaticPages::test_developers_on_mastodon
- confs/tests.py::TestCreateConference::test_mastodon_url_is_kept
Stop re-running migrations on every test run
Migrations take 17.2s per run, before the first test can start.
Details
CI should still build the database from scratch (no --reuse-db) so broken migrations are caught.
Where:
- confs/tests.py:432 TestLegacyConferenceDataMigration._migrate · 1.07 s
- confs/migrations/0031_seed_conference_tags.py:14 seed_conference_tags · 471 ms
- confs/tests.py:412 TestLegacyConferenceDataMigration._leaf_targets · 120 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:388 _get_or_create_conference · 71 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:214 _pk_batches · 63 ms
- confs/tests.py:431 TestLegacyConferenceDataMigration._migrate · 56 ms
- confs/tests.py:430 TestLegacyConferenceDataMigration._migrate · 51 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:398 migrate_legacy_pair · 37 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:325 _refresh_account_counts · 33 ms
- confs/tests.py:486 TestLegacyConferenceDataMigration._seed_overlapping_legacy_data · 23 ms
Most affected tests:
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestLegacyConferenceDataMigration::test_copies_overlapping_legacy_rows_without_clobbering_existing_data
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
Create shared test data in setUpTestData instead of setUp
setUp() runs before every test and takes 20.6s in total (password hashing 20.4s, db 158ms, factories 12ms) in 7 classes: TestPostAnnouncements (6 tests), TestFollowStarterPack (3 tests), TestConferenceApprovalVisibility (5 tests), TestApproveConference (6 tests).
Details
Since Django 3.2 attributes assigned in setUpTestData are deep-copied per test, so in-memory changes don't leak between tests. Keep objects a test modifies in the database (or re-fetches) as before.
Where:
- confs/tests.py:1044 TestCreateConference.setUp · 3.94 s
- confs/tests.py:1175 TestEditConference.setUp · 2.74 s
- confs/tests.py:1176 TestEditConference.setUp · 2.64 s
- confs/tests.py:1177 TestEditConference.setUp · 2.61 s
- confs/tests.py:1372 TestApproveConference.setUp · 2.13 s
- confs/tests.py:1371 TestApproveConference.setUp · 2.12 s
- confs/tests.py:1316 TestConferenceApprovalVisibility.setUp · 1.68 s
- confs/tests.py:1317 TestConferenceApprovalVisibility.setUp · 1.66 s
- confs/tests.py:1295 TestUnapproveConference.setUp · 491 ms
- confs/tests.py:1296 TestUnapproveConference.setUp · 487 ms
Most affected tests:
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_owner_can_edit
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestConferenceApprovalVisibility::test_owner_sees_own_pending
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- confs/tests.py::TestConferenceApprovalVisibility::test_other_user_does_not_see_pending
Repeated queries (N+1) in the code under test
10 tests run the same query 10+ times, e.g. 45x in test_invalid_selected_instance. N+1 queries slow the tests and production alike.
Details
This changes application code, so it speeds up production too; the tests stay as they are.
- 45x in test_invalid_selected_instance: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_single_facet_is_indexable: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_djangoconus_old_url_redirects: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_conference_with_utc_with_date_no_posts: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 44x in test_posts_from_yesterday_are_updated: SELECT COUNT(*) AS "__count" FROM "accounts_accountlookup" WHERE UPPER("accounts_accountlookup"."language"::text) LIKE UPPER(%s)
Most affected tests:
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- stats/tests.py::TestStats::test_posts_from_yesterday_are_updated
- stats/tests.py::TestStats::test_store_daily_stats
- stats/tests.py::TestStats::test_store_daily_stats_with_accounts
- accounts/tests.py::TestInstanceIndexingBridgy::test_follows_redirect_and_coerces_null_version
Disable django-debug-toolbar in tests
django-debug-toolbar is installed in the test settings and costs 5.9s.
Details
Where:
- confs/views.py:296 _conference_detail · 669 ms
- accounts/views.py:425 index · 413 ms
- starter_packs/views.py:862 share_starter_pack · 333 ms
- confs/tests.py:1132 TestCreateConference.test_mastodon_url_is_kept · 209 ms
- confs/views.py:128 conferences · 156 ms
- starter_packs/views.py:858 share_starter_pack · 135 ms
- confs/views.py:509 create_conference · 126 ms
- accounts/views.py:28 CountlessPage.__init__ · 125 ms
- starter_packs/views.py:60 get_editable_pack · 118 ms
- starter_packs/views.py:185 starter_packs · 111 ms
Most affected tests:
- confs/tests.py::TestConferencePage::test_conference_with_pacific_with_date
- confs/tests.py::TestCreateConference::test_mastodon_url_is_kept
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestMigratedLegacyConferences::test_dotnetconf_page
- fedidevs/tests.py::TestAnonymousCacheAndCsrf::test_anonymous_forms_work_after_csrf_bootstrap
- confs/tests.py::TestConferencePage::test_posts_after_filter_does_not_warn_naive_datetime
Make the test database server skip durability work
Tests spend 28.6s (43%) in the database. Test databases don't need to survive a crash, so the server can skip flushing to disk.
Details
Only affects crash safety of the throwaway test server; query behaviour is identical.
Where:
- confs/tests.py:432 TestLegacyConferenceDataMigration._migrate · 1.07 s
- confs/migrations/0031_seed_conference_tags.py:14 seed_conference_tags · 471 ms
- starter_packs/tests.py:397 TestToggleStarterPackAccount.test_toggle_after_limit · 453 ms
- starter_packs/views.py:862 share_starter_pack · 152 ms
- starter_packs/views.py:858 share_starter_pack · 135 ms
- accounts/views.py:28 CountlessPage.__init__ · 125 ms
- stats/models.py:20 store_daily_stats · 124 ms
- confs/tests.py:412 TestLegacyConferenceDataMigration._leaf_targets · 120 ms
- starter_packs/views.py:60 get_editable_pack · 118 ms
- accounts/views.py:295 index · 106 ms
Most affected tests:
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestLegacyConferenceDataMigration::test_copies_overlapping_legacy_rows_without_clobbering_existing_data
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
- mastodon_auth/tests.py::RegisterAppTests::test_returns_credentials_on_200
- mastodon_auth/tests.py::ParseDtTests::test_date_only_string_becomes_aware_midnight
pytest takes a long time before the first test runs
Startup takes 7.1s and collection 3.4s on every run, even when you only run one test.
Details
- mastodon_auth/tests.py (892ms)
- starter_packs/tests.py (229ms)
- accounts/tests.py (45ms)
- confs/tests.py (12ms)
- stats/tests.py (5ms)
- fedidevs/settings.py (1.5s)
- debug_toolbar/panels/sql/__init__.py (876ms)
- starter_packs/admin.py (640ms)
pytest-xdist workers finish at very different times
The busiest worker ran 20.7s of tests, the least busy 14.7s.
Details
Run migrations once, not once per worker
Each of the 4 pytest-xdist workers builds its own test database by running every migration: 18.6s in total, about 4.6s per worker. 13.9s of it repeats work another worker already did.
Details
Every copy is built by the same migrations, so a broken migration still fails the run.
- {'total': 18.583, 'builds': 4, 'workers': 4, 'per_build': 4.646}
Tests
1–200 of 301, slowest first · click a column to sort this pageMemory
Busiest worker peaked at 209 MB; all 4 together at about 676 MB| Worker | Tests | Peak | Grew by |
|---|---|---|---|
| gw0 | 139 | 192 MB | 51 MB |
| gw1 | 96 | 204 MB | 60 MB |
| gw2 | 33 | 209 MB | 33 MB |
| gw3 | 33 | 179 MB | 38 MB |
Slowest fixtures
Exact setup time across the session| Fixture | Scope | Calls | Total |
|---|---|---|---|
| _django_setup_unittest | class | 68 | 18.61 s |
| django_db_setup | session | 4 | 18.58 s |
| _unittest_setUpClass_fixture_TestSelectedInstance | class | 1 | 106 ms |
| _unittest_setUpClass_fixture_TestShareStarterPack | class | 2 | 100 ms |
| _unittest_setUpClass_fixture_TestConferencePage | class | 2 | 84 ms |
| _unittest_setUpClass_fixture_TestOwnerManagement | class | 2 | 71 ms |
| _unittest_setUpClass_fixture_TestPinAccountInStarterPack | class | 2 | 64 ms |
| _unittest_setUpClass_fixture_TestWebfingerBackfill | class | 1 | 54 ms |
| _unittest_setUpClass_fixture_TestFollowStarterPack | class | 1 | 42 ms |
| _unittest_setUpClass_fixture_TestFollowBg | class | 1 | 41 ms |
| _unittest_setUpClass_fixture_TestCreateStarterPack | class | 2 | 39 ms |
| _unittest_setUpClass_fixture_TestRemoteFollowStarterPack | class | 2 | 33 ms |
| _unittest_setUpClass_fixture_TestAddAccountsSearch | class | 1 | 28 ms |
| _unittest_setUpClass_fixture_TestStarterPacks | class | 1 | 26 ms |
| _unittest_setUpClass_fixture_TestEditStarterPack | class | 1 | 26 ms |