Passed Run of Oct 18, 11:03
pytest --flakytest-profile --flakytest-rate 2000.
Where the time goes
Test time by category- Password hashing50.46 s
- Migrations39.56 s
- Libraries38.03 s
- Database12.33 s
- DB schema4.77 s
- Mocking3.66 s
- Imports2.83 s
- Templates2.76 s
- pytest778 ms
- Factories726 ms
- Unsampled299 ms
- App code87 ms
- Waiting53 ms
- Deep copies52 ms
- Test code47 ms
- Network40 ms
- Cache36 ms
- Logging32 ms
- Serialization4 ms
Session timeline
Fix next
Ranked by the time each fix saves per runUse plain static file storage in tests
Static file storage/manifest handling costs 48.1s.
Details
Where:
- starter_packs/tests.py:840 test_api_formats · 1.78 s
- accounts/tests.py:550 assert_noindex · 1.34 s
- accounts/views.py:428 index · 1.24 s
- accounts/tests.py:555 assert_indexable · 1.01 s
- starter_packs/tests.py:780 test_order_by · 997 ms
- confs/tests.py:152 _assert_no_naive_warning · 779 ms
- confs/tests.py:1132 test_mastodon_url_is_kept · 755 ms
- starter_packs/tests.py:1036 test_no_pin_button_on_share_page · 734 ms
- confs/tests.py:884 test_conference_with_pacific_with_date · 571 ms
- confs/views.py:299 _conference_detail · 493 ms
Most affected tests:
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestCreateConference::test_mastodon_url_is_kept
- starter_packs/tests.py::TestPinAccountInStarterPack::test_no_pin_button_on_share_page
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestConferencePage::test_conference_with_pacific_with_date
- starter_packs/tests.py::TestShareStarterPack::test_api_formats_1___application_ld_json__profile__https___www_w3_org_ns_activitystreams_____application_activity_json__
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- starter_packs/tests.py::TestEditStarterPack::test_not_owned_starter_pack
Use a fast password hasher in tests
50.5s (32% of test time) is spent hashing passwords. Slow hashing protects real passwords; in tests it only burns CPU.
Details
Only change the hasher in test settings. Tests that assert on a specific hash format need updating.
Where:
- confs/tests.py:1044 setUp · 9.89 s
- confs/tests.py:1175 setUp · 8.41 s
- confs/tests.py:1177 setUp · 6.78 s
- confs/tests.py:1176 setUp · 5.52 s
- confs/tests.py:1372 setUp · 4.71 s
- confs/tests.py:1371 setUp · 4.27 s
- confs/tests.py:1317 setUp · 3.93 s
- confs/tests.py:1316 setUp · 2.93 s
- confs/tests.py:1296 setUp · 1.25 s
- fedidevs/tests.py:62 test_authenticated_html_is_private_and_keeps_csrf_header · 1.19 s
Most affected tests:
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_requires_login
- confs/tests.py::TestEditConference::test_owner_sees_edit_button_on_approved_detail
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestApproveConference::test_non_staff_forbidden
Stop re-running migrations on every test run
Migrations take 39.6s per run, before the first test can start.
Details
CI should still build the database from scratch (no --reuse-db) so broken migrations are caught.
Where:
- confs/tests.py:432 _migrate · 2.20 s
- confs/migrations/0031_seed_conference_tags.py:14 seed_conference_tags · 704 ms
- confs/tests.py:412 _leaf_targets · 267 ms
- confs/tests.py:431 _migrate · 129 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:388 _get_or_create_conference · 86 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:214 _pk_batches · 50 ms
- confs/migrations/0011_conferencepost_alter_conference_posts.py:11 update · 41 ms
- confs/migrations/0033_remove_legacy_conference_tables.py:398 migrate_legacy_pair · 38 ms
- starter_packs/migrations/0004_starterpack_published_at.py:9 set_published_at · 38 ms
- accounts/migrations/0035_fix_misskey_activitypub_id.py:22 fix_misskey_activitypub_ids · 37 ms
Most affected tests:
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestLegacyConferenceDataMigration::test_copies_overlapping_legacy_rows_without_clobbering_existing_data
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
Create shared test data in setUpTestData instead of setUp
setUp() runs before every test and takes 48.9s in total (password hashing 48.6s, db 326ms, factories 17ms) in 7 classes: TestPostAnnouncements (6 tests), TestFollowStarterPack (3 tests), TestCreateConference (10 tests), TestConferenceApprovalVisibility (5 tests).
Details
Since Django 3.2 attributes assigned in setUpTestData are deep-copied per test, so in-memory changes don't leak between tests. Keep objects a test modifies in the database (or re-fetches) as before.
Where:
- confs/tests.py:1044 setUp · 9.94 s
- confs/tests.py:1175 setUp · 8.42 s
- confs/tests.py:1177 setUp · 6.80 s
- confs/tests.py:1176 setUp · 5.54 s
- confs/tests.py:1372 setUp · 4.72 s
- confs/tests.py:1371 setUp · 4.28 s
- confs/tests.py:1317 setUp · 3.94 s
- confs/tests.py:1316 setUp · 2.94 s
- confs/tests.py:1296 setUp · 1.25 s
- confs/tests.py:1295 setUp · 883 ms
Most affected tests:
- confs/tests.py::TestEditConference::test_owner_can_edit_after_approval
- confs/tests.py::TestEditConference::test_edit_does_not_change_slug
- confs/tests.py::TestEditConference::test_requires_login
- confs/tests.py::TestEditConference::test_owner_sees_edit_button_on_approved_detail
- confs/tests.py::TestEditConference::test_other_user_forbidden
- confs/tests.py::TestEditConference::test_other_user_forbidden_on_approved_conference
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- confs/tests.py::TestApproveConference::test_non_staff_forbidden
Repeated queries (N+1) in the code under test
10 tests run the same query 10+ times, e.g. 45x in test_single_facet_is_indexable. N+1 queries slow the tests and production alike.
Details
This changes application code, so it speeds up production too; the tests stay as they are.
- 45x in test_single_facet_is_indexable: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_conference_with_utc_with_date_no_posts: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_invalid_selected_instance: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 45x in test_djangoconus_old_url_redirects: SELECT "confs_conferencetag"."id", "confs_conferencetag"."name", "confs_conferencetag"."slug", "confs_conferencetag"."icon" FROM "confs_conferencetag" WHERE "confs_conferencetag"."slug" = %s LIMIT ? FOR UPDATE
- 44x in test_posts_from_yesterday_are_updated: SELECT COUNT(*) AS "__count" FROM "accounts_accountlookup" WHERE UPPER("accounts_accountlookup"."language"::text) LIKE UPPER(%s)
Most affected tests:
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- stats/tests.py::TestStats::test_posts_from_yesterday_are_updated
- stats/tests.py::TestStats::test_store_daily_stats
- stats/tests.py::TestStats::test_store_daily_stats_with_accounts
- accounts/tests.py::TestInstanceIndexingBridgy::test_follows_redirect_and_coerces_null_version
Make the test database server skip durability work
Tests spend 56.7s (36%) in the database. Test databases don't need to survive a crash, so the server can skip flushing to disk.
Details
Only affects crash safety of the throwaway test server; query behaviour is identical.
Where:
- confs/tests.py:432 _migrate · 2.20 s
- starter_packs/tests.py:396 test_toggle_after_limit · 1.15 s
- confs/migrations/0031_seed_conference_tags.py:14 seed_conference_tags · 704 ms
- stats/models.py:20 store_daily_stats · 335 ms
- stats/models.py:12 store_daily_stats · 272 ms
- confs/tests.py:412 _leaf_targets · 267 ms
- starter_packs/tests.py:763 _set_up_orderable_accounts · 188 ms
- starter_packs/views.py:729 share_starter_pack · 157 ms
- accounts/views.py:295 index · 149 ms
- accounts/views.py:28 __init__ · 143 ms
Most affected tests:
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestMigratedLegacyConferences::test_djangoconus_old_url_redirects
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestLegacyConferenceDataMigration::test_copies_overlapping_legacy_rows_without_clobbering_existing_data
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
- starter_packs/tests.py::TestToggleStarterPackAccount::test_toggle_after_limit
- accounts/tests.py::TestInstanceIndexingBridgy::test_follows_redirect_and_coerces_null_version
Disable django-debug-toolbar in tests
django-debug-toolbar is installed in the test settings and costs 10.0s.
Details
Where:
- accounts/views.py:428 index · 1.24 s
- confs/views.py:299 _conference_detail · 493 ms
- starter_packs/views.py:865 share_starter_pack · 393 ms
- confs/views.py:512 create_conference · 227 ms
- confs/views.py:131 conferences · 178 ms
- starter_packs/views.py:188 starter_packs · 177 ms
- starter_packs/tests.py:840 test_api_formats · 174 ms
- starter_packs/views.py:729 share_starter_pack · 157 ms
- accounts/views.py:295 index · 149 ms
- accounts/tests.py:550 assert_noindex · 145 ms
Most affected tests:
- accounts/tests.py::TestAccountIndexRobots::test_single_facet_is_indexable
- accounts/tests.py::TestSelectedInstance::test_invalid_selected_instance
- confs/tests.py::TestConferencePage::test_conference_with_utc_with_date_no_posts
- confs/tests.py::TestMigratedLegacyConferences::test_dotnetconf_page
- confs/tests.py::TestCreateConference::test_rejects_more_than_three_icons
- confs/tests.py::TestEditConference::test_owner_can_view_edit_form
- accounts/tests.py::TestSelectedInstance::test_unselected_instance
- confs/tests.py::TestCreateConference::test_form_renders_for_authenticated_user
Use TestCase instead of TransactionTestCase
1.7s goes to flushing every table after tests in 2 TransactionTestCase / transaction=True / live_server tests.
Details
Tests that rely on commit behaviour, multiple connections or a live server still need a transactional test case.
Most affected tests:
- confs/tests.py::TestLegacyConferenceDataMigration::test_drops_legacy_tables_when_uniques_are_mysql_style_indexes
- accounts/tests.py::TestInstanceIndexingBridgy::test_follows_redirect_and_coerces_null_version
Tests
201–301 of 301, slowest first · click a column to sort this pageMemory
Busiest worker peaked at 212 MB; all 4 together at about 679 MB| Worker | Tests | Peak | Grew by |
|---|---|---|---|
| gw0 | 145 | 200 MB | 57 MB |
| gw1 | 32 | 212 MB | 41 MB |
| gw2 | 91 | 196 MB | 54 MB |
| gw3 | 33 | 175 MB | 40 MB |
Slowest fixtures
Exact setup time across the session| Fixture | Scope | Calls | Total |
|---|---|---|---|
| _django_setup_unittest | class | 69 | 41.32 s |
| django_db_setup | session | 4 | 41.27 s |
| _unittest_setUpClass_fixture_TestSelectedInstance | class | 1 | 276 ms |
| _unittest_setUpClass_fixture_TestConferencePage | class | 2 | 237 ms |
| _unittest_setUpClass_fixture_TestShareStarterPack | class | 2 | 175 ms |
| _unittest_setUpClass_fixture_TestOwnerManagement | class | 2 | 134 ms |
| _unittest_setUpClass_fixture_TestWebfingerBackfill | class | 1 | 114 ms |
| _unittest_setUpClass_fixture_TestPinAccountInStarterPack | class | 2 | 106 ms |
| _unittest_setUpClass_fixture_TestFollowBg | class | 1 | 84 ms |
| _unittest_setUpClass_fixture_TestEditStarterPackAccounts | class | 2 | 83 ms |
| _django_isolate_apps | function | 301 | 65 ms |
| _unittest_setUpClass_fixture_TestAddAccountsSearch | class | 1 | 62 ms |
| _unittest_setUpClass_fixture_TestRemoteFollowStarterPack | class | 2 | 60 ms |
| _unittest_setUpClass_fixture_TestDeleteStarterPack | class | 1 | 56 ms |
| _unittest_setUpClass_fixture_TestEditStarterPack | class | 1 | 52 ms |