Passed Run of Nov 14, 10:22
pytest --flakytest-profile --flakytest-rate 2000.
Where the time goes
Test time by category- Database9.03 s
- Libraries4.25 s
- Migrations1.85 s
- Password hashing1.52 s
- DB schema938 ms
- Templates806 ms
- Imports418 ms
- App code301 ms
- Deep copies283 ms
- Serialization200 ms
- Unsampled159 ms
- pytest152 ms
- Factories29 ms
- Test code25 ms
- Waiting17 ms
- Logging12 ms
- Mocking4 ms
- Network3 ms
- Cache2 ms
Session timeline
Fix next
Ranked by the time each fix saves per runUse plain static file storage in tests
Static file storage/manifest handling costs 9.2s.
Details
Where:
- apps/api/tests.py:487 ProfileUploadTest.upload · 491 ms
- apps/mcp/tests.py:67 McpTest.post · 398 ms
- ft/tests.py:210 TestAccountSettings.test_change_password · 327 ms
- apps/guides/content.py:112 render · 319 ms
- apps/orgs/tests.py:28 SignupTest.test_signing_up_creates_an_organization · 225 ms
- ft/tests.py:166 TestAccountPages.test_signup · 181 ms
- ft/tests.py:205 TestAccountSettings.test_change_password · 162 ms
- apps/projects/tests.py:320 ProjectRunPage.test_other_peoples_project · 156 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 153 ms
- apps/projects/urls.py:692 <module> · 149 ms
Most affected tests:
- apps/api/tests.py::ProfileUploadTest::test_missing_parts_mark_the_profile_incomplete
- ft/tests.py::TestAccountSettings::test_change_password
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/orgs/tests.py::SignupTest::test_signing_up_creates_an_organization
- apps/api/tests.py::TestApiIndex::test_index
- ft/tests.py::TestAccountPages::test_signup
- apps/api/tests.py::ProfileUploadTest::test_reupload_split_differently
- apps/projects/tests.py::ProjectRunPage::test_other_peoples_project
Disable django-debug-toolbar in tests
django-debug-toolbar is installed in the test settings and costs 5.8s.
Details
Where:
- ft/tests.py:210 TestAccountSettings.test_change_password · 327 ms
- apps/guides/content.py:112 render · 319 ms
- apps/orgs/tests.py:28 SignupTest.test_signing_up_creates_an_organization · 213 ms
- ft/tests.py:166 TestAccountPages.test_signup · 181 ms
- ft/tests.py:205 TestAccountSettings.test_change_password · 162 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 153 ms
- apps/projects/urls.py:692 <module> · 149 ms
- apps/projects/urls.py:498 project_profile · 136 ms
- apps/projects/urls.py:40 can_manage_project · 125 ms
- apps/orgs/views.py:33 org_shell · 115 ms
Most affected tests:
- ft/tests.py::TestAccountSettings::test_change_password
- apps/api/tests.py::ProfileUploadTest::test_missing_parts_mark_the_profile_incomplete
- apps/orgs/tests.py::SignupTest::test_signing_up_creates_an_organization
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/api/tests.py::TestApiIndex::test_index
- ft/tests.py::TestAccountPages::test_signup
- apps/api/tests.py::ProfileUploadTest::test_upload_adds_migrations_per_worker_insight
- apps/projects/tests.py::ProjectPagesTest::test_dashboard
Make the test database server skip durability work
Tests spend 11.8s (59%) in the database. Test databases don't need to survive a crash, so the server can skip flushing to disk.
Details
Only affects crash safety of the throwaway test server; query behaviour is identical.
Where:
- ft/db/postgresql/base.py:39 DatabaseWrapper.get_new_connection · 743 ms
- apps/projects/models.py:466 Profile.save · 291 ms
- apps/projects/profiles.py:352 write_snapshots · 155 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 151 ms
- apps/projects/environments.py:63 get_or_create_environment · 134 ms
- apps/projects/urls.py:40 can_manage_project · 125 ms
- apps/projects/models.py:230 TestManager.upsert · 123 ms
- apps/projects/tests.py:954 ProjectPagesTest.setUp · 122 ms
- apps/projects/rollups.py:88 add_perfs · 121 ms
- apps/orgs/views.py:33 org_shell · 115 ms
Most affected tests:
- apps/api/tests.py::TestApiIndex::test_index
- apps/projects/tests.py::SeedDemoCommandTest::test_seed_demo
- apps/projects/tests.py::ProjectPagesTest::test_configure_page
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_progress_end
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_progress_before_start
- apps/api/tests.py::IngestStartEndpointTest::test_results_are_counted_and_the_payload_dropped
Repeated queries (N+1) in the code under test
27 tests run the same query 10+ times, e.g. 180x in test_seed_demo. N+1 queries slow the tests and production alike.
Details
This changes application code, so it speeds up production too; the tests stay as they are.
- 180x in test_seed_demo: SELECT "projects_test"."id", "projects_test"."name", "projects_test"."project_id", "projects_test"."slug", "projects_test"."date_last_run", "projects_test"."date_first_seen", "projects_test"."first_commit_id", "projects_test"."muted" FROM "projects_test" WHERE ("projects_test"."name" = %s AND "proj…
- 22x in test_dashboard_filters: SELECT "projects_environment"."id", "projects_environment"."project_id", "projects_environment"."name", "projects_environment"."kind", "projects_environment"."provider", "projects_environment"."details", "projects_environment"."date_first_seen", "projects_environment"."date_last_seen" FROM "project…
- 21x in test_configure_page: SELECT "projects_profile"."id", "projects_profile"."project_id", "projects_profile"."number", "projects_profile"."environment_id", "projects_profile"."session_id", "projects_profile"."run_id", "projects_profile"."commit_id", "projects_profile"."ingest_id", "projects_profile"."external_id", "project…
- 20x in test_index: SELECT "django_content_type"."model" AS "model" FROM "django_content_type" WHERE "django_content_type"."app_label" = %s
- 20x in test_daily_stats_count_a_profile_once_every_part_is_in: SELECT "projects_profilepart"."id", "projects_profilepart"."profile_id", "projects_profilepart"."index", "projects_profilepart"."count", "projects_profilepart"."checksum", "projects_profilepart"."status", "projects_profilepart"."error", "projects_profilepart"."tests_count", "projects_profilepart"."…
Most affected tests:
- apps/projects/tests.py::SeedDemoCommandTest::test_seed_demo
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/projects/tests.py::ProjectPagesTest::test_configure_page
- apps/api/tests.py::TestApiIndex::test_index
- apps/api/tests.py::ProfileUploadTest::test_daily_stats_count_a_profile_once_every_part_is_in
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_details_are_kept_for_failures_only
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_end_after_progress
pytest takes a long time before the first test runs
Startup takes 4.6s and collection 304ms on every run, even when you only run one test.
Stop re-running migrations on every test run
Migrations take 1.8s per run, before the first test can start.
Details
CI should still build the database from scratch (no --reuse-db) so broken migrations are caught.
Where:
- ft/db/postgresql/base.py:39 DatabaseWrapper.get_new_connection · 10 ms
- apps/projects/migrations/0012_backfill_session_environments.py:30 backfill · 3 ms
- apps/projects/migrations/0014_backfill_session_counts.py:23 backfill · 3 ms
- apps/projects/migrations/0004_project_created_by_project_date_created_and_more.py:12 add_created_by_to_project · 2 ms
- apps/projects/migrations/0016_profile_parts.py:9 move_documents · 2 ms
- apps/projects/migrations/0016_profile_parts.py:24 external_storage · 2 ms
- apps/projects/migrations/0018_profile_stats_applied.py:9 mark_applied · 2 ms
- apps/projects/migrations/0022_backfill_organizations.py:48 forwards · 2 ms
- apps/projects/migrations/0022_backfill_organizations.py:69 forwards · 2 ms
- apps/projects/migrations/0025_backfill_slugs_and_numbers.py:28 forwards · 2 ms
Most affected tests:
- apps/api/tests.py::TestApiIndex::test_index
Use a fast password hasher in tests
1.5s (8% of test time) is spent hashing passwords. Slow hashing protects real passwords; in tests it only burns CPU.
Details
Only change the hasher in test settings. Tests that assert on a specific hash format need updating.
Where:
- ft/tests.py:210 TestAccountSettings.test_change_password · 321 ms
- ft/tests.py:237 TestAccountSettings.test_projects_only_lists_your_own_even_for_superusers · 187 ms
- apps/orgs/tests.py:194 SuperuserTest.test_superusers_can_open_any_project_but_dont_list_them · 183 ms
- apps/orgs/tests.py:28 SignupTest.test_signing_up_creates_an_organization · 179 ms
- ft/tests.py:166 TestAccountPages.test_signup · 172 ms
- ft/tests.py:205 TestAccountSettings.test_change_password · 160 ms
- ft/tests.py:220 TestAccountSettings.test_change_password · 160 ms
- ft/tests.py:182 TestAccountSettings.setUpTestData · 155 ms
Most affected tests:
- ft/tests.py::TestAccountSettings::test_change_password
- ft/tests.py::TestAccountSettings::test_projects_only_lists_your_own_even_for_superusers
- apps/orgs/tests.py::SuperuserTest::test_superusers_can_open_any_project_but_dont_list_them
- apps/orgs/tests.py::SignupTest::test_signing_up_creates_an_organization
- ft/tests.py::TestAccountPages::test_signup
Use TestCase instead of TransactionTestCase
811ms goes to flushing every table after tests in 11 TransactionTestCase / transaction=True / live_server tests.
Details
Tests that rely on commit behaviour, multiple connections or a live server still need a transactional test case.
Most affected tests:
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_valid
- apps/api/tests.py::IngestStartEndpointTest::test_details_are_kept_for_failures_only
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_with_git
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_multiple_sessions
- apps/api/tests.py::ProfileUploadQueueTest::test_upload_is_processed_by_the_worker
- apps/api/tests.py::ProfileUploadLockTest::test_upload_while_a_part_is_being_processed
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_progress_before_start
Tests open new database connections over and over
Connecting to the database takes 761ms.
Details
Where:
- ft/db/postgresql/base.py:39 DatabaseWrapper.get_new_connection · 714 ms
- apps/api/tests.py:73 IngestStartEndpointTest.setUp · 7 ms
- apps/api/tests.py:76 IngestStartEndpointTest.join_workers · 7 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 3 ms
- apps/api/tests.py:928 ProfileUploadLockTest.setUp · 1 ms
- apps/api/tests.py:212 IngestStartEndpointTest.test_ingest_start_progress_end · 1 ms
- apps/api/tasks.py:266 warm_reports · 1 ms
- apps/api/tests.py:306 IngestStartEndpointTest.test_details_are_kept_for_failures_only · 1 ms
Most affected tests:
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_end_after_progress
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_progress_before_start
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_progress_end
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_results_are_counted_and_the_payload_dropped
- apps/api/tests.py::IngestStartEndpointTest::test_details_are_kept_for_failures_only
- apps/api/tests.py::ProfileUploadQueueTest::test_upload_is_processed_by_the_worker
- apps/projects/tests.py::RealProfileDocumentTest::test_call_tree
Create shared test data in setUpTestData instead of setUp
setUp() runs before every test and takes 623ms in total (db 566ms, library 48ms, serialization 3ms) in 14 classes: SettingsTest (3 tests), ProjectsPageTest (5 tests), ProjectEditPage (6 tests), ProjectDeletePage (4 tests).
Details
Since Django 3.2 attributes assigned in setUpTestData are deep-copied per test, so in-memory changes don't leak between tests. Keep objects a test modifies in the database (or re-fetches) as before.
Where:
- apps/projects/tests.py:954 ProjectPagesTest.setUp · 137 ms
- ft/db/postgresql/base.py:39 DatabaseWrapper.get_new_connection · 75 ms
- apps/projects/tests.py:143 ProjectEditPage.setUp · 59 ms
- apps/projects/tests.py:101 ProjectsPageTest.setUp · 54 ms
- apps/api/tests.py:73 IngestStartEndpointTest.setUp · 41 ms
- apps/projects/tests.py:219 ProjectCreatePage.setUp · 35 ms
- apps/projects/tests.py:338 ProjectTestsPage.setUp · 32 ms
- ft/tests.py:186 TestAccountSettings.setUp · 32 ms
- apps/projects/tests.py:1429 MemoryTest.setUp · 28 ms
- apps/projects/tests.py:269 ProjectRunsPage.setUp · 27 ms
Most affected tests:
- apps/projects/tests.py::ProjectsPageTest::test_other_peoples_project
- apps/projects/tests.py::ProjectEditPage::test_project_submit
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_multiple_sessions
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_progress_end
- apps/projects/tests.py::ProjectsPageTest::test_no_project
- apps/api/tests.py::ProfileUploadLockTest::test_upload_while_a_part_is_being_processed
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_with_git
- apps/projects/tests.py::ProjectCreatePage::test_first_project_is_onboarding
Tests
1–200 of 258, slowest first · click a column to sort this pageMemory
The test process peaked at 216 MBSlowest fixtures
Exact setup time across the session| Fixture | Scope | Calls | Total |
|---|---|---|---|
| _django_setup_unittest | class | 55 | 2.04 s |
| django_db_setup | session | 1 | 2.02 s |
| _unittest_setUpClass_fixture_ProjectPagesTest | class | 1 | 245 ms |
| _unittest_setUpClass_fixture_TestAccountSettings | class | 1 | 164 ms |
| _unittest_setUpClass_fixture_AnalyticsTest | class | 1 | 155 ms |
| _unittest_setUpClass_fixture_ProfilePagePaginationTest | class | 1 | 143 ms |
| _unittest_setUpClass_fixture_McpTest | class | 1 | 140 ms |
| _unittest_setUpClass_fixture_RealProfileDocumentTest | class | 1 | 121 ms |
| _unittest_setUpClass_fixture_FlakyReportTest | class | 1 | 69 ms |
| _unittest_setUpClass_fixture_MemoryTest | class | 1 | 68 ms |
| _unittest_setUpClass_fixture_StorageTest | class | 1 | 64 ms |
| _unittest_setUpClass_fixture_LegacyUrlTest | class | 1 | 48 ms |
| _unittest_setUpClass_fixture_UnstorableCharactersTest | class | 1 | 40 ms |
| _unittest_setUpClass_fixture_FlakyReportCacheTest | class | 1 | 37 ms |
| _unittest_setUpClass_fixture_ProjectTestsPage | class | 1 | 24 ms |