Test suite performance
Suite wall time per run
Click a point to open the runWhere the time goes
Avg test time per runSlowest tests
All tests →Most memory
All tests by memory →Fix next
From the latest run per environment, by time savedUse plain static file storage in tests
Static file storage/manifest handling costs 9.2s.
Details
Where:
- apps/api/tests.py:487 ProfileUploadTest.upload · 491 ms
- apps/mcp/tests.py:67 McpTest.post · 398 ms
- ft/tests.py:210 TestAccountSettings.test_change_password · 327 ms
- apps/guides/content.py:112 render · 319 ms
- apps/orgs/tests.py:28 SignupTest.test_signing_up_creates_an_organization · 225 ms
- ft/tests.py:166 TestAccountPages.test_signup · 181 ms
- ft/tests.py:205 TestAccountSettings.test_change_password · 162 ms
- apps/projects/tests.py:320 ProjectRunPage.test_other_peoples_project · 156 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 153 ms
- apps/projects/urls.py:692 <module> · 149 ms
Most affected tests:
- apps/api/tests.py::ProfileUploadTest::test_missing_parts_mark_the_profile_incomplete
- ft/tests.py::TestAccountSettings::test_change_password
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/orgs/tests.py::SignupTest::test_signing_up_creates_an_organization
- apps/api/tests.py::TestApiIndex::test_index
- ft/tests.py::TestAccountPages::test_signup
- apps/api/tests.py::ProfileUploadTest::test_reupload_split_differently
- apps/projects/tests.py::ProjectRunPage::test_other_peoples_project
Disable django-debug-toolbar in tests
django-debug-toolbar is installed in the test settings and costs 5.8s.
Details
Where:
- ft/tests.py:210 TestAccountSettings.test_change_password · 327 ms
- apps/guides/content.py:112 render · 319 ms
- apps/orgs/tests.py:28 SignupTest.test_signing_up_creates_an_organization · 213 ms
- ft/tests.py:166 TestAccountPages.test_signup · 181 ms
- ft/tests.py:205 TestAccountSettings.test_change_password · 162 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 153 ms
- apps/projects/urls.py:692 <module> · 149 ms
- apps/projects/urls.py:498 project_profile · 136 ms
- apps/projects/urls.py:40 can_manage_project · 125 ms
- apps/orgs/views.py:33 org_shell · 115 ms
Most affected tests:
- ft/tests.py::TestAccountSettings::test_change_password
- apps/api/tests.py::ProfileUploadTest::test_missing_parts_mark_the_profile_incomplete
- apps/orgs/tests.py::SignupTest::test_signing_up_creates_an_organization
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/api/tests.py::TestApiIndex::test_index
- ft/tests.py::TestAccountPages::test_signup
- apps/api/tests.py::ProfileUploadTest::test_upload_adds_migrations_per_worker_insight
- apps/projects/tests.py::ProjectPagesTest::test_dashboard
Make the test database server skip durability work
Tests spend 11.8s (59%) in the database. Test databases don't need to survive a crash, so the server can skip flushing to disk.
Details
Only affects crash safety of the throwaway test server; query behaviour is identical.
Where:
- ft/db/postgresql/base.py:39 DatabaseWrapper.get_new_connection · 743 ms
- apps/projects/models.py:466 Profile.save · 291 ms
- apps/projects/profiles.py:352 write_snapshots · 155 ms
- apps/api/urls.py:72 api_request.<locals>.decorator.<locals>.wrapped_view · 151 ms
- apps/projects/environments.py:63 get_or_create_environment · 134 ms
- apps/projects/urls.py:40 can_manage_project · 125 ms
- apps/projects/models.py:230 TestManager.upsert · 123 ms
- apps/projects/tests.py:954 ProjectPagesTest.setUp · 122 ms
- apps/projects/rollups.py:88 add_perfs · 121 ms
- apps/orgs/views.py:33 org_shell · 115 ms
Most affected tests:
- apps/api/tests.py::TestApiIndex::test_index
- apps/projects/tests.py::SeedDemoCommandTest::test_seed_demo
- apps/projects/tests.py::ProjectPagesTest::test_configure_page
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_start_progress_end
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_progress_before_start
- apps/api/tests.py::IngestStartEndpointTest::test_results_are_counted_and_the_payload_dropped
Repeated queries (N+1) in the code under test
27 tests run the same query 10+ times, e.g. 180x in test_seed_demo. N+1 queries slow the tests and production alike.
Details
This changes application code, so it speeds up production too; the tests stay as they are.
- 180x in test_seed_demo: SELECT "projects_test"."id", "projects_test"."name", "projects_test"."project_id", "projects_test"."slug", "projects_test"."date_last_run", "projects_test"."date_first_seen", "projects_test"."first_commit_id", "projects_test"."muted" FROM "projects_test" WHERE ("projects_test"."name" = %s AND "proj…
- 22x in test_dashboard_filters: SELECT "projects_environment"."id", "projects_environment"."project_id", "projects_environment"."name", "projects_environment"."kind", "projects_environment"."provider", "projects_environment"."details", "projects_environment"."date_first_seen", "projects_environment"."date_last_seen" FROM "project…
- 21x in test_configure_page: SELECT "projects_profile"."id", "projects_profile"."project_id", "projects_profile"."number", "projects_profile"."environment_id", "projects_profile"."session_id", "projects_profile"."run_id", "projects_profile"."commit_id", "projects_profile"."ingest_id", "projects_profile"."external_id", "project…
- 20x in test_index: SELECT "django_content_type"."model" AS "model" FROM "django_content_type" WHERE "django_content_type"."app_label" = %s
- 20x in test_daily_stats_count_a_profile_once_every_part_is_in: SELECT "projects_profilepart"."id", "projects_profilepart"."profile_id", "projects_profilepart"."index", "projects_profilepart"."count", "projects_profilepart"."checksum", "projects_profilepart"."status", "projects_profilepart"."error", "projects_profilepart"."tests_count", "projects_profilepart"."…
Most affected tests:
- apps/projects/tests.py::SeedDemoCommandTest::test_seed_demo
- apps/projects/tests.py::ProjectPagesTest::test_dashboard_filters
- apps/projects/tests.py::ProjectPagesTest::test_configure_page
- apps/api/tests.py::TestApiIndex::test_index
- apps/api/tests.py::ProfileUploadTest::test_daily_stats_count_a_profile_once_every_part_is_in
- apps/api/tests.py::IngestStartEndpointTest::test_all_test_status_types
- apps/api/tests.py::IngestStartEndpointTest::test_details_are_kept_for_failures_only
- apps/api/tests.py::IngestStartEndpointTest::test_ingest_end_after_progress
CI vs local
Tests that are much slower in CI, and whyUpload profiles from CI and from a developer machine to compare them.
Locally: FLAKYTEST_SECRET_TOKEN=… pytest --flakytest-profile
Failing and flaky tests
Environments
Every CI job and developer machine that reported in the last 7 days| Environment | Profiled runs | Avg wall time |
|---|---|---|
| CI github-actions:test | 4 | 28.25 s |
| CI github-actions | 0 | – |