summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorsaurabh <saurabhrai1717@gmail.com>2025-08-26 00:18:42 +0530
committerJacob Walls <jacobtylerwalls@gmail.com>2025-11-21 16:26:57 -0500
commita89183e63844a937aacd3ddb73c4952ef869d2cc (patch)
tree769664847a3df22f50c1ebe7332dafc937e846ee
parent846613e521104fa2f2e1c2023e4a1a9886a2ff48 (diff)
Fixed #36620 -- Added coverage workflow to summarize coverage in pull requests.
Part of GSoC 2025. Thanks Lily for mentorship, and Sarah Boyce and Jacob Walls for reviews.
-rw-r--r--.github/workflows/coverage.yml104
-rw-r--r--docs/internals/contributing/writing-code/submitting-patches.txt3
-rw-r--r--docs/internals/contributing/writing-code/unit-tests.txt43
-rw-r--r--zizmor.yml1
4 files changed, 151 insertions, 0 deletions
diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml
new file mode 100644
index 0000000000..848710369f
--- /dev/null
+++ b/.github/workflows/coverage.yml
@@ -0,0 +1,104 @@
+name: Coverage
+
+on:
+ pull_request_target:
+ paths:
+ - 'django/**/*.py'
+ - 'tests/**/*.py'
+ branches:
+ - main
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+permissions:
+ contents: read
+ pull-requests: write
+
+jobs:
+ diff-coverage:
+ if: github.repository == 'django/django'
+ name: Diff Coverage (Windows)
+ runs-on: windows-latest
+ steps:
+ - name: Checkout
+ uses: actions/checkout@v5
+ with:
+ fetch-depth: 0
+ persist-credentials: false
+
+ - name: Set up Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: '3.14'
+ cache: 'pip'
+ cache-dependency-path: 'tests/requirements/py3.txt'
+
+ - name: Install dependencies
+ run: |
+ python -m pip install --upgrade pip wheel
+ python -m pip install -r tests/requirements/py3.txt -e .
+ python -m pip install 'coverage[toml]' diff-cover
+
+ - name: Run tests with coverage
+ env:
+ PYTHONPATH: ${{ github.workspace }}/tests
+ COVERAGE_PROCESS_START: ${{ github.workspace }}/tests/.coveragerc
+ RUNTESTS_DIR: ${{ github.workspace }}/tests
+ run: |
+ python -Wall tests/runtests.py -v2
+
+ - name: Generate coverage report
+ if: success()
+ env:
+ COVERAGE_RCFILE: ${{ github.workspace }}/tests/.coveragerc
+ RUNTESTS_DIR: ${{ github.workspace }}/tests
+ run: |
+ python -m coverage combine
+ python -m coverage report --show-missing
+ python -m coverage xml -o tests/coverage.xml
+
+ - name: Run diff-cover
+ if: success()
+ run: |
+ if (Test-Path 'tests/coverage.xml') {
+ diff-cover tests/coverage.xml --compare-branch=origin/main --fail-under=0 > tests/diff-cover-report.md
+ } else {
+ Set-Content -Path tests/diff-cover-report.md -Value 'No coverage.xml found; skipping diff-cover.'
+ }
+
+ - name: Post/update PR comment
+ if: success()
+ uses: actions/github-script@v8
+ with:
+ script: |
+ const fs = require('fs');
+ const reportPath = 'tests/diff-cover-report.md';
+ let body = 'No coverage data available.';
+ if (fs.existsSync(reportPath)) {
+ body = fs.readFileSync(reportPath, 'utf8');
+ }
+ const commentBody = '### 📊 Coverage Report for Changed Files\n\n```\n' + body + '\n```\n\n**Note:** Missing lines are warnings only. Some lines may not be covered by SQLite tests as they are database-specific.\n\nFor more information about code coverage on pull requests, see the [contributing documentation](https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/unit-tests/#code-coverage-on-pull-requests).';
+
+ const { data: comments } = await github.rest.issues.listComments({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ issue_number: context.issue.number,
+ });
+ for (const c of comments) {
+ if ((c.body || '').includes('📊 Coverage Report for Changed Files')) {
+ await github.rest.issues.deleteComment({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ comment_id: c.id,
+ });
+ }
+ }
+
+ await github.rest.issues.createComment({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ issue_number: context.issue.number,
+ body: commentBody,
+ });
diff --git a/docs/internals/contributing/writing-code/submitting-patches.txt b/docs/internals/contributing/writing-code/submitting-patches.txt
index 841a2109dc..428acffba1 100644
--- a/docs/internals/contributing/writing-code/submitting-patches.txt
+++ b/docs/internals/contributing/writing-code/submitting-patches.txt
@@ -438,6 +438,9 @@ All code changes
* If the change is backwards incompatible in any way, is there a note
in the release notes (``docs/releases/A.B.txt``)?
* Is Django's test suite passing?
+* If there is a :ref:`code coverage report <code-coverage-on-pull-requests>`
+ comment on the pull request, have you reviewed the missing coverage in
+ context (considering database/platform-specific limitations)?
* If the change affects the Django admin or rendered HTML output, has
:ref:`accessibility testing <accessibility-testing-baseline>` been done?
diff --git a/docs/internals/contributing/writing-code/unit-tests.txt b/docs/internals/contributing/writing-code/unit-tests.txt
index cba4ba7397..f304970a29 100644
--- a/docs/internals/contributing/writing-code/unit-tests.txt
+++ b/docs/internals/contributing/writing-code/unit-tests.txt
@@ -394,6 +394,49 @@ settings file defines ``coverage_html`` as the output directory for the report
and also excludes several directories not relevant to the results
(test code or external code included in Django).
+.. _code-coverage-on-pull-requests:
+
+Code coverage on pull requests
+------------------------------
+
+Django's continuous integration (CI) system automatically runs code coverage
+analysis on pull requests and posts a comment with a diff coverage report. This
+helps reviewers see which lines in the changed code are covered by tests.
+
+**What the coverage report shows:**
+
+The coverage report posted on pull requests uses `diff-cover`_ to analyze only
+the lines that were changed or added in the PR. It shows:
+
+* Lines that are covered by tests (✓)
+* Lines that are not covered by tests (✗)
+* Lines that cannot be covered (e.g., comments, blank lines)
+
+.. _diff-cover: https://github.com/Bachmann1234/diff_cover
+
+**Important limitations:**
+
+When reviewing coverage reports on pull requests, keep these limitations in
+mind:
+
+* **Database-specific code:** The CI coverage job runs tests using SQLite on
+ Windows. Code paths specific to other databases (PostgreSQL, MySQL, Oracle)
+ will appear as "not covered" even if database-specific tests exist. This is
+ expected and acceptable.
+
+* **Platform-specific code:** Similarly, code that only runs on certain
+ operating systems (Linux, macOS) will appear as not covered when run on
+ Windows.
+
+* **Coverage doesn't equal quality:** A line being "covered" only means it was
+ executed during tests. It doesn't guarantee the line is well-tested or that
+ all edge cases are handled. During review, assess test quality beyond just
+ coverage numbers.
+
+
+Missing coverage should be considered a warning rather than a blocker and
+should be evaluated in context.
+
.. _contrib-apps:
Contrib apps
diff --git a/zizmor.yml b/zizmor.yml
index 76e53f73cc..e0401fa716 100644
--- a/zizmor.yml
+++ b/zizmor.yml
@@ -1,6 +1,7 @@
rules:
dangerous-triggers:
ignore:
+ - coverage.yml
- labels.yml
- new_contributor_pr.yml
unpinned-uses: