Fixed #23775 -- Added docs for --bisect and --pair runtests options.

Thanks Baptiste Mispelon for review.

1 files changed, 48 insertions, 2 deletions

diff --git a/docs/internals/contributing/writing-code/unit-tests.txt b/docs/internals/contributing/writing-code/unit-tests.txt
index dead3578bf..1e69ac259e 100644
--- a/docs/internals/contributing/writing-code/unit-tests.txt
+++ b/docs/internals/contributing/writing-code/unit-tests.txt
@@ -101,6 +101,8 @@ character set. If your database server doesn't use UTF-8 as a default charset,
 you will need to include a value for :setting:`TEST_CHARSET` in the settings
 dictionary for the applicable database.
 
+.. _runtests-specifying-labels:
+
 Running only some of the tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -248,8 +250,8 @@ method as above:
 Troubleshooting
 ---------------
 
-Many test failures with ``UnicodeEncodeError``.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Many test failures with ``UnicodeEncodeError``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 If the ``locales`` package is not installed, some tests will fail with a
 ``UnicodeEncodeError``.
@@ -260,3 +262,47 @@ You can resolve this on Debian-based systems, for example, by running:
 
     $ apt-get install locales
     $ dpkg-reconfigure locales
+
+Tests that only fail in combination
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In case a test passes when run in isolation but fails within the whole suite,
+we have some tools to help analyze the problem.
+
+The ``--bisect`` option of ``runtests.py`` will run the failing test while halving the test set it
+is run together with on each iteration, often making it possible to identify a
+small number of tests that may be related to the failure.
+
+For example, suppose that the failing test that works on its own is
+``ModelTest.test_eq``, then using:
+
+.. code-block:: bash
+
+   $ ./runtests.py --bisect basic.tests.ModelTest.test_eq
+
+will try to determine a test that interferes with the given one. First, the
+test is run with the first half of the test suite. If a failure occurs, the
+first half of the test suite is split in two groups and each group is then run
+with the specified test. If there is no failure with the first half of the test
+suite, the second half of the test suite is run with the specified test and
+split appropriately as described earlier. The process repeats until the set of
+failing tests is minimized.
+
+The ``--pair`` option runs the given test alongside every other test from the
+suite, letting you check if another test has side-effects that cause the
+failure. So:
+
+.. code-block:: bash
+
+   $ ./runtests.py --pair basic.tests.ModelTest.test_eq
+
+will pair ``test_eq`` with every test label.
+
+With both ``--bisect`` and ``--pair``, if you already suspect which cases
+might be responsible for the failure, you may limit tests to be cross-analyzed
+by :ref:`specifying further test labels <runtests-specifying-labels>` after
+the first one:
+
+.. code-block:: bash
+
+   $ ./runtests.py --pair basic.tests.ModelTest.test_eq queries transactions