Fixed #22487: Optional rollback emulation for migrated apps

6 files changed, 79 insertions, 6 deletions

diff --git a/docs/ref/migration-operations.txt b/docs/ref/migration-operations.txt
index 40ef3849ac..95c9a65020 100644
--- a/docs/ref/migration-operations.txt
+++ b/docs/ref/migration-operations.txt
@@ -199,8 +199,9 @@ model::
         # We get the model from the versioned app registry;
         # if we directly import it, it'll be the wrong version
         Country = apps.get_model("myapp", "Country")
-        Country.objects.create(name="USA", code="us")
-        Country.objects.create(name="France", code="fr")
+        db_alias = schema_editor.connection.alias
+        Country.objects.create(name="USA", code="us", using=db_alias)
+        Country.objects.create(name="France", code="fr", using=db_alias)
 
     class Migration(migrations.Migration):
 
@@ -236,6 +237,14 @@ Oracle). This should be safe, but may cause a crash if you attempt to use
 the ``schema_editor`` provided on these backends; in this case, please
 set ``atomic=False``.
 
+.. warning::
+
+    RunPython does not magically alter the connection of the models for you;
+    any model methods you call will go to the default database unless you
+    give them the current database alias (available from
+    ``schema_editor.connection.alias``, where ``schema_editor`` is the second
+    argument to your function).
+
 SeparateDatabaseAndState
 ------------------------
 
diff --git a/docs/ref/settings.txt b/docs/ref/settings.txt
index 683d0f4b0f..64f2d6e213 100644
--- a/docs/ref/settings.txt
+++ b/docs/ref/settings.txt
@@ -2078,6 +2078,24 @@ Default: ``'django.test.runner.DiscoverRunner'``
 The name of the class to use for starting the test suite. See
 :ref:`other-testing-frameworks`.
 
+.. setting:: TEST_NON_SERIALIZED_APPS
+
+TEST_NON_SERIALIZED_APPS
+------------------------
+
+Default: ``[]``
+
+In order to restore the database state between tests for TransactionTestCases
+and database backends without transactions, Django will :ref:`serialize the
+contents of all apps with migrations <test-case-serialized-rollback>` when it
+starts the test run so it can then reload from that copy before tests that
+need it.
+
+This slows down the startup time of the test runner; if you have apps that
+you know don't need this feature, you can add their full names in here (e.g.
+``django.contrib.contenttypes``) to exclude them from this serialization
+process.
+
 .. setting:: THOUSAND_SEPARATOR
 
 THOUSAND_SEPARATOR
diff --git a/docs/releases/1.7.txt b/docs/releases/1.7.txt
index e1bc3e515c..cbef6b43f5 100644
--- a/docs/releases/1.7.txt
+++ b/docs/releases/1.7.txt
@@ -63,6 +63,10 @@ but a few of the key features are:
 * ``initial_data`` fixtures are no longer loaded for apps with migrations; if
   you want to load initial data for an app, we suggest you do it in a migration.
 
+* Test rollback behaviour is different for apps with migrations; in particular,
+  Django will no longer emulate rollbacks on non-transactional databases or
+  inside ``TransactionTestCase`` :ref:`unless specifically asked <test-case-serialized-rollback>`.
+
 App-loading refactor
 ~~~~~~~~~~~~~~~~~~~~
 
diff --git a/docs/topics/testing/advanced.txt b/docs/topics/testing/advanced.txt
index 13041c1945..7c8124052d 100644
--- a/docs/topics/testing/advanced.txt
+++ b/docs/topics/testing/advanced.txt
@@ -485,7 +485,7 @@ django.db.connection.creation
 The creation module of the database backend also provides some utilities that
 can be useful during testing.
 
-.. function:: create_test_db([verbosity=1, autoclobber=False, keepdb=False])
+.. function:: create_test_db([verbosity=1, autoclobber=False, keepdb=False, serialize=True])
 
     Creates a new test database and runs ``migrate`` against it.
 
@@ -507,6 +507,12 @@ can be useful during testing.
     a new database will be created, prompting the user to remove
     the existing one, if present.
 
+    ``serialize`` determines if Django serializes the database into an
+    in-memory JSON string before running tests (used to restore the database
+    state between tests if you don't have transactions). You can set this to
+    False to significantly speed up creation time if you know you don't need
+    data persistance outside of test fixtures.
+
     Returns the name of the test database that it created.
 
     ``create_test_db()`` has the side effect of modifying the value of
diff --git a/docs/topics/testing/overview.txt b/docs/topics/testing/overview.txt
index 2d9808f0bc..026bbd8c88 100644
--- a/docs/topics/testing/overview.txt
+++ b/docs/topics/testing/overview.txt
@@ -234,6 +234,33 @@ the Django test runner reorders tests in the following way:
     database by a given :class:`~django.test.TransactionTestCase` test, they
     must be updated to be able to run independently.
 
+.. _test-case-serialized-rollback:
+
+Rollback emulation
+------------------
+
+Any initial data loaded in migrations will only be available in ``TestCase``
+tests and not in ``TransactionTestCase`` tests, and additionally only on
+backends where transactions are supported (the most important exception being
+MyISAM).
+
+Django can re-load that data for you on a per-testcase basis by
+setting the ``serialized_rollback`` option to ``True`` in the body of the
+``TestCase`` or ``TransactionTestCase``, but note that this will slow down
+that test suite by approximately 3x.
+
+Third-party apps or those developing against MyISAM will need to set this;
+in general, however, you should be developing your own projects against a
+transactional database and be using ``TestCase`` for most tests, and thus
+not need this setting.
+
+The initial serialization is usually very quick, but if you wish to exclude
+some apps from this process (and speed up test runs slightly), you may add
+those apps to :setting:`TEST_NON_SERIALIZED_APPS`.
+
+Apps without migrations are not affected; ``initial_data`` fixtures are
+reloaded as usual.
+
 Other test conditions
 ---------------------
 
@@ -249,6 +276,7 @@ used. This behavior `may change`_ in the future.
 
 .. _may change: https://code.djangoproject.com/ticket/11505
 
+
 Understanding the test output
 -----------------------------
 
diff --git a/docs/topics/testing/tools.txt b/docs/topics/testing/tools.txt
index c41e35d647..1d55b2e3dc 100644
--- a/docs/topics/testing/tools.txt
+++ b/docs/topics/testing/tools.txt
@@ -600,9 +600,17 @@ to test the effects of commit and rollback:
   guarantees that the rollback at the end of the test restores the database to
   its initial state.
 
-  When running on a database that does not support rollback (e.g. MySQL with the
-  MyISAM storage engine), ``TestCase`` falls back to initializing the database
-  by truncating tables and reloading initial data.
+.. warning::
+
+  ``TestCase`` running on a database that does not support rollback (e.g. MySQL with the
+  MyISAM storage engine), and all instances of ``TransactionTestCase``, will
+  roll back at the end of the test by deleting all data from the test database
+  and reloading initial data for apps without migrations.
+
+  Apps with migrations :ref:`will not see their data reloaded <test-case-serialized-rollback>`;
+  if you need this functionality (for example, third-party apps should enable
+  this) you can set ``serialized_rollback = True`` inside the
+  ``TestCase`` body.
 
 .. warning::