Fixed #17788 -- Added batch_size argument to qs.bulk_create()

The qs.bulk_create() method did not work with large batches together
with SQLite3. This commit adds a way to split the bulk into smaller
batches. The default batch size is unlimited except for SQLite3 where
the batch size is limited to 999 SQL parameters per batch.

Thanks to everybody who participated in the discussions at Trac.

2 files changed, 11 insertions, 14 deletions

diff --git a/docs/ref/models/querysets.txt b/docs/ref/models/querysets.txt
index 0a9005ad26..8c188c67c3 100644
--- a/docs/ref/models/querysets.txt
+++ b/docs/ref/models/querysets.txt
@@ -1350,7 +1350,7 @@ has a side effect on your data. For more, see `Safe methods`_ in the HTTP spec.
 bulk_create
 ~~~~~~~~~~~
 
-.. method:: bulk_create(objs)
+.. method:: bulk_create(objs, batch_size=None)
 
 .. versionadded:: 1.4
 
@@ -1372,20 +1372,12 @@ This has a number of caveats though:
 * If the model's primary key is an :class:`~django.db.models.AutoField` it
   does not retrieve and set the primary key attribute, as ``save()`` does.
 
-.. admonition:: Limits of SQLite
+The ``batch_size`` parameter controls how many objects are created in single
+query. The default is to create all objects in one batch, except for SQLite
+where the default is such that at maximum 999 variables per query is used.
 
-    SQLite sets a limit on the number of parameters per SQL statement. The
-    maximum is defined by the SQLITE_MAX_VARIABLE_NUMBER_ compilation option,
-    which defaults to 999. For instance, if your model has 8 fields (including
-    the primary key), you cannot create more than 999 // 8 = 124 instances at
-    a time. If you exceed this limit, you'll get an exception::
-
-        django.db.utils.DatabaseError: too many SQL variables
-
-    If your application's performance requirements exceed SQLite's limits, you
-    should switch to another database engine, such as PostgreSQL.
-
-.. _SQLITE_MAX_VARIABLE_NUMBER: http://sqlite.org/limits.html#max_variable_number
+.. versionadded:: 1.5
+    The ``batch_size`` parameter was added in version 1.5.
 
 count
 ~~~~~
diff --git a/docs/releases/1.5.txt b/docs/releases/1.5.txt
index b863d102ec..fd9ae4f038 100644
--- a/docs/releases/1.5.txt
+++ b/docs/releases/1.5.txt
@@ -106,6 +106,11 @@ Django 1.5 also includes several smaller improvements worth noting:
 * The :ref:`receiver <connecting-receiver-functions>` decorator is now able to
   connect to more than one signal by supplying a list of signals.
 
+* :meth:`QuerySet.bulk_create()
+  <django.db.models.query.QuerySet.bulk_create>` has now a batch_size
+  argument. By default the batch_size is unlimited except for SQLite where
+  single batch is limited so that 999 parameters per query isn't exceeded.
+
 Backwards incompatible changes in 1.5
 =====================================