Fixed #33646 -- Added async-compatible interface to QuerySet.

Thanks Simon Charette for reviews.

Co-authored-by: Carlton Gibson <carlton.gibson@noumenal.es>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>

2 files changed, 121 insertions, 13 deletions

diff --git a/docs/topics/async.txt b/docs/topics/async.txt
index 101bbeeabb..9e2785c351 100644
--- a/docs/topics/async.txt
+++ b/docs/topics/async.txt
@@ -61,27 +61,39 @@ In both ASGI and WSGI mode, you can still safely use asynchronous support to
 run code concurrently rather than serially. This is especially handy when
 dealing with external APIs or data stores.
 
-If you want to call a part of Django that is still synchronous, like the ORM,
-you will need to wrap it in a :func:`sync_to_async` call. For example::
+If you want to call a part of Django that is still synchronous, you will need
+to wrap it in a :func:`sync_to_async` call. For example::
 
     from asgiref.sync import sync_to_async
 
-    results = await sync_to_async(Blog.objects.get, thread_sensitive=True)(pk=123)
+    results = await sync_to_async(sync_function, thread_sensitive=True)(pk=123)
 
-You may find it easier to move any ORM code into its own function and call that
-entire function using :func:`sync_to_async`. For example::
+If you accidentally try to call a part of Django that is synchronous-only
+from an async view, you will trigger Django's
+:ref:`asynchronous safety protection <async-safety>` to protect your data from
+corruption.
 
-    from asgiref.sync import sync_to_async
+Queries & the ORM
+-----------------
 
-    def _get_blog(pk):
-        return Blog.objects.select_related('author').get(pk=pk)
+.. versionadded:: 4.1
 
-    get_blog = sync_to_async(_get_blog, thread_sensitive=True)
+With some exceptions, Django can run ORM queries asynchronously as well::
 
-If you accidentally try to call a part of Django that is still synchronous-only
-from an async view, you will trigger Django's
-:ref:`asynchronous safety protection <async-safety>` to protect your data from
-corruption.
+    async for author in Author.objects.filter(name__startswith="A"):
+        book = await author.books.afirst()
+
+Detailed notes can be found in :ref:`async-queries`, but in short:
+
+* All ``QuerySet`` methods that cause a SQL query to occur have an
+  ``a``-prefixed asynchronous variant.
+
+* ``async for`` is supported on all QuerySets (including the output of
+  ``values()`` and ``values_list()``.)
+
+Transactions do not yet work in async mode. If you have a piece of code that
+needs transactions behavior, we recommend you write that piece as a single
+synchronous function and call it using :func:`sync_to_async`.
 
 Performance
 -----------
diff --git a/docs/topics/db/queries.txt b/docs/topics/db/queries.txt
index 8118532513..af26462367 100644
--- a/docs/topics/db/queries.txt
+++ b/docs/topics/db/queries.txt
@@ -849,6 +849,102 @@ being evaluated and therefore populate the cache::
     Simply printing the queryset will not populate the cache. This is because
     the call to ``__repr__()`` only returns a slice of the entire queryset.
 
+.. _async-queries:
+
+Asynchronous queries
+====================
+
+.. versionadded:: 4.1
+
+If you are writing asynchronous views or code, you cannot use the ORM for
+queries in quite the way we have described above, as you cannot call *blocking*
+synchronous code from asynchronous code - it will block up the event loop
+(or, more likely, Django will notice and raise a ``SynchronousOnlyOperation``
+to stop that from happening).
+
+Fortunately, you can do many queries using Django's asynchronous query APIs.
+Every method that might block - such as ``get()`` or ``delete()`` - has an
+asynchronous variant (``aget()`` or ``adelete()``), and when you iterate over
+results, you can use asynchronous iteration (``async for``) instead.
+
+Query iteration
+---------------
+
+.. versionadded:: 4.1
+
+The default way of iterating over a query - with ``for`` - will result in a
+blocking database query behind the scenes as Django loads the results at
+iteration time. To fix this, you can swap to ``async for``::
+
+    async for entry in Authors.objects.filter(name__startswith="A"):
+        ...
+
+Be aware that you also can't do other things that might iterate over the
+queryset, such as wrapping ``list()`` around it to force its evaluation (you
+can use ``async for`` in a comprehension, if you want it).
+
+Because ``QuerySet`` methods like ``filter()`` and ``exclude()`` do not
+actually run the query - they set up the queryset to run when it's iterated
+over - you can use those freely in asynchronous code. For a guide to which
+methods can keep being used like this, and which have asynchronous versions,
+read the next section.
+
+``QuerySet`` and manager methods
+--------------------------------
+
+.. versionadded:: 4.1
+
+Some methods on managers and querysets - like ``get()`` and ``first()`` - force
+execution of the queryset and are blocking. Some, like ``filter()`` and
+``exclude()``, don't force execution and so are safe to run from asynchronous
+code. But how are you supposed to tell the difference?
+
+While you could poke around and see if there is an ``a``-prefixed version of
+the method (for example, we have ``aget()`` but not ``afilter()``), there is a
+more logical way - look up what kind of method it is in the
+:doc:`QuerySet reference </ref/models/querysets>`.
+
+In there, you'll find the methods on QuerySets grouped into two sections:
+
+* *Methods that return new querysets*: These are the non-blocking ones,
+  and don't have asynchronous versions. You're free to use these in any
+  situation, though read the notes on ``defer()`` and ``only()`` before you use
+  them.
+
+* *Methods that do not return querysets*: These are the blocking ones, and
+  have asynchronous versions - the asynchronous name for each is noted in its
+  documentation, though our standard pattern is to add an ``a`` prefix.
+
+Using this distinction, you can work out when you need to use asynchronous
+versions, and when you don't. For example, here's a valid asynchronous query::
+
+    user = await User.objects.filter(username=my_input).afirst()
+
+``filter()`` returns a queryset, and so it's fine to keep chaining it inside an
+asynchronous environment, whereas ``first()`` evaluates and returns a model
+instance - thus, we change to ``afirst()``, and use ``await`` at the front of
+the whole expression in order to call it in an asynchronous-friendly way.
+
+.. note::
+
+    If you forget to put the ``await`` part in, you may see errors like
+    *"coroutine object has no attribute x"* or *"<coroutine …>"* strings in
+    place of your model instances. If you ever see these, you are missing an
+    ``await`` somewhere to turn that coroutine into a real value.
+
+Transactions
+------------
+
+.. versionadded:: 4.1
+
+Transactions are **not** currently supported with asynchronous queries and
+updates. You will find that trying to use one raises
+``SynchronousOnlyOperation``.
+
+If you wish to use a transaction, we suggest you write your ORM code inside a
+separate, synchronous function and then call that using sync_to_async - see
+:doc:/topics/async` for more.
+
 .. _querying-jsonfield:
 
 Querying ``JSONField``