Fixed #25367 -- Allowed boolean expressions in QuerySet.filter() and exclude().

This allows using expressions that have an output_field that is a
BooleanField to be used directly in a queryset filters, or in the
When() clauses of a Case() expression.

Thanks Josh Smeaton, Tim Graham, Simon Charette, Mariusz Felisiak, and
Adam Johnson for reviews.

Co-Authored-By: NyanKiyoshi <hello@vanille.bid>

3 files changed, 75 insertions, 20 deletions

diff --git a/docs/ref/models/conditional-expressions.txt b/docs/ref/models/conditional-expressions.txt
index f9e681f667..e88554dec6 100644
--- a/docs/ref/models/conditional-expressions.txt
+++ b/docs/ref/models/conditional-expressions.txt
@@ -42,9 +42,15 @@ We'll be using the following model in the subsequent examples::
 A ``When()`` object is used to encapsulate a condition and its result for use
 in the conditional expression. Using a ``When()`` object is similar to using
 the :meth:`~django.db.models.query.QuerySet.filter` method. The condition can
-be specified using :ref:`field lookups <field-lookups>` or
-:class:`~django.db.models.Q` objects. The result is provided using the ``then``
-keyword.
+be specified using :ref:`field lookups <field-lookups>`,
+:class:`~django.db.models.Q` objects, or :class:`~django.db.models.Expression`
+objects that have an ``output_field`` that is a
+:class:`~django.db.models.BooleanField`. The result is provided using the
+``then`` keyword.
+
+.. versionchanged:: 3.0
+
+    Support for boolean :class:`~django.db.models.Expression` was added.
 
 Some examples::
 
@@ -60,6 +66,12 @@ Some examples::
     >>> # Complex conditions can be created using Q objects
     >>> When(Q(name__startswith="John") | Q(name__startswith="Paul"),
     ...      then='name')
+    >>> # Condition can be created using boolean expressions.
+    >>> from django.db.models import Exists, OuterRef
+    >>> non_unique_account_type = Client.objects.filter(
+    ...     account_type=OuterRef('account_type'),
+    ... ).exclude(pk=OuterRef('pk')).values('pk')
+    >>> When(Exists(non_unique_account_type), then=Value('non unique'))
 
 Keep in mind that each of these values can be an expression.
 
@@ -158,9 +170,9 @@ registered more than a year ago::
 Advanced queries
 ================
 
-Conditional expressions can be used in annotations, aggregations, lookups, and
-updates. They can also be combined and nested with other expressions. This
-allows you to make powerful conditional queries.
+Conditional expressions can be used in annotations, aggregations, filters,
+lookups, and updates. They can also be combined and nested with other
+expressions. This allows you to make powerful conditional queries.
 
 Conditional update
 ------------------
@@ -236,3 +248,29 @@ On other databases, this is emulated using a ``CASE`` statement:
 
 The two SQL statements are functionally equivalent but the more explicit
 ``FILTER`` may perform better.
+
+Conditional filter
+------------------
+
+.. versionadded:: 3.0
+
+When a conditional expression returns a boolean value, it is possible to use it
+directly in filters. This means that it will not be added to the ``SELECT``
+columns, but you can still use it to filter results::
+
+    >>> non_unique_account_type = Client.objects.filter(
+    ...     account_type=OuterRef('account_type'),
+    ... ).exclude(pk=OuterRef('pk')).values('pk')
+    >>> Client.objects.filter(~Exists(non_unique_account_type))
+
+In SQL terms, that evaluates to:
+
+.. code-block:: sql
+
+    SELECT ...
+    FROM client c0
+    WHERE NOT EXISTS (
+      SELECT c1.id
+      FROM client c1
+      WHERE c1.account_type = c0.account_type AND NOT c1.id = c0.id
+    )
diff --git a/docs/ref/models/expressions.txt b/docs/ref/models/expressions.txt
index ccd75d62a9..16dabf8177 100644
--- a/docs/ref/models/expressions.txt
+++ b/docs/ref/models/expressions.txt
@@ -5,10 +5,11 @@ Query Expressions
 .. currentmodule:: django.db.models
 
 Query expressions describe a value or a computation that can be used as part of
-an update, create, filter, order by, annotation, or aggregate. There are a
-number of built-in expressions (documented below) that can be used to help you
-write queries. Expressions can be combined, or in some cases nested, to form
-more complex computations.
+an update, create, filter, order by, annotation, or aggregate. When an
+expression outputs a boolean value, it may be used directly in filters. There
+are a number of built-in expressions (documented below) that can be used to
+help you write queries. Expressions can be combined, or in some cases nested,
+to form more complex computations.
 
 Supported arithmetic
 ====================
@@ -69,6 +70,12 @@ Some examples
     CharField.register_lookup(Length)
     Company.objects.order_by('name__length')
 
+    # Boolean expression can be used directly in filters.
+    from django.db.models import Exists
+    Company.objects.filter(
+        Exists(Employee.objects.filter(company=OuterRef('pk'), salary__gt=10))
+    )
+
 Built-in Expressions
 ====================
 
@@ -626,22 +633,25 @@ degrade performance, it's automatically removed.
 
 You can query using ``NOT EXISTS`` with ``~Exists()``.
 
-Filtering on a ``Subquery`` expression
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Filtering on a ``Subquery()`` or ``Exists()`` expressions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-It's not possible to filter directly using ``Subquery`` and ``Exists``, e.g.::
+``Subquery()`` that returns a boolean value and ``Exists()`` may be used as a
+``condition`` in :class:`~django.db.models.expressions.When` expressions, or to
+directly filter a queryset::
 
+    >>> recent_comments = Comment.objects.filter(...)  # From above
     >>> Post.objects.filter(Exists(recent_comments))
-    ...
-    TypeError: 'Exists' object is not iterable
 
+This will ensure that the subquery will not be added to the ``SELECT`` columns,
+which may result in a better performance.
 
-You must filter on a subquery expression by first annotating the queryset
-and then filtering based on that annotation::
+.. versionchanged:: 3.0
 
-    >>> Post.objects.annotate(
-    ...     recent_comment=Exists(recent_comments),
-    ... ).filter(recent_comment=True)
+    In previous versions of Django, it was necessary to first annotate and then
+    filter against the annotation. This resulted in the annotated value always
+    being present in the query result, and often resulted in a query that took
+    more time to execute.
 
 Using aggregates within a ``Subquery`` expression
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/docs/releases/3.0.txt b/docs/releases/3.0.txt
index 722baeeeaa..f6ec8f8cc7 100644
--- a/docs/releases/3.0.txt
+++ b/docs/releases/3.0.txt
@@ -74,6 +74,13 @@ enable adding exclusion constraints on PostgreSQL. Constraints are added to
 models using the
 :attr:`Meta.constraints <django.db.models.Options.constraints>` option.
 
+Filter expressions
+------------------
+
+Expressions that outputs :class:`~django.db.models.BooleanField` may now be
+used directly in ``QuerySet`` filters, without having to first annotate and
+then filter against the annotation.
+
 Minor features
 --------------