Fixed #18757, #14462, #21565 -- Reworked database-python type conversions

Complete rework of translating data values from database

Deprecation of SubfieldBase, removal of resolve_columns and
convert_values in favour of a more general converter based approach and
public API Field.from_db_value(). Now works seamlessly with aggregation,
.values() and raw queries.

Thanks to akaariai in particular for extensive advice and inspiration,
also to shaib, manfre and timograham for their reviews.

1 files changed, 37 insertions, 17 deletions

diff --git a/docs/ref/models/fields.txt b/docs/ref/models/fields.txt
index e06a359ec8..d7f15d6643 100644
--- a/docs/ref/models/fields.txt
+++ b/docs/ref/models/fields.txt
@@ -1532,7 +1532,7 @@ Field API reference
     ``Field`` is an abstract class that represents a database table column.
     Django uses fields to create the database table (:meth:`db_type`), to map
     Python types to database (:meth:`get_prep_value`) and vice-versa
-    (:meth:`to_python`), and to apply :doc:`/ref/models/lookups`
+    (:meth:`from_db_value`), and to apply :doc:`/ref/models/lookups`
     (:meth:`get_prep_lookup`).
 
     A field is thus a fundamental piece in different Django APIs, notably,
@@ -1609,17 +1609,26 @@ Field API reference
 
         See :ref:`converting-query-values-to-database-values` for usage.
 
-    When loading data, :meth:`to_python` is used:
+    When loading data, :meth:`from_db_value` is used:
 
-    .. method:: to_python(value)
+    .. method:: from_db_value(value, connection)
+
+        .. versionadded:: 1.8
+
+        Converts a value as returned by the database to a Python object. It is
+        the reverse of :meth:`get_prep_value`.
 
-        Converts a value as returned by the database (or a serializer) to a
-        Python object. It is the reverse of :meth:`get_prep_value`.
+        This method is not used for most built-in fields as the database
+        backend already returns the correct Python type, or the backend itself
+        does the conversion.
 
-        The default implementation returns ``value``, which is the common case
-        when the database backend already returns the correct Python type.
+        See :ref:`converting-values-to-python-objects` for usage.
 
-        See :ref:`converting-database-values-to-python-objects` for usage.
+        .. note::
+        
+            For performance reasons, ``from_db_value`` is not implemented as a
+            no-op on fields which do not require it (all Django fields).
+            Consequently you may not call ``super`` in your definition.
 
     When saving, :meth:`pre_save` and :meth:`get_db_prep_save` are used:
 
@@ -1644,15 +1653,6 @@ Field API reference
 
         See :ref:`preprocessing-values-before-saving` for usage.
 
-    Besides saving to the database, the field also needs to know how to
-    serialize its value (inverse of :meth:`to_python`):
-
-    .. method:: value_to_string(obj)
-
-        Converts ``obj`` to a string. Used to serialize the value of the field.
-
-        See :ref:`converting-model-field-to-serialization` for usage.
-
     When a lookup is used on a field, the value may need to be "prepared".
     Django exposes two methods for this:
 
@@ -1682,6 +1682,26 @@ Field API reference
         ``prepared`` describes whether the value has already been prepared with
         :meth:`get_prep_lookup`.
 
+    Fields often receive their values as a different type, either from
+    serialization or from forms.
+    
+    .. method:: to_python(value)
+
+        Converts the value into the correct Python object. It acts as the
+        reverse of :meth:`value_to_string`, and is also called in
+        :meth:`~django.db.models.Model.clean`.
+
+        See :ref:`converting-values-to-python-objects` for usage.
+
+    Besides saving to the database, the field also needs to know how to
+    serialize its value:
+
+    .. method:: value_to_string(obj)
+
+        Converts ``obj`` to a string. Used to serialize the value of the field.
+
+        See :ref:`converting-model-field-to-serialization` for usage.
+
     When using :class:`model forms <django.forms.ModelForm>`, the ``Field``
     needs to know which form field it should be represented by: