queryset-refactor: Merged changed from trunk up to [6463].

git-svn-id: http://code.djangoproject.com/svn/django/branches/queryset-refactor@6466 bcc190cf-cafb-0310-a4f2-bffc1f526a37

13 files changed, 239 insertions, 49 deletions

diff --git a/docs/databases.txt b/docs/databases.txt
index 21ff4c7434..213c2d666c 100644
--- a/docs/databases.txt
+++ b/docs/databases.txt
@@ -163,3 +163,118 @@ storage engine, you have a couple of options.
 
 .. _AlterModelOnSyncDB: http://code.djangoproject.com/wiki/AlterModelOnSyncDB
 
+
+Oracle Notes
+============
+
+Django supports `Oracle Database Server`_ versions 9i and higher.  Oracle
+version 10g or later is required to use Django's ``regex`` and ``iregex`` query
+operators.  You will also need the `cx_Oracle`_ driver, version 4.3.1 or newer.
+
+.. _`Oracle Database Server`: http://www.oracle.com/
+.. _`cx_Oracle`: http://cx-oracle.sourceforge.net/
+
+To run ``python manage.py syncdb``, you'll need to create an Oracle database
+user with CREATE TABLE, CREATE SEQUENCE, CREATE PROCEDURE, and CREATE TRIGGER
+privileges.  To run Django's test suite, the user also needs 
+CREATE and DROP DATABASE and CREATE and DROP TABLESPACE privileges.
+
+Connecting to the Database
+--------------------------
+
+Your Django settings.py file should look something like this for Oracle::
+
+    DATABASE_ENGINE = 'oracle'
+    DATABASE_NAME = 'xe'
+    DATABASE_USER = 'a_user'
+    DATABASE_PASSWORD = 'a_password'
+    DATABASE_HOST = ''
+    DATABASE_PORT = ''
+
+If you don't use a ``tnsnames.ora`` file or a similar naming method that
+recognizes the SID ("xe" in this example), then fill in both ``DATABASE_HOST``
+and ``DATABASE_PORT`` like so::
+
+    DATABASE_ENGINE = 'oracle'
+    DATABASE_NAME = 'xe'
+    DATABASE_USER = 'a_user'
+    DATABASE_PASSWORD = 'a_password'
+    DATABASE_HOST = 'dbprod01ned.mycompany.com'
+    DATABASE_PORT = '1540'
+
+You should supply both ``DATABASE_HOST`` and ``DATABASE_PORT``, or leave both
+as empty strings.
+
+Tablespace Options
+------------------
+
+A common paradigm for optimizing performance in Oracle-based systems is the
+use of `tablespaces`_ to organize disk layout. The Oracle backend supports
+this use case by adding ``db_tablespace`` options to the ``Meta`` and
+``Field`` classes.  (When using a backend that lacks support for tablespaces,
+these options are ignored.)
+
+.. _`tablespaces`: http://en.wikipedia.org/wiki/Tablespace
+
+A tablespace can be specified for the table(s) generated by a model by
+supplying the ``db_tablespace`` option inside the model's ``Meta`` class.
+Additionally, the ``db_tablespace`` option can be passed to a ``Field``
+constructor to specify an alternate tablespace for the ``Field``'s column
+index.  If no index would be created for the column, the ``db_tablespace``
+option is ignored.
+
+::
+
+    class TablespaceExample(models.Model):
+        name = models.CharField(maxlength=30, db_index=True, db_tablespace="indexes")
+        data = models.CharField(maxlength=255, db_index=True)
+        edges = models.ManyToManyField(to="self", db_tablespace="indexes")
+
+        class Meta:
+            db_tablespace = "tables"
+
+In this example, the tables generated by the ``TablespaceExample`` model
+(i.e., the model table and the many-to-many table) would be stored in the
+``tables`` tablespace.  The index for the name field and the indexes on the
+many-to-many table would be stored in the ``indexes`` tablespace.  The ``data``
+field would also generate an index, but no tablespace for it is specified, so
+it would be stored in the model tablespace ``tables`` by default.
+
+Django does not create the tablespaces for you.  Please refer to `Oracle's
+documentation`_ for details on creating and managing tablespaces.
+
+.. _`Oracle's documentation`: http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_7003.htm#SQLRF01403
+
+Naming Issues
+-------------
+
+Oracle imposes a name length limit of 30 characters.  To accommodate this, the
+backend truncates database identifiers to fit, replacing the final four
+characters of the truncated name with a repeatable MD5 hash value.
+
+NULL and Empty Strings
+----------------------
+
+Django generally prefers to use the empty string ('') rather than NULL, but
+Oracle treats both identically.  To get around this, the Oracle backend
+coerces the ``null=True`` option on fields that permit the empty string as a
+value.  When fetching from the database, it is assumed that a NULL value in
+one of these fields really means the empty string, and the data is silently
+converted to reflect this assumption.
+
+TextField Limitations
+---------------------
+
+The Oracle backend stores ``TextFields`` as ``NCLOB`` columns.  Oracle imposes
+some limitations on the usage of such LOB columns in general:
+
+  * LOB columns may not be used as primary keys.
+
+  * LOB columns may not be used in indexes.
+
+  * LOB columns may not be used in a ``SELECT DISTINCT`` list.  This means that
+    attempting to use the ``QuerySet.distinct`` method on a model that
+    includes ``TextField`` columns will result in an error when run against
+    Oracle.  A workaround to this is to keep ``TextField`` columns out of any
+    models that you foresee performing ``.distinct`` queries on, and to
+    include the ``TextField`` in a related model instead.
diff --git a/docs/db-api.txt b/docs/db-api.txt
index 61cf2d4ffd..adca8b4d5c 100644
--- a/docs/db-api.txt
+++ b/docs/db-api.txt
@@ -952,7 +952,7 @@ Example::
 If you pass ``in_bulk()`` an empty list, you'll get an empty dictionary.
 
 ``iterator()``
-~~~~~~~~~~~~
+~~~~~~~~~~~~~~
 
 Evaluates the ``QuerySet`` (by performing the query) and returns an
 `iterator`_ over the results. A ``QuerySet`` typically reads all of
@@ -1371,11 +1371,6 @@ equivalent::
     Entry.objects.filter(blog__id=3) # __exact is implied
     Entry.objects.filter(blog__pk=3) # __pk implies __id__exact
 
-.. note::
-    Because of this shortcut, you cannot have a field in your model called
-    ``pk`` that is not the primary key of the model. It will always be
-    replaced by the name of the model's primary key in queries.
-
 Lookups that span relationships
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/docs/django-admin.txt b/docs/django-admin.txt
index 0f99987bad..f098dfa988 100644
--- a/docs/django-admin.txt
+++ b/docs/django-admin.txt
@@ -735,3 +735,32 @@ distribution. It enables tab-completion of ``django-admin.py`` and
     * Press [TAB] to see all available options.
     * Type ``sql``, then [TAB], to see all available options whose names start
       with ``sql``.
+
+Customized actions
+==================
+
+**New in Django development version**
+
+If you want to add an action of your own to ``manage.py``, you can.
+Simply add a ``management/commands`` directory to your application.
+Each python module in that directory will be discovered and registered as
+a command that can be executed as an action when you run ``manage.py``::
+
+    /fancy_blog
+        __init__.py
+        models.py
+        /management
+            __init__.py
+            /commands
+                __init__.py
+                explode.py
+        views.py
+        
+In this example, ``explode`` command will be made available to any project
+that includes the ``fancy_blog`` application in ``settings.INSTALLED_APPS``.
+
+The ``explode.py`` module has only one requirement -- it must define a class
+called ``Command`` that extends ``django.core.management.base.BaseCommand``.
+
+For more details on how to define your own commands, look at the code for the
+existing ``django-admin.py`` commands, in ``/django/core/management/commands``.
diff --git a/docs/email.txt b/docs/email.txt
index 17c2b2115a..effc5e24cf 100644
--- a/docs/email.txt
+++ b/docs/email.txt
@@ -100,31 +100,31 @@ mail_admins()
 =============
 
 ``django.core.mail.mail_admins()`` is a shortcut for sending an e-mail to the
-site admins, as defined in the `ADMINS setting`_. Here's the definition::
+site admins, as defined in the `ADMINS`_ setting. Here's the definition::
 
     mail_admins(subject, message, fail_silently=False)
 
 ``mail_admins()`` prefixes the subject with the value of the
-`EMAIL_SUBJECT_PREFIX setting`_, which is ``"[Django] "`` by default.
+`EMAIL_SUBJECT_PREFIX`_ setting, which is ``"[Django] "`` by default.
 
-The "From:" header of the e-mail will be the value of the `SERVER_EMAIL setting`_.
+The "From:" header of the e-mail will be the value of the `SERVER_EMAIL`_ setting.
 
 This method exists for convenience and readability.
 
-.. _ADMINS setting: ../settings/#admins
-.. _EMAIL_SUBJECT_PREFIX setting: ../settings/#email-subject-prefix
-.. _SERVER_EMAIL setting: ../settings/#server-email
+.. _ADMINS: ../settings/#admins
+.. _EMAIL_SUBJECT_PREFIX: ../settings/#email-subject-prefix
+.. _SERVER_EMAIL: ../settings/#server-email
 
 mail_managers() function
 ========================
 
 ``django.core.mail.mail_managers()`` is just like ``mail_admins()``, except it
-sends an e-mail to the site managers, as defined in the `MANAGERS setting`_.
+sends an e-mail to the site managers, as defined in the `MANAGERS`_ setting.
 Here's the definition::
 
     mail_managers(subject, message, fail_silently=False)
 
-.. _MANAGERS setting: ../settings/#managers
+.. _MANAGERS: ../settings/#managers
 
 Examples
 ========
@@ -225,7 +225,7 @@ optional and can be set at any time prior to calling the ``send()`` method.
 
     * ``from_email``: The sender's address. Both ``fred@example.com`` and
       ``Fred <fred@example.com>`` forms are legal. If omitted, the
-      ``DEFAULT_FROM_EMAIL`` setting is used.
+      `DEFAULT_FROM_EMAIL`_ setting is used.
 
     * ``to``: A list or tuple of recipient addresses.
 
@@ -297,6 +297,8 @@ The class has the following methods:
 
         message.attach_file('/images/weather_map.png')
 
+.. _DEFAULT_FROM_EMAIL: ../settings/#default-from-email
+
 Sending alternative content types
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -315,7 +317,7 @@ To send a text and HTML combination, you could write::
     subject, from_email, to = 'hello', 'from@example.com', 'to@example.com'
     text_content = 'This is an important message.'
     html_content = '<p>This is an <strong>important</strong> message.</p>'
-    msg = EmailMultiAlternatives(subject, text_content, from_email, to)
+    msg = EmailMultiAlternatives(subject, text_content, from_email, [to])
     msg.attach_alternative(html_content, "text/html")
     msg.send()
 
diff --git a/docs/generic_views.txt b/docs/generic_views.txt
index 87b82f7adf..08ff01c372 100644
--- a/docs/generic_views.txt
+++ b/docs/generic_views.txt
@@ -800,9 +800,14 @@ specify the page number in the URL in one of two ways:
       variable. You can iterate over the list provided by ``page_range`` 
       to create a link to every page of results.
 
-These values and lists are is 1-based, not 0-based, so the first page would be
+These values and lists are 1-based, not 0-based, so the first page would be
 represented as page ``1``. 
 
+An example of the use of pagination can be found in the `object pagination`_
+example model. 
+	 
+.. _`object pagination`: ../models/pagination/
+
 **New in Django development version:** 
 
 As a special case, you are also permitted to use 
diff --git a/docs/i18n.txt b/docs/i18n.txt
index bf73c88008..2c43e7884e 100644
--- a/docs/i18n.txt
+++ b/docs/i18n.txt
@@ -456,10 +456,13 @@ otherwise, they'll be tacked together without whitespace!
 
 .. admonition:: Mind your charset
 
-    When creating a ``.po`` file with your favorite text editor, first edit
+    When creating a PO file with your favorite text editor, first edit
     the charset line (search for ``"CHARSET"``) and set it to the charset
-    you'll be using to edit the content. Generally, utf-8 should work for most
-    languages, but ``gettext`` should handle any charset you throw at it.
+    you'll be using to edit the content. Due to the way the ``gettext`` tools
+    work internally and because we want to allow non-ASCII source strings in
+    Django's core and your applications, you **must** use UTF-8 as the encoding
+    for your PO file (this means that everybody will be using the same
+    encoding, which is important when Django processes the PO files).
 
 To reexamine all source code and templates for new translation strings and
 update all message files for **all** languages, run this::
diff --git a/docs/install.txt b/docs/install.txt
index 2de8529d24..95aa82b2e3 100644
--- a/docs/install.txt
+++ b/docs/install.txt
@@ -66,6 +66,7 @@ installed.
 * If you're using SQLite, you'll need pysqlite_. Use version 2.0.3 or higher.
 
 * If you're using Oracle, you'll need cx_Oracle_, version 4.3.1 or higher.
+  You will also want to read the database-specific notes for the `Oracle backend`_.
 
 If you plan to use Django's ``manage.py syncdb`` command to
 automatically create database tables for your models, you'll need to
@@ -88,6 +89,7 @@ to create a temporary test database.
 .. _MySQL backend: ../databases/
 .. _cx_Oracle: http://cx-oracle.sourceforge.net/
 .. _Oracle: http://www.oracle.com/
+.. _Oracle backend: ../databases/#oracle-notes
 .. _testing framework: ../testing/
 
 Remove any old versions of Django
diff --git a/docs/newforms.txt b/docs/newforms.txt
index 2c8f67ce32..2005adeec8 100644
--- a/docs/newforms.txt
+++ b/docs/newforms.txt
@@ -1923,11 +1923,22 @@ of the model fields:
 .. note::
 
     If you specify ``fields`` when creating a form with ``form_for_model()``,
-    make sure that the fields that are *not* specified can provide default
-    values, or are allowed to have a value of ``None``. If a field isn't
-    specified on a form, the object created from the form can't provide
-    a value for that attribute, which will prevent the new instance from
-    being saved.
+    then the fields that are *not* specified will not be set by the form's
+    ``save()`` method. Django will prevent any attempt to save an incomplete
+    model, so if the model does not allow the missing fields to be empty, and
+    does not provide a default value for the missing fields, any attempt to
+    ``save()`` a ``form_for_model`` with missing fields will fail. To avoid
+    this failure, you must use ``save(commit=False)`` and manually set any
+    extra required fields::
+
+        instance = form.save(commit=False)
+        instance.required_field = 'new value'
+        instance.save()
+
+    See the `section on saving forms`_ for more details on using
+    ``save(commit=False)``.
+
+.. _section on saving forms: `The save() method`_
 
 Overriding the default field types
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2064,9 +2075,9 @@ More coming soon
 ================
 
 That's all the documentation for now. For more, see the file
-http://code.djangoproject.com/browser/django/trunk/tests/regressiontests/forms/tests.py
+http://code.djangoproject.com/browser/django/trunk/tests/regressiontests/forms
 -- the unit tests for ``django.newforms``. This can give you a good idea of
-what's possible.
+what's possible. (Each submodule there contains separate tests.)
 
 If you're really itching to learn and use this library, please be patient.
 We're working hard on finishing both the code and documentation.
diff --git a/docs/request_response.txt b/docs/request_response.txt
index bf914fb5ff..7806886841 100644
--- a/docs/request_response.txt
+++ b/docs/request_response.txt
@@ -190,7 +190,7 @@ necessary because some HTML form elements, notably
 That means you can't change attributes of ``request.POST`` and ``request.GET``
 directly.
 
-``QueryDict`` implements the all standard dictionary methods, because it's a
+``QueryDict`` implements all the standard dictionary methods, because it's a
 subclass of dictionary. Exceptions are outlined here:
 
     * ``__getitem__(key)`` -- Returns the value for the given key. If the key
diff --git a/docs/sessions.txt b/docs/sessions.txt
index 7fc607bb13..96e8d36854 100644
--- a/docs/sessions.txt
+++ b/docs/sessions.txt
@@ -205,7 +205,7 @@ An API is available to manipulate session data outside of a view::
     datetime.datetime(2005, 8, 20, 13, 35, 0)
     >>> s.save()
 
-If you're using the ``django.contrib.sessions.engine.db`` backend, each
+If you're using the ``django.contrib.sessions.backends.db`` backend, each
 session is just a normal Django model. The ``Session`` model is defined in
 ``django/contrib/sessions/models.py``. Because it's a normal model, you can
 access sessions using the normal Django database API::
diff --git a/docs/settings.txt b/docs/settings.txt
index e40374a822..7ad1f6441d 100644
--- a/docs/settings.txt
+++ b/docs/settings.txt
@@ -363,7 +363,7 @@ regular expression which will hide from the DEBUG view anything that contains
 be able to give backtraces without seeing sensitive (or offensive) settings.
 
 Still, note that there are always going to be sections of your debug output that
-are inapporpriate for public consumption. File paths, configuration options, and
+are inappropriate for public consumption. File paths, configuration options, and
 the like all give attackers extra information about your server. Never deploy a
 site with ``DEBUG`` turned on.
 
diff --git a/docs/templates_python.txt b/docs/templates_python.txt
index 232f54061f..bd105888ce 100644
--- a/docs/templates_python.txt
+++ b/docs/templates_python.txt
@@ -316,7 +316,7 @@ optional, third positional argument, ``processors``. In this example, the
         }, [ip_address_processor])
         return t.render(c)
 
-Note::
+.. note::
     If you're using Django's ``render_to_response()`` shortcut to populate a
     template with the contents of a dictionary, your template will be passed a
     ``Context`` instance by default (not a ``RequestContext``). To use a
@@ -928,10 +928,36 @@ current context, available in the ``render`` method::
 ``resolve_variable`` will try to resolve ``blog_entry.date_updated`` and then
 format it accordingly.
 
-.. note::
-    The ``resolve_variable()`` function will throw a ``VariableDoesNotExist``
-    exception if it cannot resolve the string passed to it in the current
-    context of the page.
+.. admonition:: New in development version:
+
+    Variable resolution has changed in the development version of Django.
+    ``template.resolve_variable()`` is still available, but has been deprecated
+    in favor of a new ``template.Variable`` class. Using this class will usually
+    be more efficient than calling ``template.resolve_variable``
+    
+    To use the ``Variable`` class, simply instantiate it with the name of the
+    variable to be resolved, and then call ``variable.resolve(context)``. So,
+    in the development version, the above example would be more correctly
+    written as:
+    
+    .. parsed-literal::
+    
+        class FormatTimeNode(template.Node):
+            def __init__(self, date_to_be_formatted, format_string):
+                self.date_to_be_formatted = **Variable(date_to_be_formatted)**
+                self.format_string = format_string
+        
+            def render(self, context):
+                try:
+                    actual_date = **self.date_to_be_formatted.resolve(context)**
+                    return actual_date.strftime(self.format_string)
+                except template.VariableDoesNotExist:
+                    return ''
+    
+    Changes are highlighted in bold.
+
+Variable resolution will throw a ``VariableDoesNotExist`` exception if it cannot
+resolve the string passed to it in the current context of the page.
 
 Shortcut for simple tags
 ~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/docs/unicode.txt b/docs/unicode.txt
index 1ab255970c..a0e2648f57 100644
--- a/docs/unicode.txt
+++ b/docs/unicode.txt
@@ -110,19 +110,22 @@ Conversion functions
 The ``django.utils.encoding`` module contains a few functions that are handy
 for converting back and forth between Unicode and bytestrings.
 
-    * ``smart_unicode(s, encoding='utf-8', errors='strict')`` converts its
-      input to a Unicode string. The ``encoding`` parameter specifies the input
-      encoding. (For example, Django uses this internally when processing form
-      input data, which might not be UTF-8 encoded.) The ``errors`` parameter
-      takes any of the values that are accepted by Python's ``unicode()``
-      function for its error handling.
+    * ``smart_unicode(s, encoding='utf-8', strings_only=False, errors='strict')``
+      converts its input to a Unicode string. The ``encoding`` parameter
+      specifies the input encoding. (For example, Django uses this internally
+      when processing form input data, which might not be UTF-8 encoded.) The
+      ``strings_only`` parameter, if set to True, will result in Python
+      numbers, booleans and ``None`` not being converted to a string (they keep
+      their original types). The ``errors`` parameter takes any of the values
+      that are accepted by Python's ``unicode()`` function for its error
+      handling.
 
       If you pass ``smart_unicode()`` an object that has a ``__unicode__``
       method, it will use that method to do the conversion.
 
-    * ``force_unicode(s, encoding='utf-8', errors='strict')`` is identical to
-      ``smart_unicode()`` in almost all cases. The difference is when the
-      first argument is a `lazy translation`_ instance. While
+    * ``force_unicode(s, encoding='utf-8', strings_only=False, errors='strict')``
+      is identical to ``smart_unicode()`` in almost all cases. The difference
+      is when the first argument is a `lazy translation`_ instance. While
       ``smart_unicode()`` preserves lazy translations, ``force_unicode()``
       forces those objects to a Unicode string (causing the translation to
       occur). Normally, you'll want to use ``smart_unicode()``. However,
@@ -132,11 +135,10 @@ for converting back and forth between Unicode and bytestrings.
 
     * ``smart_str(s, encoding='utf-8', strings_only=False, errors='strict')``
       is essentially the opposite of ``smart_unicode()``. It forces the first
-      argument to a bytestring. The ``strings_only`` parameter, if set to True,
-      will result in Python integers, booleans and ``None`` not being
-      converted to a string (they keep their original types). This is slightly
-      different semantics from Python's builtin ``str()`` function, but the
-      difference is needed in a few places within Django's internals.
+      argument to a bytestring. The ``strings_only`` parameter has the same
+      behaviour as for ``smart_unicode()`` and ``force_unicode()``. This is
+      slightly different semantics from Python's builtin ``str()`` function,
+      but the difference is needed in a few places within Django's internals.
 
 Normally, you'll only need to use ``smart_unicode()``. Call it as early as
 possible on any input data that might be either Unicode or a bytestring, and