boulder-oracle-sprint: Merged to [5462]

git-svn-id: http://code.djangoproject.com/svn/django/branches/boulder-oracle-sprint@5463 bcc190cf-cafb-0310-a4f2-bffc1f526a37

9 files changed, 370 insertions, 35 deletions

diff --git a/docs/add_ons.txt b/docs/add_ons.txt
index 4f45d99d9a..ffc4f7420f 100644
--- a/docs/add_ons.txt
+++ b/docs/add_ons.txt
@@ -6,8 +6,9 @@ Django aims to follow Python's `"batteries included" philosophy`_. It ships
 with a variety of extra, optional tools that solve common Web-development
 problems.
 
-This code lives in ``django/contrib`` in the Django distribution. Here's a
-rundown of the packages in ``contrib``:
+This code lives in ``django/contrib`` in the Django distribution. This document
+gives a rundown of the packages in ``contrib``, along with any dependencies
+those packages have.
 
 .. admonition:: Note
 
@@ -26,6 +27,8 @@ The automatic Django administrative interface. For more information, see
 
 .. _Tutorial 2: ../tutorial02/
 
+Requires the auth_ and contenttypes_ contrib packages to be installed.
+
 auth
 ====
 
@@ -144,6 +147,8 @@ See the `flatpages documentation`_.
 
 .. _flatpages documentation: ../flatpages/
 
+Requires the sites_ contrib package to be installed as well.
+
 localflavor
 ===========
 
diff --git a/docs/authentication.txt b/docs/authentication.txt
index 091a30a895..12b61db538 100644
--- a/docs/authentication.txt
+++ b/docs/authentication.txt
@@ -730,7 +730,7 @@ Django developers are currently discussing.
 Default permissions
 -------------------
 
-Three basic permissions -- add, create and delete -- are automatically created
+Three basic permissions -- add, change and delete -- are automatically created
 for each Django model that has a ``class Admin`` set. Behind the scenes, these
 permissions are added to the ``auth_permission`` database table when you run
 ``manage.py syncdb``.
diff --git a/docs/db-api.txt b/docs/db-api.txt
index 6f13b3a467..49e4ffa739 100644
--- a/docs/db-api.txt
+++ b/docs/db-api.txt
@@ -134,6 +134,15 @@ the database until you explicitly call ``save()``.
 
 The ``save()`` method has no return value.
 
+Updating ``ForeignKey`` fields works exactly the same way; simply assign an
+object of the right type to the field in question::
+
+    joe = Author.objects.create(name="Joe")
+    entry.author = joe
+    entry.save()
+    
+Django will complain if you try to assign an object of the wrong type.
+
 How Django knows to UPDATE vs. INSERT
 -------------------------------------
 
@@ -379,7 +388,7 @@ The lookup parameters (``**kwargs``) should be in the format described in
 `Field lookups`_ below. Multiple parameters are joined via ``AND`` in the
 underlying SQL statement, and the whole thing is enclosed in a ``NOT()``.
 
-This example excludes all entries whose ``pub_date`` is the current date/time
+This example excludes all entries whose ``pub_date`` is later than 2005-1-3
 AND whose ``headline`` is "Hello"::
 
     Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3), headline='Hello')
@@ -389,8 +398,8 @@ In SQL terms, that evaluates to::
     SELECT ...
     WHERE NOT (pub_date > '2005-1-3' AND headline = 'Hello')
 
-This example excludes all entries whose ``pub_date`` is the current date/time
-OR whose ``headline`` is "Hello"::
+This example excludes all entries whose ``pub_date`` is later than 2005-1-3
+AND whose headline is NOT "Hello"::
 
     Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3)).exclude(headline='Hello')
 
@@ -1229,8 +1238,8 @@ whose ``headline`` contains ``'Lennon'``::
 
     Blog.objects.filter(entry__headline__contains='Lennon')
 
-Escaping parenthesis and underscores in LIKE statements
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Escaping percent signs and underscores in LIKE statements
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The field lookups that equate to ``LIKE`` SQL statements (``iexact``,
 ``contains``, ``icontains``, ``startswith``, ``istartswith``, ``endswith``
diff --git a/docs/fastcgi.txt b/docs/fastcgi.txt
index 119688096f..81888bba76 100644
--- a/docs/fastcgi.txt
+++ b/docs/fastcgi.txt
@@ -203,7 +203,7 @@ This is probably the most common case, if you're using Django's admin site::
       DocumentRoot /home/user/public_html
       Alias /media /home/user/python/django/contrib/admin/media
       RewriteEngine On
-      RewriteRule ^/(media.*)$ /$1 [QSA,L]
+      RewriteRule ^/(media.*)$ /$1 [QSA,L,PT]
       RewriteCond %{REQUEST_FILENAME} !-f
       RewriteRule ^/(.*)$ /mysite.fcgi/$1 [QSA,L]
     </VirtualHost>
diff --git a/docs/man/django-admin.1 b/docs/man/django-admin.1
new file mode 100644
index 0000000000..fec5053ccb
--- /dev/null
+++ b/docs/man/django-admin.1
@@ -0,0 +1,162 @@
+.TH "django-admin.py" "1" "June 2007" "Django Project" ""
+.SH "NAME"
+django\-admin.py \- Utility script for the Django web framework
+.SH "SYNOPSIS"
+.B django\-admin.py
+.I <action>
+.B [options]
+.sp
+.SH "DESCRIPTION"
+This utility script provides commands for creation and maintenance of Django
+projects and apps.
+.sp
+With the exception of
+.BI startproject,
+all commands listed below can also be performed with the
+.BI manage.py
+script found at the top level of each Django project directory.
+.sp
+.SH "ACTIONS"
+.TP
+.BI "adminindex [" "appname ..." "]"
+Prints the admin\-index template snippet for the given app name(s).
+.TP
+.BI "createcachetable [" "tablename" "]"
+Creates the table needed to use the SQL cache backend
+.TP
+.B dbshell
+Runs the command\-line client for the current
+.BI DATABASE_ENGINE.
+.TP
+.B diffsettings
+Displays differences between the current
+.B settings.py
+and Django's default settings. Settings that don't appear in the defaults are
+followed by "###".
+.TP
+.B inspectdb
+Introspects the database tables in the database specified in settings.py and outputs a Django
+model module.
+.TP
+.BI "install [" "appname ..." "]"
+Executes
+.B sqlall
+for the given app(s) in the current database.
+.TP
+.BI "reset [" "appname ..." "]"
+Executes
+.B sqlreset
+for the given app(s) in the current database.
+.TP
+.BI "runfcgi [" "KEY=val" "] [" "KEY=val" "] " "..."
+Runs this project as a FastCGI application. Requires flup. Use
+.B runfcgi help
+for help on the KEY=val pairs.
+.TP
+.BI "runserver [" "\-\-noreload" "] [" "\-\-adminmedia=ADMIN_MEDIA_PATH" "] [" "port|ipaddr:port" "]"
+Starts a lightweight Web server for development.
+.TP
+.BI "shell [" "\-\-plain" "]"
+Runs a Python interactive interpreter. Tries to use IPython, if it's available.
+The
+.BI \-\-plain
+option forces the use of the standard Python interpreter even when IPython is
+installed.
+.TP
+.BI "sql [" "appname ..." "]"
+Prints the CREATE TABLE SQL statements for the given app name(s).
+.TP
+.BI "sqlall [" "appname ..." "]"
+Prints the CREATE TABLE, initial\-data and CREATE INDEX SQL statements for the
+given model module name(s).
+.TP
+.BI "sqlclear [" "appname ..." "]"
+Prints the DROP TABLE SQL statements for the given app name(s).
+.TP
+.BI "sqlindexes [" "appname ..." "]"
+Prints the CREATE INDEX SQL statements for the given model module name(s).
+.TP
+.BI "sqlinitialdata [" "appname ..." "]"
+Prints the initial INSERT SQL statements for the given app name(s).
+.TP
+.BI "sqlreset [" "appname ..." "]"
+Prints the DROP TABLE SQL, then the CREATE TABLE SQL, for the given app
+name(s).
+.TP
+.BI "sqlsequencereset [" "appname ..." "]"
+Prints the SQL statements for resetting PostgreSQL sequences for the
+given app name(s).
+.TP
+.BI "startapp [" "appname" "]"
+Creates a Django app directory structure for the given app name in
+the current directory.
+.TP
+.BI "startproject [" "projectname" "]"
+Creates a Django project directory structure for the given project name
+in the current directory.
+.TP
+.BI syncdb
+Creates the database tables for all apps in INSTALLED_APPS whose tables
+haven't already been created.
+.TP
+.BI "test [" "\-\-verbosity" "] [" "appname ..." "]"
+Runs the test suite for the specified applications, or the entire project if
+no apps are specified
+.TP
+.BI validate
+Validates all installed models.
+.SH "OPTIONS"
+.TP
+.I \-\-version
+Show program's version number and exit.
+.TP
+.I \-h, \-\-help
+Show this help message and exit.
+.TP
+.I \-\-settings=SETTINGS
+Python path to settings module, e.g. "myproject.settings.main". If
+this isn't provided, the DJANGO_SETTINGS_MODULE environment variable
+will be used.
+.TP
+.I \-\-pythonpath=PYTHONPATH
+Lets you manually add a directory the Python path,
+e.g. "/home/djangoprojects/myproject".
+.TP
+.I \-\-plain
+Use plain Python, not IPython, for the "shell" command.
+.TP
+.I \-\-noinput
+Do not prompt the user for input.
+.TP
+.I \-\-noreload
+Disable the development server's auto\-reloader.
+.TP
+.I \-\-verbosity=VERBOSITY
+Verbosity level: 0=minimal output, 1=normal output, 2=all output.
+.TP
+.I \-\-adminmedia=ADMIN_MEDIA_PATH
+Specifies the directory from which to serve admin media when using the development server.
+
+.SH "ENVIRONMENT"
+.TP
+.I DJANGO_SETTINGS_MODULE
+In the absence of the
+.BI \-\-settings
+option, this environment variable defines the settings module to be read.
+It should be in Python-import form, e.g. "myproject.settings".
+
+.SH "SEE ALSO"
+Full descriptions of all these options, with examples, as well as documentation
+for the rest of the Django framework, can be found on the Django site:
+.sp
+.I http://www.djangoproject.com/documentation/
+.sp
+or in the distributed documentation.
+.SH "AUTHORS/CREDITS"
+Originally developed at World Online in Lawrence, Kansas, USA. Refer to the
+AUTHORS file in the Django distribution for contributors.
+.sp
+.SH "LICENSE"
+New BSD license. For the full license text refer to the LICENSE file in the
+Django distribution.
+
diff --git a/docs/model-api.txt b/docs/model-api.txt
index de765f3517..2794046ebd 100644
--- a/docs/model-api.txt
+++ b/docs/model-api.txt
@@ -447,6 +447,11 @@ and doesn't give a 404 response).
 
 The admin represents this as an ``<input type="text">`` (a single-line input).
 
+``URLField`` takes an optional argument, ``maxlength``, the maximum length (in
+characters) of the field. The maxlength is enforced at the database level and
+in Django's validation. If you don't specify ``maxlength``, a default of 200
+is used.
+
 ``USStateField``
 ~~~~~~~~~~~~~~~~
 
@@ -1877,14 +1882,15 @@ rows. Example::
         row = cursor.fetchone()
         return row
 
-``connection`` and ``cursor`` simply use the standard `Python DB-API`_. If
-you're not familiar with the Python DB-API, note that the SQL statement in
-``cursor.execute()`` uses placeholders, ``"%s"``, rather than adding parameters
-directly within the SQL. If you use this technique, the underlying database
-library will automatically add quotes and escaping to your parameter(s) as
-necessary. (Also note that Django expects the ``"%s"`` placeholder, *not* the
-``"?"`` placeholder, which is used by the SQLite Python bindings. This is for
-the sake of consistency and sanity.)
+``connection`` and ``cursor`` mostly implement the standard `Python DB-API`_
+(except when it comes to `transaction handling`_). If you're not familiar with
+the Python DB-API, note that the SQL statement in ``cursor.execute()`` uses
+placeholders, ``"%s"``, rather than adding parameters directly within the SQL.
+If you use this technique, the underlying database library will automatically
+add quotes and escaping to your parameter(s) as necessary. (Also note that
+Django expects the ``"%s"`` placeholder, *not* the ``"?"`` placeholder, which is
+used by the SQLite Python bindings. This is for the sake of consistency and
+sanity.)
 
 A final note: If all you want to do is a custom ``WHERE`` clause, you can just
 just the ``where``, ``tables`` and ``params`` arguments to the standard lookup
@@ -1892,6 +1898,7 @@ API. See `Other lookup options`_.
 
 .. _Python DB-API: http://www.python.org/peps/pep-0249.html
 .. _Other lookup options: ../db-api/#extra-params-select-where-tables
+.. _transaction handling: ../transactions/
 
 Overriding default model methods
 --------------------------------
diff --git a/docs/modpython.txt b/docs/modpython.txt
index 43c5791be5..388a6168f3 100644
--- a/docs/modpython.txt
+++ b/docs/modpython.txt
@@ -51,9 +51,17 @@ whereas ``<Location>`` points at places in the URL structure of a Web site.
 ``<Directory>`` would be meaningless here.
 
 Also, if you've manually altered your ``PYTHONPATH`` to put your Django project
-on it, you'll need to tell mod_python::
+on it, you'll need to tell mod_python:
 
-    PythonPath "['/path/to/project'] + sys.path"
+.. parsed-literal::
+
+    <Location "/mysite/">
+        SetHandler python-program
+        PythonHandler django.core.handlers.modpython
+        SetEnv DJANGO_SETTINGS_MODULE mysite.settings
+        PythonDebug On
+        **PythonPath "['/path/to/project'] + sys.path"**
+    </Location>
 
 .. caution::
 
diff --git a/docs/newforms.txt b/docs/newforms.txt
index bb6c179648..e37d76643a 100644
--- a/docs/newforms.txt
+++ b/docs/newforms.txt
@@ -299,12 +299,19 @@ required. In this example, the data dictionary doesn't include a value for the
 In this above example, the ``cleaned_data`` value for ``nick_name`` is set to an
 empty string, because ``nick_name`` is ``CharField``, and ``CharField``\s treat
 empty values as an empty string. Each field type knows what its "blank" value
-is -- e.g., for ``DateField``, it's ``None`` instead of the empty string.
+is -- e.g., for ``DateField``, it's ``None`` instead of the empty string. For
+full details on each field's behavior in this case, see the "Empty value" note
+for each field in the "Built-in ``Field`` classes" section below.
+
+You can write code to perform validation for particular form fields (based on
+their name) or for the form as a whole (considering combinations of various
+fields). More information about this is in the `Custom form and field
+validation`_ section, below.
 
 Behavior of unbound forms
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
-It's meaningless to request "clean" data in a form with no data, but, for the
+It's meaningless to request "cleaned" data in a form with no data, but, for the
 record, here's what happens with unbound forms::
 
     >>> f = ContactForm()
@@ -606,8 +613,13 @@ Using forms in views and templates
 ----------------------------------
 
 Let's put this all together and use the ``ContactForm`` example in a Django
-view and template. This example view displays the contact form by default and
-validates/processes it if accessed via a POST request::
+view and template.
+
+Simple view example
+~~~~~~~~~~~~~~~~~~~
+
+This example view displays the contact form by default and validates/processes
+it if accessed via a POST request::
 
     def contact(request):
         if request.method == 'POST':
@@ -619,12 +631,12 @@ validates/processes it if accessed via a POST request::
             form = ContactForm()
         return render_to_response('contact.html', {'form': form})
 
-Simple template output
-~~~~~~~~~~~~~~~~~~~~~~
+Simple template example
+~~~~~~~~~~~~~~~~~~~~~~~
 
-The template, ``contact.html``, is responsible for displaying the form as HTML.
-To do this, we can use the techniques outlined in the "Outputting forms as HTML"
-section above.
+The template in the above view example, ``contact.html``, is responsible for
+displaying the form as HTML. To do this, we can use the techniques outlined in
+the "Outputting forms as HTML" section above.
 
 The simplest way to display a form's HTML is to use the variable on its own,
 like this::
@@ -677,7 +689,7 @@ The easiest way is to iterate over the form's fields, with
 
 This iteration technique is useful if you want to apply the same HTML
 formatting to each field, or if you don't know the names of the form fields
-ahead of time. Note that the fields will be listed in the order in which
+ahead of time. Note that the fields will be iterated over in the order in which
 they're defined in the ``Form`` class.
 
 Alternatively, you can arrange the form's fields explicitly, by name. Do that
@@ -701,7 +713,10 @@ For example::
 Subclassing forms
 -----------------
 
-If you subclass a custom ``Form`` class, the resulting ``Form`` class will
+If you have multiple ``Form`` classes that share fields, you can use
+subclassing to remove redundancy.
+
+When you subclass a custom ``Form`` class, the resulting subclass will
 include all fields of the parent class(es), followed by the fields you define
 in the subclass.
 
@@ -1202,6 +1217,114 @@ custom ``Field`` classes. To do this, just create a subclass of
 mentioned above (``required``, ``label``, ``initial``, ``widget``,
 ``help_text``).
 
+Custom form and field validation
+---------------------------------
+
+Form validation happens when the data is cleaned. If you want to customise
+this process, there are various places you can change, each one serving a
+different purpose. Thee types of cleaning methods are run during form
+processing. These are normally executed when you call the ``is_valid()``
+method on a form. There are other things that can kick of cleaning and
+validation (accessing the ``errors`` attribute or calling ``full_clean()``
+directly), but normally they won't be needed.
+
+In general, any cleaning method can raise ``ValidationError`` if there is a
+problem with the data it is processing, passing the relevant error message to
+the ``ValidationError`` constructor. If no ``ValidationError`` is raised, the
+method should return the cleaned (normalised) data as a Python object.
+
+If you detect multiple errors during a cleaning method and wish to signal all
+of them to the form submittor, it is possible to pass a list of errors to the
+``ValidationError`` constructor.
+
+The three types of cleaning methods are:
+
+    * The ``clean()`` method on a Field subclass. This is responsible
+      for cleaning the data in a way that is generic for that type of field.
+      For example, a FloatField will turn the data into a Python ``float`` or
+      raise a ``ValidationError``.
+
+    * The ``clean_<fieldname>()`` method in a form subclass -- where
+      ``<fieldname>`` is replaced with the name of the form field attribute.
+      This method does any cleaning that is specific to that particular
+      attribute, unrelated to the type of field that it is. This method is not
+      passed any parameters. You will need to look up the value of the field
+      in ``self.cleaned_data`` and remember that it will be a Python object
+      at this point, not the original string submitted in the form (it will be
+      in ``cleaned_data`` because the general field ``clean()`` method, above,
+      has already cleaned the data once).
+
+      For example, if you wanted to validate that the contents of a
+      ``CharField`` called ``serialnumber`` was unique,
+      ``clean_serialnumber()`` would be the right place to do this. You don't
+      need a specific field (it's just a ``CharField``), but you want a
+      formfield-specific piece of validation and, possibly,
+      cleaning/normalizing the data.
+
+    * The Form subclass's ``clean()`` method. This method can perform
+      any validation that requires access to multiple fields from the form at
+      once. This is where you might put in things to check that if field ``A``
+      is supplied, field ``B`` must contain a valid email address and the
+      like. The data that this method returns is the final ``cleaned_data``
+      attribute for the form, so don't forget to return the full list of
+      cleaned data if you override this method (by default, ``Form.clean()``
+      just returns ``self.cleaned_data``).
+
+      Note that any errors raised by your ``Form.clean()`` override will not
+      be associated with any field in particular. They go into a special
+      "field" (called ``__all__``, which you can access via the
+      ``non_field_errors()`` method if you need to.
+
+These methods are run in the order given above, one field at a time.  That is,
+for each field in the form (in the order they are declared in the form
+definition), the ``Field.clean()`` method (or it's override) is run, then
+``clean_<fieldname>()``. Finally, once those two methods are run for every
+field, the ``Form.clean()`` method, or it's override, is executed.
+
+As mentioned above, any of these methods can raise a ``ValidationError``. For
+any field, if the ``Field.clean()`` method raises a ``ValidationError``, any
+field-specific cleaning method is not called. However, the cleaning methods
+for all remaining fields are still executed.
+
+The ``clean()`` method for the ``Form`` class or subclass is always run. If
+that method raises a ``ValidationError``, ``cleaned_data`` will be an empty
+dictionary.
+
+The previous paragraph means that if you are overriding ``Form.clean()``, you
+should iterate through ``self.cleaned_data.items()``, possibly considering the
+``_errors`` dictionary attribute on the form as well. In this way, you will
+already know which fields have passed thei individual validation requirements.
+
+A simple example
+~~~~~~~~~~~~~~~~
+
+Here's a simple example of a custom field that validates its input is a string
+containing comma-separated e-mail addresses, with at least one address. We'll
+keep it simple and assume e-mail validation is contained in a function called
+``is_valid_email()``. The full class::
+
+    from django import newforms as forms
+
+    class MultiEmailField(forms.Field):
+        def clean(self, value):
+            emails = value.split(',')
+            for email in emails:
+                if not is_valid_email(email):
+                    raise forms.ValidationError('%s is not a valid e-mail address.' % email)
+            if not emails:
+                raise forms.ValidationError('Enter at least one e-mail address.')
+            return emails
+
+Let's alter the ongoing ``ContactForm`` example to demonstrate how you'd use
+this in a form. Simply use ``MultiEmailField`` instead of ``forms.EmailField``,
+like so::
+
+    class ContactForm(forms.Form):
+        subject = forms.CharField(max_length=100)
+        message = forms.CharField()
+        senders = MultiEmailField()
+        cc_myself = forms.BooleanField()
+
 Generating forms for models
 ===========================
 
diff --git a/docs/templates.txt b/docs/templates.txt
index a293bdcff2..cb8e238f43 100644
--- a/docs/templates.txt
+++ b/docs/templates.txt
@@ -91,9 +91,12 @@ Filters can be "chained." The output of one filter is applied to the next.
 ``{{ text|escape|linebreaks }}`` is a common idiom for escaping text contents,
 then converting line breaks to ``<p>`` tags.
 
-Some filters take arguments. A filter argument looks like this:
-``{{ bio|truncatewords:"30" }}``. This will display the first 30 words of the
-``bio`` variable. Filter arguments always are in double quotes.
+Some filters take arguments. A filter argument looks like this: ``{{
+bio|truncatewords:30 }}``. This will display the first 30 words of the ``bio``
+variable.
+
+Filter arguments that contain spaces must be quoted; for example, to join a list
+with commas and spaced you'd use ``{{ list|join:", " }}``.
 
 The `Built-in filter reference`_ below describes all the built-in filters.
 
@@ -444,7 +447,7 @@ for
 ~~~
 
 Loop over each item in an array.  For example, to display a list of athletes
-given ``athlete_list``::
+provided in ``athlete_list``::
 
     <ul>
     {% for athlete in athlete_list %}
@@ -452,7 +455,25 @@ given ``athlete_list``::
     {% endfor %}
     </ul>
 
-You can also loop over a list in reverse by using ``{% for obj in list reversed %}``.
+You can loop over a list in reverse by using ``{% for obj in list reversed %}``.
+
+**New in Django development version**
+If you need to loop over a list of lists, you can unpack the values
+in eachs sub-list into a set of known names. For example, if your context contains
+a list of (x,y) coordinates called ``points``, you could use the following
+to output the list of points:: 
+
+    {% for x, y in points %}
+        There is a point at {{ x }},{{ y }}
+    {% endfor %}
+    
+This can also be useful if you need to access the items in a dictionary. 
+For example, if your context contained a dictionary ``data``, the following
+would display the keys and values of the dictionary::
+
+    {% for key, value in data.items %}
+        {{ key }}: {{ value }}
+    {% endfor %}
 
 The for loop sets a number of variables available within the loop: