[1.3.X] Altered the behavior of URLField to avoid a potential DOS vector, and to avoid potential leakage of local filesystem data. A security announcement will be made shortly.

Backport of r16760 from trunk.

git-svn-id: http://code.djangoproject.com/svn/django/branches/releases/1.3.X@16763 bcc190cf-cafb-0310-a4f2-bffc1f526a37

4 files changed, 37 insertions, 13 deletions

diff --git a/docs/internals/deprecation.txt b/docs/internals/deprecation.txt
index c7f8bcbcc0..3f0f998b4a 100644
--- a/docs/internals/deprecation.txt
+++ b/docs/internals/deprecation.txt
@@ -108,6 +108,12 @@ their deprecation, as per the :ref:`Django deprecation policy
           beyond that of a simple ``TextField`` since the removal of oldforms.
           All uses of ``XMLField`` can be replaced with ``TextField``.
 
+       * ``django.db.models.fields.URLField.verify_exists`` has been
+         deprecated due to intractable security and performance
+         issues. Validation behavior has been removed in 1.4, and the
+         argument will be removed in 1.5.
+
+
     * 1.5
         * The ``mod_python`` request handler has been deprecated since the 1.3
           release. The ``mod_wsgi`` handler should be used instead.
diff --git a/docs/ref/forms/fields.txt b/docs/ref/forms/fields.txt
index 59a6df82d0..11647b33b2 100644
--- a/docs/ref/forms/fields.txt
+++ b/docs/ref/forms/fields.txt
@@ -756,6 +756,11 @@ Takes the following optional arguments:
     If ``True``, the validator will attempt to load the given URL, raising
     ``ValidationError`` if the page gives a 404. Defaults to ``False``.
 
+.. deprecated:: 1.3.1
+
+   ``verify_exists`` was deprecated for security reasons and will be
+   removed in 1.4. This deprecation also removes ``validator_user_agent``.
+
 .. attribute:: URLField.validator_user_agent
 
     String used as the user-agent used when checking for a URL's existence.
diff --git a/docs/ref/models/fields.txt b/docs/ref/models/fields.txt
index 2fb5d494b1..36e2b109b8 100644
--- a/docs/ref/models/fields.txt
+++ b/docs/ref/models/fields.txt
@@ -831,14 +831,21 @@ shortcuts.
 ``URLField``
 ------------
 
-.. class:: URLField([verify_exists=True, max_length=200, **options])
+.. class:: URLField([verify_exists=False, max_length=200, **options])
 
 A :class:`CharField` for a URL. Has one extra optional argument:
 
+.. deprecated:: 1.3.1
+
+   ``verify_exists`` is deprecated for security reasons as of 1.3.1
+   and will be removed in 1.4. Prior to 1.3.1, the default value was
+   ``True``.
+
 .. attribute:: URLField.verify_exists
 
-    If ``True`` (the default), the URL given will be checked for existence
-    (i.e., the URL actually loads and doesn't give a 404 response).
+    If ``True``, the URL given will be checked for existence (i.e.,
+    the URL actually loads and doesn't give a 404 response) using a
+    ``HEAD`` request. Redirects are allowed, but will not be followed.
 
     Note that when you're using the single-threaded development server,
     validating a URL being served by the same server will hang. This should not
diff --git a/docs/ref/settings.txt b/docs/ref/settings.txt
index 175e50818c..18155f19fc 100644
--- a/docs/ref/settings.txt
+++ b/docs/ref/settings.txt
@@ -1892,16 +1892,6 @@ to ensure your processes are running in the correct environment.
 
 .. _See available choices: http://www.postgresql.org/docs/8.1/static/datetime-keywords.html#DATETIME-TIMEZONE-SET-TABLE
 
-.. setting:: URL_VALIDATOR_USER_AGENT
-
-URL_VALIDATOR_USER_AGENT
-------------------------
-
-Default: ``Django/<version> (http://www.djangoproject.com/)``
-
-The string to use as the ``User-Agent`` header when checking to see if URLs
-exist (see the ``verify_exists`` option on :class:`~django.db.models.URLField`).
-
 .. setting:: USE_ETAGS
 
 USE_ETAGS
@@ -2095,3 +2085,19 @@ TEST_DATABASE_NAME
    This setting has been replaced by :setting:`TEST_NAME` in
    :setting:`DATABASES`.
 
+
+
+URL_VALIDATOR_USER_AGENT
+------------------------
+
+.. deprecated:: 1.3.1
+   This setting has been removed due to intractable performance and
+   security problems.
+
+Default: ``Django/<version> (http://www.djangoproject.com/)``
+
+The string to use as the ``User-Agent`` header when checking to see if
+URLs exist (see the ``verify_exists`` option on
+:class:`~django.db.models.URLField`). This setting was deprecated in
+1.3.1 along with ``verify_exists`` and will be removed in 1.4.
+