Fixed #6262 -- Added a cached template loader, and modified existing template loaders and tag to be cacheable. Thanks to Mike Malone for the patch.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@11862 bcc190cf-cafb-0310-a4f2-bffc1f526a37

6 files changed, 234 insertions, 22 deletions

diff --git a/docs/howto/custom-template-tags.txt b/docs/howto/custom-template-tags.txt
index c6f76772de..774d12dc44 100644
--- a/docs/howto/custom-template-tags.txt
+++ b/docs/howto/custom-template-tags.txt
@@ -463,6 +463,85 @@ new ``Context`` in this example, the results would have *always* been
 automatically escaped, which may not be the desired behavior if the template
 tag is used inside a ``{% autoescape off %}`` block.
 
+.. _template_tag_thread_safety:
+
+Thread-safety considerations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 1.2
+
+Once a node is parsed, its ``render`` method may be called any number of times.
+Since Django is sometimes run in multi-threaded environments, a single node may
+be simultaneously rendering with different contexts in response to two separate
+requests. Therefore, it's important to make sure your template tags are thread
+safe.
+
+To make sure your template tags are thread safe, you should never store state
+information on the node itself. For example, Django provides a builtin ``cycle``
+template tag that cycles among a list of given strings each time it's rendered::
+
+    {% for o in some_list %}
+        <tr class="{% cycle 'row1' 'row2' %}>
+            ...
+        </tr>
+    {% endfor %}
+
+A naive implementation of ``CycleNode`` might look something like this::
+
+    class CycleNode(Node):
+        def __init__(self, cyclevars):
+            self.cycle_iter = itertools.cycle(cyclevars)
+        def render(self, context):
+            return self.cycle_iter.next()
+
+But, suppose we have two templates rendering the template snippet from above at
+the same time:
+
+    1. Thread 1 performs its first loop iteration, ``CycleNode.render()``
+       returns 'row1'
+    2. Thread 2 performs its first loop iteration, ``CycleNode.render()``
+       returns 'row2'
+    3. Thread 1 performs its second loop iteration, ``CycleNode.render()``
+       returns 'row1'
+    4. Thread 2 performs its second loop iteration, ``CycleNode.render()``
+       returns 'row2'
+
+The CycleNode is iterating, but it's iterating globally. As far as Thread 1
+and Thread 2 are concerned, it's always returning the same value. This is
+obviously not what we want!
+
+To address this problem, Django provides a ``render_context`` that's associated
+with the ``context`` of the template that is currently being rendered. The
+``render_context`` behaves like a Python dictionary, and should be used to store
+``Node`` state between invocations of the ``render`` method.
+
+Let's refactor our ``CycleNode`` implementation to use the ``render_context``::
+
+    class CycleNode(Node):
+        def __init__(self, cyclevars):
+            self.cyclevars = cyclevars
+        def render(self, context):
+            if self not in context.render_context:
+                context.render_context[self] = itertools.cycle(self.cyclevars)
+            cycle_iter = context.render_context[self]
+            return cycle_iter.next()
+
+Note that it's perfectly safe to store global information that will not change
+throughout the life of the ``Node`` as an attribute. In the case of
+``CycleNode``, the ``cyclevars`` argument doesn't change after the ``Node`` is
+instantiated, so we don't need to put it in the ``render_context``. But state
+information that is specific to the template that is currently being rendered,
+like the current iteration of the ``CycleNode``, should be stored in the
+``render_context``.
+
+.. note::
+    Notice how we used ``self`` to scope the ``CycleNode`` specific information
+    within the ``render_context``. There may be multiple ``CycleNodes`` in a
+    given template, so we need to be careful not to clobber another node's state
+    information. The easiest way to do this is to always use ``self`` as the key
+    into ``render_context``. If you're keeping track of several state variables,
+    make ``render_context[self]`` a dictionary.
+
 Registering the tag
 ~~~~~~~~~~~~~~~~~~~
 
diff --git a/docs/internals/deprecation.txt b/docs/internals/deprecation.txt
index d93340ebbe..b54ae6e61a 100644
--- a/docs/internals/deprecation.txt
+++ b/docs/internals/deprecation.txt
@@ -36,14 +36,20 @@ their deprecation, as per the :ref:`Django deprecation policy
           manager in the ``User`` model (``user.message_set``), and the
           associated methods (``user.message_set.create()`` and
           ``user.get_and_delete_messages()``), which have
-          been deprecated since the 1.2 release, will be removed.  The 
-          :ref:`messages framework <ref-contrib-messages>` should be used 
+          been deprecated since the 1.2 release, will be removed.  The
+          :ref:`messages framework <ref-contrib-messages>` should be used
           instead.
 
         * Authentication backends need to support the ``obj`` parameter for
           permission checking. The ``supports_object_permissions`` variable
           is not checked any longer and can be removed.
 
+        * The ability to specify a callable template loader rather than a
+          ``Loader`` class will be removed, as will the ``load_template_source``
+          functions that are included with the built in template loaders for
+          backwards compatibility. These have been deprecated since the 1.2
+          release.
+
     * 2.0
         * ``django.views.defaults.shortcut()``. This function has been moved
           to ``django.contrib.contenttypes.views.shortcut()`` as part of the
diff --git a/docs/ref/contrib/sitemaps.txt b/docs/ref/contrib/sitemaps.txt
index e7cd224e3e..82a4d15cf4 100644
--- a/docs/ref/contrib/sitemaps.txt
+++ b/docs/ref/contrib/sitemaps.txt
@@ -36,7 +36,7 @@ To install the sitemap app, follow these steps:
     1. Add ``'django.contrib.sitemaps'`` to your :setting:`INSTALLED_APPS`
        setting.
        
-    2. Make sure ``'django.template.loaders.app_directories.load_template_source'``
+    2. Make sure ``'django.template.loaders.app_directories.Loader'``
        is in your :setting:`TEMPLATE_LOADERS` setting. It's in there by default,
        so you'll only need to change this if you've changed that setting.
 
@@ -45,7 +45,7 @@ To install the sitemap app, follow these steps:
 
 (Note: The sitemap application doesn't install any database tables. The only
 reason it needs to go into :setting:`INSTALLED_APPS` is so that the
-:func:`~django.template.loaders.app_directories.load_template_source` template
+:func:`~django.template.loaders.app_directories.Loader` template
 loader can find the default templates.)
 
 Initialization
diff --git a/docs/ref/settings.txt b/docs/ref/settings.txt
index 9730c1ddf4..8ff15097ac 100644
--- a/docs/ref/settings.txt
+++ b/docs/ref/settings.txt
@@ -819,7 +819,7 @@ MESSAGE_LEVEL
 
 Default: `messages.INFO`
 
-Sets the minimum message level that will be recorded by the messages 
+Sets the minimum message level that will be recorded by the messages
 framework. See the :ref:`messages documentation <ref-contrib-messages>` for
 more details.
 
@@ -1150,11 +1150,14 @@ TEMPLATE_LOADERS
 
 Default::
 
-     ('django.template.loaders.filesystem.load_template_source',
-      'django.template.loaders.app_directories.load_template_source')
+     ('django.template.loaders.filesystem.Loader',
+      'django.template.loaders.app_directories.Loader')
 
-A tuple of callables (as strings) that know how to import templates from
-various sources. See :ref:`ref-templates-api`.
+A tuple of template loader classes, specified as strings. Each ``Loader`` class
+knows how to import templates from a particular sources. Optionally, a tuple can be
+used instead of a string. The first item in the tuple should be the ``Loader``'s
+module, subsequent items are passed to the ``Loader`` during initialization. See
+:ref:`ref-templates-api`.
 
 .. setting:: TEMPLATE_STRING_IF_INVALID
 
diff --git a/docs/ref/templates/api.txt b/docs/ref/templates/api.txt
index 077325b48e..fa42949eea 100644
--- a/docs/ref/templates/api.txt
+++ b/docs/ref/templates/api.txt
@@ -322,7 +322,7 @@ and return a dictionary of items to be merged into the context. By default,
    cannot be turned off by the :setting:`TEMPLATE_CONTEXT_PROCESSORS` setting.
 
 .. versionadded:: 1.2
-   The ``'messages'`` context processor was added.  For more information, see 
+   The ``'messages'`` context processor was added.  For more information, see
    the :ref:`messages documentation <ref-contrib-messages>`.
 
 Each processor is applied in order. That means, if one processor adds a
@@ -379,7 +379,7 @@ If :setting:`TEMPLATE_CONTEXT_PROCESSORS` contains this processor, every
 
 .. versionchanged:: 1.2
    Prior to version 1.2, the ``messages`` variable was a lazy accessor for
-   ``user.get_and_delete_messages()``. It has been changed to include any 
+   ``user.get_and_delete_messages()``. It has been changed to include any
    messages added via the :ref:`messages framework <ref-contrib-messages`.
 
 django.core.context_processors.debug
@@ -448,7 +448,7 @@ If :setting:`TEMPLATE_CONTEXT_PROCESSORS` contains this processor, every
    context processor.  For backwards compatibility the ``'auth'`` context
    processor will continue to supply the ``messages`` variable until Django
    1.4.  If you use the ``messages`` variable, your project will work with
-   either (or both) context processors, but it is recommended to add 
+   either (or both) context processors, but it is recommended to add
    ``django.contrib.messages.context_processors.messages`` so your project
    will be prepared for the future upgrade.
 
@@ -571,11 +571,11 @@ by editing your :setting:`TEMPLATE_LOADERS` setting. :setting:`TEMPLATE_LOADERS`
 should be a tuple of strings, where each string represents a template loader.
 Here are the template loaders that come with Django:
 
-``django.template.loaders.filesystem.load_template_source``
+``django.template.loaders.filesystem.Loader``
     Loads templates from the filesystem, according to :setting:`TEMPLATE_DIRS`.
     This loader is enabled by default.
 
-``django.template.loaders.app_directories.load_template_source``
+``django.template.loaders.app_directories.Loader``
     Loads templates from Django apps on the filesystem. For each app in
     :setting:`INSTALLED_APPS`, the loader looks for a ``templates``
     subdirectory. If the directory exists, Django looks for templates in there.
@@ -599,12 +599,43 @@ Here are the template loaders that come with Django:
 
     This loader is enabled by default.
 
-``django.template.loaders.eggs.load_template_source``
+``django.template.loaders.eggs.Loader``
     Just like ``app_directories`` above, but it loads templates from Python
     eggs rather than from the filesystem.
 
     This loader is disabled by default.
 
+``django.template.loaders.cached.Loader``
+    By default, the templating system will read and compile your templates every
+    time they need to be rendered. While the Django templating system is quite
+    fast, the overhead from reading and compiling templates can add up.
+
+    The cached template loader is a class-based loader that you configure with
+    a list of other loaders that it should wrap. The wrapped loaders are used to
+    locate unknown templates when they are first encountered. The cached loader
+    then stores the compiled ``Template`` in memory. The cached ``Template``
+    instance is returned for subsequent requests to load the same template.
+
+    For example, to enable template caching with the ``filesystem`` and
+    ``app_directories`` template loaders you might use the following settings::
+
+        TEMPLATE_LOADERS = (
+            ('django.template.loaders.cached.Loader', (
+                'django.template.loaders.filesystem.Loader',
+                'django.template.loaders.app_directories.Loader',
+            )),
+        )
+
+    .. note::
+        All of the built-in Django template tags are safe to use with the cached
+        loader, but if you're using custom template tags that come from third
+        party packages, or that you wrote yourself, you should ensure that the
+        ``Node`` implementation for each tag is thread-safe. For more
+        information, see
+        :ref:`template tag thread safety considerations<template_tag_thread_safety>`.
+
+    This loader is disabled by default.
+
 Django uses the template loaders in order according to the
 :setting:`TEMPLATE_LOADERS` setting. It uses each loader until a loader finds a
 match.
@@ -667,3 +698,68 @@ settings you wish to specify. You might want to consider setting at least
 and :setting:`TEMPLATE_DEBUG`. All available settings are described in the
 :ref:`settings documentation <ref-settings>`, and any setting starting with
 ``TEMPLATE_`` is of obvious interest.
+
+.. _topic-template-alternate-language:
+
+Using an alternative template language
+======================================
+
+.. versionadded 1.2
+
+The Django ``Template`` and ``Loader`` classes implement a simple API for
+loading and rendering templates. By providing some simple wrapper classes that
+implement this API we can use third party template systems like `Jinja2
+<http://jinja.pocoo.org/2/>`_ or `Cheetah <http://www.cheetahtemplate.org/>`_. This
+allows us to use third-party template libraries without giving up useful Django
+features like the Django ``Context`` object and handy shortcuts like
+``render_to_response()``.
+
+The core component of the Django templating system is the ``Template`` class.
+This class has a very simple interface: it has a constructor that takes a single
+positional argument specifying the template string, and a ``render()`` method
+that takes a ``django.template.context.Context`` object and returns a string
+containing the rendered response.
+
+Suppose we're using a template language that defines a ``Template`` object with
+a ``render()`` method that takes a dictionary rather than a ``Context`` object.
+We can write a simple wrapper that implements the Django ``Template`` interface::
+
+    import some_template_language
+    class Template(some_template_language.Template):
+        def render(self, context):
+            # flatten the Django Context into a single dictionary.
+            context_dict = {}
+            for d in context.dicts:
+                context_dict.update(d)
+            return super(Template, self).render(context_dict)
+
+That's all that's required to make our fictional ``Template`` class compatible
+with the Django loading and rendering system!
+
+The next step is to write a ``Loader`` class that returns instances of our custom
+template class instead of the default ``django.template.Template``. Custom ``Loader``
+classes should inherit from ``django.template.loader.BaseLoader`` and override
+the ``load_template_source()`` method, which takes a ``template_name`` argument,
+loads the template from disk (or elsewhere), and returns a tuple:
+``(template_string, template_origin)``.
+
+The ``load_template()`` method of the ``Loader`` class retrieves the template
+string by calling ``load_template_source()``, instantiates a ``Template`` from
+the template source, and returns a tuple: ``(template, template_origin)``. Since
+this is the method that actually instantiates the ``Template``, we'll need to
+override it to use our custom template class instead. We can inherit from the
+builtin ``django.template.loaders.app_directories.Loader`` to take advantage of
+the ``load_template_source()`` method implemented there::
+
+    from django.template.loaders import app_directories
+    class Loader(app_directories.Loader):
+        is_usable = True
+
+        def load_template(self, template_name, template_dirs=None):
+            source, origin = self.load_template_source(template_name, template_dirs)
+            template = Template(source)
+            return template, origin
+
+Finally, we need to modify our project settings, telling Django to use our custom
+loader. Now we can write all of our templates in our alternative template
+language while continuing to use the rest of the Django templating system.
diff --git a/docs/releases/1.2.txt b/docs/releases/1.2.txt
index 22560a1f1f..62f536d733 100644
--- a/docs/releases/1.2.txt
+++ b/docs/releases/1.2.txt
@@ -76,6 +76,19 @@ changes:
 
        __members__ = property(lambda self: self.__dir__())
 
+Stateful template tags
+----------------------
+
+Template tags that store rendering state on the node itself may experience
+problems if they are used with the new :ref:`cached
+template loader<template-loaders>`.
+
+All of the built-in Django template tags are safe to use with the cached
+loader, but if you're using custom template tags that come from third
+party packages, or that you wrote yourself, you should ensure that the
+``Node`` implementation for each tag is thread-safe. For more
+information, see
+:ref:`template tag thread safety considerations<template_tag_thread_safety>`.
 
 .. _deprecated-features-1.2:
 
@@ -130,11 +143,11 @@ additional arguments, those arguments can be passed to the
 :meth:`~django.core.mail.get_connection()` call::
 
     connection = get_connection('django.core.mail.backends.smtp', hostname='localhost', port=1234)
-    
+
 User Messages API
 -----------------
 
-The API for storing messages in the user ``Message`` model (via 
+The API for storing messages in the user ``Message`` model (via
 ``user.message_set.create``) is now deprecated and will be removed in Django
 1.4 according to the standard :ref:`release process <internals-release-process>`.
 
@@ -147,20 +160,20 @@ with the following::
     from django.contrib import messages
     messages.add_message(request, messages.INFO, 'a message')
 
-Additionally, if you make use of the method, you need to replace the 
+Additionally, if you make use of the method, you need to replace the
 following::
 
     for message in user.get_and_delete_messages():
         ...
-    
+
 with::
 
     from django.contrib import messages
     for message in messages.get_messages(request):
         ...
-    
-For more information, see the full 
-:ref:`messages documentation <ref-contrib-messages>`. You should begin to 
+
+For more information, see the full
+:ref:`messages documentation <ref-contrib-messages>`. You should begin to
 update your code to use the new API immediately.
 
 What's new in Django 1.2
@@ -239,3 +252,18 @@ Also, filters may now be used in the ``if`` expression. For example:
           class="highlight"
         {% endif %}
       >{{ message }}</div>
+
+Template caching
+----------------
+
+In previous versions of Django, every time you rendered a template it
+would be reloaded from disk. In Django 1.2, you can use a :ref:`cached
+template loader <template-loaders>` to load templates once, then use a
+cached the result for every subsequent render. This can lead to a
+significant performance improvement if your templates are broken into
+lots of smaller subtemplates (using the ``{% extends %}`` or ``{%
+include %}`` tags).
+
+As a side effect, it is now much easier to support non-Django template
+languages. For more details, see the :ref:`notes on supporting
+non-Django template languages<topic-template-alternate-language>`.